From 9defd48a06903b0c7575716f57a0b06044702665 Mon Sep 17 00:00:00 2001 From: Max Lund Date: Mon, 18 May 2026 21:24:01 -0500 Subject: [PATCH 1/8] =?UTF-8?q?Rename=20folders=20=E2=86=92=20memory=20in?= =?UTF-8?q?=20HCL,=20keep=20folder=20names=20as=20aliases?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit User-facing rename: the HCL spellings `shared_memory`, `memory`, `run_memory`, `memories =`, and `shared_memories.NAME` are now the preferred names for what were previously called folders. All of the original spellings — `shared_folder`, `folder`, `run_folder`, `folders =`, `shared_folders.NAME` — keep working unchanged as backwards-compatible aliases, so existing configs need no edits. Setting both spellings for the same slot on a single mission (e.g. `memory {}` + `folder {}`, or `memories = ...` + `folders = ...`) is rejected at config-load with a clear error so it can't silently disagree. Scope is HCL-only by design: Go types (SharedFolder, MissionFolder, etc.), the file_* tool names, and the `folder` parameter on those tools are unchanged. Docs (CLAUDE.md, missions/folders.mdx, missions/overview.mdx, config/overview.mdx, README) updated to lead with the memory spellings and note the old names as aliases. Tests: 5 new specs covering the memory spellings and the cross-name conflict errors; 2 existing specs updated for the refreshed error strings. Full `go test ./...` passes. --- CLAUDE.md | 51 +++++++------ README.md | 2 +- config/config.go | 75 +++++++++++++------ config/folder_test.go | 113 ++++++++++++++++++++++++++++- docs/content/config/overview.mdx | 2 +- docs/content/missions/_meta.js | 2 +- docs/content/missions/folders.mdx | 64 +++++++++------- docs/content/missions/overview.mdx | 6 +- 8 files changed, 236 insertions(+), 79 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 6c97006..0cd3048 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -282,39 +282,44 @@ mission "expensive_research" { The scheduler lives in `scheduler/` but its lifecycle (creation, config updates, shutdown) is managed by `cmd/serve.go`, not wsbridge. The wsbridge client receives a `ConcurrencyTracker` interface for enforcing `max_parallel` on all mission starts. The cron library used is `robfig/cron/v3`. -### Folders +### Memory (formerly "Folders") -Folders are sandboxed filesystem locations that agents access via the `file_list`, -`file_read`, `file_create`, `file_delete`, `file_search`, and `file_grep` tools. -The `folder` parameter is required on every tool call — there is no implicit -default. +Memory blocks are sandboxed filesystem locations that agents access via the +`file_list`, `file_read`, `file_create`, `file_delete`, `file_search`, and +`file_grep` tools. The `folder` parameter (kept under that name on the tools +for back-compat) is required on every tool call — there is no implicit default. Three kinds exist, with strict naming rules: -| Kind | HCL | Registered name | Scope | Persistence | -|------|-----|-----------------|-------|-------------| -| Shared | top-level `shared_folder "name" { ... }` | user-chosen name | referenced by any mission via `folders = [shared_folders.name]` | persists | -| Mission | `folder { ... }` inside a mission | literal `"mission"` | one per mission | persists across runs | -| Run | `run_folder { ... }` inside a mission | literal `"run"` | one per mission per run | ephemeral; path is `//` | +| Kind | HCL (preferred) | HCL (back-compat alias) | Registered name | Scope | Persistence | +|------|-----------------|--------------------------|-----------------|-------|-------------| +| Shared | `shared_memory "name" { ... }` (top-level) | `shared_folder "name" { ... }` | user-chosen name | referenced by any mission via `memories = [shared_memories.name]` (or `folders = [shared_folders.name]`) | persists | +| Mission | `memory { ... }` (inside a mission) | `folder { ... }` | literal `"mission"` | one per mission | persists across runs | +| Run | `run_memory { ... }` (inside a mission) | `run_folder { ... }` | literal `"run"` | one per mission per run | ephemeral; path is `//` | -The names `"mission"` and `"run"` are reserved — `shared_folder` cannot use them -(enforced in `config/shared_folder.go`). +The names `"mission"` and `"run"` are reserved — `shared_memory` / +`shared_folder` cannot use them (enforced in `config/shared_folder.go`). + +The old `folders` / `folder` / `run_folder` / `shared_folder` / `shared_folders` +spellings keep working unchanged. Using *both* spellings for the same slot on +one mission (e.g. `memory {}` and `folder {}`, or `memories =` and `folders =`) +is rejected at config-load time. ```hcl -shared_folder "reference" { +shared_memory "reference" { path = "./data/reference" editable = false # default read-only } mission "analyze" { - folders = [shared_folders.reference] + memories = [shared_memories.reference] - folder { + memory { path = "./analyses" description = "Cumulative reports — persists across runs" } - run_folder { + run_memory { base = "./runs" # optional, default ".squadron/runs" description = "Per-run scratch" cleanup = 7 # optional, auto-delete after N days; defaults to 7, set 0 to keep forever @@ -326,14 +331,18 @@ mission "analyze" { - `config.SharedFolder`, `config.MissionFolder`, `config.MissionRunFolder` in [config/shared_folder.go](config/shared_folder.go) and [config/mission.go](config/mission.go). -- Shared folders are parsed in Stage 1.5 (with `vars` context). The mission's - `folder` and `run_folder` blocks are parsed inside the mission block in Stage 5. + Both HCL spellings decode into these same structs — there is no separate + "memory" type internally. +- Shared memory blocks are parsed in Stage 1.5 (with `vars` context). The + mission's `memory`/`folder` and `run_memory`/`run_folder` blocks are parsed + inside the mission block in Stage 5. The `shared_memories.NAME` / + `shared_folders.NAME` HCL namespaces are both registered. - [mission/folder_store.go](mission/folder_store.go) is the authoritative runtime resolver. `buildFolderStore(mission, sharedFolders, missionID)` must be called **after** the mission ID is assigned in `Runner.Run()`, because the run - folder path depends on it. There is no implicit default folder — every - tool call must name the folder explicitly. -- Run folders are materialized at `//` and a sidecar + memory path depends on it. There is no implicit default memory — every tool + call must name the slot explicitly via the `folder` parameter. +- Run memory blocks are materialized at `//` and a sidecar `.squadron-run.json` records `created_at` + `cleanup_days`. - `mission.SweepExpiredRunFolders(base)` walks a base directory and deletes any subfolder whose sidecar says it is past its cleanup deadline. It runs diff --git a/README.md b/README.md index 378cbe1..6809c1a 100644 --- a/README.md +++ b/README.md @@ -111,7 +111,7 @@ Honest tradeoffs on each comparison page. - **[MCP host](https://docs.squadron.sh/config/mcp_host)** — run Squadron as an MCP server for Claude Desktop, Claude Code, Cursor, and other MCP clients. - **[Schedules & triggers](https://docs.squadron.sh/missions/schedules)** — cron-based or HTTP-triggered missions, with per-mission concurrency limits. - **[Budgets](https://docs.squadron.sh/missions/budgets)** — token + dollar caps per mission or per task. -- **[Folders](https://docs.squadron.sh/missions/folders)** — sandboxed filesystem locations agents can read/write, with shared, mission-scoped, and ephemeral run-scoped variants. +- **[Memory](https://docs.squadron.sh/missions/folders)** — sandboxed filesystem locations agents can read/write, with shared, mission-scoped, and ephemeral run-scoped variants. (HCL: `shared_memory`, `memory`, `run_memory` — the old `shared_folder`/`folder`/`run_folder` spellings still work.) - **[Gateways](https://docs.squadron.sh/config/gateways)** — managed subprocesses that bridge Squadron to Slack, Discord, Teams, or any custom system via the Gateway SDK. - **Encrypted vault** — secrets stored at rest with AES-256-GCM + Argon2id; passphrase in the OS keychain. - **Single binary, no runtime deps.** Docker images on every release. diff --git a/config/config.go b/config/config.go index 63b1e44..c8bb07b 100644 --- a/config/config.go +++ b/config/config.go @@ -721,6 +721,7 @@ func loadFromFiles(files []string) (*Config, error) { {Type: "storage"}, {Type: "command_center"}, {Type: "shared_folder", LabelNames: []string{"name"}}, + {Type: "shared_memory", LabelNames: []string{"name"}}, {Type: "mcp_host"}, {Type: "mcp", LabelNames: []string{"name"}}, {Type: "skill", LabelNames: []string{"name"}}, @@ -752,7 +753,7 @@ func loadFromFiles(files []string) (*Config, error) { pb.Storage = append(pb.Storage, block) case "command_center": pb.CommandCenter = append(pb.CommandCenter, block) - case "shared_folder": + case "shared_folder", "shared_memory": pb.SharedFolders = append(pb.SharedFolders, block) case "mcp_host": pb.MCPHost = append(pb.MCPHost, block) @@ -949,7 +950,10 @@ func loadFromFiles(files []string) (*Config, error) { } } - // Parse shared_folder blocks (optional, with vars context) + // Parse shared_folder / shared_memory blocks (optional, with vars context). + // Both block types share the same SharedFolder struct — shared_memory is + // the preferred spelling; shared_folder is kept as a backwards-compatible + // alias. var allSharedFolders []SharedFolder for _, pb := range allParsedBlocks { for _, block := range pb.SharedFolders { @@ -957,7 +961,7 @@ func loadFromFiles(files []string) (*Config, error) { fb.Name = block.Labels[0] diags := gohcl.DecodeBody(block.Body, varsCtx, &fb) if diags.HasErrors() { - return nil, fmt.Errorf("shared_folder '%s': %w", fb.Name, diags) + return nil, fmt.Errorf("%s '%s': %w", block.Type, fb.Name, diags) } allSharedFolders = append(allSharedFolders, fb) } @@ -1144,13 +1148,17 @@ func loadFromFiles(files []string) (*Config, error) { // Build agents context (add to full context) agentsCtx := buildAgentsContext(skillsCtx, allAgents) - // Add shared_folders namespace for mission references + // Add shared_folders / shared_memories namespaces for mission references. + // Both names point at the same map — shared_memories is the preferred + // spelling; shared_folders is kept as a backwards-compatible alias. if len(allSharedFolders) > 0 { folderMap := make(map[string]cty.Value) for _, f := range allSharedFolders { folderMap[f.Name] = cty.StringVal(f.Name) } - agentsCtx.Variables["shared_folders"] = cty.ObjectVal(folderMap) + ns := cty.ObjectVal(folderMap) + agentsCtx.Variables["shared_folders"] = ns + agentsCtx.Variables["shared_memories"] = ns } // Stage 5: Load missions (with vars + models + tools + agents + shared_folders context) @@ -1771,7 +1779,8 @@ func parseMissionBlock(block *hcl.Block, ctx *hcl.EvalContext) (*Mission, error) Attributes: []hcl.AttributeSchema{ {Name: "agents", Required: true}, {Name: "directive"}, - {Name: "folders"}, + {Name: "folders"}, // deprecated alias for "memories" + {Name: "memories"}, // shared memory references {Name: "max_parallel"}, {Name: "inputs"}, // shorthand: inputs = { field = string("desc", { default = "val" }) } }, @@ -1782,8 +1791,10 @@ func parseMissionBlock(block *hcl.Block, ctx *hcl.EvalContext) (*Mission, error) {Type: "input", LabelNames: []string{"name"}}, // verbose input blocks still supported {Type: "dataset", LabelNames: []string{"name"}}, {Type: "secret", LabelNames: []string{"name"}}, - {Type: "folder"}, - {Type: "run_folder"}, + {Type: "folder"}, // deprecated alias for "memory" + {Type: "memory"}, // dedicated mission memory (reserved name "mission") + {Type: "run_folder"}, // deprecated alias for "run_memory" + {Type: "run_memory"}, // per-run ephemeral memory (reserved name "run") {Type: "schedule"}, {Type: "trigger"}, {Type: "budget"}, @@ -1934,49 +1945,69 @@ func parseMissionBlock(block *hcl.Block, ctx *hcl.EvalContext) (*Mission, error) directive = val.AsString() } - // Parse optional folders attribute (list of shared folder names) + // Parse optional memories / folders attribute (list of shared memory names). + // "memories" is the preferred spelling; "folders" is kept as a + // backwards-compatible alias. Setting both is an error. var missionFolders []string - if foldersAttr, ok := missionContent.Attributes["folders"]; ok { - foldersVal, diags := foldersAttr.Expr.Value(ctx) + memoriesAttr, hasMemories := missionContent.Attributes["memories"] + foldersAttr, hasFolders := missionContent.Attributes["folders"] + if hasMemories && hasFolders { + return nil, fmt.Errorf("mission '%s': set either 'memories' or 'folders', not both", missionName) + } + if attr := memoriesAttr; hasMemories { + v, diags := attr.Expr.Value(ctx) + if diags.HasErrors() { + return nil, fmt.Errorf("mission '%s' memories: %w", missionName, diags) + } + for it := v.ElementIterator(); it.Next(); { + _, e := it.Element() + missionFolders = append(missionFolders, e.AsString()) + } + } else if hasFolders { + v, diags := foldersAttr.Expr.Value(ctx) if diags.HasErrors() { return nil, fmt.Errorf("mission '%s' folders: %w", missionName, diags) } - for it := foldersVal.ElementIterator(); it.Next(); { - _, v := it.Element() - missionFolders = append(missionFolders, v.AsString()) + for it := v.ElementIterator(); it.Next(); { + _, e := it.Element() + missionFolders = append(missionFolders, e.AsString()) } } - // Parse optional folder block (dedicated mission folder, reserved name "mission") + // Parse optional memory / folder block (dedicated mission memory, reserved + // name "mission"). "memory" is preferred; "folder" is kept as a + // backwards-compatible alias. At most one of either may appear. var missionFolder *MissionFolder for _, folderBlock := range missionContent.Blocks { - if folderBlock.Type != "folder" { + if folderBlock.Type != "folder" && folderBlock.Type != "memory" { continue } if missionFolder != nil { - return nil, fmt.Errorf("mission '%s': only one folder block allowed", missionName) + return nil, fmt.Errorf("mission '%s': only one memory/folder block allowed", missionName) } var mf MissionFolder diags := gohcl.DecodeBody(folderBlock.Body, ctx, &mf) if diags.HasErrors() { - return nil, fmt.Errorf("mission '%s' folder: %w", missionName, diags) + return nil, fmt.Errorf("mission '%s' %s: %w", missionName, folderBlock.Type, diags) } missionFolder = &mf } - // Parse optional run_folder block (per-run ephemeral folder, reserved name "run") + // Parse optional run_memory / run_folder block (per-run ephemeral memory, + // reserved name "run"). "run_memory" is preferred; "run_folder" is kept as + // a backwards-compatible alias. var missionRunFolder *MissionRunFolder for _, rfBlock := range missionContent.Blocks { - if rfBlock.Type != "run_folder" { + if rfBlock.Type != "run_folder" && rfBlock.Type != "run_memory" { continue } if missionRunFolder != nil { - return nil, fmt.Errorf("mission '%s': only one run_folder block allowed", missionName) + return nil, fmt.Errorf("mission '%s': only one run_memory/run_folder block allowed", missionName) } var rf MissionRunFolder diags := gohcl.DecodeBody(rfBlock.Body, ctx, &rf) if diags.HasErrors() { - return nil, fmt.Errorf("mission '%s' run_folder: %w", missionName, diags) + return nil, fmt.Errorf("mission '%s' %s: %w", missionName, rfBlock.Type, diags) } if rf.Cleanup == nil { v := DefaultRunFolderCleanupDays diff --git a/config/folder_test.go b/config/folder_test.go index 6622fa9..69a5680 100644 --- a/config/folder_test.go +++ b/config/folder_test.go @@ -9,6 +9,83 @@ import ( var _ = Describe("Folders", func() { + Describe("memory spellings (preferred)", func() { + It("parses shared_memory + memories + memory + run_memory together", func() { + hcl := fullBaseHCL() + ` +shared_memory "research" { + path = "./data" + description = "Research docs" + editable = true +} +mission "m" { + commander { model = models.anthropic.claude_sonnet_4 } + agents = [agents.test_agent] + memories = [shared_memories.research] + memory { + path = "./persistent" + description = "Persistent" + } + run_memory { + description = "Per-run scratch" + } + task "t" { objective = "go" } +} +` + _, f := writeFixture("config.hcl", hcl) + cfg, err := config.LoadFile(f) + Expect(err).NotTo(HaveOccurred()) + Expect(cfg.SharedFolders).To(HaveLen(1)) + Expect(cfg.SharedFolders[0].Name).To(Equal("research")) + Expect(cfg.Missions[0].Folders).To(ConsistOf("research")) + Expect(cfg.Missions[0].Folder).NotTo(BeNil()) + Expect(cfg.Missions[0].Folder.Path).To(Equal("./persistent")) + Expect(cfg.Missions[0].RunFolder).NotTo(BeNil()) + Expect(cfg.Missions[0].RunFolder.Description).To(Equal("Per-run scratch")) + }) + + It("allows mixing old and new spellings across the file", func() { + // shared_folder + memories + memory mixed with run_folder + hcl := fullBaseHCL() + ` +shared_folder "research" { + path = "./data" +} +mission "m" { + commander { model = models.anthropic.claude_sonnet_4 } + agents = [agents.test_agent] + memories = [shared_memories.research] + memory { path = "./persistent" } + run_folder { description = "old name" } + task "t" { objective = "go" } +} +` + _, f := writeFixture("config.hcl", hcl) + cfg, err := config.LoadFile(f) + Expect(err).NotTo(HaveOccurred()) + Expect(cfg.Missions[0].Folders).To(ConsistOf("research")) + Expect(cfg.Missions[0].Folder).NotTo(BeNil()) + Expect(cfg.Missions[0].RunFolder).NotTo(BeNil()) + }) + + It("rejects setting both 'memories' and 'folders' on the same mission", func() { + hcl := fullBaseHCL() + ` +shared_memory "research" { + path = "./data" +} +mission "m" { + commander { model = models.anthropic.claude_sonnet_4 } + agents = [agents.test_agent] + memories = [shared_memories.research] + folders = [shared_folders.research] + task "t" { objective = "go" } +} +` + _, f := writeFixture("config.hcl", hcl) + _, err := config.LoadFile(f) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("set either 'memories' or 'folders'")) + }) + }) + Describe("shared_folder", func() { It("parses a shared_folder block", func() { hcl := fullBaseHCL() + ` @@ -89,7 +166,7 @@ mission "m" { _, f := writeFixture("config.hcl", hcl) _, err := config.LoadFile(f) Expect(err).To(HaveOccurred()) - Expect(err.Error()).To(ContainSubstring("only one folder block allowed")) + Expect(err.Error()).To(ContainSubstring("only one memory/folder block allowed")) }) }) @@ -166,7 +243,7 @@ mission "m" { _, f := writeFixture("config.hcl", hcl) _, err := config.LoadFile(f) Expect(err).To(HaveOccurred()) - Expect(err.Error()).To(ContainSubstring("only one run_folder block allowed")) + Expect(err.Error()).To(ContainSubstring("only one run_memory/run_folder block allowed")) }) It("rejects a negative cleanup value", func() { @@ -189,6 +266,38 @@ mission "m" { Expect(rf.Validate()).To(Succeed()) }) + It("rejects mixing folder and memory blocks on the same mission", func() { + hcl := fullBaseHCL() + ` +mission "m" { + commander { model = models.anthropic.claude_sonnet_4 } + agents = [agents.test_agent] + folder { path = "./a" } + memory { path = "./b" } + task "t" { objective = "go" } +} +` + _, f := writeFixture("config.hcl", hcl) + _, err := config.LoadFile(f) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("only one memory/folder block allowed")) + }) + + It("rejects mixing run_folder and run_memory blocks on the same mission", func() { + hcl := fullBaseHCL() + ` +mission "m" { + commander { model = models.anthropic.claude_sonnet_4 } + agents = [agents.test_agent] + run_folder { base = "./a" } + run_memory { base = "./b" } + task "t" { objective = "go" } +} +` + _, f := writeFixture("config.hcl", hcl) + _, err := config.LoadFile(f) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("only one run_memory/run_folder block allowed")) + }) + It("allows both folder and run_folder on the same mission", func() { hcl := fullBaseHCL() + ` mission "m" { diff --git a/docs/content/config/overview.mdx b/docs/content/config/overview.mdx index efe0788..64423a2 100644 --- a/docs/content/config/overview.mdx +++ b/docs/content/config/overview.mdx @@ -60,7 +60,7 @@ agent "assistant" { | `mcp_host` | Start a built-in [MCP host](/config/mcp_host) to expose Squadron to AI clients | | `mission` | Define multi-task missions | | `commander` | Commander server connection config | -| `shared_folder` | Shared file folders accessible to missions | +| `shared_memory` | Shared filesystem locations accessible to missions (formerly `shared_folder`; both spellings accepted) | ## Expressions diff --git a/docs/content/missions/_meta.js b/docs/content/missions/_meta.js index f483d7f..3b5386f 100644 --- a/docs/content/missions/_meta.js +++ b/docs/content/missions/_meta.js @@ -5,7 +5,7 @@ export default { routing: 'Routing', datasets: 'Datasets', iteration: 'Iteration', - folders: 'Folders', + folders: 'Memory', 'internal-tools': 'Internal Tools', budgets: 'Budgets', schedules: 'Schedules & Triggers', diff --git a/docs/content/missions/folders.mdx b/docs/content/missions/folders.mdx index dd7557b..ea5008f 100644 --- a/docs/content/missions/folders.mdx +++ b/docs/content/missions/folders.mdx @@ -1,10 +1,10 @@ --- -title: Folders +title: Memory --- -# Folders +# Memory -Folders are filesystem locations that agents can read from and write to. Squadron gives you three kinds, each suited to a different use case: +Memory blocks are filesystem locations that agents can read from and write to. Squadron gives you three kinds, each suited to a different use case: | Kind | Use it for | Lifetime | |------|------------|----------| @@ -12,22 +12,30 @@ Folders are filesystem locations that agents can read from and write to. Squadro | **Mission** | A mission's own long-lived state — archives, accumulated output | Persists | | **Run** | Scratch space for the work happening right now | One per run | -Agents reach folders through the `file_list`, `file_read`, `file_create`, `file_delete`, `file_search`, and `file_grep` tools, naming the folder on each call. +Agents reach memory through the `file_list`, `file_read`, `file_create`, `file_delete`, `file_search`, and `file_grep` tools, naming the slot on each call. -A mission can use any number of shared folders, plus one of each mission-scoped kind. The names `mission` and `run` are reserved. +A mission can use any number of shared memory blocks, plus one of each mission-scoped kind. The names `mission` and `run` are reserved. -## Shared Folders +> **Naming note.** Memory blocks were originally called "folders". The old HCL +> spellings — `shared_folder`, `folder`, `run_folder`, `folders`, +> `shared_folders` — still work as backwards-compatible aliases of the new +> `shared_memory`, `memory`, `run_memory`, `memories`, `shared_memories` names. +> Use whichever spelling you prefer; new code should prefer the memory +> spellings. Mixing both spellings for the *same* slot on one mission is an +> error. The tools themselves still take a parameter literally called `folder`. + +## Shared Memory Declared at the top level so multiple missions can share the same data: ```hcl -shared_folder "research" { +shared_memory "research" { path = "./data/research" description = "Shared research documents" editable = true } -shared_folder "reference" { +shared_memory "reference" { path = "./data/reference" # no editable flag → read-only } @@ -44,18 +52,18 @@ Missions opt in by listing them, then refer to them by name in tool calls: ```hcl mission "analyze" { - folders = [shared_folders.research, shared_folders.reference] + memories = [shared_memories.research, shared_memories.reference] # agents call e.g. file_read with folder = "research" } ``` -## The Mission Folder +## The Mission Memory -The mission's own dedicated folder. Use it for things you want to keep around: a running log, an archive of finished reports, accumulated state across runs. +The mission's own dedicated slot. Use it for things you want to keep around: a running log, an archive of finished reports, accumulated state across runs. ```hcl mission "analyze" { - folder { + memory { path = "./analyses" description = "Cumulative analysis output across every run" } @@ -71,21 +79,21 @@ Agents reach it under the name `mission`: | Attribute | Type | Description | |-----------|------|-------------| | `path` | string | Directory path (created if missing) | -| `description` | string | Shown to agents alongside the folder name | +| `description` | string | Shown to agents alongside the slot name | -## The Run Folder +## The Run Memory A clean workspace for one mission run. Each invocation gets its own subdirectory, isolated from previous and concurrent runs — perfect for intermediate files, working drafts, or anything you don't want bleeding into the next run. ```hcl mission "analyze" { - run_folder { + run_memory { description = "Scratch space for this run" } } ``` -By default, run folders are deleted 7 days after the run started. Set `cleanup` to override the window, or `cleanup = 0` to keep folders forever. +By default, run memory is deleted 7 days after the run started. Set `cleanup` to override the window, or `cleanup = 0` to keep it forever. Agents reach it under the name `run`: @@ -95,15 +103,15 @@ Agents reach it under the name `run`: | Attribute | Type | Description | |-----------|------|-------------| -| `base` | string | Where run folders live. Defaults to `.squadron/runs` — change it if you want them somewhere else | -| `description` | string | Shown to agents alongside the folder name | -| `cleanup` | integer | Delete the folder this many days after the run started. Defaults to `7`; set `0` to keep forever | +| `base` | string | Where run directories live. Defaults to `.squadron/runs` — change it if you want them somewhere else | +| `description` | string | Shown to agents alongside the slot name | +| `cleanup` | integer | Delete the directory this many days after the run started. Defaults to `7`; set `0` to keep forever | -Run folders stick around after the mission ends — useful for inspecting what an agent produced, and required for `squadron mission --resume ` to find its workspace. `cleanup` is just a max age; nothing is deleted while a run is in flight. +Run directories stick around after the mission ends — useful for inspecting what an agent produced, and required for `squadron mission --resume ` to find its workspace. `cleanup` is just a max age; nothing is deleted while a run is in flight. ## Tool Reference -All six folder tools take a required `folder` parameter: +All six file tools take a required `folder` parameter — that's the slot name (a shared memory name, or the reserved `"mission"` / `"run"`): | Tool | Purpose | |------|---------| @@ -114,12 +122,12 @@ All six folder tools take a required `folder` parameter: | `file_search` | Recursively search for files by filename regex | | `file_grep` | Search file contents by regex | -Paths are always relative to the folder root. Absolute paths and `..` escapes are rejected. +Paths are always relative to the slot's root directory. Absolute paths and `..` escapes are rejected. ## Full Example ```hcl -shared_folder "reference" { +shared_memory "reference" { path = "./data/reference" description = "Reference materials shared across missions" } @@ -127,25 +135,25 @@ shared_folder "reference" { mission "research" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.researcher] - folders = [shared_folders.reference] + memories = [shared_memories.reference] - folder { + memory { path = "./research_archive" description = "Finished reports, one per run" } - run_folder { + run_memory { description = "Working files for the in-flight run" cleanup = 14 # override the 7-day default } task "gather" { - objective = "Gather sources from the reference folder and save notes to the run folder" + objective = "Gather sources from the reference memory and save notes to the run memory" } task "publish" { depends_on = [tasks.gather] - objective = "Write the final report into the mission folder" + objective = "Write the final report into the mission memory" } } ``` diff --git a/docs/content/missions/overview.mdx b/docs/content/missions/overview.mdx index 3f97de8..6541dd0 100644 --- a/docs/content/missions/overview.mdx +++ b/docs/content/missions/overview.mdx @@ -43,9 +43,9 @@ mission "data_pipeline" { | `input` | block | Mission input parameters (repeatable) | | `task` | block | Task definitions (repeatable) | | `dataset` | block | Dataset definitions (optional) | -| `folders` | list | Shared folder references, e.g. `[shared_folders.data]` (see [Folders](/missions/folders)) | -| `folder` | block | Persistent mission-scoped folder, registered as `"mission"` | -| `run_folder` | block | Per-run ephemeral folder, registered as `"run"` | +| `memories` | list | Shared memory references, e.g. `[shared_memories.data]` (see [Memory](/missions/folders)). Old name: `folders` (still accepted). | +| `memory` | block | Persistent mission-scoped memory, registered as `"mission"`. Old name: `folder` (still accepted). | +| `run_memory` | block | Per-run ephemeral memory, registered as `"run"`. Old name: `run_folder` (still accepted). | | `schedule` | block | Automatic run schedules (optional, repeatable) | | `trigger` | block | Webhook trigger (optional) | | `max_parallel` | number | Max concurrent instances (default: 3) | From f67ead6ebec1e2a99b1fd2796c6c7d54af4ec071 Mon Sep 17 00:00:00 2001 From: Max Lund Date: Mon, 25 May 2026 21:30:09 -0500 Subject: [PATCH 2/8] Rework memory DSL: single block type, derived paths, drop legacy spellings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Builds on the previous folders→memory rename to land a cleaner DSL the user pushed for: Top-level shared memory is just `memory "name" { ... }` (no `shared_` prefix). Mission-scoped memory collapses to a single `memory { type = "persistent" | "ephemeral" }` block — at most one of each per mission; `type` defaults to persistent. Paths are removed from the DSL entirely. Squadron owns every memory location under /memories/: - shared → memories/shared// - persistent → memories/mission//persistent/ - ephemeral → memories/mission//run// The old DSL surfaces are no longer accepted — `shared_folder`, `shared_memory`, `folder { ... }`, `run_folder { ... }`, `run_memory { ... }`, the `folders = ...` attribute, and a `path` attribute on any memory block all produce explicit errors pointing at the new syntax. Changes: - config/memory.go: new Memory (top-level) and MissionMemory (with Type field) types; drops Path everywhere. SharedFolder, MissionFolder, and MissionRunFolder are gone. - config/mission.go: Folders/Folder/RunFolder fields replaced by Memories/PersistentMemory/EphemeralMemory; Validate signature takes []Memory. - config/config.go: top-level schema gains labeled `memory` blocks, drops the old block types from the live parse path (still detected to surface a clear error). Mission schema accepts one `memories` attr and zero or more `memory {}` blocks with type discrimination. - mission/folder_store.go: paths derived from paths.SquadronHome() via new SharedMemoryPath / PersistentMemoryPath / EphemeralMemoryPath helpers. SweepExpiredEphemeralMemories walks memories/mission/*/run/* — no per-mission config lookup required. - cmd/engage.go runFolderCleanupLoop simplified to call the new sweep once per tick. - wsbridge: file browser resolves memory paths through the new helpers instead of reading user-supplied Path. Wire protocol unchanged. - Tests rewritten for the new DSL and new path scheme (using paths.SetHome/ResetHome for isolation). - Docs: CLAUDE.md memory section, missions/folders.mdx, and the missions/config overview tables updated to describe the new DSL. --- CLAUDE.md | 90 ++++---- README.md | 2 +- cmd/engage.go | 33 +-- config/config.go | 186 +++++++++-------- config/folder_test.go | 296 +++++++++++++------------- config/memory.go | 81 +++++++ config/mission.go | 85 +++----- config/shared_folder.go | 35 ---- docs/content/config/overview.mdx | 2 +- docs/content/missions/folders.mdx | 62 +++--- docs/content/missions/overview.mdx | 5 +- mission/folder_store.go | 237 +++++++++++++-------- mission/folder_store_test.go | 325 +++++++++++++++++------------ mission/runner.go | 21 +- wsbridge/convert.go | 41 +--- wsbridge/shared_folder.go | 133 +++++++----- 16 files changed, 891 insertions(+), 743 deletions(-) create mode 100644 config/memory.go delete mode 100644 config/shared_folder.go diff --git a/CLAUDE.md b/CLAUDE.md index 0cd3048..418b117 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -282,73 +282,79 @@ mission "expensive_research" { The scheduler lives in `scheduler/` but its lifecycle (creation, config updates, shutdown) is managed by `cmd/serve.go`, not wsbridge. The wsbridge client receives a `ConcurrencyTracker` interface for enforcing `max_parallel` on all mission starts. The cron library used is `robfig/cron/v3`. -### Memory (formerly "Folders") +### Memory Memory blocks are sandboxed filesystem locations that agents access via the `file_list`, `file_read`, `file_create`, `file_delete`, `file_search`, and -`file_grep` tools. The `folder` parameter (kept under that name on the tools -for back-compat) is required on every tool call — there is no implicit default. +`file_grep` tools. The `folder` parameter (the literal historical name on the +tools) is required on every tool call — there is no implicit default. -Three kinds exist, with strict naming rules: +Squadron owns the paths: every slot lives under `/memories/` +and you do **not** specify `path` anywhere. The three kinds map to three +fixed location patterns: -| Kind | HCL (preferred) | HCL (back-compat alias) | Registered name | Scope | Persistence | -|------|-----------------|--------------------------|-----------------|-------|-------------| -| Shared | `shared_memory "name" { ... }` (top-level) | `shared_folder "name" { ... }` | user-chosen name | referenced by any mission via `memories = [shared_memories.name]` (or `folders = [shared_folders.name]`) | persists | -| Mission | `memory { ... }` (inside a mission) | `folder { ... }` | literal `"mission"` | one per mission | persists across runs | -| Run | `run_memory { ... }` (inside a mission) | `run_folder { ... }` | literal `"run"` | one per mission per run | ephemeral; path is `//` | +| Kind | HCL | Slot name agents use | On-disk path | +|------|-----|-----------------------|--------------| +| Shared | top-level `memory "name" { ... }` | the HCL label | `/memories/shared//` | +| Persistent (mission) | `memory { type = "persistent" }` inside a mission (also the default if `type` is omitted) | literal `"mission"` | `/memories/mission//persistent/` | +| Ephemeral (per-run) | `memory { type = "ephemeral" }` inside a mission | literal `"run"` | `/memories/mission//run//` | -The names `"mission"` and `"run"` are reserved — `shared_memory` / -`shared_folder` cannot use them (enforced in `config/shared_folder.go`). +A mission may declare at most one persistent and one ephemeral memory. The +names `"mission"` and `"run"` are reserved — a top-level `memory "mission"` +or `memory "run"` block is rejected. -The old `folders` / `folder` / `run_folder` / `shared_folder` / `shared_folders` -spellings keep working unchanged. Using *both* spellings for the same slot on -one mission (e.g. `memory {}` and `folder {}`, or `memories =` and `folders =`) -is rejected at config-load time. +The old DSL surfaces — `shared_folder`/`shared_memory` blocks, `folder`/ +`run_folder`/`run_memory` blocks, the `folders = ...` attribute — are **not** +accepted. The parser surfaces a clear error pointing at the new syntax. Path +attributes anywhere on a memory block are rejected too. ```hcl -shared_memory "reference" { - path = "./data/reference" - editable = false # default read-only +memory "reference" { + description = "Shared reference materials" + editable = false # default read-only } mission "analyze" { - memories = [shared_memories.reference] + memories = [memories.reference] memory { - path = "./analyses" + # type defaults to "persistent" description = "Cumulative reports — persists across runs" } - run_memory { - base = "./runs" # optional, default ".squadron/runs" + memory { + type = "ephemeral" description = "Per-run scratch" - cleanup = 7 # optional, auto-delete after N days; defaults to 7, set 0 to keep forever + cleanup = 7 # optional; defaults to 7, set 0 to keep forever } } ``` **Implementation:** -- `config.SharedFolder`, `config.MissionFolder`, `config.MissionRunFolder` in - [config/shared_folder.go](config/shared_folder.go) and [config/mission.go](config/mission.go). - Both HCL spellings decode into these same structs — there is no separate - "memory" type internally. -- Shared memory blocks are parsed in Stage 1.5 (with `vars` context). The - mission's `memory`/`folder` and `run_memory`/`run_folder` blocks are parsed - inside the mission block in Stage 5. The `shared_memories.NAME` / - `shared_folders.NAME` HCL namespaces are both registered. +- `config.Memory` (top-level) and `config.MissionMemory` (mission-scoped, with + a `Type` field) in [config/memory.go](config/memory.go); `MissionMemory` + fields land on the mission as `PersistentMemory` and `EphemeralMemory` + (`config.Mission` in [config/mission.go](config/mission.go)). +- Top-level `memory "name"` blocks are parsed in Stage 1.5 (with `vars` + context); each mission's `memory { ... }` blocks are parsed inside the + mission block in Stage 5. The `memories.NAME` HCL namespace exposes the + shared-memory labels to mission attributes. - [mission/folder_store.go](mission/folder_store.go) is the authoritative - runtime resolver. `buildFolderStore(mission, sharedFolders, missionID)` must be - called **after** the mission ID is assigned in `Runner.Run()`, because the run - memory path depends on it. There is no implicit default memory — every tool - call must name the slot explicitly via the `folder` parameter. -- Run memory blocks are materialized at `//` and a sidecar - `.squadron-run.json` records `created_at` + `cleanup_days`. -- `mission.SweepExpiredRunFolders(base)` walks a base directory and deletes any - subfolder whose sidecar says it is past its cleanup deadline. It runs - opportunistically at the start of every `Runner.Run()` (for that mission's - base) and on an hourly ticker in `cmd/engage.go` (across every mission in the - current config). + runtime resolver. Paths are derived from `paths.SquadronHome()`: + `SharedMemoryPath(name)`, `PersistentMemoryPath(missionName)`, and + `EphemeralMemoryPath(missionName, missionInstanceID)`. + `buildFolderStore(mission, memories, missionInstanceID)` must be called + **after** the mission instance ID is assigned in `Runner.Run()` because the + ephemeral path depends on it. +- Each ephemeral directory gets a sidecar `.squadron-run.json` recording + `created_at` + `cleanup_days` so the sweep can find expired ones. +- `mission.SweepExpiredEphemeralMemories()` walks + `/memories/mission/*/run/*`, deleting any per-run directory + whose sidecar is past its cleanup deadline. It runs opportunistically at + the start of every `Runner.Run()` (for missions with an ephemeral memory) + and on an hourly ticker in `cmd/engage.go`. No config lookup needed — the + filesystem layout is self-describing. ### Mission-Scoped Agents diff --git a/README.md b/README.md index 6809c1a..8831cd7 100644 --- a/README.md +++ b/README.md @@ -111,7 +111,7 @@ Honest tradeoffs on each comparison page. - **[MCP host](https://docs.squadron.sh/config/mcp_host)** — run Squadron as an MCP server for Claude Desktop, Claude Code, Cursor, and other MCP clients. - **[Schedules & triggers](https://docs.squadron.sh/missions/schedules)** — cron-based or HTTP-triggered missions, with per-mission concurrency limits. - **[Budgets](https://docs.squadron.sh/missions/budgets)** — token + dollar caps per mission or per task. -- **[Memory](https://docs.squadron.sh/missions/folders)** — sandboxed filesystem locations agents can read/write, with shared, mission-scoped, and ephemeral run-scoped variants. (HCL: `shared_memory`, `memory`, `run_memory` — the old `shared_folder`/`folder`/`run_folder` spellings still work.) +- **[Memory](https://docs.squadron.sh/missions/folders)** — sandboxed filesystem locations agents can read/write. Three kinds: shared (top-level `memory "name"`), persistent mission, and ephemeral per-run (`memory { type = "persistent" | "ephemeral" }`). Squadron owns the paths. - **[Gateways](https://docs.squadron.sh/config/gateways)** — managed subprocesses that bridge Squadron to Slack, Discord, Teams, or any custom system via the Gateway SDK. - **Encrypted vault** — secrets stored at rest with AES-256-GCM + Argon2id; passphrase in the OS keychain. - **Single binary, no runtime deps.** Docker images on every release. diff --git a/cmd/engage.go b/cmd/engage.go index 108be90..e6ffbb0 100644 --- a/cmd/engage.go +++ b/cmd/engage.go @@ -901,33 +901,14 @@ func openBrowser(url string) { browser.Open(url) } -// runFolderCleanupLoop periodically sweeps expired per-run mission folders -// for every mission in the current config that declares a run_folder with -// cleanup > 0. It runs once immediately, then hourly, and exits when -// shutdown is closed. -func runFolderCleanupLoop(shutdown <-chan struct{}, getCfg func() *config.Config) { +// runFolderCleanupLoop periodically sweeps expired per-run ephemeral +// memory directories. The sweep walks the entire memories tree, so it +// doesn't need to know which missions are configured. It runs once +// immediately, then hourly, and exits when shutdown is closed. +func runFolderCleanupLoop(shutdown <-chan struct{}, _ func() *config.Config) { sweep := func() { - cfg := getCfg() - if cfg == nil { - return - } - seen := make(map[string]bool) - for i := range cfg.Missions { - rf := cfg.Missions[i].RunFolder - if rf == nil { - continue - } - if rf.Cleanup != nil && *rf.Cleanup == 0 { - continue // user explicitly opted out of cleanup - } - base := mission.ResolvedRunFolderBase(rf) - if seen[base] { - continue - } - seen[base] = true - if _, err := mission.SweepExpiredRunFolders(base); err != nil { - log.Printf("run folder cleanup: sweep %q: %v", base, err) - } + if _, err := mission.SweepExpiredEphemeralMemories(); err != nil { + log.Printf("ephemeral memory cleanup: %v", err) } } diff --git a/config/config.go b/config/config.go index c8bb07b..949d3f4 100644 --- a/config/config.go +++ b/config/config.go @@ -47,8 +47,8 @@ type Config struct { // renamed to `mcp_host { ... }`). nil when the block is absent. MCPHost *MCPHostConfig `hcl:"-"` - // File browser configurations (optional) - SharedFolders []SharedFolder `hcl:"-"` + // Top-level shared memory blocks (memory "name" { ... }). + Memories []Memory `hcl:"-"` // LoadedPlugins holds the loaded plugin clients, keyed by plugin name LoadedPlugins map[string]*plugin.PluginClient `hcl:"-"` @@ -437,9 +437,9 @@ func (c *Config) Validate() error { } } - for _, fb := range c.SharedFolders { - if err := fb.Validate(); err != nil { - return fmt.Errorf("shared_folder '%s': %w", fb.Name, err) + for _, m := range c.Memories { + if err := m.Validate(); err != nil { + return fmt.Errorf("memory '%s': %w", m.Name, err) } } @@ -615,7 +615,7 @@ func (c *Config) Validate() error { // Validate missions for i := range c.Missions { - if err := c.Missions[i].Validate(c.Models, c.Agents, c.SharedFolders, allMissionNames); err != nil { + if err := c.Missions[i].Validate(c.Models, c.Agents, c.Memories, allMissionNames); err != nil { return fmt.Errorf("mission '%s': %w", c.Missions[i].Name, err) } } @@ -684,7 +684,7 @@ type parsedBlocks struct { Missions []*hcl.Block Storage []*hcl.Block CommandCenter []*hcl.Block - SharedFolders []*hcl.Block + Memories []*hcl.Block MCPHost []*hcl.Block Skills []*hcl.Block Gateways []*hcl.Block @@ -720,6 +720,9 @@ func loadFromFiles(files []string) (*Config, error) { {Type: "mission", LabelNames: []string{"name"}}, {Type: "storage"}, {Type: "command_center"}, + {Type: "memory", LabelNames: []string{"name"}}, + // Detected for nicer errors only — the parse-pass below + // rejects them with a pointer to the new `memory` block. {Type: "shared_folder", LabelNames: []string{"name"}}, {Type: "shared_memory", LabelNames: []string{"name"}}, {Type: "mcp_host"}, @@ -753,8 +756,12 @@ func loadFromFiles(files []string) (*Config, error) { pb.Storage = append(pb.Storage, block) case "command_center": pb.CommandCenter = append(pb.CommandCenter, block) + case "memory": + pb.Memories = append(pb.Memories, block) case "shared_folder", "shared_memory": - pb.SharedFolders = append(pb.SharedFolders, block) + // Collected only so we can produce a clear error in the + // parse pass below. + pb.Memories = append(pb.Memories, block) case "mcp_host": pb.MCPHost = append(pb.MCPHost, block) case "mcp": @@ -950,20 +957,22 @@ func loadFromFiles(files []string) (*Config, error) { } } - // Parse shared_folder / shared_memory blocks (optional, with vars context). - // Both block types share the same SharedFolder struct — shared_memory is - // the preferred spelling; shared_folder is kept as a backwards-compatible - // alias. - var allSharedFolders []SharedFolder + // Parse top-level `memory "name" { ... }` blocks (with vars context). + // `shared_folder` and `shared_memory` are no longer supported — if a user + // writes one we surface an explicit error pointing at the new syntax. + var allMemories []Memory for _, pb := range allParsedBlocks { - for _, block := range pb.SharedFolders { - var fb SharedFolder - fb.Name = block.Labels[0] - diags := gohcl.DecodeBody(block.Body, varsCtx, &fb) + for _, block := range pb.Memories { + if block.Type != "memory" { + return nil, fmt.Errorf("%s %q at %s: this block type is no longer supported — use `memory %q { ... }` instead (path is now derived automatically; remove the `path` attribute)", block.Type, block.Labels[0], block.DefRange, block.Labels[0]) + } + var m Memory + m.Name = block.Labels[0] + diags := gohcl.DecodeBody(block.Body, varsCtx, &m) if diags.HasErrors() { - return nil, fmt.Errorf("%s '%s': %w", block.Type, fb.Name, diags) + return nil, fmt.Errorf("memory '%s': %w", m.Name, diags) } - allSharedFolders = append(allSharedFolders, fb) + allMemories = append(allMemories, m) } } @@ -1148,20 +1157,17 @@ func loadFromFiles(files []string) (*Config, error) { // Build agents context (add to full context) agentsCtx := buildAgentsContext(skillsCtx, allAgents) - // Add shared_folders / shared_memories namespaces for mission references. - // Both names point at the same map — shared_memories is the preferred - // spelling; shared_folders is kept as a backwards-compatible alias. - if len(allSharedFolders) > 0 { - folderMap := make(map[string]cty.Value) - for _, f := range allSharedFolders { - folderMap[f.Name] = cty.StringVal(f.Name) + // Add `memories` namespace for mission references: `memories.NAME` resolves + // to the memory's name as a string. + if len(allMemories) > 0 { + memMap := make(map[string]cty.Value) + for _, m := range allMemories { + memMap[m.Name] = cty.StringVal(m.Name) } - ns := cty.ObjectVal(folderMap) - agentsCtx.Variables["shared_folders"] = ns - agentsCtx.Variables["shared_memories"] = ns + agentsCtx.Variables["memories"] = cty.ObjectVal(memMap) } - // Stage 5: Load missions (with vars + models + tools + agents + shared_folders context) + // Stage 5: Load missions (with vars + models + tools + agents + memories context) // First pass: collect all mission names so router targets can reference missions.* missionNames := make(map[string]cty.Value) for _, pb := range allParsedBlocks { @@ -1211,7 +1217,7 @@ func loadFromFiles(files []string) (*Config, error) { Storage: &storageConfig, CommandCenter: commandCenterConfig, MCPHost: mcpHostConfig, - SharedFolders: allSharedFolders, + Memories: allMemories, LoadedPlugins: loadedPlugins, LoadedMCPClients: loadedMCPClients, LoadedMCPErrors: loadedMCPErrors, @@ -1779,10 +1785,11 @@ func parseMissionBlock(block *hcl.Block, ctx *hcl.EvalContext) (*Mission, error) Attributes: []hcl.AttributeSchema{ {Name: "agents", Required: true}, {Name: "directive"}, - {Name: "folders"}, // deprecated alias for "memories" - {Name: "memories"}, // shared memory references + {Name: "memories"}, // shared memory references: memories = [memories.foo] {Name: "max_parallel"}, {Name: "inputs"}, // shorthand: inputs = { field = string("desc", { default = "val" }) } + // Detected so we can produce a nicer error than "unsupported argument". + {Name: "folders"}, }, Blocks: []hcl.BlockHeaderSchema{ {Type: "commander"}, @@ -1791,13 +1798,14 @@ func parseMissionBlock(block *hcl.Block, ctx *hcl.EvalContext) (*Mission, error) {Type: "input", LabelNames: []string{"name"}}, // verbose input blocks still supported {Type: "dataset", LabelNames: []string{"name"}}, {Type: "secret", LabelNames: []string{"name"}}, - {Type: "folder"}, // deprecated alias for "memory" - {Type: "memory"}, // dedicated mission memory (reserved name "mission") - {Type: "run_folder"}, // deprecated alias for "run_memory" - {Type: "run_memory"}, // per-run ephemeral memory (reserved name "run") + {Type: "memory"}, // mission-scoped memory: memory { type = "persistent"|"ephemeral" } {Type: "schedule"}, {Type: "trigger"}, {Type: "budget"}, + // Detected so we can produce a nicer error than the parser's default. + {Type: "folder"}, + {Type: "run_folder"}, + {Type: "run_memory"}, }, }) if diags.HasErrors() { @@ -1945,75 +1953,71 @@ func parseMissionBlock(block *hcl.Block, ctx *hcl.EvalContext) (*Mission, error) directive = val.AsString() } - // Parse optional memories / folders attribute (list of shared memory names). - // "memories" is the preferred spelling; "folders" is kept as a - // backwards-compatible alias. Setting both is an error. - var missionFolders []string - memoriesAttr, hasMemories := missionContent.Attributes["memories"] - foldersAttr, hasFolders := missionContent.Attributes["folders"] - if hasMemories && hasFolders { - return nil, fmt.Errorf("mission '%s': set either 'memories' or 'folders', not both", missionName) + // Reject the old `folders = ...` attribute with a clear pointer at the new + // syntax — we deliberately broke compatibility here. + if _, ok := missionContent.Attributes["folders"]; ok { + return nil, fmt.Errorf("mission '%s': the `folders` attribute is no longer supported — use `memories = [memories.NAME, ...]` instead", missionName) } - if attr := memoriesAttr; hasMemories { + + // Parse optional memories attribute (list of shared memory names). + var missionMemories []string + if attr, ok := missionContent.Attributes["memories"]; ok { v, diags := attr.Expr.Value(ctx) if diags.HasErrors() { return nil, fmt.Errorf("mission '%s' memories: %w", missionName, diags) } for it := v.ElementIterator(); it.Next(); { _, e := it.Element() - missionFolders = append(missionFolders, e.AsString()) - } - } else if hasFolders { - v, diags := foldersAttr.Expr.Value(ctx) - if diags.HasErrors() { - return nil, fmt.Errorf("mission '%s' folders: %w", missionName, diags) - } - for it := v.ElementIterator(); it.Next(); { - _, e := it.Element() - missionFolders = append(missionFolders, e.AsString()) + missionMemories = append(missionMemories, e.AsString()) } } - // Parse optional memory / folder block (dedicated mission memory, reserved - // name "mission"). "memory" is preferred; "folder" is kept as a - // backwards-compatible alias. At most one of either may appear. - var missionFolder *MissionFolder - for _, folderBlock := range missionContent.Blocks { - if folderBlock.Type != "folder" && folderBlock.Type != "memory" { - continue - } - if missionFolder != nil { - return nil, fmt.Errorf("mission '%s': only one memory/folder block allowed", missionName) - } - var mf MissionFolder - diags := gohcl.DecodeBody(folderBlock.Body, ctx, &mf) - if diags.HasErrors() { - return nil, fmt.Errorf("mission '%s' %s: %w", missionName, folderBlock.Type, diags) + // Reject the old `folder { ... }` / `run_folder { ... }` / `run_memory { ... }` + // blocks with a clear pointer at the new syntax. + for _, b := range missionContent.Blocks { + switch b.Type { + case "folder": + return nil, fmt.Errorf("mission '%s': the `folder { ... }` block is no longer supported — use `memory { type = \"persistent\" }` instead (path is now derived automatically)", missionName) + case "run_folder": + return nil, fmt.Errorf("mission '%s': the `run_folder { ... }` block is no longer supported — use `memory { type = \"ephemeral\" }` instead", missionName) + case "run_memory": + return nil, fmt.Errorf("mission '%s': the `run_memory { ... }` block is no longer supported — use `memory { type = \"ephemeral\" }` instead", missionName) } - missionFolder = &mf } - // Parse optional run_memory / run_folder block (per-run ephemeral memory, - // reserved name "run"). "run_memory" is preferred; "run_folder" is kept as - // a backwards-compatible alias. - var missionRunFolder *MissionRunFolder - for _, rfBlock := range missionContent.Blocks { - if rfBlock.Type != "run_folder" && rfBlock.Type != "run_memory" { + // Parse zero or more `memory { ... }` blocks. A mission may declare at most + // one persistent and one ephemeral memory. + var persistentMemory, ephemeralMemory *MissionMemory + for _, mb := range missionContent.Blocks { + if mb.Type != "memory" { continue } - if missionRunFolder != nil { - return nil, fmt.Errorf("mission '%s': only one run_memory/run_folder block allowed", missionName) - } - var rf MissionRunFolder - diags := gohcl.DecodeBody(rfBlock.Body, ctx, &rf) + var mm MissionMemory + diags := gohcl.DecodeBody(mb.Body, ctx, &mm) if diags.HasErrors() { - return nil, fmt.Errorf("mission '%s' %s: %w", missionName, rfBlock.Type, diags) + return nil, fmt.Errorf("mission '%s' memory: %w", missionName, diags) + } + if mm.Type == "" { + mm.Type = MemoryTypePersistent } - if rf.Cleanup == nil { - v := DefaultRunFolderCleanupDays - rf.Cleanup = &v + switch mm.Type { + case MemoryTypePersistent: + if persistentMemory != nil { + return nil, fmt.Errorf("mission '%s': only one persistent memory block allowed", missionName) + } + persistentMemory = &mm + case MemoryTypeEphemeral: + if ephemeralMemory != nil { + return nil, fmt.Errorf("mission '%s': only one ephemeral memory block allowed", missionName) + } + if mm.Cleanup == nil { + v := DefaultEphemeralCleanupDays + mm.Cleanup = &v + } + ephemeralMemory = &mm + default: + return nil, fmt.Errorf("mission '%s' memory: type must be %q or %q (got %q)", missionName, MemoryTypePersistent, MemoryTypeEphemeral, mm.Type) } - missionRunFolder = &rf } // Parse schedule blocks (optional, multiple allowed) @@ -2080,9 +2084,9 @@ func parseMissionBlock(block *hcl.Block, ctx *hcl.EvalContext) (*Mission, error) Commander: missionCommander, Agents: missionAgents, LocalAgents: localAgents, - Folders: missionFolders, - Folder: missionFolder, - RunFolder: missionRunFolder, + Memories: missionMemories, + PersistentMemory: persistentMemory, + EphemeralMemory: ephemeralMemory, Schedules: schedules, Trigger: trigger, MaxParallel: maxParallel, diff --git a/config/folder_test.go b/config/folder_test.go index 69a5680..1e37038 100644 --- a/config/folder_test.go +++ b/config/folder_test.go @@ -7,44 +7,47 @@ import ( . "github.com/onsi/gomega" ) -var _ = Describe("Folders", func() { +var _ = Describe("Memory", func() { - Describe("memory spellings (preferred)", func() { - It("parses shared_memory + memories + memory + run_memory together", func() { + Describe("top-level memory block", func() { + It("parses a memory block and exposes it via memories.NAME", func() { hcl := fullBaseHCL() + ` -shared_memory "research" { - path = "./data" +memory "research" { description = "Research docs" + label = "Research" editable = true } mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] - memories = [shared_memories.research] - memory { - path = "./persistent" - description = "Persistent" - } - run_memory { - description = "Per-run scratch" - } + memories = [memories.research] task "t" { objective = "go" } } ` _, f := writeFixture("config.hcl", hcl) cfg, err := config.LoadFile(f) Expect(err).NotTo(HaveOccurred()) - Expect(cfg.SharedFolders).To(HaveLen(1)) - Expect(cfg.SharedFolders[0].Name).To(Equal("research")) - Expect(cfg.Missions[0].Folders).To(ConsistOf("research")) - Expect(cfg.Missions[0].Folder).NotTo(BeNil()) - Expect(cfg.Missions[0].Folder.Path).To(Equal("./persistent")) - Expect(cfg.Missions[0].RunFolder).NotTo(BeNil()) - Expect(cfg.Missions[0].RunFolder.Description).To(Equal("Per-run scratch")) + Expect(cfg.Memories).To(HaveLen(1)) + Expect(cfg.Memories[0].Name).To(Equal("research")) + Expect(cfg.Memories[0].Editable).To(BeTrue()) + Expect(cfg.Missions[0].Memories).To(ConsistOf("research")) + }) + + It("rejects the reserved name 'mission'", func() { + m := config.Memory{Name: "mission"} + err := m.Validate() + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("reserved")) + }) + + It("rejects the reserved name 'run'", func() { + m := config.Memory{Name: "run"} + err := m.Validate() + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("reserved")) }) - It("allows mixing old and new spellings across the file", func() { - // shared_folder + memories + memory mixed with run_folder + It("rejects the old shared_folder block with a pointer at the new syntax", func() { hcl := fullBaseHCL() + ` shared_folder "research" { path = "./data" @@ -52,21 +55,17 @@ shared_folder "research" { mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] - memories = [shared_memories.research] - memory { path = "./persistent" } - run_folder { description = "old name" } task "t" { objective = "go" } } ` _, f := writeFixture("config.hcl", hcl) - cfg, err := config.LoadFile(f) - Expect(err).NotTo(HaveOccurred()) - Expect(cfg.Missions[0].Folders).To(ConsistOf("research")) - Expect(cfg.Missions[0].Folder).NotTo(BeNil()) - Expect(cfg.Missions[0].RunFolder).NotTo(BeNil()) + _, err := config.LoadFile(f) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("no longer supported")) + Expect(err.Error()).To(ContainSubstring("memory \"research\"")) }) - It("rejects setting both 'memories' and 'folders' on the same mission", func() { + It("rejects the old shared_memory block too", func() { hcl := fullBaseHCL() + ` shared_memory "research" { path = "./data" @@ -74,73 +73,40 @@ shared_memory "research" { mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] - memories = [shared_memories.research] - folders = [shared_folders.research] task "t" { objective = "go" } } ` _, f := writeFixture("config.hcl", hcl) _, err := config.LoadFile(f) Expect(err).To(HaveOccurred()) - Expect(err.Error()).To(ContainSubstring("set either 'memories' or 'folders'")) + Expect(err.Error()).To(ContainSubstring("no longer supported")) }) - }) - Describe("shared_folder", func() { - It("parses a shared_folder block", func() { + It("rejects the old `path` attribute on a memory block", func() { hcl := fullBaseHCL() + ` -shared_folder "research" { - path = "./data" - description = "Research docs" - editable = true +memory "research" { + path = "./data" } mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] - folders = [shared_folders.research] task "t" { objective = "go" } } ` _, f := writeFixture("config.hcl", hcl) - cfg, err := config.LoadFile(f) - Expect(err).NotTo(HaveOccurred()) - Expect(cfg.SharedFolders).To(HaveLen(1)) - Expect(cfg.SharedFolders[0].Name).To(Equal("research")) - Expect(cfg.SharedFolders[0].Editable).To(BeTrue()) - Expect(cfg.Missions[0].Folders).To(ConsistOf("research")) - }) - - It("rejects the reserved name 'mission'", func() { - sf := config.SharedFolder{Name: "mission", Path: "./x"} - err := sf.Validate() - Expect(err).To(HaveOccurred()) - Expect(err.Error()).To(ContainSubstring("reserved")) - }) - - It("rejects the reserved name 'run'", func() { - sf := config.SharedFolder{Name: "run", Path: "./x"} - err := sf.Validate() - Expect(err).To(HaveOccurred()) - Expect(err.Error()).To(ContainSubstring("reserved")) - }) - - It("rejects an empty path", func() { - sf := config.SharedFolder{Name: "ok", Path: ""} - err := sf.Validate() + _, err := config.LoadFile(f) Expect(err).To(HaveOccurred()) - Expect(err.Error()).To(ContainSubstring("path is required")) }) }) - Describe("mission folder block", func() { - It("parses a dedicated folder block", func() { + Describe("mission memory block", func() { + It("parses a persistent memory block (default type)", func() { hcl := fullBaseHCL() + ` mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] - folder { - path = "./persistent" - description = "Persistent" + memory { + description = "Long-term notes" } task "t" { objective = "go" } } @@ -148,36 +114,38 @@ mission "m" { _, f := writeFixture("config.hcl", hcl) cfg, err := config.LoadFile(f) Expect(err).NotTo(HaveOccurred()) - Expect(cfg.Missions[0].Folder).NotTo(BeNil()) - Expect(cfg.Missions[0].Folder.Path).To(Equal("./persistent")) - Expect(cfg.Missions[0].Folder.Description).To(Equal("Persistent")) + Expect(cfg.Missions[0].PersistentMemory).NotTo(BeNil()) + Expect(cfg.Missions[0].PersistentMemory.Type).To(Equal(config.MemoryTypePersistent)) + Expect(cfg.Missions[0].PersistentMemory.Description).To(Equal("Long-term notes")) + Expect(cfg.Missions[0].EphemeralMemory).To(BeNil()) }) - It("rejects multiple folder blocks on the same mission", func() { + It("parses an explicit persistent type", func() { hcl := fullBaseHCL() + ` mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] - folder { path = "./a" } - folder { path = "./b" } + memory { + type = "persistent" + description = "x" + } task "t" { objective = "go" } } ` _, f := writeFixture("config.hcl", hcl) - _, err := config.LoadFile(f) - Expect(err).To(HaveOccurred()) - Expect(err.Error()).To(ContainSubstring("only one memory/folder block allowed")) + cfg, err := config.LoadFile(f) + Expect(err).NotTo(HaveOccurred()) + Expect(cfg.Missions[0].PersistentMemory).NotTo(BeNil()) }) - }) - Describe("mission run_folder block", func() { - It("parses a run_folder with defaults", func() { + It("parses an ephemeral memory block with default cleanup", func() { hcl := fullBaseHCL() + ` mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] - run_folder { - description = "Per-run scratch" + memory { + type = "ephemeral" + description = "Scratch" } task "t" { objective = "go" } } @@ -185,22 +153,21 @@ mission "m" { _, f := writeFixture("config.hcl", hcl) cfg, err := config.LoadFile(f) Expect(err).NotTo(HaveOccurred()) - Expect(cfg.Missions[0].RunFolder).NotTo(BeNil()) - Expect(cfg.Missions[0].RunFolder.Base).To(Equal("")) - Expect(cfg.Missions[0].RunFolder.Description).To(Equal("Per-run scratch")) - // Cleanup omitted → Validate() filled in the default - Expect(cfg.Missions[0].RunFolder.Cleanup).NotTo(BeNil()) - Expect(*cfg.Missions[0].RunFolder.Cleanup).To(Equal(config.DefaultRunFolderCleanupDays)) + Expect(cfg.Missions[0].EphemeralMemory).NotTo(BeNil()) + Expect(cfg.Missions[0].EphemeralMemory.Type).To(Equal(config.MemoryTypeEphemeral)) + Expect(cfg.Missions[0].EphemeralMemory.Cleanup).NotTo(BeNil()) + Expect(*cfg.Missions[0].EphemeralMemory.Cleanup).To(Equal(config.DefaultEphemeralCleanupDays)) + Expect(cfg.Missions[0].PersistentMemory).To(BeNil()) }) - It("parses a run_folder with custom base and cleanup", func() { + It("preserves an explicit cleanup = 0 on ephemeral memory", func() { hcl := fullBaseHCL() + ` mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] - run_folder { - base = "./custom_runs" - cleanup = 14 + memory { + type = "ephemeral" + cleanup = 0 } task "t" { objective = "go" } } @@ -208,116 +175,161 @@ mission "m" { _, f := writeFixture("config.hcl", hcl) cfg, err := config.LoadFile(f) Expect(err).NotTo(HaveOccurred()) - Expect(cfg.Missions[0].RunFolder.Base).To(Equal("./custom_runs")) - Expect(*cfg.Missions[0].RunFolder.Cleanup).To(Equal(14)) + Expect(cfg.Missions[0].EphemeralMemory.Cleanup).NotTo(BeNil()) + Expect(*cfg.Missions[0].EphemeralMemory.Cleanup).To(Equal(0)) }) - It("preserves an explicit cleanup of zero (never delete)", func() { + It("allows one persistent + one ephemeral on the same mission", func() { hcl := fullBaseHCL() + ` mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] - run_folder { - cleanup = 0 - } + memory { type = "persistent" } + memory { type = "ephemeral" } task "t" { objective = "go" } } ` _, f := writeFixture("config.hcl", hcl) cfg, err := config.LoadFile(f) Expect(err).NotTo(HaveOccurred()) - Expect(cfg.Missions[0].RunFolder.Cleanup).NotTo(BeNil()) - Expect(*cfg.Missions[0].RunFolder.Cleanup).To(Equal(0)) + Expect(cfg.Missions[0].PersistentMemory).NotTo(BeNil()) + Expect(cfg.Missions[0].EphemeralMemory).NotTo(BeNil()) + }) + + It("rejects two persistent memory blocks", func() { + hcl := fullBaseHCL() + ` +mission "m" { + commander { model = models.anthropic.claude_sonnet_4 } + agents = [agents.test_agent] + memory { type = "persistent" } + memory { type = "persistent" } + task "t" { objective = "go" } +} +` + _, f := writeFixture("config.hcl", hcl) + _, err := config.LoadFile(f) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("only one persistent memory")) }) - It("rejects multiple run_folder blocks", func() { + It("rejects two ephemeral memory blocks", func() { hcl := fullBaseHCL() + ` mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] - run_folder { base = "./a" } - run_folder { base = "./b" } + memory { type = "ephemeral" } + memory { type = "ephemeral" } task "t" { objective = "go" } } ` _, f := writeFixture("config.hcl", hcl) _, err := config.LoadFile(f) Expect(err).To(HaveOccurred()) - Expect(err.Error()).To(ContainSubstring("only one run_memory/run_folder block allowed")) + Expect(err.Error()).To(ContainSubstring("only one ephemeral memory")) }) - It("rejects a negative cleanup value", func() { - neg := -1 - rf := config.MissionRunFolder{Cleanup: &neg} - err := rf.Validate() + It("rejects an unknown type", func() { + hcl := fullBaseHCL() + ` +mission "m" { + commander { model = models.anthropic.claude_sonnet_4 } + agents = [agents.test_agent] + memory { type = "weird" } + task "t" { objective = "go" } +} +` + _, f := writeFixture("config.hcl", hcl) + _, err := config.LoadFile(f) Expect(err).To(HaveOccurred()) - Expect(err.Error()).To(ContainSubstring("cleanup")) + Expect(err.Error()).To(ContainSubstring(`type must be "persistent" or "ephemeral"`)) }) - It("allows cleanup of zero (never)", func() { - zero := 0 - rf := config.MissionRunFolder{Cleanup: &zero} - Expect(rf.Validate()).To(Succeed()) - Expect(*rf.Cleanup).To(Equal(0)) + It("rejects cleanup on persistent memory", func() { + mm := &config.MissionMemory{Type: "persistent", Cleanup: ptrInt(7)} + Expect(mm.Validate()).To(MatchError(ContainSubstring("cleanup is only valid on ephemeral memory"))) }) - It("Validate accepts an unset Cleanup (parser fills the default)", func() { - rf := config.MissionRunFolder{} - Expect(rf.Validate()).To(Succeed()) + It("rejects negative cleanup", func() { + mm := &config.MissionMemory{Type: "ephemeral", Cleanup: ptrInt(-1)} + Expect(mm.Validate()).To(MatchError(ContainSubstring("cleanup must be >= 0"))) }) - It("rejects mixing folder and memory blocks on the same mission", func() { + It("rejects the old `path` attribute on a mission memory block", func() { hcl := fullBaseHCL() + ` mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] - folder { path = "./a" } - memory { path = "./b" } + memory { path = "./x" } task "t" { objective = "go" } } ` _, f := writeFixture("config.hcl", hcl) _, err := config.LoadFile(f) Expect(err).To(HaveOccurred()) - Expect(err.Error()).To(ContainSubstring("only one memory/folder block allowed")) }) + }) - It("rejects mixing run_folder and run_memory blocks on the same mission", func() { + Describe("deprecated DSL surfaces", func() { + It("rejects the old `folder { ... }` block with a pointer at the new syntax", func() { hcl := fullBaseHCL() + ` mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] - run_folder { base = "./a" } - run_memory { base = "./b" } + folder { path = "./x" } task "t" { objective = "go" } } ` _, f := writeFixture("config.hcl", hcl) _, err := config.LoadFile(f) Expect(err).To(HaveOccurred()) - Expect(err.Error()).To(ContainSubstring("only one run_memory/run_folder block allowed")) + Expect(err.Error()).To(ContainSubstring("`folder { ... }` block is no longer supported")) }) - It("allows both folder and run_folder on the same mission", func() { + It("rejects the old `run_folder { ... }` block", func() { hcl := fullBaseHCL() + ` mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] - folder { - path = "./persist" - } - run_folder { - base = "./runs" - cleanup = 7 - } + run_folder { base = "./x" } task "t" { objective = "go" } } ` _, f := writeFixture("config.hcl", hcl) - cfg, err := config.LoadFile(f) - Expect(err).NotTo(HaveOccurred()) - Expect(cfg.Missions[0].Folder).NotTo(BeNil()) - Expect(cfg.Missions[0].RunFolder).NotTo(BeNil()) + _, err := config.LoadFile(f) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("`run_folder { ... }` block is no longer supported")) + }) + + It("rejects the old `run_memory { ... }` block", func() { + hcl := fullBaseHCL() + ` +mission "m" { + commander { model = models.anthropic.claude_sonnet_4 } + agents = [agents.test_agent] + run_memory { description = "x" } + task "t" { objective = "go" } +} +` + _, f := writeFixture("config.hcl", hcl) + _, err := config.LoadFile(f) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("`run_memory { ... }` block is no longer supported")) + }) + + It("rejects the old `folders = ...` attribute", func() { + hcl := fullBaseHCL() + ` +memory "ref" {} +mission "m" { + commander { model = models.anthropic.claude_sonnet_4 } + agents = [agents.test_agent] + folders = [memories.ref] + task "t" { objective = "go" } +} +` + _, f := writeFixture("config.hcl", hcl) + _, err := config.LoadFile(f) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("`folders` attribute is no longer supported")) }) }) }) + +func ptrInt(v int) *int { return &v } diff --git a/config/memory.go b/config/memory.go new file mode 100644 index 0000000..4adbf69 --- /dev/null +++ b/config/memory.go @@ -0,0 +1,81 @@ +package config + +import "fmt" + +// Reserved slot names for mission-scoped memory. Tool calls reference these +// via the `folder` parameter (e.g. `folder: "mission"` or `folder: "run"`). +const ( + PersistentSlotName = "mission" + EphemeralSlotName = "run" +) + +// Valid values for the `type` attribute on a mission-scoped `memory` block. +const ( + MemoryTypePersistent = "persistent" + MemoryTypeEphemeral = "ephemeral" +) + +// DefaultEphemeralCleanupDays is the auto-delete window applied to an +// ephemeral mission memory when `cleanup` is not set. +const DefaultEphemeralCleanupDays = 7 + +// Memory describes a top-level shared memory block: +// +// memory "research" { +// description = "..." +// label = "..." +// editable = true +// } +// +// The storage path is derived by the runtime — it lives at +// `/memories/shared//` — so no `path` is accepted from +// HCL. +type Memory struct { + Name string `hcl:"name,label"` + Description string `hcl:"description,optional"` + Label string `hcl:"label,optional"` + Editable bool `hcl:"editable,optional"` +} + +// Validate enforces naming rules. The literal names "mission" and "run" are +// reserved for the mission-scoped memory slots and must not be reused by a +// shared memory. +func (m *Memory) Validate() error { + if m.Name == PersistentSlotName || m.Name == EphemeralSlotName { + return fmt.Errorf("name %q is reserved for mission-scoped memory", m.Name) + } + return nil +} + +// MissionMemory describes a `memory { ... }` block inside a mission. A +// mission may declare at most one persistent and one ephemeral memory. +// +// The storage path is derived by the runtime from the mission name and (for +// ephemeral) the mission instance ID, so no `path` is accepted from HCL. +// +// Cleanup is a pointer so we can distinguish "user did not set it" (apply +// default of 7 days, only meaningful for ephemeral) from "user set 0" (keep +// forever). +type MissionMemory struct { + Type string `hcl:"type,optional"` // "persistent" (default) or "ephemeral" + Description string `hcl:"description,optional"` + Cleanup *int `hcl:"cleanup,optional"` // ephemeral only; days before auto-delete, 0 = never +} + +// Validate normalizes Type (defaulting to persistent), enforces the cleanup +// rules, and rejects unknown type values. +func (mm *MissionMemory) Validate() error { + if mm.Type == "" { + mm.Type = MemoryTypePersistent + } + if mm.Type != MemoryTypePersistent && mm.Type != MemoryTypeEphemeral { + return fmt.Errorf("type must be %q or %q (got %q)", MemoryTypePersistent, MemoryTypeEphemeral, mm.Type) + } + if mm.Type == MemoryTypePersistent && mm.Cleanup != nil { + return fmt.Errorf("cleanup is only valid on ephemeral memory") + } + if mm.Cleanup != nil && *mm.Cleanup < 0 { + return fmt.Errorf("cleanup must be >= 0 (days)") + } + return nil +} diff --git a/config/mission.go b/config/mission.go index 14438a2..a946ee4 100644 --- a/config/mission.go +++ b/config/mission.go @@ -74,46 +74,7 @@ type OutputField struct { Properties []OutputField `json:"properties,omitempty"` } -// MissionFolder represents a dedicated folder for a mission. -// Registered under the reserved name "mission". Persists across runs. -type MissionFolder struct { - Path string `hcl:"path"` - Description string `hcl:"description,optional"` -} - -// Validate checks that the mission folder configuration is valid -func (mf *MissionFolder) Validate() error { - if mf.Path == "" { - return fmt.Errorf("path is required") - } - return nil -} - -// DefaultRunFolderCleanupDays is the auto-delete window applied when a -// run_folder block does not specify `cleanup`. -const DefaultRunFolderCleanupDays = 7 - -// MissionRunFolder represents a per-run ephemeral folder for a mission. -// Registered under the reserved name "run". A fresh subdirectory is created -// under Base for each mission run, keyed by mission ID. -// -// Cleanup is a pointer so we can distinguish "user didn't set it" (apply -// default of 7 days) from "user set 0" (keep forever). -type MissionRunFolder struct { - Base string `hcl:"base,optional"` // parent directory; defaults to ".squadron/runs" - Description string `hcl:"description,optional"` - Cleanup *int `hcl:"cleanup,optional"` // days after creation before auto-delete; nil = default (7), 0 = never -} - -// Validate rejects negative cleanup values. Default-filling happens at parse -// time (see config.go) so callers reading a parsed Mission see a complete -// struct without needing to validate first. -func (rf *MissionRunFolder) Validate() error { - if rf.Cleanup != nil && *rf.Cleanup < 0 { - return fmt.Errorf("cleanup must be >= 0 (days)") - } - return nil -} +// (Memory and MissionMemory live in memory.go.) // Schedule defines a time-based trigger for a mission. // Three modes (mutually exclusive): @@ -348,9 +309,9 @@ type Mission struct { Tasks []Task `hcl:"task,block"` Inputs []MissionInput // Parsed from input blocks Datasets []Dataset // Parsed from dataset blocks - Folders []string // Shared folder names referenced by this mission - Folder *MissionFolder // Optional dedicated mission folder (reserved name "mission") - RunFolder *MissionRunFolder // Optional per-run ephemeral folder (reserved name "run") + Memories []string // Shared memory names referenced by this mission + PersistentMemory *MissionMemory // Optional dedicated mission memory (reserved slot "mission") + EphemeralMemory *MissionMemory // Optional per-run ephemeral memory (reserved slot "run") Schedules []Schedule `json:"schedules,omitempty"` Trigger *Trigger `json:"trigger,omitempty"` MaxParallel int `json:"maxParallel,omitempty"` // default 3 @@ -394,7 +355,7 @@ type TaskRoute struct { } // Validate checks that the mission configuration is valid -func (w *Mission) Validate(models []Model, agents []Agent, sharedFolders []SharedFolder, allMissionNames map[string]bool) error { +func (w *Mission) Validate(models []Model, agents []Agent, memories []Memory, allMissionNames map[string]bool) error { if w.Name == "" { return fmt.Errorf("mission name is required") } @@ -513,28 +474,34 @@ func (w *Mission) Validate(models []Model, agents []Agent, sharedFolders []Share } } - // Validate folder references - folderNames := make(map[string]bool) - for _, sf := range sharedFolders { - folderNames[sf.Name] = true + // Validate shared memory references + memoryNames := make(map[string]bool) + for _, m := range memories { + memoryNames[m.Name] = true } - for _, folderRef := range w.Folders { - if !folderNames[folderRef] { - return fmt.Errorf("shared folder '%s' not found", folderRef) + for _, ref := range w.Memories { + if !memoryNames[ref] { + return fmt.Errorf("shared memory '%s' not found", ref) } } - // Validate dedicated folder if present - if w.Folder != nil { - if err := w.Folder.Validate(); err != nil { - return fmt.Errorf("folder: %w", err) + // Validate persistent mission memory if present + if w.PersistentMemory != nil { + if err := w.PersistentMemory.Validate(); err != nil { + return fmt.Errorf("persistent memory: %w", err) + } + if w.PersistentMemory.Type != MemoryTypePersistent { + return fmt.Errorf("persistent memory: internal error — wrong type %q", w.PersistentMemory.Type) } } - // Validate run folder if present - if w.RunFolder != nil { - if err := w.RunFolder.Validate(); err != nil { - return fmt.Errorf("run_folder: %w", err) + // Validate ephemeral mission memory if present + if w.EphemeralMemory != nil { + if err := w.EphemeralMemory.Validate(); err != nil { + return fmt.Errorf("ephemeral memory: %w", err) + } + if w.EphemeralMemory.Type != MemoryTypeEphemeral { + return fmt.Errorf("ephemeral memory: internal error — wrong type %q", w.EphemeralMemory.Type) } } diff --git a/config/shared_folder.go b/config/shared_folder.go deleted file mode 100644 index b29f409..0000000 --- a/config/shared_folder.go +++ /dev/null @@ -1,35 +0,0 @@ -package config - -import ( - "fmt" - "os" - - "squadron/internal/paths" -) - -type SharedFolder struct { - Name string `hcl:"name,label"` - Path string `hcl:"path"` - Label string `hcl:"label,optional"` - Description string `hcl:"description,optional"` - Editable bool `hcl:"editable,optional"` -} - -func (fb *SharedFolder) Validate() error { - if fb.Name == "mission" || fb.Name == "run" { - return fmt.Errorf("name %q is reserved for mission-scoped folders", fb.Name) - } - if fb.Path == "" { - return fmt.Errorf("path is required") - } - absPath, err := paths.ResolveFolderPath(fb.Path) - if err != nil { - return fmt.Errorf("invalid path: %w", err) - } - // Create the directory if it doesn't exist (common in container mode) - if err := os.MkdirAll(absPath, 0755); err != nil { - return fmt.Errorf("failed to create folder path: %w", err) - } - fb.Path = absPath - return nil -} diff --git a/docs/content/config/overview.mdx b/docs/content/config/overview.mdx index 64423a2..5e2cc1c 100644 --- a/docs/content/config/overview.mdx +++ b/docs/content/config/overview.mdx @@ -60,7 +60,7 @@ agent "assistant" { | `mcp_host` | Start a built-in [MCP host](/config/mcp_host) to expose Squadron to AI clients | | `mission` | Define multi-task missions | | `commander` | Commander server connection config | -| `shared_memory` | Shared filesystem locations accessible to missions (formerly `shared_folder`; both spellings accepted) | +| `memory` | Shared filesystem locations accessible to missions (paths managed under `/memories/shared/`) | ## Expressions diff --git a/docs/content/missions/folders.mdx b/docs/content/missions/folders.mdx index ea5008f..03421b0 100644 --- a/docs/content/missions/folders.mdx +++ b/docs/content/missions/folders.mdx @@ -4,46 +4,39 @@ title: Memory # Memory -Memory blocks are filesystem locations that agents can read from and write to. Squadron gives you three kinds, each suited to a different use case: +Memory blocks are filesystem locations that agents can read from and write to. Squadron manages the storage paths for you — you declare what memory you need, and Squadron decides where on disk it lives, under `/memories/`. -| Kind | Use it for | Lifetime | -|------|------------|----------| -| **Shared** | Reference data or output that many missions touch | Persists | -| **Mission** | A mission's own long-lived state — archives, accumulated output | Persists | -| **Run** | Scratch space for the work happening right now | One per run | +Three kinds, each with a fixed location pattern: -Agents reach memory through the `file_list`, `file_read`, `file_create`, `file_delete`, `file_search`, and `file_grep` tools, naming the slot on each call. +| Kind | HCL | Slot name agents use | On-disk path | +|------|-----|-----------------------|--------------| +| **Shared** | top-level `memory "name" { ... }` | the HCL label | `/memories/shared//` | +| **Persistent** (mission) | `memory { type = "persistent" }` inside a mission | `"mission"` | `/memories/mission//persistent/` | +| **Ephemeral** (per-run) | `memory { type = "ephemeral" }` inside a mission | `"run"` | `/memories/mission//run//` | + +A mission may declare at most one persistent and one ephemeral memory. The names `"mission"` and `"run"` are reserved — a shared `memory "mission"` or `memory "run"` block is rejected. -A mission can use any number of shared memory blocks, plus one of each mission-scoped kind. The names `mission` and `run` are reserved. +Agents reach memory through the `file_list`, `file_read`, `file_create`, `file_delete`, `file_search`, and `file_grep` tools, naming the slot on each call. -> **Naming note.** Memory blocks were originally called "folders". The old HCL -> spellings — `shared_folder`, `folder`, `run_folder`, `folders`, -> `shared_folders` — still work as backwards-compatible aliases of the new -> `shared_memory`, `memory`, `run_memory`, `memories`, `shared_memories` names. -> Use whichever spelling you prefer; new code should prefer the memory -> spellings. Mixing both spellings for the *same* slot on one mission is an -> error. The tools themselves still take a parameter literally called `folder`. +> **No `path` attribute.** Squadron owns memory locations. You don't write `path = "./somewhere"` anywhere — paths are derived from the mission name (and, for ephemeral, the per-run instance ID). ## Shared Memory Declared at the top level so multiple missions can share the same data: ```hcl -shared_memory "research" { - path = "./data/research" +memory "research" { description = "Shared research documents" editable = true } -shared_memory "reference" { - path = "./data/reference" +memory "reference" { # no editable flag → read-only } ``` | Attribute | Type | Description | |-----------|------|-------------| -| `path` | string | Directory path (created if missing) | | `description` | string | Human-readable description shown to agents | | `label` | string | Optional label (falls back to description) | | `editable` | bool | If `true`, agents may write. Default `false` (read-only) | @@ -52,19 +45,19 @@ Missions opt in by listing them, then refer to them by name in tool calls: ```hcl mission "analyze" { - memories = [shared_memories.research, shared_memories.reference] + memories = [memories.research, memories.reference] # agents call e.g. file_read with folder = "research" } ``` -## The Mission Memory +## Persistent Mission Memory The mission's own dedicated slot. Use it for things you want to keep around: a running log, an archive of finished reports, accumulated state across runs. ```hcl mission "analyze" { memory { - path = "./analyses" + # type defaults to "persistent" when omitted description = "Cumulative analysis output across every run" } } @@ -78,22 +71,23 @@ Agents reach it under the name `mission`: | Attribute | Type | Description | |-----------|------|-------------| -| `path` | string | Directory path (created if missing) | +| `type` | string | `"persistent"` (default) or `"ephemeral"` | | `description` | string | Shown to agents alongside the slot name | -## The Run Memory +## Ephemeral (Per-Run) Memory A clean workspace for one mission run. Each invocation gets its own subdirectory, isolated from previous and concurrent runs — perfect for intermediate files, working drafts, or anything you don't want bleeding into the next run. ```hcl mission "analyze" { - run_memory { + memory { + type = "ephemeral" description = "Scratch space for this run" } } ``` -By default, run memory is deleted 7 days after the run started. Set `cleanup` to override the window, or `cleanup = 0` to keep it forever. +By default, ephemeral memory is deleted 7 days after the run started. Set `cleanup` to override the window, or `cleanup = 0` to keep it forever. Agents reach it under the name `run`: @@ -103,11 +97,11 @@ Agents reach it under the name `run`: | Attribute | Type | Description | |-----------|------|-------------| -| `base` | string | Where run directories live. Defaults to `.squadron/runs` — change it if you want them somewhere else | +| `type` | string | Must be `"ephemeral"` | | `description` | string | Shown to agents alongside the slot name | | `cleanup` | integer | Delete the directory this many days after the run started. Defaults to `7`; set `0` to keep forever | -Run directories stick around after the mission ends — useful for inspecting what an agent produced, and required for `squadron mission --resume ` to find its workspace. `cleanup` is just a max age; nothing is deleted while a run is in flight. +Ephemeral directories stick around after the mission ends — useful for inspecting what an agent produced, and required for `squadron mission --resume ` to find its workspace. `cleanup` is just a max age; nothing is deleted while a run is in flight. ## Tool Reference @@ -127,22 +121,22 @@ Paths are always relative to the slot's root directory. Absolute paths and `..` ## Full Example ```hcl -shared_memory "reference" { - path = "./data/reference" +memory "reference" { description = "Reference materials shared across missions" } mission "research" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.researcher] - memories = [shared_memories.reference] + memories = [memories.reference] memory { - path = "./research_archive" + # persistent by default — survives across every run description = "Finished reports, one per run" } - run_memory { + memory { + type = "ephemeral" description = "Working files for the in-flight run" cleanup = 14 # override the 7-day default } diff --git a/docs/content/missions/overview.mdx b/docs/content/missions/overview.mdx index 6541dd0..4bbe4c4 100644 --- a/docs/content/missions/overview.mdx +++ b/docs/content/missions/overview.mdx @@ -43,9 +43,8 @@ mission "data_pipeline" { | `input` | block | Mission input parameters (repeatable) | | `task` | block | Task definitions (repeatable) | | `dataset` | block | Dataset definitions (optional) | -| `memories` | list | Shared memory references, e.g. `[shared_memories.data]` (see [Memory](/missions/folders)). Old name: `folders` (still accepted). | -| `memory` | block | Persistent mission-scoped memory, registered as `"mission"`. Old name: `folder` (still accepted). | -| `run_memory` | block | Per-run ephemeral memory, registered as `"run"`. Old name: `run_folder` (still accepted). | +| `memories` | list | Shared memory references, e.g. `[memories.data]` (see [Memory](/missions/folders)) | +| `memory` | block | Mission-scoped memory; `type = "persistent"` (default) registers as `"mission"`, `type = "ephemeral"` registers as `"run"`. At most one of each per mission. | | `schedule` | block | Automatic run schedules (optional, repeatable) | | `trigger` | block | Webhook trigger (optional) | | `max_parallel` | number | Max concurrent instances (default: 3) | diff --git a/mission/folder_store.go b/mission/folder_store.go index ef8bd03..ca8bd05 100644 --- a/mission/folder_store.go +++ b/mission/folder_store.go @@ -14,12 +14,16 @@ import ( "squadron/internal/paths" ) -// DefaultRunFolderBase is the parent directory used when a run_folder block -// does not specify `base`. -const DefaultRunFolderBase = ".squadron/runs" +// memoriesSubdir is the directory under SquadronHome that holds every +// materialized memory slot. Layout: +// +// /memories/shared// +// /memories/mission//persistent/ +// /memories/mission//run// +const memoriesSubdir = "memories" -// runMetadataFile is the sidecar written inside each materialized run folder -// so the cleanup sweep can tell when the folder was created. +// runMetadataFile is the sidecar written inside each materialized ephemeral +// memory directory so the cleanup sweep can tell when it was created. const runMetadataFile = ".squadron-run.json" type runMetadata struct { @@ -29,24 +33,68 @@ type runMetadata struct { CleanupDays int `json:"cleanup_days,omitempty"` } -// ResolvedRunFolderBase returns the base directory for a run_folder, -// substituting the default when unset. -func ResolvedRunFolderBase(rf *config.MissionRunFolder) string { - if rf == nil || rf.Base == "" { - return DefaultRunFolderBase +// MemoriesRoot returns `/memories`, the parent of every +// materialized memory slot. Exported so callers like the cleanup loop can +// pivot off the same base. +func MemoriesRoot() (string, error) { + home, err := paths.SquadronHome() + if err != nil { + return "", err + } + return filepath.Join(home, memoriesSubdir), nil +} + +// SharedMemoryPath returns the on-disk path for a top-level shared memory +// named `name`. +func SharedMemoryPath(name string) (string, error) { + root, err := MemoriesRoot() + if err != nil { + return "", err + } + return filepath.Join(root, "shared", name), nil +} + +// PersistentMemoryPath returns the on-disk path for a mission's persistent +// memory slot. Path is stable across runs of the same mission. +func PersistentMemoryPath(missionName string) (string, error) { + root, err := MemoriesRoot() + if err != nil { + return "", err } - return rf.Base + return filepath.Join(root, "mission", missionName, "persistent"), nil } -// resolvedCleanup returns the cleanup window in days for a run_folder. -// Reads the parsed pointer when set (Validate fills in the default at config -// load time); falls back to the default for callers that hand-build a struct -// and skip Validate (notably tests). -func resolvedCleanup(rf *config.MissionRunFolder) int { - if rf == nil || rf.Cleanup == nil { - return config.DefaultRunFolderCleanupDays +// EphemeralMemoryPath returns the on-disk path for one run's ephemeral +// memory slot. Path is unique per mission run instance. +func EphemeralMemoryPath(missionName, missionInstanceID string) (string, error) { + root, err := MemoriesRoot() + if err != nil { + return "", err + } + return filepath.Join(root, "mission", missionName, "run", missionInstanceID), nil +} + +// MissionRunRoot returns `/mission//run`, the parent +// dir of every ephemeral memory directory for a given mission. Used by the +// cleanup sweep. +func MissionRunRoot(missionName string) (string, error) { + root, err := MemoriesRoot() + if err != nil { + return "", err } - return *rf.Cleanup + return filepath.Join(root, "mission", missionName, "run"), nil +} + +// resolvedEphemeralCleanup returns the cleanup window in days for an +// ephemeral mission memory. Reads the parsed pointer when set (Validate / +// the config parser fill in the default at config load time); falls back to +// the default for callers that hand-build a struct and skip Validate +// (notably tests). +func resolvedEphemeralCleanup(mm *config.MissionMemory) int { + if mm == nil || mm.Cleanup == nil { + return config.DefaultEphemeralCleanupDays + } + return *mm.Cleanup } type missionFolderStore struct { @@ -59,75 +107,78 @@ type folderEntry struct { writable bool } -// buildFolderStore creates a FolderStore from the mission config. -// missionID scopes the per-run folder path; it must be non-empty when -// mission.RunFolder is set. Returns nil if no folders are configured. -func buildFolderStore(mission *config.Mission, sharedFolders []config.SharedFolder, missionID string) (aitools.FolderStore, error) { +// buildFolderStore creates a FolderStore from the mission config and the +// declared top-level memories. missionInstanceID scopes the ephemeral memory +// path; it must be non-empty when mission.EphemeralMemory is set. Returns +// nil if no memory slots are configured. +func buildFolderStore(mission *config.Mission, memories []config.Memory, missionInstanceID string) (aitools.FolderStore, error) { store := &missionFolderStore{ folders: make(map[string]*folderEntry), } - foldersByName := make(map[string]*config.SharedFolder) - for i := range sharedFolders { - foldersByName[sharedFolders[i].Name] = &sharedFolders[i] + memByName := make(map[string]*config.Memory) + for i := range memories { + memByName[memories[i].Name] = &memories[i] } - for _, name := range mission.Folders { - if name == aitools.MissionFolderName || name == aitools.RunFolderName { - return nil, fmt.Errorf("shared folder %q uses a reserved name", name) + for _, name := range mission.Memories { + if name == config.PersistentSlotName || name == config.EphemeralSlotName { + return nil, fmt.Errorf("shared memory %q uses a reserved name", name) } - sf, ok := foldersByName[name] + mem, ok := memByName[name] if !ok { - return nil, fmt.Errorf("shared folder %q not found", name) + return nil, fmt.Errorf("shared memory %q not found", name) } - absPath, err := paths.ResolveFolderPath(sf.Path) + absPath, err := SharedMemoryPath(name) if err != nil { - return nil, fmt.Errorf("shared folder %q: invalid path: %w", name, err) + return nil, fmt.Errorf("shared memory %q: resolve path: %w", name, err) } - desc := sf.Description + if err := os.MkdirAll(absPath, 0755); err != nil { + return nil, fmt.Errorf("shared memory %q: create directory: %w", name, err) + } + desc := mem.Description if desc == "" { - desc = sf.Label + desc = mem.Label } store.folders[name] = &folderEntry{ absPath: absPath, description: desc, - writable: sf.Editable, + writable: mem.Editable, } } - if mission.Folder != nil { - absPath, err := paths.ResolveFolderPath(mission.Folder.Path) + if mission.PersistentMemory != nil { + absPath, err := PersistentMemoryPath(mission.Name) if err != nil { - return nil, fmt.Errorf("mission folder: invalid path: %w", err) + return nil, fmt.Errorf("persistent memory: resolve path: %w", err) } if err := os.MkdirAll(absPath, 0755); err != nil { - return nil, fmt.Errorf("mission folder: create directory: %w", err) + return nil, fmt.Errorf("persistent memory: create directory: %w", err) } - store.folders[aitools.MissionFolderName] = &folderEntry{ + store.folders[config.PersistentSlotName] = &folderEntry{ absPath: absPath, - description: mission.Folder.Description, + description: mission.PersistentMemory.Description, writable: true, } } - if mission.RunFolder != nil { - if missionID == "" { - return nil, fmt.Errorf("run_folder requires a mission ID") + if mission.EphemeralMemory != nil { + if missionInstanceID == "" { + return nil, fmt.Errorf("ephemeral memory requires a mission instance ID") } - absBase, err := paths.ResolveFolderPath(ResolvedRunFolderBase(mission.RunFolder)) + absPath, err := EphemeralMemoryPath(mission.Name, missionInstanceID) if err != nil { - return nil, fmt.Errorf("run_folder: invalid base: %w", err) + return nil, fmt.Errorf("ephemeral memory: resolve path: %w", err) } - absPath := filepath.Join(absBase, missionID) if err := os.MkdirAll(absPath, 0755); err != nil { - return nil, fmt.Errorf("run_folder: create directory: %w", err) + return nil, fmt.Errorf("ephemeral memory: create directory: %w", err) } - if err := writeRunMetadata(absPath, mission.Name, missionID, resolvedCleanup(mission.RunFolder)); err != nil { - return nil, fmt.Errorf("run_folder: write metadata: %w", err) + if err := writeRunMetadata(absPath, mission.Name, missionInstanceID, resolvedEphemeralCleanup(mission.EphemeralMemory)); err != nil { + return nil, fmt.Errorf("ephemeral memory: write metadata: %w", err) } - store.folders[aitools.RunFolderName] = &folderEntry{ + store.folders[config.EphemeralSlotName] = &folderEntry{ absPath: absPath, - description: mission.RunFolder.Description, + description: mission.EphemeralMemory.Description, writable: true, } } @@ -139,10 +190,10 @@ func buildFolderStore(mission *config.Mission, sharedFolders []config.SharedFold return store, nil } -// writeRunMetadata records when the run folder was created so the sweep can -// decide when to delete it. Uses O_CREATE|O_EXCL so concurrent starts and -// resumes never clobber the original timestamp — exactly one writer wins, -// others observe EEXIST and skip. +// writeRunMetadata records when the ephemeral memory directory was created +// so the sweep can decide when to delete it. Uses O_CREATE|O_EXCL so +// concurrent starts and resumes never clobber the original timestamp — +// exactly one writer wins, others observe EEXIST and skip. func writeRunMetadata(dir, missionName, missionID string, cleanupDays int) error { path := filepath.Join(dir, runMetadataFile) f, err := os.OpenFile(path, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0644) @@ -210,16 +261,21 @@ func (s *missionFolderStore) FolderInfos() []aitools.FolderInfo { return infos } -// SweepExpiredRunFolders deletes any subfolder of base whose sidecar -// (.squadron-run.json) records a created_at older than its cleanup_days. -// Folders without a sidecar, or with cleanup_days == 0, are left alone. -// Missing base is not an error — returns (nil, nil). -func SweepExpiredRunFolders(base string) (removed []string, err error) { - absBase, err := paths.ResolveFolderPath(base) +// SweepExpiredEphemeralMemories deletes any per-run ephemeral memory +// directory whose sidecar (.squadron-run.json) records a created_at older +// than its cleanup_days. Directories without a sidecar, or with +// cleanup_days == 0, are left alone. +// +// It walks `/mission/*/run/*` and considers every +// per-run directory it finds — there's no per-mission filtering, so callers +// don't need to know which missions exist. +func SweepExpiredEphemeralMemories() (removed []string, err error) { + root, err := MemoriesRoot() if err != nil { return nil, err } - entries, err := os.ReadDir(absBase) + missionsBase := filepath.Join(root, "mission") + entries, err := os.ReadDir(missionsBase) if err != nil { if os.IsNotExist(err) { return nil, nil @@ -227,31 +283,44 @@ func SweepExpiredRunFolders(base string) (removed []string, err error) { return nil, err } now := time.Now().UTC() - for _, e := range entries { - if !e.IsDir() { + for _, missionEntry := range entries { + if !missionEntry.IsDir() { continue } - runDir := filepath.Join(absBase, e.Name()) - metaPath := filepath.Join(runDir, runMetadataFile) - b, err := os.ReadFile(metaPath) + runBase := filepath.Join(missionsBase, missionEntry.Name(), "run") + runEntries, err := os.ReadDir(runBase) if err != nil { - continue + if os.IsNotExist(err) { + continue + } + return removed, err } - var meta runMetadata - if err := json.Unmarshal(b, &meta); err != nil { - continue - } - if meta.CleanupDays <= 0 { - continue - } - deadline := meta.CreatedAt.Add(time.Duration(meta.CleanupDays) * 24 * time.Hour) - if now.Before(deadline) { - continue - } - if err := os.RemoveAll(runDir); err != nil { - continue + for _, e := range runEntries { + if !e.IsDir() { + continue + } + runDir := filepath.Join(runBase, e.Name()) + metaPath := filepath.Join(runDir, runMetadataFile) + b, err := os.ReadFile(metaPath) + if err != nil { + continue + } + var meta runMetadata + if err := json.Unmarshal(b, &meta); err != nil { + continue + } + if meta.CleanupDays <= 0 { + continue + } + deadline := meta.CreatedAt.Add(time.Duration(meta.CleanupDays) * 24 * time.Hour) + if now.Before(deadline) { + continue + } + if err := os.RemoveAll(runDir); err != nil { + continue + } + removed = append(removed, runDir) } - removed = append(removed, runDir) } return removed, nil } diff --git a/mission/folder_store_test.go b/mission/folder_store_test.go index 4f9148d..67eee91 100644 --- a/mission/folder_store_test.go +++ b/mission/folder_store_test.go @@ -9,27 +9,41 @@ import ( "squadron/aitools" "squadron/config" + "squadron/internal/paths" ) -func TestBuildFolderStore_NoFolders(t *testing.T) { +// withTempHome installs a fresh SquadronHome under a t.TempDir and registers +// cleanup so the next test starts from a clean cache. +func withTempHome(t *testing.T) string { + t.Helper() + home := t.TempDir() + paths.ResetHome() + if err := paths.SetHome(home); err != nil { + t.Fatalf("set home: %v", err) + } + t.Cleanup(paths.ResetHome) + return home +} + +func TestBuildFolderStore_NoMemories(t *testing.T) { + withTempHome(t) m := &config.Mission{Name: "m"} store, err := buildFolderStore(m, nil, "mid-1") if err != nil { t.Fatalf("unexpected error: %v", err) } if store != nil { - t.Fatalf("expected nil store when no folders configured, got %+v", store) + t.Fatalf("expected nil store when no memories configured, got %+v", store) } } -func TestBuildFolderStore_MissionFolder(t *testing.T) { - dir := t.TempDir() - missionDir := filepath.Join(dir, "persistent") +func TestBuildFolderStore_PersistentMemory(t *testing.T) { + home := withTempHome(t) m := &config.Mission{ Name: "m", - Folder: &config.MissionFolder{ - Path: missionDir, + PersistentMemory: &config.MissionMemory{ + Type: "persistent", Description: "persistent", }, } @@ -42,35 +56,34 @@ func TestBuildFolderStore_MissionFolder(t *testing.T) { t.Fatal("expected non-nil store") } - // Registered under reserved name "mission", not the mission name abs, writable, err := store.ResolvePath(aitools.MissionFolderName, ".") if err != nil { t.Fatalf("ResolvePath: %v", err) } if !writable { - t.Fatal("mission folder must be writable") + t.Fatal("persistent memory must be writable") } - if !filepath.IsAbs(abs) { - t.Fatalf("resolved path should be absolute: %s", abs) + want := filepath.Join(home, "memories", "mission", "m", "persistent") + if abs != want { + t.Fatalf("persistent path: want %s, got %s", want, abs) } - // Directory was created if info, err := os.Stat(abs); err != nil || !info.IsDir() { - t.Fatalf("mission folder not created at %s: %v", abs, err) + t.Fatalf("persistent directory not created at %s: %v", abs, err) } - // The mission name is NOT a valid folder key — prevents regression + // The mission name is NOT a valid slot key — prevents regression. if _, _, err := store.ResolvePath(m.Name, "."); err == nil { t.Fatal("expected error when resolving by mission name") } } -func TestBuildFolderStore_RunFolder_CreatesSidecar(t *testing.T) { - dir := t.TempDir() +func TestBuildFolderStore_EphemeralMemory_CreatesSidecar(t *testing.T) { + home := withTempHome(t) cleanup := 7 m := &config.Mission{ Name: "m", - RunFolder: &config.MissionRunFolder{ - Base: filepath.Join(dir, "runs"), + EphemeralMemory: &config.MissionMemory{ + Type: "ephemeral", Cleanup: &cleanup, }, } @@ -85,13 +98,13 @@ func TestBuildFolderStore_RunFolder_CreatesSidecar(t *testing.T) { t.Fatalf("ResolvePath: %v", err) } if !writable { - t.Fatal("run folder must be writable") + t.Fatal("ephemeral memory must be writable") } - if filepath.Base(abs) != "mid-abc" { - t.Fatalf("run folder should be keyed by missionID, got %s", abs) + want := filepath.Join(home, "memories", "mission", "m", "run", "mid-abc") + if abs != want { + t.Fatalf("ephemeral path: want %s, got %s", want, abs) } - // Sidecar written with CleanupDays preserved metaBytes, err := os.ReadFile(filepath.Join(abs, runMetadataFile)) if err != nil { t.Fatalf("sidecar not written: %v", err) @@ -111,12 +124,12 @@ func TestBuildFolderStore_RunFolder_CreatesSidecar(t *testing.T) { } } -func TestBuildFolderStore_RunFolder_SidecarPreservedOnResume(t *testing.T) { - dir := t.TempDir() +func TestBuildFolderStore_EphemeralMemory_SidecarPreservedOnResume(t *testing.T) { + home := withTempHome(t) m := &config.Mission{ Name: "m", - RunFolder: &config.MissionRunFolder{ - Base: filepath.Join(dir, "runs"), + EphemeralMemory: &config.MissionMemory{ + Type: "ephemeral", }, } @@ -124,12 +137,11 @@ func TestBuildFolderStore_RunFolder_SidecarPreservedOnResume(t *testing.T) { if _, err := buildFolderStore(m, nil, "mid-1"); err != nil { t.Fatalf("first build: %v", err) } - runDir := filepath.Join(dir, "runs", "mid-1") + runDir := filepath.Join(home, "memories", "mission", "m", "run", "mid-1") firstMetaBytes, _ := os.ReadFile(filepath.Join(runDir, runMetadataFile)) var first runMetadata _ = json.Unmarshal(firstMetaBytes, &first) - // Sleep a touch so a re-written timestamp would differ time.Sleep(10 * time.Millisecond) // Second build (same missionID = resume): sidecar must NOT be overwritten @@ -145,11 +157,12 @@ func TestBuildFolderStore_RunFolder_SidecarPreservedOnResume(t *testing.T) { } } -func TestBuildFolderStore_RunFolder_RequiresMissionID(t *testing.T) { +func TestBuildFolderStore_EphemeralMemory_RequiresMissionID(t *testing.T) { + withTempHome(t) m := &config.Mission{ Name: "m", - RunFolder: &config.MissionRunFolder{ - Base: t.TempDir(), + EphemeralMemory: &config.MissionMemory{ + Type: "ephemeral", }, } _, err := buildFolderStore(m, nil, "") @@ -158,31 +171,32 @@ func TestBuildFolderStore_RunFolder_RequiresMissionID(t *testing.T) { } } -func TestBuildFolderStore_RejectsReservedSharedFolderNames(t *testing.T) { +func TestBuildFolderStore_RejectsReservedSharedMemoryNames(t *testing.T) { + withTempHome(t) for _, reserved := range []string{"mission", "run"} { m := &config.Mission{ - Name: "m", - Folders: []string{reserved}, + Name: "m", + Memories: []string{reserved}, } - shared := []config.SharedFolder{ - {Name: reserved, Path: t.TempDir()}, + mems := []config.Memory{ + {Name: reserved}, } - _, err := buildFolderStore(m, shared, "mid-1") + _, err := buildFolderStore(m, mems, "mid-1") if err == nil { - t.Fatalf("expected error for reserved shared folder name %q", reserved) + t.Fatalf("expected error for reserved shared memory name %q", reserved) } } } -func TestBuildFolderStore_BothMissionAndRunFolder(t *testing.T) { - dir := t.TempDir() +func TestBuildFolderStore_BothPersistentAndEphemeral(t *testing.T) { + withTempHome(t) m := &config.Mission{ Name: "m", - Folder: &config.MissionFolder{ - Path: filepath.Join(dir, "persist"), + PersistentMemory: &config.MissionMemory{ + Type: "persistent", }, - RunFolder: &config.MissionRunFolder{ - Base: filepath.Join(dir, "runs"), + EphemeralMemory: &config.MissionMemory{ + Type: "ephemeral", }, } store, err := buildFolderStore(m, nil, "mid-1") @@ -190,18 +204,72 @@ func TestBuildFolderStore_BothMissionAndRunFolder(t *testing.T) { t.Fatalf("unexpected error: %v", err) } if _, _, err := store.ResolvePath(aitools.MissionFolderName, "."); err != nil { - t.Fatalf("mission folder should resolve: %v", err) + t.Fatalf("persistent should resolve: %v", err) } if _, _, err := store.ResolvePath(aitools.RunFolderName, "."); err != nil { - t.Fatalf("run folder should resolve: %v", err) + t.Fatalf("ephemeral should resolve: %v", err) + } +} + +func TestBuildFolderStore_SharedMemory(t *testing.T) { + home := withTempHome(t) + m := &config.Mission{ + Name: "m", + Memories: []string{"research"}, + } + mems := []config.Memory{ + {Name: "research", Description: "Research notes", Editable: true}, + } + + store, err := buildFolderStore(m, mems, "mid-1") + if err != nil { + t.Fatalf("build: %v", err) + } + + abs, writable, err := store.ResolvePath("research", ".") + if err != nil { + t.Fatalf("resolve: %v", err) + } + if !writable { + t.Fatal("editable shared memory must resolve as writable") + } + want := filepath.Join(home, "memories", "shared", "research") + if abs != want { + t.Fatalf("shared memory path: want %s, got %s", want, abs) + } + if info, err := os.Stat(abs); err != nil || !info.IsDir() { + t.Fatalf("shared memory directory not created: %v", err) + } +} + +func TestBuildFolderStore_SharedMemory_ReadOnly(t *testing.T) { + withTempHome(t) + m := &config.Mission{ + Name: "m", + Memories: []string{"reference"}, + } + mems := []config.Memory{ + {Name: "reference"}, // editable defaults to false + } + + store, err := buildFolderStore(m, mems, "mid-1") + if err != nil { + t.Fatalf("build: %v", err) + } + _, writable, err := store.ResolvePath("reference", ".") + if err != nil { + t.Fatalf("resolve: %v", err) + } + if writable { + t.Fatal("default shared memory must be read-only") } } func TestResolvePath_EmptyFolderNameRejected(t *testing.T) { - dir := t.TempDir() + withTempHome(t) m := &config.Mission{ - Name: "m", - Folder: &config.MissionFolder{Path: dir}, + Name: "m", + PersistentMemory: &config.MissionMemory{Type: "persistent"}, } store, err := buildFolderStore(m, nil, "mid-1") if err != nil { @@ -213,10 +281,10 @@ func TestResolvePath_EmptyFolderNameRejected(t *testing.T) { } func TestResolvePath_RejectsPathEscape(t *testing.T) { - dir := t.TempDir() + withTempHome(t) m := &config.Mission{ - Name: "m", - Folder: &config.MissionFolder{Path: dir}, + Name: "m", + PersistentMemory: &config.MissionMemory{Type: "persistent"}, } store, err := buildFolderStore(m, nil, "mid-1") if err != nil { @@ -228,10 +296,10 @@ func TestResolvePath_RejectsPathEscape(t *testing.T) { } func TestResolvePath_UnknownFolder(t *testing.T) { - dir := t.TempDir() + withTempHome(t) m := &config.Mission{ - Name: "m", - Folder: &config.MissionFolder{Path: dir}, + Name: "m", + PersistentMemory: &config.MissionMemory{Type: "persistent"}, } store, err := buildFolderStore(m, nil, "mid-1") if err != nil { @@ -242,19 +310,20 @@ func TestResolvePath_UnknownFolder(t *testing.T) { } } -// --- SweepExpiredRunFolders ------------------------------------------------ +// --- SweepExpiredEphemeralMemories ---------------------------------------- -// writeRun creates a run folder with a sidecar recording the given created_at -// and cleanup_days. Useful for driving the sweep. -func writeRun(t *testing.T, base, name string, createdAt time.Time, cleanupDays int) string { +// writeEphemeral builds a fake per-run ephemeral memory directory under +// /memories/mission//run/, with a sidecar recording +// the given created_at and cleanup_days. +func writeEphemeral(t *testing.T, home, missionName, runID string, createdAt time.Time, cleanupDays int) string { t.Helper() - dir := filepath.Join(base, name) + dir := filepath.Join(home, "memories", "mission", missionName, "run", runID) if err := os.MkdirAll(dir, 0755); err != nil { t.Fatal(err) } meta := runMetadata{ - Mission: "m", - MissionID: name, + Mission: missionName, + MissionID: runID, CreatedAt: createdAt, CleanupDays: cleanupDays, } @@ -265,11 +334,11 @@ func writeRun(t *testing.T, base, name string, createdAt time.Time, cleanupDays return dir } -func TestSweepExpiredRunFolders_DeletesExpired(t *testing.T) { - base := t.TempDir() - expired := writeRun(t, base, "old", time.Now().Add(-8*24*time.Hour), 7) +func TestSweep_DeletesExpired(t *testing.T) { + home := withTempHome(t) + expired := writeEphemeral(t, home, "m", "old", time.Now().Add(-8*24*time.Hour), 7) - removed, err := SweepExpiredRunFolders(base) + removed, err := SweepExpiredEphemeralMemories() if err != nil { t.Fatalf("sweep: %v", err) } @@ -277,15 +346,15 @@ func TestSweepExpiredRunFolders_DeletesExpired(t *testing.T) { t.Fatalf("expected 1 removal, got %v", removed) } if _, err := os.Stat(expired); !os.IsNotExist(err) { - t.Fatalf("expired folder should be gone: %v", err) + t.Fatalf("expired directory should be gone: %v", err) } } -func TestSweepExpiredRunFolders_KeepsUnexpired(t *testing.T) { - base := t.TempDir() - fresh := writeRun(t, base, "new", time.Now().Add(-2*24*time.Hour), 7) +func TestSweep_KeepsUnexpired(t *testing.T) { + home := withTempHome(t) + fresh := writeEphemeral(t, home, "m", "new", time.Now().Add(-2*24*time.Hour), 7) - removed, err := SweepExpiredRunFolders(base) + removed, err := SweepExpiredEphemeralMemories() if err != nil { t.Fatalf("sweep: %v", err) } @@ -293,15 +362,15 @@ func TestSweepExpiredRunFolders_KeepsUnexpired(t *testing.T) { t.Fatalf("expected nothing removed, got %v", removed) } if _, err := os.Stat(fresh); err != nil { - t.Fatalf("fresh folder should still exist: %v", err) + t.Fatalf("fresh directory should still exist: %v", err) } } -func TestSweepExpiredRunFolders_IgnoresZeroCleanup(t *testing.T) { - base := t.TempDir() - keep := writeRun(t, base, "forever", time.Now().Add(-365*24*time.Hour), 0) +func TestSweep_IgnoresZeroCleanup(t *testing.T) { + home := withTempHome(t) + keep := writeEphemeral(t, home, "m", "forever", time.Now().Add(-365*24*time.Hour), 0) - removed, err := SweepExpiredRunFolders(base) + removed, err := SweepExpiredEphemeralMemories() if err != nil { t.Fatalf("sweep: %v", err) } @@ -309,62 +378,84 @@ func TestSweepExpiredRunFolders_IgnoresZeroCleanup(t *testing.T) { t.Fatalf("expected nothing removed when cleanup=0, got %v", removed) } if _, err := os.Stat(keep); err != nil { - t.Fatalf("folder with cleanup=0 should be preserved: %v", err) + t.Fatalf("directory with cleanup=0 should be preserved: %v", err) } } -func TestSweepExpiredRunFolders_IgnoresFoldersWithoutSidecar(t *testing.T) { - base := t.TempDir() - manual := filepath.Join(base, "hand_made") +func TestSweep_IgnoresDirectoriesWithoutSidecar(t *testing.T) { + home := withTempHome(t) + manual := filepath.Join(home, "memories", "mission", "m", "run", "hand_made") if err := os.MkdirAll(manual, 0755); err != nil { t.Fatal(err) } - removed, err := SweepExpiredRunFolders(base) + removed, err := SweepExpiredEphemeralMemories() if err != nil { t.Fatalf("sweep: %v", err) } if len(removed) != 0 { - t.Fatalf("sweep must leave un-marked folders alone, got %v", removed) + t.Fatalf("sweep must leave un-marked directories alone, got %v", removed) } if _, err := os.Stat(manual); err != nil { - t.Fatalf("manually created folder should still exist: %v", err) + t.Fatalf("manually created directory should still exist: %v", err) } } -func TestSweepExpiredRunFolders_MissingBaseIsNotAnError(t *testing.T) { - base := filepath.Join(t.TempDir(), "does", "not", "exist") - removed, err := SweepExpiredRunFolders(base) +func TestSweep_MissingRootIsNotAnError(t *testing.T) { + withTempHome(t) // home is set, but memories/ subtree doesn't exist yet + removed, err := SweepExpiredEphemeralMemories() if err != nil { - t.Fatalf("sweep should tolerate missing base: %v", err) + t.Fatalf("sweep should tolerate missing root: %v", err) } if removed != nil { t.Fatalf("expected nil removed, got %v", removed) } } -// TestSweepThenRebuildRoundTrip mirrors the real flow: an old run folder -// exists with a sidecar backdated past its cleanup window, the sweep deletes -// it, then a new buildFolderStore (different missionID) creates a fresh run -// folder with a current sidecar — and the old one is gone. -func TestSweepThenRebuildRoundTrip(t *testing.T) { - base := filepath.Join(t.TempDir(), "runs") +func TestSweep_WalksAcrossMissions(t *testing.T) { + home := withTempHome(t) + // Two missions, one expired run each + a := writeEphemeral(t, home, "alpha", "run1", time.Now().Add(-10*24*time.Hour), 2) + b := writeEphemeral(t, home, "beta", "run1", time.Now().Add(-10*24*time.Hour), 2) + keep := writeEphemeral(t, home, "alpha", "run2", time.Now().Add(-1*24*time.Hour), 2) + + removed, err := SweepExpiredEphemeralMemories() + if err != nil { + t.Fatalf("sweep: %v", err) + } + if len(removed) != 2 { + t.Fatalf("expected 2 removals across missions, got %v", removed) + } + for _, gone := range []string{a, b} { + if _, err := os.Stat(gone); !os.IsNotExist(err) { + t.Fatalf("expected %s gone: %v", gone, err) + } + } + if _, err := os.Stat(keep); err != nil { + t.Fatalf("unexpired directory should remain: %v", err) + } +} - // Simulate a run from days ago that's past its cleanup deadline. - stale := writeRun(t, base, "old-run", time.Now().Add(-5*24*time.Hour), 2) +// TestSweepThenRebuildRoundTrip mirrors the real flow: an old ephemeral +// memory exists with a sidecar backdated past its cleanup window, the sweep +// deletes it, then a new buildFolderStore (different missionID) creates a +// fresh ephemeral memory with a current sidecar — and the old one is gone. +func TestSweepThenRebuildRoundTrip(t *testing.T) { + home := withTempHome(t) + stale := writeEphemeral(t, home, "demo", "old-run", time.Now().Add(-5*24*time.Hour), 2) - if _, err := SweepExpiredRunFolders(base); err != nil { + if _, err := SweepExpiredEphemeralMemories(); err != nil { t.Fatalf("sweep: %v", err) } if _, err := os.Stat(stale); !os.IsNotExist(err) { - t.Fatalf("stale run should have been deleted: %v", err) + t.Fatalf("stale ephemeral should have been deleted: %v", err) } cleanup := 2 m := &config.Mission{ - Name: "folders_demo", - RunFolder: &config.MissionRunFolder{ - Base: base, + Name: "demo", + EphemeralMemory: &config.MissionMemory{ + Type: "ephemeral", Cleanup: &cleanup, }, } @@ -376,8 +467,9 @@ func TestSweepThenRebuildRoundTrip(t *testing.T) { if err != nil { t.Fatalf("resolve: %v", err) } - if filepath.Base(fresh) != "new-run" { - t.Fatalf("fresh run should be keyed by new missionID, got %s", fresh) + want := filepath.Join(home, "memories", "mission", "demo", "run", "new-run") + if fresh != want { + t.Fatalf("fresh path: want %s, got %s", want, fresh) } metaBytes, err := os.ReadFile(filepath.Join(fresh, runMetadataFile)) @@ -393,35 +485,6 @@ func TestSweepThenRebuildRoundTrip(t *testing.T) { } } -func TestResolvedRunFolderBase(t *testing.T) { - if got := ResolvedRunFolderBase(nil); got != DefaultRunFolderBase { - t.Fatalf("nil rf: want %q, got %q", DefaultRunFolderBase, got) - } - if got := ResolvedRunFolderBase(&config.MissionRunFolder{}); got != DefaultRunFolderBase { - t.Fatalf("empty base: want %q, got %q", DefaultRunFolderBase, got) - } - if got := ResolvedRunFolderBase(&config.MissionRunFolder{Base: "/custom"}); got != "/custom" { - t.Fatalf("explicit base: want %q, got %q", "/custom", got) - } -} - -func TestBuildFolderStore_CreatesNestedBase(t *testing.T) { - // Base directory with nested missing parents — MkdirAll should create them all. - base := filepath.Join(t.TempDir(), "a", "b", "c", "runs") - m := &config.Mission{ - Name: "m", - RunFolder: &config.MissionRunFolder{ - Base: base, - }, - } - if _, err := buildFolderStore(m, nil, "mid-1"); err != nil { - t.Fatalf("build: %v", err) - } - if info, err := os.Stat(filepath.Join(base, "mid-1")); err != nil || !info.IsDir() { - t.Fatalf("nested run folder not created: %v", err) - } -} - func TestWriteRunMetadata_PreservesOnReentry(t *testing.T) { // Direct test of the O_CREATE|O_EXCL sidecar write: calling twice with // different cleanupDays must not overwrite the original. diff --git a/mission/runner.go b/mission/runner.go index 3b820e1..711d41b 100644 --- a/mission/runner.go +++ b/mission/runner.go @@ -288,9 +288,10 @@ func NewRunner(cfg *config.Config, configPath string, missionName string, inputs r.secretInfos = secretInfos } - // Folder store is built later in Run() once missionID is known — the per-run - // folder path depends on it. Shared + persistent folders only would let us - // build here, but deferring is simpler than branching on mission.RunFolder. + // Folder store is built later in Run() once missionID is known — the + // ephemeral memory path depends on it. Shared + persistent memory alone + // would let us build here, but deferring is simpler than branching on + // mission.EphemeralMemory. return r, nil } @@ -567,14 +568,14 @@ func (r *Runner) Run(ctx context.Context, streamer streamers.MissionHandler) err r.resolvedDatasets = nil } - // Folder store depends on missionID (for run_folder path), so build it - // here rather than in NewRunner. Sweep expired run folders async — the - // result doesn't affect this run's correctness, only disk usage. - if r.mission.RunFolder != nil { - base := ResolvedRunFolderBase(r.mission.RunFolder) - go func() { _, _ = SweepExpiredRunFolders(base) }() + // Folder store depends on missionID (for ephemeral memory path), so build + // it here rather than in NewRunner. Sweep expired ephemeral memories + // async — the result doesn't affect this run's correctness, only disk + // usage. + if r.mission.EphemeralMemory != nil { + go func() { _, _ = SweepExpiredEphemeralMemories() }() } - folderStore, err := buildFolderStore(r.mission, r.cfg.SharedFolders, missionID) + folderStore, err := buildFolderStore(r.mission, r.cfg.Memories, missionID) if err != nil { return fmt.Errorf("mission '%s': build folder store: %w", r.mission.Name, err) } diff --git a/wsbridge/convert.go b/wsbridge/convert.go index 4974798..8f2f81c 100644 --- a/wsbridge/convert.go +++ b/wsbridge/convert.go @@ -252,43 +252,10 @@ func ConfigToInstanceConfig(cfg *config.Config) protocol.InstanceConfig { }) } - // Build shared folder → missions map - sharedMissions := map[string][]string{} - for _, m := range cfg.Missions { - for _, folderName := range m.Folders { - sharedMissions[folderName] = append(sharedMissions[folderName], m.Name) - } - } - - for _, fb := range cfg.SharedFolders { - label := fb.Label - if label == "" { - label = fb.Name - } - ic.SharedFolders = append(ic.SharedFolders, protocol.SharedFolderInfo{ - Name: fb.Name, - Path: fb.Path, - Label: label, - Description: fb.Description, - Editable: fb.Editable, - IsShared: true, - Missions: sharedMissions[fb.Name], - }) - } - - // Add dedicated mission folders - for _, m := range cfg.Missions { - if m.Folder != nil { - ic.SharedFolders = append(ic.SharedFolders, protocol.SharedFolderInfo{ - Name: m.Name, - Path: m.Folder.Path, - Label: m.Name, - Description: m.Folder.Description, - Editable: true, - IsShared: false, - Missions: []string{m.Name}, - }) - } + // Collect shared memories + per-mission persistent memories. Paths are + // derived from SquadronHome so we don't need user-supplied paths. + if mems, err := collectMemoryInfos(cfg); err == nil { + ic.SharedFolders = append(ic.SharedFolders, mems...) } return ic diff --git a/wsbridge/shared_folder.go b/wsbridge/shared_folder.go index c585d65..4c28d73 100644 --- a/wsbridge/shared_folder.go +++ b/wsbridge/shared_folder.go @@ -13,42 +13,61 @@ import ( "github.com/mlund01/squadron-wire/protocol" "squadron/config" + "squadron/mission" ) -// resolveSharedFolderPath looks up a folder by name (shared folders + mission folders) +// resolvedMemory describes one materialized memory slot for the UI: a +// human-friendly name, the absolute path it lives at, and whether agents are +// allowed to write to it. +type resolvedMemory struct { + name string + path string + editable bool +} + +// resolveSharedFolderPath looks up a memory slot by name (top-level shared +// memories first, then per-mission persistent memories keyed by mission name) // and safely resolves a relative path within it. -func (c *Client) resolveSharedFolderPath(folderName, relPath string) (*config.SharedFolder, string, error) { +func (c *Client) resolveSharedFolderPath(folderName, relPath string) (*resolvedMemory, string, error) { cfg := c.getConfig() - // Check shared folders first - for i := range cfg.SharedFolders { - if cfg.SharedFolders[i].Name == folderName { - folder := &cfg.SharedFolders[i] - path, err := c.resolveSafePath(folder.Path, relPath) - return folder, path, err + // Check shared memories first. + for i := range cfg.Memories { + if cfg.Memories[i].Name == folderName { + absPath, err := mission.SharedMemoryPath(folderName) + if err != nil { + return nil, "", fmt.Errorf("resolve shared memory %q: %w", folderName, err) + } + rm := &resolvedMemory{ + name: cfg.Memories[i].Name, + path: absPath, + editable: cfg.Memories[i].Editable, + } + path, err := c.resolveSafePath(absPath, relPath) + return rm, path, err } } - // Check mission dedicated folders (keyed by mission name) + // Check mission persistent memory (keyed by mission name). for _, m := range cfg.Missions { - if m.Folder != nil && m.Name == folderName { - sf := &config.SharedFolder{ - Name: m.Name, - Path: m.Folder.Path, - Editable: true, + if m.PersistentMemory != nil && m.Name == folderName { + absPath, err := mission.PersistentMemoryPath(m.Name) + if err != nil { + return nil, "", fmt.Errorf("resolve persistent memory for %q: %w", m.Name, err) } - path, err := c.resolveSafePath(m.Folder.Path, relPath) - return sf, path, err + rm := &resolvedMemory{name: m.Name, path: absPath, editable: true} + path, err := c.resolveSafePath(absPath, relPath) + return rm, path, err } } - return nil, "", fmt.Errorf("folder %q not found", folderName) + return nil, "", fmt.Errorf("memory %q not found", folderName) } func (c *Client) resolveSafePath(basePath, relPath string) (string, error) { rootPath, err := filepath.Abs(basePath) if err != nil { - return "", fmt.Errorf("invalid folder path: %w", err) + return "", fmt.Errorf("invalid memory path: %w", err) } if relPath == "" { @@ -62,7 +81,7 @@ func (c *Client) resolveSafePath(basePath, relPath string) (string, error) { fullPath := filepath.Join(rootPath, cleaned) if !strings.HasPrefix(fullPath, rootPath) { - return "", fmt.Errorf("path escapes folder root") + return "", fmt.Errorf("path escapes memory root") } return fullPath, nil @@ -71,50 +90,70 @@ func (c *Client) resolveSafePath(basePath, relPath string) (string, error) { func (c *Client) handleListSharedFolders(env *protocol.Envelope) (*protocol.Envelope, error) { cfg := c.getConfig() - // Build shared folder → missions map + folders, err := collectMemoryInfos(cfg) + if err != nil { + return nil, err + } + + return protocol.NewResponse(env.RequestID, protocol.TypeListSharedFoldersResult, + &protocol.ListSharedFoldersResultPayload{Folders: folders}) +} + +// collectMemoryInfos walks the config and turns every memory slot +// (shared + per-mission persistent) into a protocol.SharedFolderInfo. Used +// by both the standalone list_shared_folders RPC and the bulk +// instance-info payload in convert.go. +func collectMemoryInfos(cfg *config.Config) ([]protocol.SharedFolderInfo, error) { + // Build shared memory → missions map sharedMissions := map[string][]string{} for _, m := range cfg.Missions { - for _, folderName := range m.Folders { - sharedMissions[folderName] = append(sharedMissions[folderName], m.Name) + for _, name := range m.Memories { + sharedMissions[name] = append(sharedMissions[name], m.Name) } } var folders []protocol.SharedFolderInfo - // Add shared folders - for _, fb := range cfg.SharedFolders { - label := fb.Label + for _, mem := range cfg.Memories { + label := mem.Label if label == "" { - label = fb.Name + label = mem.Name + } + path, err := mission.SharedMemoryPath(mem.Name) + if err != nil { + return nil, fmt.Errorf("shared memory %q: %w", mem.Name, err) } folders = append(folders, protocol.SharedFolderInfo{ - Name: fb.Name, - Path: fb.Path, + Name: mem.Name, + Path: path, Label: label, - Description: fb.Description, - Editable: fb.Editable, + Description: mem.Description, + Editable: mem.Editable, IsShared: true, - Missions: sharedMissions[fb.Name], + Missions: sharedMissions[mem.Name], }) } - // Add dedicated mission folders for _, m := range cfg.Missions { - if m.Folder != nil { - folders = append(folders, protocol.SharedFolderInfo{ - Name: m.Name, - Path: m.Folder.Path, - Label: m.Name, - Description: m.Folder.Description, - Editable: true, - IsShared: false, - Missions: []string{m.Name}, - }) + if m.PersistentMemory == nil { + continue } + path, err := mission.PersistentMemoryPath(m.Name) + if err != nil { + return nil, fmt.Errorf("persistent memory for %q: %w", m.Name, err) + } + folders = append(folders, protocol.SharedFolderInfo{ + Name: m.Name, + Path: path, + Label: m.Name, + Description: m.PersistentMemory.Description, + Editable: true, + IsShared: false, + Missions: []string{m.Name}, + }) } - return protocol.NewResponse(env.RequestID, protocol.TypeListSharedFoldersResult, - &protocol.ListSharedFoldersResultPayload{Folders: folders}) + return folders, nil } func (c *Client) handleBrowseDirectory(env *protocol.Envelope) (*protocol.Envelope, error) { @@ -220,15 +259,15 @@ func (c *Client) handleWriteBrowseFile(env *protocol.Envelope) (*protocol.Envelo return nil, fmt.Errorf("decode write_browse_file: %w", err) } - folder, fullPath, err := c.resolveSharedFolderPath(payload.BrowserName, payload.RelPath) + mem, fullPath, err := c.resolveSharedFolderPath(payload.BrowserName, payload.RelPath) if err != nil { return protocol.NewResponse(env.RequestID, protocol.TypeWriteBrowseFileResult, &protocol.WriteBrowseFileResultPayload{Success: false, Error: err.Error()}) } - if !folder.Editable { + if !mem.editable { return protocol.NewResponse(env.RequestID, protocol.TypeWriteBrowseFileResult, - &protocol.WriteBrowseFileResultPayload{Success: false, Error: "shared folder is read-only"}) + &protocol.WriteBrowseFileResultPayload{Success: false, Error: "memory is read-only"}) } if err := os.WriteFile(fullPath, []byte(payload.Content), 0644); err != nil { From 329c9a86aba26a5089f7f9b19de179a9a78ea635 Mon Sep 17 00:00:00 2001 From: Max Lund Date: Mon, 25 May 2026 21:53:43 -0500 Subject: [PATCH 3/8] =?UTF-8?q?Rename=20folder=E2=86=92memory=20across=20t?= =?UTF-8?q?ools,=20internals,=20and=20docs?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit User-visible side: - The file_* tools (file_list, file_read, file_create, file_delete, file_search, file_grep) now take a `memory` parameter instead of `folder`. Tool schemas + descriptions agents see say "memory slot" everywhere — no more "folder" leaking into the LLM-facing surface. - Agent system-prompt section is now "Available Memory" (was "Available Folders") with the same `memory` parameter wording. - The no-code-multi-agent-workflow guide is updated to the new DSL (it was still showing `folder { path = "./briefs" }`, which would now fail to parse). - All in-app docs that referenced "folder" in user-visible copy now say "memory". Go internals renamed for coherence with the DSL: - aitools.FolderStore → aitools.MemoryStore - aitools.FolderInfo → aitools.MemoryInfo - aitools.Folder{List,Read,Create,Delete,Search,Grep}Tool → aitools.Memory{List,Read,Create,Delete,Search,Grep}Tool - aitools.MissionFolderName → aitools.PersistentMemoryName - aitools.RunFolderName → aitools.EphemeralMemoryName - mission.buildFolderStore → mission.buildMemoryStore - mission.missionFolderStore → mission.missionMemoryStore - agent.AgentOptions.FolderStore → MemoryStore (similarly on CommanderOptions, AgentManagerConfig, internal fields) - prompts.FormatFolderContext → prompts.FormatMemoryContext - cmd.runFolderCleanupLoop → cmd.runMemoryCleanupLoop Files renamed: - aitools/folder_tools.go → aitools/memory_tools.go - mission/folder_store.go → mission/memory_store.go - mission/folder_store_test.go → mission/memory_store_test.go - wsbridge/shared_folder.go → wsbridge/memory_browser.go - config/folder_test.go → config/memory_test.go Intentionally NOT renamed: - protocol.SharedFolderInfo / protocol.TypeListSharedFolders and the matching wsbridge handler — these are wire-protocol names owned by the external squadron-wire repo. Renaming would require a coordinated cross-repo change. - internal/paths.ResolveFolderPath — generic path utility used outside the memory subsystem. - Generic English uses of "folder" in CLI help text describing OS directories (e.g. "Dump documentation to a local folder"). - The docs URL /missions/folders stays for link stability; the page itself renders as "Memory" in the nav. All tests pass. --- CLAUDE.md | 8 +- agent/agent.go | 28 +- agent/agent_manager.go | 8 +- agent/commander.go | 30 +-- agent/internal/prompts/prompts.go | 20 +- aitools/{folder_tools.go => memory_tools.go} | 244 +++++++++--------- cmd/engage.go | 10 +- config/memory.go | 2 +- config/{folder_test.go => memory_test.go} | 0 docs/content/declarative-agent-framework.mdx | 2 +- .../guides/no-code-multi-agent-workflow.mdx | 6 +- docs/content/missions/folders.mdx | 8 +- mission/{folder_store.go => memory_store.go} | 54 ++-- ...der_store_test.go => memory_store_test.go} | 66 ++--- mission/runner.go | 28 +- .../{shared_folder.go => memory_browser.go} | 24 +- 16 files changed, 272 insertions(+), 266 deletions(-) rename aitools/{folder_tools.go => memory_tools.go} (70%) rename config/{folder_test.go => memory_test.go} (100%) rename mission/{folder_store.go => memory_store.go} (85%) rename mission/{folder_store_test.go => memory_store_test.go} (86%) rename wsbridge/{shared_folder.go => memory_browser.go} (91%) diff --git a/CLAUDE.md b/CLAUDE.md index 418b117..6ee5c59 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -286,15 +286,15 @@ The scheduler lives in `scheduler/` but its lifecycle (creation, config updates, Memory blocks are sandboxed filesystem locations that agents access via the `file_list`, `file_read`, `file_create`, `file_delete`, `file_search`, and -`file_grep` tools. The `folder` parameter (the literal historical name on the -tools) is required on every tool call — there is no implicit default. +`file_grep` tools. Each call takes a required `memory` parameter naming the +slot — there is no implicit default. Squadron owns the paths: every slot lives under `/memories/` and you do **not** specify `path` anywhere. The three kinds map to three fixed location patterns: -| Kind | HCL | Slot name agents use | On-disk path | -|------|-----|-----------------------|--------------| +| Kind | HCL | `memory` arg agents pass | On-disk path | +|------|-----|---------------------------|--------------| | Shared | top-level `memory "name" { ... }` | the HCL label | `/memories/shared//` | | Persistent (mission) | `memory { type = "persistent" }` inside a mission (also the default if `type` is omitted) | literal `"mission"` | `/memories/mission//persistent/` | | Ephemeral (per-run) | `memory { type = "ephemeral" }` inside a mission | literal `"run"` | `/memories/mission//run//` | diff --git a/agent/agent.go b/agent/agent.go index 47b0d0f..cf1bcd4 100644 --- a/agent/agent.go +++ b/agent/agent.go @@ -77,8 +77,8 @@ type Options struct { SecretInfos []SecretInfo // SecretValues contains actual secret values for tool call injection SecretValues map[string]string - // FolderStore provides file folder access for the mission (optional) - FolderStore aitools.FolderStore + // MemoryStore provides file memory access for the mission (optional) + MemoryStore aitools.MemoryStore // OnCompaction is called when context compaction occurs (optional, mission context only) OnCompaction func(inputTokens int, tokenLimit int, messagesCompacted int, turnRetention int) // OnSessionTurn is called after each LLM turn with telemetry data (optional) @@ -178,14 +178,14 @@ func New(ctx context.Context, opts Options) (*Agent, error) { } } - // Add folder tools if FolderStore is available - if opts.FolderStore != nil { - tools["file_list"] = &aitools.FolderListTool{Store: opts.FolderStore} - tools["file_read"] = &aitools.FolderReadTool{Store: opts.FolderStore} - tools["file_create"] = &aitools.FolderCreateTool{Store: opts.FolderStore} - tools["file_delete"] = &aitools.FolderDeleteTool{Store: opts.FolderStore} - tools["file_search"] = &aitools.FolderSearchTool{Store: opts.FolderStore} - tools["file_grep"] = &aitools.FolderGrepTool{Store: opts.FolderStore} + // Add memory tools if MemoryStore is available + if opts.MemoryStore != nil { + tools["file_list"] = &aitools.MemoryListTool{Store: opts.MemoryStore} + tools["file_read"] = &aitools.MemoryReadTool{Store: opts.MemoryStore} + tools["file_create"] = &aitools.MemoryCreateTool{Store: opts.MemoryStore} + tools["file_delete"] = &aitools.MemoryDeleteTool{Store: opts.MemoryStore} + tools["file_search"] = &aitools.MemorySearchTool{Store: opts.MemoryStore} + tools["file_grep"] = &aitools.MemoryGrepTool{Store: opts.MemoryStore} } // Resolve skills and add load_skill tool @@ -243,10 +243,10 @@ func New(ctx context.Context, opts Options) (*Agent, error) { } } - // Add folder context if FolderStore is available - if opts.FolderStore != nil { - if folderPrompt := prompts.FormatFolderContext(opts.FolderStore); folderPrompt != "" { - systemPrompts = append(systemPrompts, folderPrompt) + // Add memory context if MemoryStore is available + if opts.MemoryStore != nil { + if memoryPrompt := prompts.FormatMemoryContext(opts.MemoryStore); memoryPrompt != "" { + systemPrompts = append(systemPrompts, memoryPrompt) } } diff --git a/agent/agent_manager.go b/agent/agent_manager.go index 42bdac4..aeb4a70 100644 --- a/agent/agent_manager.go +++ b/agent/agent_manager.go @@ -34,7 +34,7 @@ type AgentManager struct { cfg *config.Config secretInfos []SecretInfo secretValues map[string]string - folderStore aitools.FolderStore + memoryStore aitools.MemoryStore sessionLogger SessionLogger taskID string missionID string @@ -55,7 +55,7 @@ type AgentManagerConfig struct { Config *config.Config SecretInfos []SecretInfo SecretValues map[string]string - FolderStore aitools.FolderStore + MemoryStore aitools.MemoryStore SessionLogger SessionLogger TaskID string MissionID string @@ -83,7 +83,7 @@ func NewAgentManager(cfg AgentManagerConfig) *AgentManager { cfg: cfg.Config, secretInfos: cfg.SecretInfos, secretValues: cfg.SecretValues, - folderStore: cfg.FolderStore, + memoryStore: cfg.MemoryStore, sessionLogger: cfg.SessionLogger, taskID: cfg.TaskID, missionID: cfg.MissionID, @@ -278,7 +278,7 @@ func (m *AgentManager) createAgent(ctx context.Context, agentCfg *config.Agent) DatasetStore: datasetStore, SecretInfos: m.secretInfos, SecretValues: m.secretValues, - FolderStore: m.folderStore, + MemoryStore: m.memoryStore, OnCompaction: onCompaction, OnSessionTurn: onSessionTurn, PricingOverrides: m.pricingOverrides, diff --git a/agent/commander.go b/agent/commander.go index 4b00b37..b3f362c 100644 --- a/agent/commander.go +++ b/agent/commander.go @@ -75,8 +75,8 @@ type CommanderOptions struct { // SequentialDataset contains all items for sequential iteration processing // When set, the commander handles all items in a single session using dataset_next/submit_output tools SequentialDataset []cty.Value - // FolderStore provides file folder access for the mission (optional) - FolderStore aitools.FolderStore + // MemoryStore provides file memory access for the mission (optional) + MemoryStore aitools.MemoryStore // Compaction settings for the commander session (nil if disabled) Compaction *CompactionConfig // PruneOn triggers pruning when conversation reaches this many turns (0 = disabled) @@ -342,7 +342,7 @@ type Commander struct { agentMgr *AgentManager // Manages agent lifecycle (creation, session, resume) pricingOverrides map[string]*llm.ModelPricing subtasksSet bool // Whether set_subtasks has been called - folderStore aitools.FolderStore // Folder access for missions (nil if not configured) + memoryStore aitools.MemoryStore // Memory access for missions (nil if not configured) compaction *CompactionConfig // Compaction settings (nil if disabled) pruneOn int // Trigger pruning at this many turns (0 = disabled) pruneTo int // Prune down to this many turns @@ -482,17 +482,17 @@ func NewCommander(ctx context.Context, opts CommanderOptions) (*Commander, error sup.tools["result_keys"] = &aitools.ResultKeysTool{Store: resultStore} sup.tools["result_chunk"] = &aitools.ResultChunkTool{Store: resultStore} - // Add folder tools if FolderStore is available - if opts.FolderStore != nil { - sup.folderStore = opts.FolderStore - sup.tools["file_list"] = &aitools.FolderListTool{Store: opts.FolderStore} - sup.tools["file_read"] = &aitools.FolderReadTool{Store: opts.FolderStore} - sup.tools["file_create"] = &aitools.FolderCreateTool{Store: opts.FolderStore} - sup.tools["file_delete"] = &aitools.FolderDeleteTool{Store: opts.FolderStore} - sup.tools["file_search"] = &aitools.FolderSearchTool{Store: opts.FolderStore} - sup.tools["file_grep"] = &aitools.FolderGrepTool{Store: opts.FolderStore} - if folderPrompt := prompts.FormatFolderContext(opts.FolderStore); folderPrompt != "" { - session.AddSystemPrompt(folderPrompt) + // Add memory tools if MemoryStore is available + if opts.MemoryStore != nil { + sup.memoryStore = opts.MemoryStore + sup.tools["file_list"] = &aitools.MemoryListTool{Store: opts.MemoryStore} + sup.tools["file_read"] = &aitools.MemoryReadTool{Store: opts.MemoryStore} + sup.tools["file_create"] = &aitools.MemoryCreateTool{Store: opts.MemoryStore} + sup.tools["file_delete"] = &aitools.MemoryDeleteTool{Store: opts.MemoryStore} + sup.tools["file_search"] = &aitools.MemorySearchTool{Store: opts.MemoryStore} + sup.tools["file_grep"] = &aitools.MemoryGrepTool{Store: opts.MemoryStore} + if memoryPrompt := prompts.FormatMemoryContext(opts.MemoryStore); memoryPrompt != "" { + session.AddSystemPrompt(memoryPrompt) } } @@ -725,7 +725,7 @@ func (s *Commander) SetToolCallbacks(callbacks *CommanderToolCallbacks, depSumma Config: s.cfg, SecretInfos: s.secretInfos, SecretValues: s.secretValues, - FolderStore: s.folderStore, + MemoryStore: s.memoryStore, SessionLogger: s.sessionLogger, TaskID: s.callbacksTaskID, MissionID: s.callbacksMissionID, diff --git a/agent/internal/prompts/prompts.go b/agent/internal/prompts/prompts.go index eb732f5..ae81e4a 100644 --- a/agent/internal/prompts/prompts.go +++ b/agent/internal/prompts/prompts.go @@ -231,20 +231,20 @@ func formatAgents(agents []AgentInfo) string { return sb.String() } -// FormatFolderContext builds a system prompt section describing available folders. -func FormatFolderContext(store aitools.FolderStore) string { +// FormatMemoryContext builds a system prompt section describing available memory slots. +func FormatMemoryContext(store aitools.MemoryStore) string { if store == nil { return "" } - infos := store.FolderInfos() + infos := store.MemoryInfos() if len(infos) == 0 { return "" } var sb strings.Builder - sb.WriteString("## Available Folders\n\n") - sb.WriteString("You have access to file folders via the file_list, file_read, file_create, file_delete, file_search, and file_grep tools.\n") - sb.WriteString("The `folder` parameter is required on every call — pick one of the names below.\n\n") + sb.WriteString("## Available Memory\n\n") + sb.WriteString("You have access to memory slots via the file_list, file_read, file_create, file_delete, file_search, and file_grep tools.\n") + sb.WriteString("The `memory` parameter is required on every call — pick one of the slot names below.\n\n") for _, info := range infos { access := "read-only" @@ -253,10 +253,10 @@ func FormatFolderContext(store aitools.FolderStore) string { } label := "" switch info.Name { - case aitools.MissionFolderName: - label = " (persistent mission folder — survives across runs)" - case aitools.RunFolderName: - label = " (per-run folder — fresh directory for this mission run)" + case aitools.PersistentMemoryName: + label = " (persistent mission memory — survives across runs)" + case aitools.EphemeralMemoryName: + label = " (ephemeral per-run memory — fresh for this mission run)" } desc := "" if info.Description != "" { diff --git a/aitools/folder_tools.go b/aitools/memory_tools.go similarity index 70% rename from aitools/folder_tools.go rename to aitools/memory_tools.go index 3b1c7a6..4779525 100644 --- a/aitools/folder_tools.go +++ b/aitools/memory_tools.go @@ -1,8 +1,8 @@ package aitools import ( - "context" "bufio" + "context" "encoding/json" "fmt" "io" @@ -12,31 +12,32 @@ import ( "strings" ) -// Reserved folder names for mission-scoped folders. Callers addressing the -// persistent mission folder and the per-run folder must use these names. +// Reserved slot names for mission-scoped memory. Callers addressing the +// persistent mission memory and the per-run ephemeral memory must use these +// names as the `memory` parameter. const ( - MissionFolderName = "mission" - RunFolderName = "run" + PersistentMemoryName = "mission" + EphemeralMemoryName = "run" ) -// FolderStore provides folder access for missions. -// Implementations resolve folder names to paths and enforce security. -type FolderStore interface { - // ResolvePath resolves a folder name + relative path to an absolute path. - // Returns: absolute path, writable flag, error. - ResolvePath(folderName string, relPath string) (string, bool, error) - // FolderInfos returns info about all available folders. - FolderInfos() []FolderInfo +// MemoryStore provides memory access for missions. Implementations resolve a +// memory slot name to an absolute path and enforce read/write semantics. +type MemoryStore interface { + // ResolvePath resolves a memory slot name + relative path to an absolute + // path. Returns: absolute path, writable flag, error. + ResolvePath(memoryName string, relPath string) (string, bool, error) + // MemoryInfos returns info about all available memory slots. + MemoryInfos() []MemoryInfo } -// FolderInfo describes an available folder. -type FolderInfo struct { +// MemoryInfo describes an available memory slot. +type MemoryInfo struct { Name string Description string Writable bool } -// validateRelPath ensures a path is relative and doesn't escape the folder root. +// validateRelPath ensures a path is relative and doesn't escape the memory root. func validateRelPath(relPath string) error { if relPath == "" { return fmt.Errorf("path is required") @@ -44,44 +45,49 @@ func validateRelPath(relPath string) error { cleaned := filepath.Clean(relPath) if filepath.IsAbs(cleaned) || strings.HasPrefix(cleaned, "..") || strings.Contains(cleaned, string(filepath.Separator)+"..") { - return fmt.Errorf("invalid path: must be relative and within folder") + return fmt.Errorf("invalid path: must be relative and within the memory root") } return nil } -// resolveFolderPath is a helper that resolves the folder and validates the path. -func resolveFolderPath(store FolderStore, folderName, relPath string) (string, bool, error) { +// resolveMemoryPath is a helper that resolves the memory slot and validates +// the relative path. +func resolveMemoryPath(store MemoryStore, name, relPath string) (string, bool, error) { if err := validateRelPath(relPath); err != nil { return "", false, err } - return store.ResolvePath(folderName, relPath) + return store.ResolvePath(name, relPath) } +// memoryParamDescription is reused across every file tool's `memory` +// parameter so the agent sees a consistent description. +const memoryParamDescription = "Memory slot to operate in. Use \"mission\" for the persistent mission memory, \"run\" for the per-run ephemeral memory, or a shared memory name." + // ============================================================================= -// folder_list — List files and directories +// file_list — List files and directories // ============================================================================= -type FolderListTool struct { - Store FolderStore +type MemoryListTool struct { + Store MemoryStore } -func (t *FolderListTool) ToolName() string { return "file_list" } +func (t *MemoryListTool) ToolName() string { return "file_list" } -func (t *FolderListTool) ToolDescription() string { - return "List files and directories in a folder. Returns names, types (file/dir), and sizes. Results are paginated (default 100 per page). Use 'offset' to get subsequent pages." +func (t *MemoryListTool) ToolDescription() string { + return "List files and directories in a memory slot. Returns names, types (file/dir), and sizes. Results are paginated (default 100 per page). Use 'offset' to get subsequent pages." } -func (t *FolderListTool) ToolPayloadSchema() Schema { +func (t *MemoryListTool) ToolPayloadSchema() Schema { return Schema{ Type: TypeObject, Properties: PropertyMap{ - "folder": { + "memory": { Type: TypeString, - Description: "Folder name. Use \"mission\" for the persistent mission folder, \"run\" for the per-run ephemeral folder, or a shared folder name.", + Description: memoryParamDescription, }, "path": { Type: TypeString, - Description: "Relative subdirectory path within the folder. Omit to list the root.", + Description: "Relative subdirectory path within the memory slot. Omit to list the root.", }, "recursive": { Type: TypeBoolean, @@ -96,12 +102,12 @@ func (t *FolderListTool) ToolPayloadSchema() Schema { Description: "Number of entries to skip (for pagination). Default 0.", }, }, - Required: []string{"folder"}, + Required: []string{"memory"}, } } -type folderListParams struct { - Folder string `json:"folder"` +type memoryListParams struct { + Memory string `json:"memory"` Path string `json:"path"` Recursive bool `json:"recursive"` Limit int `json:"limit"` @@ -110,8 +116,8 @@ type folderListParams struct { const defaultListLimit = 100 -func (t *FolderListTool) Call(ctx context.Context, params string) string { - var p folderListParams +func (t *MemoryListTool) Call(ctx context.Context, params string) string { + var p memoryListParams if err := json.Unmarshal([]byte(params), &p); err != nil { return "Error: invalid parameters - " + err.Error() } @@ -124,9 +130,9 @@ func (t *FolderListTool) Call(ctx context.Context, params string) string { var absPath string var err error if p.Path == "" { - absPath, _, err = t.Store.ResolvePath(p.Folder, ".") + absPath, _, err = t.Store.ResolvePath(p.Memory, ".") } else { - absPath, _, err = resolveFolderPath(t.Store, p.Folder, p.Path) + absPath, _, err = resolveMemoryPath(t.Store, p.Memory, p.Path) } if err != nil { return "Error: " + err.Error() @@ -248,30 +254,30 @@ func formatSize(bytes int64) string { } // ============================================================================= -// folder_read — Read file content +// file_read — Read file content // ============================================================================= -type FolderReadTool struct { - Store FolderStore +type MemoryReadTool struct { + Store MemoryStore } -func (t *FolderReadTool) ToolName() string { return "file_read" } +func (t *MemoryReadTool) ToolName() string { return "file_read" } -func (t *FolderReadTool) ToolDescription() string { - return "Read the contents of a file in a folder. Optionally limit to the first N lines or N bytes." +func (t *MemoryReadTool) ToolDescription() string { + return "Read the contents of a file in a memory slot. Optionally limit to the first N lines or N bytes." } -func (t *FolderReadTool) ToolPayloadSchema() Schema { +func (t *MemoryReadTool) ToolPayloadSchema() Schema { return Schema{ Type: TypeObject, Properties: PropertyMap{ - "folder": { + "memory": { Type: TypeString, - Description: "Folder name. Use \"mission\" for the persistent mission folder, \"run\" for the per-run ephemeral folder, or a shared folder name.", + Description: memoryParamDescription, }, "path": { Type: TypeString, - Description: "Relative file path within the folder.", + Description: "Relative file path within the memory slot.", }, "max_lines": { Type: TypeInteger, @@ -282,12 +288,12 @@ func (t *FolderReadTool) ToolPayloadSchema() Schema { Description: "Return only the first N bytes. 0 or omit for no limit.", }, }, - Required: []string{"folder", "path"}, + Required: []string{"memory", "path"}, } } -type folderReadParams struct { - Folder string `json:"folder"` +type memoryReadParams struct { + Memory string `json:"memory"` Path string `json:"path"` MaxLines int `json:"max_lines"` MaxBytes int `json:"max_bytes"` @@ -295,8 +301,8 @@ type folderReadParams struct { const maxReadSize = 10 * 1024 * 1024 // 10MB -func (t *FolderReadTool) Call(ctx context.Context, params string) string { - var p folderReadParams +func (t *MemoryReadTool) Call(ctx context.Context, params string) string { + var p memoryReadParams if err := json.Unmarshal([]byte(params), &p); err != nil { return "Error: invalid parameters - " + err.Error() } @@ -305,7 +311,7 @@ func (t *FolderReadTool) Call(ctx context.Context, params string) string { return "Error: path is required" } - absPath, _, err := resolveFolderPath(t.Store, p.Folder, p.Path) + absPath, _, err := resolveMemoryPath(t.Store, p.Memory, p.Path) if err != nil { return "Error: " + err.Error() } @@ -356,30 +362,30 @@ func (t *FolderReadTool) Call(ctx context.Context, params string) string { } // ============================================================================= -// folder_create — Create or write to a file +// file_create — Create or write to a file // ============================================================================= -type FolderCreateTool struct { - Store FolderStore +type MemoryCreateTool struct { + Store MemoryStore } -func (t *FolderCreateTool) ToolName() string { return "file_create" } +func (t *MemoryCreateTool) ToolName() string { return "file_create" } -func (t *FolderCreateTool) ToolDescription() string { - return "Create or write to a file in a folder. By default, creates a new file (fails if it already exists). Use 'overwrite' to replace an existing file, or 'append' to add to an existing file." +func (t *MemoryCreateTool) ToolDescription() string { + return "Create or write to a file in a memory slot. By default, creates a new file (fails if it already exists). Use 'overwrite' to replace an existing file, or 'append' to add to an existing file." } -func (t *FolderCreateTool) ToolPayloadSchema() Schema { +func (t *MemoryCreateTool) ToolPayloadSchema() Schema { return Schema{ Type: TypeObject, Properties: PropertyMap{ - "folder": { + "memory": { Type: TypeString, - Description: "Folder name. Use \"mission\" for the persistent mission folder, \"run\" for the per-run ephemeral folder, or a shared folder name.", + Description: memoryParamDescription, }, "path": { Type: TypeString, - Description: "Relative file path within the folder.", + Description: "Relative file path within the memory slot.", }, "content": { Type: TypeString, @@ -394,20 +400,20 @@ func (t *FolderCreateTool) ToolPayloadSchema() Schema { Description: "If true, overwrite the file if it already exists. Ignored when 'append' is true.", }, }, - Required: []string{"folder", "path", "content"}, + Required: []string{"memory", "path", "content"}, } } -type folderCreateParams struct { - Folder string `json:"folder"` +type memoryCreateParams struct { + Memory string `json:"memory"` Path string `json:"path"` Content string `json:"content"` Append bool `json:"append"` Overwrite bool `json:"overwrite"` } -func (t *FolderCreateTool) Call(ctx context.Context, params string) string { - var p folderCreateParams +func (t *MemoryCreateTool) Call(ctx context.Context, params string) string { + var p memoryCreateParams if err := json.Unmarshal([]byte(params), &p); err != nil { return "Error: invalid parameters - " + err.Error() } @@ -416,13 +422,13 @@ func (t *FolderCreateTool) Call(ctx context.Context, params string) string { return "Error: path is required" } - absPath, writable, err := resolveFolderPath(t.Store, p.Folder, p.Path) + absPath, writable, err := resolveMemoryPath(t.Store, p.Memory, p.Path) if err != nil { return "Error: " + err.Error() } if !writable { - return "Error: folder is read-only" + return "Error: memory slot is read-only" } if p.Append { @@ -459,43 +465,43 @@ func (t *FolderCreateTool) Call(ctx context.Context, params string) string { } // ============================================================================= -// folder_delete — Delete a file +// file_delete — Delete a file // ============================================================================= -type FolderDeleteTool struct { - Store FolderStore +type MemoryDeleteTool struct { + Store MemoryStore } -func (t *FolderDeleteTool) ToolName() string { return "file_delete" } +func (t *MemoryDeleteTool) ToolName() string { return "file_delete" } -func (t *FolderDeleteTool) ToolDescription() string { - return "Delete a file in a folder. Only files can be deleted, not directories." +func (t *MemoryDeleteTool) ToolDescription() string { + return "Delete a file in a memory slot. Only files can be deleted, not directories." } -func (t *FolderDeleteTool) ToolPayloadSchema() Schema { +func (t *MemoryDeleteTool) ToolPayloadSchema() Schema { return Schema{ Type: TypeObject, Properties: PropertyMap{ - "folder": { + "memory": { Type: TypeString, - Description: "Folder name. Use \"mission\" for the persistent mission folder, \"run\" for the per-run ephemeral folder, or a shared folder name.", + Description: memoryParamDescription, }, "path": { Type: TypeString, - Description: "Relative file path within the folder.", + Description: "Relative file path within the memory slot.", }, }, - Required: []string{"folder", "path"}, + Required: []string{"memory", "path"}, } } -type folderDeleteParams struct { - Folder string `json:"folder"` +type memoryDeleteParams struct { + Memory string `json:"memory"` Path string `json:"path"` } -func (t *FolderDeleteTool) Call(ctx context.Context, params string) string { - var p folderDeleteParams +func (t *MemoryDeleteTool) Call(ctx context.Context, params string) string { + var p memoryDeleteParams if err := json.Unmarshal([]byte(params), &p); err != nil { return "Error: invalid parameters - " + err.Error() } @@ -504,13 +510,13 @@ func (t *FolderDeleteTool) Call(ctx context.Context, params string) string { return "Error: path is required" } - absPath, writable, err := resolveFolderPath(t.Store, p.Folder, p.Path) + absPath, writable, err := resolveMemoryPath(t.Store, p.Memory, p.Path) if err != nil { return "Error: " + err.Error() } if !writable { - return "Error: folder is read-only" + return "Error: memory slot is read-only" } info, err := os.Stat(absPath) @@ -532,27 +538,27 @@ func (t *FolderDeleteTool) Call(ctx context.Context, params string) string { // file_search — Search for files by name pattern // ============================================================================= -type FolderSearchTool struct { - Store FolderStore +type MemorySearchTool struct { + Store MemoryStore } -func (t *FolderSearchTool) ToolName() string { return "file_search" } +func (t *MemorySearchTool) ToolName() string { return "file_search" } -func (t *FolderSearchTool) ToolDescription() string { - return "Search for files by name using a regex pattern. Returns matching file paths with sizes. Searches recursively by default. Results are paginated (default 50)." +func (t *MemorySearchTool) ToolDescription() string { + return "Search for files by name within a memory slot using a regex pattern. Returns matching file paths with sizes. Searches recursively by default. Results are paginated (default 50)." } -func (t *FolderSearchTool) ToolPayloadSchema() Schema { +func (t *MemorySearchTool) ToolPayloadSchema() Schema { return Schema{ Type: TypeObject, Properties: PropertyMap{ - "folder": { + "memory": { Type: TypeString, - Description: "Folder name. Use \"mission\" for the persistent mission folder, \"run\" for the per-run ephemeral folder, or a shared folder name.", + Description: memoryParamDescription, }, "path": { Type: TypeString, - Description: "Relative path to search within. Omit to search the folder root.", + Description: "Relative path to search within. Omit to search the memory root.", }, "pattern": { Type: TypeString, @@ -567,12 +573,12 @@ func (t *FolderSearchTool) ToolPayloadSchema() Schema { Description: "Number of results to skip (for pagination). Default 0.", }, }, - Required: []string{"folder", "pattern"}, + Required: []string{"memory", "pattern"}, } } -type folderSearchParams struct { - Folder string `json:"folder"` +type memorySearchParams struct { + Memory string `json:"memory"` Path string `json:"path"` Pattern string `json:"pattern"` Limit int `json:"limit"` @@ -581,8 +587,8 @@ type folderSearchParams struct { const defaultSearchLimit = 50 -func (t *FolderSearchTool) Call(ctx context.Context, params string) string { - var p folderSearchParams +func (t *MemorySearchTool) Call(ctx context.Context, params string) string { + var p memorySearchParams if err := json.Unmarshal([]byte(params), &p); err != nil { return "Error: invalid parameters - " + err.Error() } @@ -603,9 +609,9 @@ func (t *FolderSearchTool) Call(ctx context.Context, params string) string { // Resolve search root var absPath string if p.Path == "" { - absPath, _, err = t.Store.ResolvePath(p.Folder, ".") + absPath, _, err = t.Store.ResolvePath(p.Memory, ".") } else { - absPath, _, err = resolveFolderPath(t.Store, p.Folder, p.Path) + absPath, _, err = resolveMemoryPath(t.Store, p.Memory, p.Path) } if err != nil { return "Error: " + err.Error() @@ -681,27 +687,27 @@ func (t *FolderSearchTool) Call(ctx context.Context, params string) string { // file_grep — Search file contents with regex // ============================================================================= -type FolderGrepTool struct { - Store FolderStore +type MemoryGrepTool struct { + Store MemoryStore } -func (t *FolderGrepTool) ToolName() string { return "file_grep" } +func (t *MemoryGrepTool) ToolName() string { return "file_grep" } -func (t *FolderGrepTool) ToolDescription() string { - return "Search file contents using a regex pattern. Returns matching lines with file paths and line numbers. Results are paginated (default 50 matches)." +func (t *MemoryGrepTool) ToolDescription() string { + return "Search file contents within a memory slot using a regex pattern. Returns matching lines with file paths and line numbers. Results are paginated (default 50 matches)." } -func (t *FolderGrepTool) ToolPayloadSchema() Schema { +func (t *MemoryGrepTool) ToolPayloadSchema() Schema { return Schema{ Type: TypeObject, Properties: PropertyMap{ - "folder": { + "memory": { Type: TypeString, - Description: "Folder name. Use \"mission\" for the persistent mission folder, \"run\" for the per-run ephemeral folder, or a shared folder name.", + Description: memoryParamDescription, }, "path": { Type: TypeString, - Description: "Relative path to search within. Omit to search the folder root.", + Description: "Relative path to search within. Omit to search the memory root.", }, "pattern": { Type: TypeString, @@ -720,12 +726,12 @@ func (t *FolderGrepTool) ToolPayloadSchema() Schema { Description: "Number of matches to skip (for pagination). Default 0.", }, }, - Required: []string{"folder", "pattern"}, + Required: []string{"memory", "pattern"}, } } -type folderGrepParams struct { - Folder string `json:"folder"` +type memoryGrepParams struct { + Memory string `json:"memory"` Path string `json:"path"` Pattern string `json:"pattern"` Recursive bool `json:"recursive"` @@ -735,8 +741,8 @@ type folderGrepParams struct { const defaultGrepLimit = 50 -func (t *FolderGrepTool) Call(ctx context.Context, params string) string { - var p folderGrepParams +func (t *MemoryGrepTool) Call(ctx context.Context, params string) string { + var p memoryGrepParams if err := json.Unmarshal([]byte(params), &p); err != nil { return "Error: invalid parameters - " + err.Error() } @@ -757,9 +763,9 @@ func (t *FolderGrepTool) Call(ctx context.Context, params string) string { // Resolve search root var absPath string if p.Path == "" { - absPath, _, err = t.Store.ResolvePath(p.Folder, ".") + absPath, _, err = t.Store.ResolvePath(p.Memory, ".") } else { - absPath, _, err = resolveFolderPath(t.Store, p.Folder, p.Path) + absPath, _, err = resolveMemoryPath(t.Store, p.Memory, p.Path) } if err != nil { return "Error: " + err.Error() diff --git a/cmd/engage.go b/cmd/engage.go index e6ffbb0..cf8a205 100644 --- a/cmd/engage.go +++ b/cmd/engage.go @@ -392,9 +392,9 @@ func runEngage(cmd *cobra.Command, args []string) { client.Close() }() - // Periodic sweep of expired per-run mission folders. Runs hourly, reads - // the live config so new run_folder bases show up after a reload. - go runFolderCleanupLoop(shutdown, client.GetConfig) + // Periodic sweep of expired per-run ephemeral memory directories. + // Runs hourly; walks the filesystem so the live config isn't needed. + go runMemoryCleanupLoop(shutdown) // Even without valid config we still try to connect — the command center // can show vars and config files so the user can fix things from the UI. @@ -901,11 +901,11 @@ func openBrowser(url string) { browser.Open(url) } -// runFolderCleanupLoop periodically sweeps expired per-run ephemeral +// runMemoryCleanupLoop periodically sweeps expired per-run ephemeral // memory directories. The sweep walks the entire memories tree, so it // doesn't need to know which missions are configured. It runs once // immediately, then hourly, and exits when shutdown is closed. -func runFolderCleanupLoop(shutdown <-chan struct{}, _ func() *config.Config) { +func runMemoryCleanupLoop(shutdown <-chan struct{}) { sweep := func() { if _, err := mission.SweepExpiredEphemeralMemories(); err != nil { log.Printf("ephemeral memory cleanup: %v", err) diff --git a/config/memory.go b/config/memory.go index 4adbf69..228cc41 100644 --- a/config/memory.go +++ b/config/memory.go @@ -3,7 +3,7 @@ package config import "fmt" // Reserved slot names for mission-scoped memory. Tool calls reference these -// via the `folder` parameter (e.g. `folder: "mission"` or `folder: "run"`). +// via the `memory` parameter (e.g. `memory: "mission"` or `memory: "run"`). const ( PersistentSlotName = "mission" EphemeralSlotName = "run" diff --git a/config/folder_test.go b/config/memory_test.go similarity index 100% rename from config/folder_test.go rename to config/memory_test.go diff --git a/docs/content/declarative-agent-framework.mdx b/docs/content/declarative-agent-framework.mdx index 167ca72..32e5c57 100644 --- a/docs/content/declarative-agent-framework.mdx +++ b/docs/content/declarative-agent-framework.mdx @@ -18,7 +18,7 @@ A framework is declarative when: 1. **The workflow is data, not code.** You can serialize the whole thing as a config file, send it to someone else, and they can run it without the source repo. 2. **A runtime — not your code — drives execution.** The runtime decides when to call the model, what to retry, what to checkpoint, when to fan out, when to resume. 3. **Changes are reviewable as config diffs.** Adding a step, reordering dependencies, swapping a model — all show up as a small, readable patch. -4. **There is no hidden state.** All state is either declared (variables, datasets, folders) or persisted by the runtime (sessions, route decisions, outputs). +4. **There is no hidden state.** All state is either declared (variables, datasets, memory) or persisted by the runtime (sessions, route decisions, outputs). This is the same shift that happened in infrastructure (Terraform replaced Bash + Ansible scripts), in CI (declarative YAML replaced Makefile-driven pipelines), and in deployment (Kubernetes manifests replaced bespoke deploy scripts). The pattern: when the orchestration part of a workflow is gnarlier than the work itself, push the orchestration into a runtime and let humans edit data. diff --git a/docs/content/guides/no-code-multi-agent-workflow.mdx b/docs/content/guides/no-code-multi-agent-workflow.mdx index 73b1bde..3abfe9d 100644 --- a/docs/content/guides/no-code-multi-agent-workflow.mdx +++ b/docs/content/guides/no-code-multi-agent-workflow.mdx @@ -69,7 +69,7 @@ mission "daily_brief" { topic = string("Topic to brief on", { default = "AI agent frameworks" }) } - folder { path = "./briefs"; description = "Daily brief output" } + memory { description = "Daily brief output (persistent across runs)" } budget { tokens = 200000; dollars = 5 } @@ -96,12 +96,12 @@ mission "daily_brief" { task "deep_dive" { agents = [agents.summarizer] - objective = "Write a 500-word deep-dive on the most important item. Save to the briefs folder." + objective = "Write a 500-word deep-dive on the most important item. Save it into the mission memory." } task "summarize" { agents = [agents.summarizer] - objective = "Write a 200-word digest of all items. Save to the briefs folder." + objective = "Write a 200-word digest of all items. Save it into the mission memory." } } ``` diff --git a/docs/content/missions/folders.mdx b/docs/content/missions/folders.mdx index 03421b0..ea03e29 100644 --- a/docs/content/missions/folders.mdx +++ b/docs/content/missions/folders.mdx @@ -46,7 +46,7 @@ Missions opt in by listing them, then refer to them by name in tool calls: ```hcl mission "analyze" { memories = [memories.research, memories.reference] - # agents call e.g. file_read with folder = "research" + # agents call e.g. file_read with memory = "research" } ``` @@ -66,7 +66,7 @@ mission "analyze" { Agents reach it under the name `mission`: ```json -{ "folder": "mission", "path": "2026-04-23/report.md", "content": "..." } +{ "memory": "mission", "path": "2026-04-23/report.md", "content": "..." } ``` | Attribute | Type | Description | @@ -92,7 +92,7 @@ By default, ephemeral memory is deleted 7 days after the run started. Set `clean Agents reach it under the name `run`: ```json -{ "folder": "run", "path": "notes.txt", "content": "..." } +{ "memory": "run", "path": "notes.txt", "content": "..." } ``` | Attribute | Type | Description | @@ -105,7 +105,7 @@ Ephemeral directories stick around after the mission ends — useful for inspect ## Tool Reference -All six file tools take a required `folder` parameter — that's the slot name (a shared memory name, or the reserved `"mission"` / `"run"`): +All six file tools take a required `memory` parameter — that's the slot name (a shared memory name, or the reserved `"mission"` / `"run"`): | Tool | Purpose | |------|---------| diff --git a/mission/folder_store.go b/mission/memory_store.go similarity index 85% rename from mission/folder_store.go rename to mission/memory_store.go index ca8bd05..91353a1 100644 --- a/mission/folder_store.go +++ b/mission/memory_store.go @@ -97,23 +97,23 @@ func resolvedEphemeralCleanup(mm *config.MissionMemory) int { return *mm.Cleanup } -type missionFolderStore struct { - folders map[string]*folderEntry +type missionMemoryStore struct { + slots map[string]*memorySlot } -type folderEntry struct { +type memorySlot struct { absPath string description string writable bool } -// buildFolderStore creates a FolderStore from the mission config and the -// declared top-level memories. missionInstanceID scopes the ephemeral memory -// path; it must be non-empty when mission.EphemeralMemory is set. Returns -// nil if no memory slots are configured. -func buildFolderStore(mission *config.Mission, memories []config.Memory, missionInstanceID string) (aitools.FolderStore, error) { - store := &missionFolderStore{ - folders: make(map[string]*folderEntry), +// buildMemoryStore creates an aitools.MemoryStore from the mission config and +// the declared top-level memories. missionInstanceID scopes the ephemeral +// memory path; it must be non-empty when mission.EphemeralMemory is set. +// Returns nil if no memory slots are configured. +func buildMemoryStore(mission *config.Mission, memories []config.Memory, missionInstanceID string) (aitools.MemoryStore, error) { + store := &missionMemoryStore{ + slots: make(map[string]*memorySlot), } memByName := make(map[string]*config.Memory) @@ -140,7 +140,7 @@ func buildFolderStore(mission *config.Mission, memories []config.Memory, mission if desc == "" { desc = mem.Label } - store.folders[name] = &folderEntry{ + store.slots[name] = &memorySlot{ absPath: absPath, description: desc, writable: mem.Editable, @@ -155,7 +155,7 @@ func buildFolderStore(mission *config.Mission, memories []config.Memory, mission if err := os.MkdirAll(absPath, 0755); err != nil { return nil, fmt.Errorf("persistent memory: create directory: %w", err) } - store.folders[config.PersistentSlotName] = &folderEntry{ + store.slots[config.PersistentSlotName] = &memorySlot{ absPath: absPath, description: mission.PersistentMemory.Description, writable: true, @@ -176,14 +176,14 @@ func buildFolderStore(mission *config.Mission, memories []config.Memory, mission if err := writeRunMetadata(absPath, mission.Name, missionInstanceID, resolvedEphemeralCleanup(mission.EphemeralMemory)); err != nil { return nil, fmt.Errorf("ephemeral memory: write metadata: %w", err) } - store.folders[config.EphemeralSlotName] = &folderEntry{ + store.slots[config.EphemeralSlotName] = &memorySlot{ absPath: absPath, description: mission.EphemeralMemory.Description, writable: true, } } - if len(store.folders) == 0 { + if len(store.slots) == 0 { return nil, nil } @@ -218,14 +218,14 @@ func writeRunMetadata(dir, missionName, missionID string, cleanupDays int) error return err } -func (s *missionFolderStore) ResolvePath(folderName string, relPath string) (string, bool, error) { - if folderName == "" { - return "", false, fmt.Errorf("folder name is required (available: %v)", s.availableNames()) +func (s *missionMemoryStore) ResolvePath(memoryName string, relPath string) (string, bool, error) { + if memoryName == "" { + return "", false, fmt.Errorf("memory name is required (available: %v)", s.availableNames()) } - entry, ok := s.folders[folderName] + entry, ok := s.slots[memoryName] if !ok { - return "", false, fmt.Errorf("folder %q not found. Available: %v", folderName, s.availableNames()) + return "", false, fmt.Errorf("memory %q not found. Available: %v", memoryName, s.availableNames()) } cleaned := filepath.Clean(relPath) @@ -235,24 +235,24 @@ func (s *missionFolderStore) ResolvePath(folderName string, relPath string) (str fullPath := filepath.Join(entry.absPath, cleaned) if !strings.HasPrefix(fullPath, entry.absPath) { - return "", false, fmt.Errorf("path escapes folder root") + return "", false, fmt.Errorf("path escapes memory root") } return fullPath, entry.writable, nil } -func (s *missionFolderStore) availableNames() []string { - names := make([]string, 0, len(s.folders)) - for name := range s.folders { +func (s *missionMemoryStore) availableNames() []string { + names := make([]string, 0, len(s.slots)) + for name := range s.slots { names = append(names, name) } return names } -func (s *missionFolderStore) FolderInfos() []aitools.FolderInfo { - var infos []aitools.FolderInfo - for name, entry := range s.folders { - infos = append(infos, aitools.FolderInfo{ +func (s *missionMemoryStore) MemoryInfos() []aitools.MemoryInfo { + var infos []aitools.MemoryInfo + for name, entry := range s.slots { + infos = append(infos, aitools.MemoryInfo{ Name: name, Description: entry.description, Writable: entry.writable, diff --git a/mission/folder_store_test.go b/mission/memory_store_test.go similarity index 86% rename from mission/folder_store_test.go rename to mission/memory_store_test.go index 67eee91..03a38a6 100644 --- a/mission/folder_store_test.go +++ b/mission/memory_store_test.go @@ -25,10 +25,10 @@ func withTempHome(t *testing.T) string { return home } -func TestBuildFolderStore_NoMemories(t *testing.T) { +func TestBuildMemoryStore_NoMemories(t *testing.T) { withTempHome(t) m := &config.Mission{Name: "m"} - store, err := buildFolderStore(m, nil, "mid-1") + store, err := buildMemoryStore(m, nil, "mid-1") if err != nil { t.Fatalf("unexpected error: %v", err) } @@ -37,7 +37,7 @@ func TestBuildFolderStore_NoMemories(t *testing.T) { } } -func TestBuildFolderStore_PersistentMemory(t *testing.T) { +func TestBuildMemoryStore_PersistentMemory(t *testing.T) { home := withTempHome(t) m := &config.Mission{ @@ -48,7 +48,7 @@ func TestBuildFolderStore_PersistentMemory(t *testing.T) { }, } - store, err := buildFolderStore(m, nil, "mid-1") + store, err := buildMemoryStore(m, nil, "mid-1") if err != nil { t.Fatalf("unexpected error: %v", err) } @@ -56,7 +56,7 @@ func TestBuildFolderStore_PersistentMemory(t *testing.T) { t.Fatal("expected non-nil store") } - abs, writable, err := store.ResolvePath(aitools.MissionFolderName, ".") + abs, writable, err := store.ResolvePath(aitools.PersistentMemoryName, ".") if err != nil { t.Fatalf("ResolvePath: %v", err) } @@ -77,7 +77,7 @@ func TestBuildFolderStore_PersistentMemory(t *testing.T) { } } -func TestBuildFolderStore_EphemeralMemory_CreatesSidecar(t *testing.T) { +func TestBuildMemoryStore_EphemeralMemory_CreatesSidecar(t *testing.T) { home := withTempHome(t) cleanup := 7 m := &config.Mission{ @@ -88,12 +88,12 @@ func TestBuildFolderStore_EphemeralMemory_CreatesSidecar(t *testing.T) { }, } - store, err := buildFolderStore(m, nil, "mid-abc") + store, err := buildMemoryStore(m, nil, "mid-abc") if err != nil { t.Fatalf("unexpected error: %v", err) } - abs, writable, err := store.ResolvePath(aitools.RunFolderName, ".") + abs, writable, err := store.ResolvePath(aitools.EphemeralMemoryName, ".") if err != nil { t.Fatalf("ResolvePath: %v", err) } @@ -124,7 +124,7 @@ func TestBuildFolderStore_EphemeralMemory_CreatesSidecar(t *testing.T) { } } -func TestBuildFolderStore_EphemeralMemory_SidecarPreservedOnResume(t *testing.T) { +func TestBuildMemoryStore_EphemeralMemory_SidecarPreservedOnResume(t *testing.T) { home := withTempHome(t) m := &config.Mission{ Name: "m", @@ -134,7 +134,7 @@ func TestBuildFolderStore_EphemeralMemory_SidecarPreservedOnResume(t *testing.T) } // First build: sidecar written - if _, err := buildFolderStore(m, nil, "mid-1"); err != nil { + if _, err := buildMemoryStore(m, nil, "mid-1"); err != nil { t.Fatalf("first build: %v", err) } runDir := filepath.Join(home, "memories", "mission", "m", "run", "mid-1") @@ -145,7 +145,7 @@ func TestBuildFolderStore_EphemeralMemory_SidecarPreservedOnResume(t *testing.T) time.Sleep(10 * time.Millisecond) // Second build (same missionID = resume): sidecar must NOT be overwritten - if _, err := buildFolderStore(m, nil, "mid-1"); err != nil { + if _, err := buildMemoryStore(m, nil, "mid-1"); err != nil { t.Fatalf("second build: %v", err) } secondMetaBytes, _ := os.ReadFile(filepath.Join(runDir, runMetadataFile)) @@ -157,7 +157,7 @@ func TestBuildFolderStore_EphemeralMemory_SidecarPreservedOnResume(t *testing.T) } } -func TestBuildFolderStore_EphemeralMemory_RequiresMissionID(t *testing.T) { +func TestBuildMemoryStore_EphemeralMemory_RequiresMissionID(t *testing.T) { withTempHome(t) m := &config.Mission{ Name: "m", @@ -165,13 +165,13 @@ func TestBuildFolderStore_EphemeralMemory_RequiresMissionID(t *testing.T) { Type: "ephemeral", }, } - _, err := buildFolderStore(m, nil, "") + _, err := buildMemoryStore(m, nil, "") if err == nil { t.Fatal("expected error when missionID is empty") } } -func TestBuildFolderStore_RejectsReservedSharedMemoryNames(t *testing.T) { +func TestBuildMemoryStore_RejectsReservedSharedMemoryNames(t *testing.T) { withTempHome(t) for _, reserved := range []string{"mission", "run"} { m := &config.Mission{ @@ -181,14 +181,14 @@ func TestBuildFolderStore_RejectsReservedSharedMemoryNames(t *testing.T) { mems := []config.Memory{ {Name: reserved}, } - _, err := buildFolderStore(m, mems, "mid-1") + _, err := buildMemoryStore(m, mems, "mid-1") if err == nil { t.Fatalf("expected error for reserved shared memory name %q", reserved) } } } -func TestBuildFolderStore_BothPersistentAndEphemeral(t *testing.T) { +func TestBuildMemoryStore_BothPersistentAndEphemeral(t *testing.T) { withTempHome(t) m := &config.Mission{ Name: "m", @@ -199,19 +199,19 @@ func TestBuildFolderStore_BothPersistentAndEphemeral(t *testing.T) { Type: "ephemeral", }, } - store, err := buildFolderStore(m, nil, "mid-1") + store, err := buildMemoryStore(m, nil, "mid-1") if err != nil { t.Fatalf("unexpected error: %v", err) } - if _, _, err := store.ResolvePath(aitools.MissionFolderName, "."); err != nil { + if _, _, err := store.ResolvePath(aitools.PersistentMemoryName, "."); err != nil { t.Fatalf("persistent should resolve: %v", err) } - if _, _, err := store.ResolvePath(aitools.RunFolderName, "."); err != nil { + if _, _, err := store.ResolvePath(aitools.EphemeralMemoryName, "."); err != nil { t.Fatalf("ephemeral should resolve: %v", err) } } -func TestBuildFolderStore_SharedMemory(t *testing.T) { +func TestBuildMemoryStore_SharedMemory(t *testing.T) { home := withTempHome(t) m := &config.Mission{ Name: "m", @@ -221,7 +221,7 @@ func TestBuildFolderStore_SharedMemory(t *testing.T) { {Name: "research", Description: "Research notes", Editable: true}, } - store, err := buildFolderStore(m, mems, "mid-1") + store, err := buildMemoryStore(m, mems, "mid-1") if err != nil { t.Fatalf("build: %v", err) } @@ -242,7 +242,7 @@ func TestBuildFolderStore_SharedMemory(t *testing.T) { } } -func TestBuildFolderStore_SharedMemory_ReadOnly(t *testing.T) { +func TestBuildMemoryStore_SharedMemory_ReadOnly(t *testing.T) { withTempHome(t) m := &config.Mission{ Name: "m", @@ -252,7 +252,7 @@ func TestBuildFolderStore_SharedMemory_ReadOnly(t *testing.T) { {Name: "reference"}, // editable defaults to false } - store, err := buildFolderStore(m, mems, "mid-1") + store, err := buildMemoryStore(m, mems, "mid-1") if err != nil { t.Fatalf("build: %v", err) } @@ -265,18 +265,18 @@ func TestBuildFolderStore_SharedMemory_ReadOnly(t *testing.T) { } } -func TestResolvePath_EmptyFolderNameRejected(t *testing.T) { +func TestResolvePath_EmptyMemoryNameRejected(t *testing.T) { withTempHome(t) m := &config.Mission{ Name: "m", PersistentMemory: &config.MissionMemory{Type: "persistent"}, } - store, err := buildFolderStore(m, nil, "mid-1") + store, err := buildMemoryStore(m, nil, "mid-1") if err != nil { t.Fatalf("build: %v", err) } if _, _, err := store.ResolvePath("", "."); err == nil { - t.Fatal("expected error when folder name is empty") + t.Fatal("expected error when memory name is empty") } } @@ -286,7 +286,7 @@ func TestResolvePath_RejectsPathEscape(t *testing.T) { Name: "m", PersistentMemory: &config.MissionMemory{Type: "persistent"}, } - store, err := buildFolderStore(m, nil, "mid-1") + store, err := buildMemoryStore(m, nil, "mid-1") if err != nil { t.Fatalf("build: %v", err) } @@ -295,18 +295,18 @@ func TestResolvePath_RejectsPathEscape(t *testing.T) { } } -func TestResolvePath_UnknownFolder(t *testing.T) { +func TestResolvePath_UnknownMemory(t *testing.T) { withTempHome(t) m := &config.Mission{ Name: "m", PersistentMemory: &config.MissionMemory{Type: "persistent"}, } - store, err := buildFolderStore(m, nil, "mid-1") + store, err := buildMemoryStore(m, nil, "mid-1") if err != nil { t.Fatalf("build: %v", err) } if _, _, err := store.ResolvePath("does_not_exist", "."); err == nil { - t.Fatal("expected error for unknown folder") + t.Fatal("expected error for unknown memory") } } @@ -438,7 +438,7 @@ func TestSweep_WalksAcrossMissions(t *testing.T) { // TestSweepThenRebuildRoundTrip mirrors the real flow: an old ephemeral // memory exists with a sidecar backdated past its cleanup window, the sweep -// deletes it, then a new buildFolderStore (different missionID) creates a +// deletes it, then a new buildMemoryStore (different missionID) creates a // fresh ephemeral memory with a current sidecar — and the old one is gone. func TestSweepThenRebuildRoundTrip(t *testing.T) { home := withTempHome(t) @@ -459,11 +459,11 @@ func TestSweepThenRebuildRoundTrip(t *testing.T) { Cleanup: &cleanup, }, } - store, err := buildFolderStore(m, nil, "new-run") + store, err := buildMemoryStore(m, nil, "new-run") if err != nil { t.Fatalf("build: %v", err) } - fresh, _, err := store.ResolvePath(aitools.RunFolderName, ".") + fresh, _, err := store.ResolvePath(aitools.EphemeralMemoryName, ".") if err != nil { t.Fatalf("resolve: %v", err) } diff --git a/mission/runner.go b/mission/runner.go index 711d41b..23b2d14 100644 --- a/mission/runner.go +++ b/mission/runner.go @@ -63,8 +63,8 @@ type Runner struct { resumeMissionID string // Non-empty when resuming a prior mission rawInputs map[string]string // Raw input strings for persistence/resume - // Folder access for mission - folderStore aitools.FolderStore + // Memory access for mission + memoryStore aitools.MemoryStore // Conditional routing state routerPending []routerActivation // queue of tasks activated by routers @@ -288,7 +288,7 @@ func NewRunner(cfg *config.Config, configPath string, missionName string, inputs r.secretInfos = secretInfos } - // Folder store is built later in Run() once missionID is known — the + // Memory store is built later in Run() once missionID is known — the // ephemeral memory path depends on it. Shared + persistent memory alone // would let us build here, but deferring is simpler than branching on // mission.EphemeralMemory. @@ -568,18 +568,18 @@ func (r *Runner) Run(ctx context.Context, streamer streamers.MissionHandler) err r.resolvedDatasets = nil } - // Folder store depends on missionID (for ephemeral memory path), so build + // Memory store depends on missionID (for ephemeral memory path), so build // it here rather than in NewRunner. Sweep expired ephemeral memories // async — the result doesn't affect this run's correctness, only disk // usage. if r.mission.EphemeralMemory != nil { go func() { _, _ = SweepExpiredEphemeralMemories() }() } - folderStore, err := buildFolderStore(r.mission, r.cfg.Memories, missionID) + memoryStore, err := buildMemoryStore(r.mission, r.cfg.Memories, missionID) if err != nil { - return fmt.Errorf("mission '%s': build folder store: %w", r.mission.Name, err) + return fmt.Errorf("mission '%s': build memory store: %w", r.mission.Name, err) } - r.folderStore = folderStore + r.memoryStore = memoryStore streamer.MissionStarted(r.mission.Name, missionID, len(r.mission.Tasks)) @@ -1002,7 +1002,7 @@ func (r *Runner) resaturateCommanders(ctx context.Context, completedTaskNames [] SecretInfos: r.secretInfos, SecretValues: r.secretValues, IsIteration: isIterated, - FolderStore: r.folderStore, + MemoryStore: r.memoryStore, Compaction: r.commanderCompaction(), PruneOn: r.commanderPruneOn(), PruneTo: r.commanderPruneTo(), @@ -1056,7 +1056,7 @@ func (r *Runner) resaturateCommanders(ctx context.Context, completedTaskNames [] SecretInfos: r.secretInfos, SecretValues: r.secretValues, DatasetStore: r, - FolderStore: r.folderStore, + MemoryStore: r.memoryStore, HumanBridge: r.humanBridge, }, agentLLMMsgs) if err != nil { @@ -1140,7 +1140,7 @@ func (r *Runner) restoreAgentSessions(ctx context.Context, sup *agent.Commander, SecretInfos: r.secretInfos, SecretValues: r.secretValues, DatasetStore: r, - FolderStore: r.folderStore, + MemoryStore: r.memoryStore, HumanBridge: r.humanBridge, }, llmMsgs) if err != nil { @@ -1263,7 +1263,7 @@ func (r *Runner) runTask(ctx context.Context, task config.Task, missionID string SecretValues: r.secretValues, IsIteration: false, DebugFile: debugFile, - FolderStore: r.folderStore, + MemoryStore: r.memoryStore, Compaction: r.commanderCompaction(), PruneOn: r.commanderPruneOn(), PruneTo: r.commanderPruneTo(), @@ -2045,7 +2045,7 @@ Continue until dataset_next returns "exhausted".`, len(items), taskObjective) IsParallel: false, DebugFile: debugFile, SequentialDataset: items, - FolderStore: r.folderStore, + MemoryStore: r.memoryStore, Compaction: r.commanderCompaction(), PruneOn: r.commanderPruneOn(), PruneTo: r.commanderPruneTo(), @@ -2499,7 +2499,7 @@ Continue until dataset_next returns "exhausted".`, len(remainingItems), taskObje IsParallel: false, DebugFile: debugFile, SequentialDataset: remainingItems, - FolderStore: r.folderStore, + MemoryStore: r.memoryStore, Compaction: r.commanderCompaction(), PruneOn: r.commanderPruneOn(), PruneTo: r.commanderPruneTo(), @@ -2742,7 +2742,7 @@ func (r *Runner) runSingleIteration(ctx context.Context, task config.Task, index IsIteration: true, IsParallel: task.Iterator.Parallel, DebugFile: debugFile, - FolderStore: r.folderStore, + MemoryStore: r.memoryStore, Compaction: r.commanderCompaction(), PruneOn: r.commanderPruneOn(), PruneTo: r.commanderPruneTo(), diff --git a/wsbridge/shared_folder.go b/wsbridge/memory_browser.go similarity index 91% rename from wsbridge/shared_folder.go rename to wsbridge/memory_browser.go index 4c28d73..33a53f0 100644 --- a/wsbridge/shared_folder.go +++ b/wsbridge/memory_browser.go @@ -25,18 +25,18 @@ type resolvedMemory struct { editable bool } -// resolveSharedFolderPath looks up a memory slot by name (top-level shared +// resolveMemoryPath looks up a memory slot by name (top-level shared // memories first, then per-mission persistent memories keyed by mission name) // and safely resolves a relative path within it. -func (c *Client) resolveSharedFolderPath(folderName, relPath string) (*resolvedMemory, string, error) { +func (c *Client) resolveMemoryPath(memoryName, relPath string) (*resolvedMemory, string, error) { cfg := c.getConfig() // Check shared memories first. for i := range cfg.Memories { - if cfg.Memories[i].Name == folderName { - absPath, err := mission.SharedMemoryPath(folderName) + if cfg.Memories[i].Name == memoryName { + absPath, err := mission.SharedMemoryPath(memoryName) if err != nil { - return nil, "", fmt.Errorf("resolve shared memory %q: %w", folderName, err) + return nil, "", fmt.Errorf("resolve shared memory %q: %w", memoryName, err) } rm := &resolvedMemory{ name: cfg.Memories[i].Name, @@ -50,7 +50,7 @@ func (c *Client) resolveSharedFolderPath(folderName, relPath string) (*resolvedM // Check mission persistent memory (keyed by mission name). for _, m := range cfg.Missions { - if m.PersistentMemory != nil && m.Name == folderName { + if m.PersistentMemory != nil && m.Name == memoryName { absPath, err := mission.PersistentMemoryPath(m.Name) if err != nil { return nil, "", fmt.Errorf("resolve persistent memory for %q: %w", m.Name, err) @@ -61,7 +61,7 @@ func (c *Client) resolveSharedFolderPath(folderName, relPath string) (*resolvedM } } - return nil, "", fmt.Errorf("memory %q not found", folderName) + return nil, "", fmt.Errorf("memory %q not found", memoryName) } func (c *Client) resolveSafePath(basePath, relPath string) (string, error) { @@ -162,7 +162,7 @@ func (c *Client) handleBrowseDirectory(env *protocol.Envelope) (*protocol.Envelo return nil, fmt.Errorf("decode browse_directory: %w", err) } - _, fullPath, err := c.resolveSharedFolderPath(payload.BrowserName, payload.RelPath) + _, fullPath, err := c.resolveMemoryPath(payload.BrowserName, payload.RelPath) if err != nil { return nil, err } @@ -203,7 +203,7 @@ func (c *Client) handleReadBrowseFile(env *protocol.Envelope) (*protocol.Envelop return nil, fmt.Errorf("decode read_browse_file: %w", err) } - _, fullPath, err := c.resolveSharedFolderPath(payload.BrowserName, payload.RelPath) + _, fullPath, err := c.resolveMemoryPath(payload.BrowserName, payload.RelPath) if err != nil { return nil, err } @@ -259,7 +259,7 @@ func (c *Client) handleWriteBrowseFile(env *protocol.Envelope) (*protocol.Envelo return nil, fmt.Errorf("decode write_browse_file: %w", err) } - mem, fullPath, err := c.resolveSharedFolderPath(payload.BrowserName, payload.RelPath) + mem, fullPath, err := c.resolveMemoryPath(payload.BrowserName, payload.RelPath) if err != nil { return protocol.NewResponse(env.RequestID, protocol.TypeWriteBrowseFileResult, &protocol.WriteBrowseFileResultPayload{Success: false, Error: err.Error()}) @@ -285,7 +285,7 @@ func (c *Client) handleDownloadFile(env *protocol.Envelope) (*protocol.Envelope, return nil, fmt.Errorf("decode download_file: %w", err) } - _, fullPath, err := c.resolveSharedFolderPath(payload.BrowserName, payload.RelPath) + _, fullPath, err := c.resolveMemoryPath(payload.BrowserName, payload.RelPath) if err != nil { return nil, err } @@ -311,7 +311,7 @@ func (c *Client) handleDownloadDirectory(env *protocol.Envelope) (*protocol.Enve return nil, fmt.Errorf("decode download_directory: %w", err) } - _, fullPath, err := c.resolveSharedFolderPath(payload.BrowserName, payload.RelPath) + _, fullPath, err := c.resolveMemoryPath(payload.BrowserName, payload.RelPath) if err != nil { return nil, err } From 995347e7281efcdedda93f899c71ab939923a411 Mon Sep 17 00:00:00 2001 From: Max Lund Date: Mon, 25 May 2026 22:02:50 -0500 Subject: [PATCH 4/8] Drop transient shared_memory / run_memory detection MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit shared_memory and run_memory were intermediate names introduced earlier in this PR's history (between shared_folder → shared_memory → memory, and run_folder → run_memory → memory { type = "ephemeral" }) and never shipped to any user. The "no longer supported" errors and the corresponding detection-only entries in the HCL block schemas don't need to defend against names nobody has ever written in the wild. Removed: - shared_memory entry from the top-level block schema + parse switch - run_memory entry from the mission block schema + parse switch - The two unit tests that asserted those errors - CLAUDE.md mention of shared_memory in the broken-DSL list Kept (these were actually shipped names): shared_folder, folder, run_folder, folders=. Their explicit-error paths stay. --- CLAUDE.md | 8 ++++---- config/config.go | 18 +++++++----------- config/memory_test.go | 32 -------------------------------- 3 files changed, 11 insertions(+), 47 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 6ee5c59..f2dcc85 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -303,10 +303,10 @@ A mission may declare at most one persistent and one ephemeral memory. The names `"mission"` and `"run"` are reserved — a top-level `memory "mission"` or `memory "run"` block is rejected. -The old DSL surfaces — `shared_folder`/`shared_memory` blocks, `folder`/ -`run_folder`/`run_memory` blocks, the `folders = ...` attribute — are **not** -accepted. The parser surfaces a clear error pointing at the new syntax. Path -attributes anywhere on a memory block are rejected too. +The old DSL surfaces — `shared_folder` blocks, `folder` / `run_folder` +blocks, the `folders = ...` attribute — are **not** accepted. The parser +surfaces a clear error pointing at the new syntax. Path attributes anywhere +on a memory block are rejected too. ```hcl memory "reference" { diff --git a/config/config.go b/config/config.go index 949d3f4..378d235 100644 --- a/config/config.go +++ b/config/config.go @@ -721,10 +721,9 @@ func loadFromFiles(files []string) (*Config, error) { {Type: "storage"}, {Type: "command_center"}, {Type: "memory", LabelNames: []string{"name"}}, - // Detected for nicer errors only — the parse-pass below - // rejects them with a pointer to the new `memory` block. + // Detected for a nicer error only — the parse-pass below + // rejects it with a pointer to the new `memory` block. {Type: "shared_folder", LabelNames: []string{"name"}}, - {Type: "shared_memory", LabelNames: []string{"name"}}, {Type: "mcp_host"}, {Type: "mcp", LabelNames: []string{"name"}}, {Type: "skill", LabelNames: []string{"name"}}, @@ -758,7 +757,7 @@ func loadFromFiles(files []string) (*Config, error) { pb.CommandCenter = append(pb.CommandCenter, block) case "memory": pb.Memories = append(pb.Memories, block) - case "shared_folder", "shared_memory": + case "shared_folder": // Collected only so we can produce a clear error in the // parse pass below. pb.Memories = append(pb.Memories, block) @@ -958,8 +957,8 @@ func loadFromFiles(files []string) (*Config, error) { } // Parse top-level `memory "name" { ... }` blocks (with vars context). - // `shared_folder` and `shared_memory` are no longer supported — if a user - // writes one we surface an explicit error pointing at the new syntax. + // `shared_folder` is no longer supported — if a user writes one we + // surface an explicit error pointing at the new syntax. var allMemories []Memory for _, pb := range allParsedBlocks { for _, block := range pb.Memories { @@ -1805,7 +1804,6 @@ func parseMissionBlock(block *hcl.Block, ctx *hcl.EvalContext) (*Mission, error) // Detected so we can produce a nicer error than the parser's default. {Type: "folder"}, {Type: "run_folder"}, - {Type: "run_memory"}, }, }) if diags.HasErrors() { @@ -1972,16 +1970,14 @@ func parseMissionBlock(block *hcl.Block, ctx *hcl.EvalContext) (*Mission, error) } } - // Reject the old `folder { ... }` / `run_folder { ... }` / `run_memory { ... }` - // blocks with a clear pointer at the new syntax. + // Reject the old `folder { ... }` / `run_folder { ... }` blocks with a + // clear pointer at the new syntax. for _, b := range missionContent.Blocks { switch b.Type { case "folder": return nil, fmt.Errorf("mission '%s': the `folder { ... }` block is no longer supported — use `memory { type = \"persistent\" }` instead (path is now derived automatically)", missionName) case "run_folder": return nil, fmt.Errorf("mission '%s': the `run_folder { ... }` block is no longer supported — use `memory { type = \"ephemeral\" }` instead", missionName) - case "run_memory": - return nil, fmt.Errorf("mission '%s': the `run_memory { ... }` block is no longer supported — use `memory { type = \"ephemeral\" }` instead", missionName) } } diff --git a/config/memory_test.go b/config/memory_test.go index 1e37038..2832f60 100644 --- a/config/memory_test.go +++ b/config/memory_test.go @@ -65,23 +65,6 @@ mission "m" { Expect(err.Error()).To(ContainSubstring("memory \"research\"")) }) - It("rejects the old shared_memory block too", func() { - hcl := fullBaseHCL() + ` -shared_memory "research" { - path = "./data" -} -mission "m" { - commander { model = models.anthropic.claude_sonnet_4 } - agents = [agents.test_agent] - task "t" { objective = "go" } -} -` - _, f := writeFixture("config.hcl", hcl) - _, err := config.LoadFile(f) - Expect(err).To(HaveOccurred()) - Expect(err.Error()).To(ContainSubstring("no longer supported")) - }) - It("rejects the old `path` attribute on a memory block", func() { hcl := fullBaseHCL() + ` memory "research" { @@ -299,21 +282,6 @@ mission "m" { Expect(err.Error()).To(ContainSubstring("`run_folder { ... }` block is no longer supported")) }) - It("rejects the old `run_memory { ... }` block", func() { - hcl := fullBaseHCL() + ` -mission "m" { - commander { model = models.anthropic.claude_sonnet_4 } - agents = [agents.test_agent] - run_memory { description = "x" } - task "t" { objective = "go" } -} -` - _, f := writeFixture("config.hcl", hcl) - _, err := config.LoadFile(f) - Expect(err).To(HaveOccurred()) - Expect(err.Error()).To(ContainSubstring("`run_memory { ... }` block is no longer supported")) - }) - It("rejects the old `folders = ...` attribute", func() { hcl := fullBaseHCL() + ` memory "ref" {} From 541fad6ce1da0fc175b6792b67e95b942f2a5b2c Mon Sep 17 00:00:00 2001 From: Max Lund Date: Mon, 25 May 2026 22:23:11 -0500 Subject: [PATCH 5/8] Split memory by purpose: memory (persistent) + scratchpad (ephemeral) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replaces the type-discriminated `memory { type = "persistent" | "ephemeral" }` block with two semantically distinct block types: memory { ... } — persistent storage that survives across runs. At most one per mission. Slot name agents pass: "memory". scratchpad { ... } — ephemeral per-run working space. At most one per mission. Fresh directory per mission run; sidecar- driven cleanup sweep deletes ones past their configured window. Slot name: "scratchpad". Top-level `memory "name" { ... }` (shared, multiple) is unchanged. The tool parameter agents pass is renamed from `memory` to `slot`, since the slot values are now literally "memory" and "scratchpad" (a `memory: "memory"` JSON payload was awkward). Same six tools (file_list, file_read, ...), same behavior — only the parameter name changed. On-disk layout: /memories/shared// — shared memory /memories/mission// — mission memory (was .../persistent/) /scratchpads/// — scratchpad (was .../run//) Scratchpads live under their own top-level subdir so the cleanup sweep can walk one tree without scanning memory directories that should never be touched. Go-side renames: config.MissionMemory — now just { Description } (no Type field) config.MissionScratchpad — NEW { Description, Cleanup } Mission.PersistentMemory → Mission.Memory (*MissionMemory) Mission.EphemeralMemory → Mission.Scratchpad (*MissionScratchpad) config.PersistentSlotName → config.MemorySlotName ("memory") config.EphemeralSlotName → config.ScratchpadSlotName ("scratchpad") config.MemoryType{Persistent,Ephemeral} — removed config.DefaultEphemeralCleanupDays → DefaultScratchpadCleanupDays aitools.PersistentMemoryName → MemorySlotName aitools.EphemeralMemoryName → ScratchpadSlotName aitools.{tool}.Memory field → .Slot (JSON tag "memory" → "slot") mission.PersistentMemoryPath → MissionMemoryPath mission.EphemeralMemoryPath → MissionScratchpadPath mission.MissionRunRoot — removed (subsumed by ScratchpadsRoot) mission.SweepExpiredEphemeralMemories → SweepExpiredScratchpads cmd.runMemoryCleanupLoop → cmd.runScratchpadCleanupLoop Tests in config/memory_test.go and mission/memory_store_test.go fully rewritten for the new block types and path scheme. CLAUDE.md and the public docs (missions/folders.mdx, missions/overview.mdx, sidebar label, README features list) updated to describe the new design. Full `go test ./...` passes. --- CLAUDE.md | 91 ++++++++-------- README.md | 2 +- agent/internal/prompts/prompts.go | 12 +-- aitools/memory_tools.go | 135 ++++++++++++------------ cmd/engage.go | 16 +-- config/config.go | 59 +++++------ config/memory.go | 77 +++++++------- config/memory_test.go | 136 +++++++++++------------- config/mission.go | 28 ++--- docs/content/missions/_meta.js | 2 +- docs/content/missions/folders.mdx | 60 +++++------ docs/content/missions/overview.mdx | 5 +- mission/memory_store.go | 158 ++++++++++++++-------------- mission/memory_store_test.go | 160 +++++++++++++---------------- mission/runner.go | 17 ++- wsbridge/memory_browser.go | 16 +-- 16 files changed, 468 insertions(+), 506 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index f2dcc85..01727c9 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -282,31 +282,40 @@ mission "expensive_research" { The scheduler lives in `scheduler/` but its lifecycle (creation, config updates, shutdown) is managed by `cmd/serve.go`, not wsbridge. The wsbridge client receives a `ConcurrencyTracker` interface for enforcing `max_parallel` on all mission starts. The cron library used is `robfig/cron/v3`. -### Memory +### Memory + Scratchpad -Memory blocks are sandboxed filesystem locations that agents access via the -`file_list`, `file_read`, `file_create`, `file_delete`, `file_search`, and -`file_grep` tools. Each call takes a required `memory` parameter naming the -slot — there is no implicit default. +There are two kinds of mission-scoped file storage, plus a top-level shared +kind: -Squadron owns the paths: every slot lives under `/memories/` -and you do **not** specify `path` anywhere. The three kinds map to three -fixed location patterns: +- **Memory** — persistent storage that survives across runs. Declared with + a top-level `memory "name" { ... }` (shared, multiple) or a mission-scoped + `memory { ... }` (private to the mission, at most one). +- **Scratchpad** — ephemeral per-run working space. Declared with a + mission-scoped `scratchpad { ... }` block (at most one per mission). A + fresh directory is created for each mission run and the sweep deletes ones + past their `cleanup` window. -| Kind | HCL | `memory` arg agents pass | On-disk path | -|------|-----|---------------------------|--------------| -| Shared | top-level `memory "name" { ... }` | the HCL label | `/memories/shared//` | -| Persistent (mission) | `memory { type = "persistent" }` inside a mission (also the default if `type` is omitted) | literal `"mission"` | `/memories/mission//persistent/` | -| Ephemeral (per-run) | `memory { type = "ephemeral" }` inside a mission | literal `"run"` | `/memories/mission//run//` | +Agents access both kinds via the same `file_list`, `file_read`, `file_create`, +`file_delete`, `file_search`, `file_grep` tools. Each call takes a required +`slot` parameter naming which slot to operate in — there is no implicit +default. -A mission may declare at most one persistent and one ephemeral memory. The -names `"mission"` and `"run"` are reserved — a top-level `memory "mission"` -or `memory "run"` block is rejected. +Squadron owns the on-disk paths; the HCL never accepts a `path` attribute. +The three slot kinds map to three fixed location patterns: + +| Kind | HCL | `slot` agents pass | On-disk path | +|------|-----|---------------------|--------------| +| Shared memory | top-level `memory "name" { ... }` | the HCL label | `/memories/shared//` | +| Mission memory | `memory { ... }` inside a mission | literal `"memory"` | `/memories/mission//` | +| Mission scratchpad | `scratchpad { ... }` inside a mission | literal `"scratchpad"` | `/scratchpads///` | + +The slot names `"memory"` and `"scratchpad"` are reserved — a top-level +`memory "memory"` or `memory "scratchpad"` block is rejected. The old DSL surfaces — `shared_folder` blocks, `folder` / `run_folder` blocks, the `folders = ...` attribute — are **not** accepted. The parser surfaces a clear error pointing at the new syntax. Path attributes anywhere -on a memory block are rejected too. +on a memory/scratchpad block are rejected too. ```hcl memory "reference" { @@ -318,13 +327,11 @@ mission "analyze" { memories = [memories.reference] memory { - # type defaults to "persistent" description = "Cumulative reports — persists across runs" } - memory { - type = "ephemeral" - description = "Per-run scratch" + scratchpad { + description = "Per-run working space" cleanup = 7 # optional; defaults to 7, set 0 to keep forever } } @@ -332,29 +339,31 @@ mission "analyze" { **Implementation:** -- `config.Memory` (top-level) and `config.MissionMemory` (mission-scoped, with - a `Type` field) in [config/memory.go](config/memory.go); `MissionMemory` - fields land on the mission as `PersistentMemory` and `EphemeralMemory` - (`config.Mission` in [config/mission.go](config/mission.go)). +- `config.Memory` (top-level shared), `config.MissionMemory` (mission's + persistent), and `config.MissionScratchpad` (mission's ephemeral) all + live in [config/memory.go](config/memory.go). On a parsed mission they + land as `Memories []string`, `Memory *MissionMemory`, and + `Scratchpad *MissionScratchpad` (`config.Mission` in + [config/mission.go](config/mission.go)). - Top-level `memory "name"` blocks are parsed in Stage 1.5 (with `vars` - context); each mission's `memory { ... }` blocks are parsed inside the - mission block in Stage 5. The `memories.NAME` HCL namespace exposes the - shared-memory labels to mission attributes. -- [mission/folder_store.go](mission/folder_store.go) is the authoritative + context); each mission's `memory { ... }` and `scratchpad { ... }` blocks + are parsed inside the mission block in Stage 5. The `memories.NAME` HCL + namespace exposes the shared-memory labels to mission attributes. +- [mission/memory_store.go](mission/memory_store.go) is the authoritative runtime resolver. Paths are derived from `paths.SquadronHome()`: - `SharedMemoryPath(name)`, `PersistentMemoryPath(missionName)`, and - `EphemeralMemoryPath(missionName, missionInstanceID)`. - `buildFolderStore(mission, memories, missionInstanceID)` must be called - **after** the mission instance ID is assigned in `Runner.Run()` because the - ephemeral path depends on it. -- Each ephemeral directory gets a sidecar `.squadron-run.json` recording + `SharedMemoryPath(name)`, `MissionMemoryPath(missionName)`, and + `MissionScratchpadPath(missionName, missionInstanceID)`. + `buildMemoryStore(mission, memories, missionInstanceID)` must be called + **after** the mission instance ID is assigned in `Runner.Run()` because + the scratchpad path depends on it. +- Each scratchpad directory gets a sidecar `.squadron-run.json` recording `created_at` + `cleanup_days` so the sweep can find expired ones. -- `mission.SweepExpiredEphemeralMemories()` walks - `/memories/mission/*/run/*`, deleting any per-run directory - whose sidecar is past its cleanup deadline. It runs opportunistically at - the start of every `Runner.Run()` (for missions with an ephemeral memory) - and on an hourly ticker in `cmd/engage.go`. No config lookup needed — the - filesystem layout is self-describing. +- `mission.SweepExpiredScratchpads()` walks + `/scratchpads/*/*`, deleting any per-run directory whose + sidecar is past its cleanup deadline. It runs opportunistically at the + start of every `Runner.Run()` (for missions with a scratchpad) and on an + hourly ticker in `cmd/engage.go`. No config lookup needed — the filesystem + layout is self-describing. ### Mission-Scoped Agents diff --git a/README.md b/README.md index 8831cd7..bf6e6ed 100644 --- a/README.md +++ b/README.md @@ -111,7 +111,7 @@ Honest tradeoffs on each comparison page. - **[MCP host](https://docs.squadron.sh/config/mcp_host)** — run Squadron as an MCP server for Claude Desktop, Claude Code, Cursor, and other MCP clients. - **[Schedules & triggers](https://docs.squadron.sh/missions/schedules)** — cron-based or HTTP-triggered missions, with per-mission concurrency limits. - **[Budgets](https://docs.squadron.sh/missions/budgets)** — token + dollar caps per mission or per task. -- **[Memory](https://docs.squadron.sh/missions/folders)** — sandboxed filesystem locations agents can read/write. Three kinds: shared (top-level `memory "name"`), persistent mission, and ephemeral per-run (`memory { type = "persistent" | "ephemeral" }`). Squadron owns the paths. +- **[Memory & Scratchpad](https://docs.squadron.sh/missions/folders)** — sandboxed filesystem locations agents can read/write. Persistent memory (shared with `memory "name"` or per-mission with `memory { }`) plus per-run `scratchpad { }`. Squadron owns the paths. - **[Gateways](https://docs.squadron.sh/config/gateways)** — managed subprocesses that bridge Squadron to Slack, Discord, Teams, or any custom system via the Gateway SDK. - **Encrypted vault** — secrets stored at rest with AES-256-GCM + Argon2id; passphrase in the OS keychain. - **Single binary, no runtime deps.** Docker images on every release. diff --git a/agent/internal/prompts/prompts.go b/agent/internal/prompts/prompts.go index ae81e4a..68bfb3e 100644 --- a/agent/internal/prompts/prompts.go +++ b/agent/internal/prompts/prompts.go @@ -242,9 +242,9 @@ func FormatMemoryContext(store aitools.MemoryStore) string { } var sb strings.Builder - sb.WriteString("## Available Memory\n\n") - sb.WriteString("You have access to memory slots via the file_list, file_read, file_create, file_delete, file_search, and file_grep tools.\n") - sb.WriteString("The `memory` parameter is required on every call — pick one of the slot names below.\n\n") + sb.WriteString("## Available Slots\n\n") + sb.WriteString("You have access to file storage slots via the file_list, file_read, file_create, file_delete, file_search, and file_grep tools.\n") + sb.WriteString("The `slot` parameter is required on every call — pick one of the slot names below.\n\n") for _, info := range infos { access := "read-only" @@ -253,10 +253,10 @@ func FormatMemoryContext(store aitools.MemoryStore) string { } label := "" switch info.Name { - case aitools.PersistentMemoryName: + case aitools.MemorySlotName: label = " (persistent mission memory — survives across runs)" - case aitools.EphemeralMemoryName: - label = " (ephemeral per-run memory — fresh for this mission run)" + case aitools.ScratchpadSlotName: + label = " (ephemeral per-run scratchpad — fresh for this mission run)" } desc := "" if info.Description != "" { diff --git a/aitools/memory_tools.go b/aitools/memory_tools.go index 4779525..388c749 100644 --- a/aitools/memory_tools.go +++ b/aitools/memory_tools.go @@ -12,32 +12,33 @@ import ( "strings" ) -// Reserved slot names for mission-scoped memory. Callers addressing the -// persistent mission memory and the per-run ephemeral memory must use these -// names as the `memory` parameter. +// Reserved slot names for the mission-scoped storage slots. Agents address +// them via the `slot` parameter on the file tools (alongside any shared +// memory names declared at the top level). const ( - PersistentMemoryName = "mission" - EphemeralMemoryName = "run" + MemorySlotName = "memory" + ScratchpadSlotName = "scratchpad" ) -// MemoryStore provides memory access for missions. Implementations resolve a -// memory slot name to an absolute path and enforce read/write semantics. +// MemoryStore provides slot access for missions. Implementations resolve a +// slot name (the mission's "memory" / "scratchpad", or a shared memory's +// label) to an absolute path and enforce read/write semantics. type MemoryStore interface { - // ResolvePath resolves a memory slot name + relative path to an absolute - // path. Returns: absolute path, writable flag, error. - ResolvePath(memoryName string, relPath string) (string, bool, error) - // MemoryInfos returns info about all available memory slots. + // ResolvePath resolves a slot name + relative path to an absolute path. + // Returns: absolute path, writable flag, error. + ResolvePath(slotName string, relPath string) (string, bool, error) + // MemoryInfos returns info about all available slots. MemoryInfos() []MemoryInfo } -// MemoryInfo describes an available memory slot. +// MemoryInfo describes an available slot. type MemoryInfo struct { Name string Description string Writable bool } -// validateRelPath ensures a path is relative and doesn't escape the memory root. +// validateRelPath ensures a path is relative and doesn't escape the slot root. func validateRelPath(relPath string) error { if relPath == "" { return fmt.Errorf("path is required") @@ -45,23 +46,23 @@ func validateRelPath(relPath string) error { cleaned := filepath.Clean(relPath) if filepath.IsAbs(cleaned) || strings.HasPrefix(cleaned, "..") || strings.Contains(cleaned, string(filepath.Separator)+"..") { - return fmt.Errorf("invalid path: must be relative and within the memory root") + return fmt.Errorf("invalid path: must be relative and within the slot root") } return nil } -// resolveMemoryPath is a helper that resolves the memory slot and validates -// the relative path. -func resolveMemoryPath(store MemoryStore, name, relPath string) (string, bool, error) { +// resolveSlotPath is a helper that resolves the slot and validates the +// relative path. +func resolveSlotPath(store MemoryStore, name, relPath string) (string, bool, error) { if err := validateRelPath(relPath); err != nil { return "", false, err } return store.ResolvePath(name, relPath) } -// memoryParamDescription is reused across every file tool's `memory` -// parameter so the agent sees a consistent description. -const memoryParamDescription = "Memory slot to operate in. Use \"mission\" for the persistent mission memory, \"run\" for the per-run ephemeral memory, or a shared memory name." +// slotParamDescription is reused across every file tool's `slot` parameter +// so the agent sees a consistent description. +const slotParamDescription = "Slot to operate in. Use \"memory\" for the mission's persistent memory, \"scratchpad\" for its ephemeral per-run scratchpad, or a shared memory name." // ============================================================================= // file_list — List files and directories @@ -74,20 +75,20 @@ type MemoryListTool struct { func (t *MemoryListTool) ToolName() string { return "file_list" } func (t *MemoryListTool) ToolDescription() string { - return "List files and directories in a memory slot. Returns names, types (file/dir), and sizes. Results are paginated (default 100 per page). Use 'offset' to get subsequent pages." + return "List files and directories in a slot. Returns names, types (file/dir), and sizes. Results are paginated (default 100 per page). Use 'offset' to get subsequent pages." } func (t *MemoryListTool) ToolPayloadSchema() Schema { return Schema{ Type: TypeObject, Properties: PropertyMap{ - "memory": { + "slot": { Type: TypeString, - Description: memoryParamDescription, + Description: slotParamDescription, }, "path": { Type: TypeString, - Description: "Relative subdirectory path within the memory slot. Omit to list the root.", + Description: "Relative subdirectory path within the slot. Omit to list the root.", }, "recursive": { Type: TypeBoolean, @@ -102,12 +103,12 @@ func (t *MemoryListTool) ToolPayloadSchema() Schema { Description: "Number of entries to skip (for pagination). Default 0.", }, }, - Required: []string{"memory"}, + Required: []string{"slot"}, } } type memoryListParams struct { - Memory string `json:"memory"` + Slot string `json:"slot"` Path string `json:"path"` Recursive bool `json:"recursive"` Limit int `json:"limit"` @@ -130,9 +131,9 @@ func (t *MemoryListTool) Call(ctx context.Context, params string) string { var absPath string var err error if p.Path == "" { - absPath, _, err = t.Store.ResolvePath(p.Memory, ".") + absPath, _, err = t.Store.ResolvePath(p.Slot, ".") } else { - absPath, _, err = resolveMemoryPath(t.Store, p.Memory, p.Path) + absPath, _, err = resolveSlotPath(t.Store, p.Slot, p.Path) } if err != nil { return "Error: " + err.Error() @@ -264,20 +265,20 @@ type MemoryReadTool struct { func (t *MemoryReadTool) ToolName() string { return "file_read" } func (t *MemoryReadTool) ToolDescription() string { - return "Read the contents of a file in a memory slot. Optionally limit to the first N lines or N bytes." + return "Read the contents of a file in a slot. Optionally limit to the first N lines or N bytes." } func (t *MemoryReadTool) ToolPayloadSchema() Schema { return Schema{ Type: TypeObject, Properties: PropertyMap{ - "memory": { + "slot": { Type: TypeString, - Description: memoryParamDescription, + Description: slotParamDescription, }, "path": { Type: TypeString, - Description: "Relative file path within the memory slot.", + Description: "Relative file path within the slot.", }, "max_lines": { Type: TypeInteger, @@ -288,12 +289,12 @@ func (t *MemoryReadTool) ToolPayloadSchema() Schema { Description: "Return only the first N bytes. 0 or omit for no limit.", }, }, - Required: []string{"memory", "path"}, + Required: []string{"slot", "path"}, } } type memoryReadParams struct { - Memory string `json:"memory"` + Slot string `json:"slot"` Path string `json:"path"` MaxLines int `json:"max_lines"` MaxBytes int `json:"max_bytes"` @@ -311,7 +312,7 @@ func (t *MemoryReadTool) Call(ctx context.Context, params string) string { return "Error: path is required" } - absPath, _, err := resolveMemoryPath(t.Store, p.Memory, p.Path) + absPath, _, err := resolveSlotPath(t.Store, p.Slot, p.Path) if err != nil { return "Error: " + err.Error() } @@ -372,20 +373,20 @@ type MemoryCreateTool struct { func (t *MemoryCreateTool) ToolName() string { return "file_create" } func (t *MemoryCreateTool) ToolDescription() string { - return "Create or write to a file in a memory slot. By default, creates a new file (fails if it already exists). Use 'overwrite' to replace an existing file, or 'append' to add to an existing file." + return "Create or write to a file in a slot. By default, creates a new file (fails if it already exists). Use 'overwrite' to replace an existing file, or 'append' to add to an existing file." } func (t *MemoryCreateTool) ToolPayloadSchema() Schema { return Schema{ Type: TypeObject, Properties: PropertyMap{ - "memory": { + "slot": { Type: TypeString, - Description: memoryParamDescription, + Description: slotParamDescription, }, "path": { Type: TypeString, - Description: "Relative file path within the memory slot.", + Description: "Relative file path within the slot.", }, "content": { Type: TypeString, @@ -400,12 +401,12 @@ func (t *MemoryCreateTool) ToolPayloadSchema() Schema { Description: "If true, overwrite the file if it already exists. Ignored when 'append' is true.", }, }, - Required: []string{"memory", "path", "content"}, + Required: []string{"slot", "path", "content"}, } } type memoryCreateParams struct { - Memory string `json:"memory"` + Slot string `json:"slot"` Path string `json:"path"` Content string `json:"content"` Append bool `json:"append"` @@ -422,13 +423,13 @@ func (t *MemoryCreateTool) Call(ctx context.Context, params string) string { return "Error: path is required" } - absPath, writable, err := resolveMemoryPath(t.Store, p.Memory, p.Path) + absPath, writable, err := resolveSlotPath(t.Store, p.Slot, p.Path) if err != nil { return "Error: " + err.Error() } if !writable { - return "Error: memory slot is read-only" + return "Error: slot is read-only" } if p.Append { @@ -475,28 +476,28 @@ type MemoryDeleteTool struct { func (t *MemoryDeleteTool) ToolName() string { return "file_delete" } func (t *MemoryDeleteTool) ToolDescription() string { - return "Delete a file in a memory slot. Only files can be deleted, not directories." + return "Delete a file in a slot. Only files can be deleted, not directories." } func (t *MemoryDeleteTool) ToolPayloadSchema() Schema { return Schema{ Type: TypeObject, Properties: PropertyMap{ - "memory": { + "slot": { Type: TypeString, - Description: memoryParamDescription, + Description: slotParamDescription, }, "path": { Type: TypeString, - Description: "Relative file path within the memory slot.", + Description: "Relative file path within the slot.", }, }, - Required: []string{"memory", "path"}, + Required: []string{"slot", "path"}, } } type memoryDeleteParams struct { - Memory string `json:"memory"` + Slot string `json:"slot"` Path string `json:"path"` } @@ -510,13 +511,13 @@ func (t *MemoryDeleteTool) Call(ctx context.Context, params string) string { return "Error: path is required" } - absPath, writable, err := resolveMemoryPath(t.Store, p.Memory, p.Path) + absPath, writable, err := resolveSlotPath(t.Store, p.Slot, p.Path) if err != nil { return "Error: " + err.Error() } if !writable { - return "Error: memory slot is read-only" + return "Error: slot is read-only" } info, err := os.Stat(absPath) @@ -545,20 +546,20 @@ type MemorySearchTool struct { func (t *MemorySearchTool) ToolName() string { return "file_search" } func (t *MemorySearchTool) ToolDescription() string { - return "Search for files by name within a memory slot using a regex pattern. Returns matching file paths with sizes. Searches recursively by default. Results are paginated (default 50)." + return "Search for files by name within a slot using a regex pattern. Returns matching file paths with sizes. Searches recursively by default. Results are paginated (default 50)." } func (t *MemorySearchTool) ToolPayloadSchema() Schema { return Schema{ Type: TypeObject, Properties: PropertyMap{ - "memory": { + "slot": { Type: TypeString, - Description: memoryParamDescription, + Description: slotParamDescription, }, "path": { Type: TypeString, - Description: "Relative path to search within. Omit to search the memory root.", + Description: "Relative path to search within. Omit to search the slot root.", }, "pattern": { Type: TypeString, @@ -573,12 +574,12 @@ func (t *MemorySearchTool) ToolPayloadSchema() Schema { Description: "Number of results to skip (for pagination). Default 0.", }, }, - Required: []string{"memory", "pattern"}, + Required: []string{"slot", "pattern"}, } } type memorySearchParams struct { - Memory string `json:"memory"` + Slot string `json:"slot"` Path string `json:"path"` Pattern string `json:"pattern"` Limit int `json:"limit"` @@ -609,9 +610,9 @@ func (t *MemorySearchTool) Call(ctx context.Context, params string) string { // Resolve search root var absPath string if p.Path == "" { - absPath, _, err = t.Store.ResolvePath(p.Memory, ".") + absPath, _, err = t.Store.ResolvePath(p.Slot, ".") } else { - absPath, _, err = resolveMemoryPath(t.Store, p.Memory, p.Path) + absPath, _, err = resolveSlotPath(t.Store, p.Slot, p.Path) } if err != nil { return "Error: " + err.Error() @@ -694,20 +695,20 @@ type MemoryGrepTool struct { func (t *MemoryGrepTool) ToolName() string { return "file_grep" } func (t *MemoryGrepTool) ToolDescription() string { - return "Search file contents within a memory slot using a regex pattern. Returns matching lines with file paths and line numbers. Results are paginated (default 50 matches)." + return "Search file contents within a slot using a regex pattern. Returns matching lines with file paths and line numbers. Results are paginated (default 50 matches)." } func (t *MemoryGrepTool) ToolPayloadSchema() Schema { return Schema{ Type: TypeObject, Properties: PropertyMap{ - "memory": { + "slot": { Type: TypeString, - Description: memoryParamDescription, + Description: slotParamDescription, }, "path": { Type: TypeString, - Description: "Relative path to search within. Omit to search the memory root.", + Description: "Relative path to search within. Omit to search the slot root.", }, "pattern": { Type: TypeString, @@ -726,12 +727,12 @@ func (t *MemoryGrepTool) ToolPayloadSchema() Schema { Description: "Number of matches to skip (for pagination). Default 0.", }, }, - Required: []string{"memory", "pattern"}, + Required: []string{"slot", "pattern"}, } } type memoryGrepParams struct { - Memory string `json:"memory"` + Slot string `json:"slot"` Path string `json:"path"` Pattern string `json:"pattern"` Recursive bool `json:"recursive"` @@ -763,9 +764,9 @@ func (t *MemoryGrepTool) Call(ctx context.Context, params string) string { // Resolve search root var absPath string if p.Path == "" { - absPath, _, err = t.Store.ResolvePath(p.Memory, ".") + absPath, _, err = t.Store.ResolvePath(p.Slot, ".") } else { - absPath, _, err = resolveMemoryPath(t.Store, p.Memory, p.Path) + absPath, _, err = resolveSlotPath(t.Store, p.Slot, p.Path) } if err != nil { return "Error: " + err.Error() diff --git a/cmd/engage.go b/cmd/engage.go index cf8a205..ba86e34 100644 --- a/cmd/engage.go +++ b/cmd/engage.go @@ -394,7 +394,7 @@ func runEngage(cmd *cobra.Command, args []string) { // Periodic sweep of expired per-run ephemeral memory directories. // Runs hourly; walks the filesystem so the live config isn't needed. - go runMemoryCleanupLoop(shutdown) + go runScratchpadCleanupLoop(shutdown) // Even without valid config we still try to connect — the command center // can show vars and config files so the user can fix things from the UI. @@ -901,14 +901,14 @@ func openBrowser(url string) { browser.Open(url) } -// runMemoryCleanupLoop periodically sweeps expired per-run ephemeral -// memory directories. The sweep walks the entire memories tree, so it -// doesn't need to know which missions are configured. It runs once -// immediately, then hourly, and exits when shutdown is closed. -func runMemoryCleanupLoop(shutdown <-chan struct{}) { +// runScratchpadCleanupLoop periodically sweeps expired per-run scratchpad +// directories. The sweep walks the entire scratchpads tree, so it doesn't +// need to know which missions are configured. It runs once immediately, +// then hourly, and exits when shutdown is closed. +func runScratchpadCleanupLoop(shutdown <-chan struct{}) { sweep := func() { - if _, err := mission.SweepExpiredEphemeralMemories(); err != nil { - log.Printf("ephemeral memory cleanup: %v", err) + if _, err := mission.SweepExpiredScratchpads(); err != nil { + log.Printf("scratchpad cleanup: %v", err) } } diff --git a/config/config.go b/config/config.go index 378d235..89af238 100644 --- a/config/config.go +++ b/config/config.go @@ -1797,7 +1797,8 @@ func parseMissionBlock(block *hcl.Block, ctx *hcl.EvalContext) (*Mission, error) {Type: "input", LabelNames: []string{"name"}}, // verbose input blocks still supported {Type: "dataset", LabelNames: []string{"name"}}, {Type: "secret", LabelNames: []string{"name"}}, - {Type: "memory"}, // mission-scoped memory: memory { type = "persistent"|"ephemeral" } + {Type: "memory"}, // mission-scoped persistent memory (slot "memory") + {Type: "scratchpad"}, // mission-scoped ephemeral scratchpad (slot "scratchpad") {Type: "schedule"}, {Type: "trigger"}, {Type: "budget"}, @@ -1981,38 +1982,34 @@ func parseMissionBlock(block *hcl.Block, ctx *hcl.EvalContext) (*Mission, error) } } - // Parse zero or more `memory { ... }` blocks. A mission may declare at most - // one persistent and one ephemeral memory. - var persistentMemory, ephemeralMemory *MissionMemory + // Parse the optional `memory { ... }` block (persistent, one per mission) + // and `scratchpad { ... }` block (ephemeral, one per mission). + var missionMemory *MissionMemory + var missionScratchpad *MissionScratchpad for _, mb := range missionContent.Blocks { - if mb.Type != "memory" { - continue - } - var mm MissionMemory - diags := gohcl.DecodeBody(mb.Body, ctx, &mm) - if diags.HasErrors() { - return nil, fmt.Errorf("mission '%s' memory: %w", missionName, diags) - } - if mm.Type == "" { - mm.Type = MemoryTypePersistent - } - switch mm.Type { - case MemoryTypePersistent: - if persistentMemory != nil { - return nil, fmt.Errorf("mission '%s': only one persistent memory block allowed", missionName) + switch mb.Type { + case "memory": + if missionMemory != nil { + return nil, fmt.Errorf("mission '%s': only one memory block allowed", missionName) } - persistentMemory = &mm - case MemoryTypeEphemeral: - if ephemeralMemory != nil { - return nil, fmt.Errorf("mission '%s': only one ephemeral memory block allowed", missionName) + var mm MissionMemory + if diags := gohcl.DecodeBody(mb.Body, ctx, &mm); diags.HasErrors() { + return nil, fmt.Errorf("mission '%s' memory: %w", missionName, diags) } - if mm.Cleanup == nil { - v := DefaultEphemeralCleanupDays - mm.Cleanup = &v + missionMemory = &mm + case "scratchpad": + if missionScratchpad != nil { + return nil, fmt.Errorf("mission '%s': only one scratchpad block allowed", missionName) } - ephemeralMemory = &mm - default: - return nil, fmt.Errorf("mission '%s' memory: type must be %q or %q (got %q)", missionName, MemoryTypePersistent, MemoryTypeEphemeral, mm.Type) + var ms MissionScratchpad + if diags := gohcl.DecodeBody(mb.Body, ctx, &ms); diags.HasErrors() { + return nil, fmt.Errorf("mission '%s' scratchpad: %w", missionName, diags) + } + if ms.Cleanup == nil { + v := DefaultScratchpadCleanupDays + ms.Cleanup = &v + } + missionScratchpad = &ms } } @@ -2081,8 +2078,8 @@ func parseMissionBlock(block *hcl.Block, ctx *hcl.EvalContext) (*Mission, error) Agents: missionAgents, LocalAgents: localAgents, Memories: missionMemories, - PersistentMemory: persistentMemory, - EphemeralMemory: ephemeralMemory, + Memory: missionMemory, + Scratchpad: missionScratchpad, Schedules: schedules, Trigger: trigger, MaxParallel: maxParallel, diff --git a/config/memory.go b/config/memory.go index 228cc41..332bec5 100644 --- a/config/memory.go +++ b/config/memory.go @@ -2,22 +2,16 @@ package config import "fmt" -// Reserved slot names for mission-scoped memory. Tool calls reference these -// via the `memory` parameter (e.g. `memory: "mission"` or `memory: "run"`). +// Reserved slot names for mission-scoped storage. Tool calls reference these +// via the `slot` parameter (e.g. `slot: "memory"` or `slot: "scratchpad"`). const ( - PersistentSlotName = "mission" - EphemeralSlotName = "run" + MemorySlotName = "memory" + ScratchpadSlotName = "scratchpad" ) -// Valid values for the `type` attribute on a mission-scoped `memory` block. -const ( - MemoryTypePersistent = "persistent" - MemoryTypeEphemeral = "ephemeral" -) - -// DefaultEphemeralCleanupDays is the auto-delete window applied to an -// ephemeral mission memory when `cleanup` is not set. -const DefaultEphemeralCleanupDays = 7 +// DefaultScratchpadCleanupDays is the auto-delete window applied to a +// mission's scratchpad when `cleanup` is not set. +const DefaultScratchpadCleanupDays = 7 // Memory describes a top-level shared memory block: // @@ -37,44 +31,47 @@ type Memory struct { Editable bool `hcl:"editable,optional"` } -// Validate enforces naming rules. The literal names "mission" and "run" are -// reserved for the mission-scoped memory slots and must not be reused by a -// shared memory. +// Validate enforces naming rules. The literal names "memory" and "scratchpad" +// are reserved for mission-scoped slots and must not be reused by a shared +// memory. func (m *Memory) Validate() error { - if m.Name == PersistentSlotName || m.Name == EphemeralSlotName { - return fmt.Errorf("name %q is reserved for mission-scoped memory", m.Name) + if m.Name == MemorySlotName || m.Name == ScratchpadSlotName { + return fmt.Errorf("name %q is reserved for mission-scoped slots", m.Name) } return nil } -// MissionMemory describes a `memory { ... }` block inside a mission. A -// mission may declare at most one persistent and one ephemeral memory. +// MissionMemory describes the `memory { ... }` block inside a mission — +// persistent storage that survives across runs. At most one per mission. // -// The storage path is derived by the runtime from the mission name and (for -// ephemeral) the mission instance ID, so no `path` is accepted from HCL. +// Path is derived by the runtime from the mission name, so no `path` is +// accepted from HCL. +type MissionMemory struct { + Description string `hcl:"description,optional"` +} + +// Validate is a no-op today; kept so the type satisfies the same surface as +// MissionScratchpad and so future fields can grow into it. +func (mm *MissionMemory) Validate() error { return nil } + +// MissionScratchpad describes the `scratchpad { ... }` block inside a +// mission — ephemeral per-run working space. At most one per mission. A +// fresh directory is materialized for each mission instance and the cleanup +// sweep deletes ones older than `cleanup` days. +// +// Path is derived by the runtime from the mission name and mission instance +// ID, so no `path` is accepted from HCL. // // Cleanup is a pointer so we can distinguish "user did not set it" (apply -// default of 7 days, only meaningful for ephemeral) from "user set 0" (keep -// forever). -type MissionMemory struct { - Type string `hcl:"type,optional"` // "persistent" (default) or "ephemeral" +// the default) from "user set 0" (keep forever). +type MissionScratchpad struct { Description string `hcl:"description,optional"` - Cleanup *int `hcl:"cleanup,optional"` // ephemeral only; days before auto-delete, 0 = never + Cleanup *int `hcl:"cleanup,optional"` // days before auto-delete; 0 = never } -// Validate normalizes Type (defaulting to persistent), enforces the cleanup -// rules, and rejects unknown type values. -func (mm *MissionMemory) Validate() error { - if mm.Type == "" { - mm.Type = MemoryTypePersistent - } - if mm.Type != MemoryTypePersistent && mm.Type != MemoryTypeEphemeral { - return fmt.Errorf("type must be %q or %q (got %q)", MemoryTypePersistent, MemoryTypeEphemeral, mm.Type) - } - if mm.Type == MemoryTypePersistent && mm.Cleanup != nil { - return fmt.Errorf("cleanup is only valid on ephemeral memory") - } - if mm.Cleanup != nil && *mm.Cleanup < 0 { +// Validate rejects negative cleanup values. +func (ms *MissionScratchpad) Validate() error { + if ms.Cleanup != nil && *ms.Cleanup < 0 { return fmt.Errorf("cleanup must be >= 0 (days)") } return nil diff --git a/config/memory_test.go b/config/memory_test.go index 2832f60..ec3903e 100644 --- a/config/memory_test.go +++ b/config/memory_test.go @@ -7,7 +7,7 @@ import ( . "github.com/onsi/gomega" ) -var _ = Describe("Memory", func() { +var _ = Describe("Memory + Scratchpad", func() { Describe("top-level memory block", func() { It("parses a memory block and exposes it via memories.NAME", func() { @@ -33,15 +33,15 @@ mission "m" { Expect(cfg.Missions[0].Memories).To(ConsistOf("research")) }) - It("rejects the reserved name 'mission'", func() { - m := config.Memory{Name: "mission"} + It("rejects the reserved name 'memory'", func() { + m := config.Memory{Name: "memory"} err := m.Validate() Expect(err).To(HaveOccurred()) Expect(err.Error()).To(ContainSubstring("reserved")) }) - It("rejects the reserved name 'run'", func() { - m := config.Memory{Name: "run"} + It("rejects the reserved name 'scratchpad'", func() { + m := config.Memory{Name: "scratchpad"} err := m.Validate() Expect(err).To(HaveOccurred()) Expect(err.Error()).To(ContainSubstring("reserved")) @@ -82,8 +82,8 @@ mission "m" { }) }) - Describe("mission memory block", func() { - It("parses a persistent memory block (default type)", func() { + Describe("mission memory block (persistent)", func() { + It("parses a memory block on a mission", func() { hcl := fullBaseHCL() + ` mission "m" { commander { model = models.anthropic.claude_sonnet_4 } @@ -97,157 +97,147 @@ mission "m" { _, f := writeFixture("config.hcl", hcl) cfg, err := config.LoadFile(f) Expect(err).NotTo(HaveOccurred()) - Expect(cfg.Missions[0].PersistentMemory).NotTo(BeNil()) - Expect(cfg.Missions[0].PersistentMemory.Type).To(Equal(config.MemoryTypePersistent)) - Expect(cfg.Missions[0].PersistentMemory.Description).To(Equal("Long-term notes")) - Expect(cfg.Missions[0].EphemeralMemory).To(BeNil()) + Expect(cfg.Missions[0].Memory).NotTo(BeNil()) + Expect(cfg.Missions[0].Memory.Description).To(Equal("Long-term notes")) + Expect(cfg.Missions[0].Scratchpad).To(BeNil()) }) - It("parses an explicit persistent type", func() { + It("rejects two memory blocks on the same mission", func() { hcl := fullBaseHCL() + ` mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] - memory { - type = "persistent" - description = "x" - } + memory { description = "a" } + memory { description = "b" } task "t" { objective = "go" } } ` _, f := writeFixture("config.hcl", hcl) - cfg, err := config.LoadFile(f) - Expect(err).NotTo(HaveOccurred()) - Expect(cfg.Missions[0].PersistentMemory).NotTo(BeNil()) + _, err := config.LoadFile(f) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("only one memory block allowed")) }) - It("parses an ephemeral memory block with default cleanup", func() { + It("rejects a `type` attribute (was removed when the block split)", func() { hcl := fullBaseHCL() + ` mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] - memory { - type = "ephemeral" - description = "Scratch" - } + memory { type = "persistent" } task "t" { objective = "go" } } ` _, f := writeFixture("config.hcl", hcl) - cfg, err := config.LoadFile(f) - Expect(err).NotTo(HaveOccurred()) - Expect(cfg.Missions[0].EphemeralMemory).NotTo(BeNil()) - Expect(cfg.Missions[0].EphemeralMemory.Type).To(Equal(config.MemoryTypeEphemeral)) - Expect(cfg.Missions[0].EphemeralMemory.Cleanup).NotTo(BeNil()) - Expect(*cfg.Missions[0].EphemeralMemory.Cleanup).To(Equal(config.DefaultEphemeralCleanupDays)) - Expect(cfg.Missions[0].PersistentMemory).To(BeNil()) + _, err := config.LoadFile(f) + Expect(err).To(HaveOccurred()) }) - It("preserves an explicit cleanup = 0 on ephemeral memory", func() { + It("rejects a `path` attribute on a mission memory block", func() { hcl := fullBaseHCL() + ` mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] - memory { - type = "ephemeral" - cleanup = 0 - } + memory { path = "./x" } task "t" { objective = "go" } } ` _, f := writeFixture("config.hcl", hcl) - cfg, err := config.LoadFile(f) - Expect(err).NotTo(HaveOccurred()) - Expect(cfg.Missions[0].EphemeralMemory.Cleanup).NotTo(BeNil()) - Expect(*cfg.Missions[0].EphemeralMemory.Cleanup).To(Equal(0)) + _, err := config.LoadFile(f) + Expect(err).To(HaveOccurred()) }) + }) - It("allows one persistent + one ephemeral on the same mission", func() { + Describe("mission scratchpad block (ephemeral)", func() { + It("parses a scratchpad block with default cleanup", func() { hcl := fullBaseHCL() + ` mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] - memory { type = "persistent" } - memory { type = "ephemeral" } + scratchpad { + description = "Scratch" + } task "t" { objective = "go" } } ` _, f := writeFixture("config.hcl", hcl) cfg, err := config.LoadFile(f) Expect(err).NotTo(HaveOccurred()) - Expect(cfg.Missions[0].PersistentMemory).NotTo(BeNil()) - Expect(cfg.Missions[0].EphemeralMemory).NotTo(BeNil()) + Expect(cfg.Missions[0].Scratchpad).NotTo(BeNil()) + Expect(cfg.Missions[0].Scratchpad.Description).To(Equal("Scratch")) + Expect(cfg.Missions[0].Scratchpad.Cleanup).NotTo(BeNil()) + Expect(*cfg.Missions[0].Scratchpad.Cleanup).To(Equal(config.DefaultScratchpadCleanupDays)) + Expect(cfg.Missions[0].Memory).To(BeNil()) }) - It("rejects two persistent memory blocks", func() { + It("preserves an explicit cleanup = 0 (keep forever)", func() { hcl := fullBaseHCL() + ` mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] - memory { type = "persistent" } - memory { type = "persistent" } + scratchpad { cleanup = 0 } task "t" { objective = "go" } } ` _, f := writeFixture("config.hcl", hcl) - _, err := config.LoadFile(f) - Expect(err).To(HaveOccurred()) - Expect(err.Error()).To(ContainSubstring("only one persistent memory")) + cfg, err := config.LoadFile(f) + Expect(err).NotTo(HaveOccurred()) + Expect(cfg.Missions[0].Scratchpad.Cleanup).NotTo(BeNil()) + Expect(*cfg.Missions[0].Scratchpad.Cleanup).To(Equal(0)) }) - It("rejects two ephemeral memory blocks", func() { + It("rejects two scratchpad blocks", func() { hcl := fullBaseHCL() + ` mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] - memory { type = "ephemeral" } - memory { type = "ephemeral" } + scratchpad {} + scratchpad {} task "t" { objective = "go" } } ` _, f := writeFixture("config.hcl", hcl) _, err := config.LoadFile(f) Expect(err).To(HaveOccurred()) - Expect(err.Error()).To(ContainSubstring("only one ephemeral memory")) + Expect(err.Error()).To(ContainSubstring("only one scratchpad block allowed")) + }) + + It("rejects a negative cleanup", func() { + neg := -1 + ms := &config.MissionScratchpad{Cleanup: &neg} + Expect(ms.Validate()).To(MatchError(ContainSubstring("cleanup must be >= 0"))) }) - It("rejects an unknown type", func() { + It("rejects a `path` attribute", func() { hcl := fullBaseHCL() + ` mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] - memory { type = "weird" } + scratchpad { path = "./x" } task "t" { objective = "go" } } ` _, f := writeFixture("config.hcl", hcl) _, err := config.LoadFile(f) Expect(err).To(HaveOccurred()) - Expect(err.Error()).To(ContainSubstring(`type must be "persistent" or "ephemeral"`)) - }) - - It("rejects cleanup on persistent memory", func() { - mm := &config.MissionMemory{Type: "persistent", Cleanup: ptrInt(7)} - Expect(mm.Validate()).To(MatchError(ContainSubstring("cleanup is only valid on ephemeral memory"))) - }) - - It("rejects negative cleanup", func() { - mm := &config.MissionMemory{Type: "ephemeral", Cleanup: ptrInt(-1)} - Expect(mm.Validate()).To(MatchError(ContainSubstring("cleanup must be >= 0"))) }) + }) - It("rejects the old `path` attribute on a mission memory block", func() { + Describe("memory + scratchpad on the same mission", func() { + It("allows both", func() { hcl := fullBaseHCL() + ` mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] - memory { path = "./x" } + memory { description = "long-term" } + scratchpad { description = "per-run" } task "t" { objective = "go" } } ` _, f := writeFixture("config.hcl", hcl) - _, err := config.LoadFile(f) - Expect(err).To(HaveOccurred()) + cfg, err := config.LoadFile(f) + Expect(err).NotTo(HaveOccurred()) + Expect(cfg.Missions[0].Memory).NotTo(BeNil()) + Expect(cfg.Missions[0].Scratchpad).NotTo(BeNil()) }) }) @@ -299,5 +289,3 @@ mission "m" { }) }) }) - -func ptrInt(v int) *int { return &v } diff --git a/config/mission.go b/config/mission.go index a946ee4..f556e62 100644 --- a/config/mission.go +++ b/config/mission.go @@ -309,9 +309,9 @@ type Mission struct { Tasks []Task `hcl:"task,block"` Inputs []MissionInput // Parsed from input blocks Datasets []Dataset // Parsed from dataset blocks - Memories []string // Shared memory names referenced by this mission - PersistentMemory *MissionMemory // Optional dedicated mission memory (reserved slot "mission") - EphemeralMemory *MissionMemory // Optional per-run ephemeral memory (reserved slot "run") + Memories []string // Shared memory names referenced by this mission + Memory *MissionMemory // Optional persistent mission memory (reserved slot "memory") + Scratchpad *MissionScratchpad // Optional per-run ephemeral scratchpad (reserved slot "scratchpad") Schedules []Schedule `json:"schedules,omitempty"` Trigger *Trigger `json:"trigger,omitempty"` MaxParallel int `json:"maxParallel,omitempty"` // default 3 @@ -485,23 +485,17 @@ func (w *Mission) Validate(models []Model, agents []Agent, memories []Memory, al } } - // Validate persistent mission memory if present - if w.PersistentMemory != nil { - if err := w.PersistentMemory.Validate(); err != nil { - return fmt.Errorf("persistent memory: %w", err) - } - if w.PersistentMemory.Type != MemoryTypePersistent { - return fmt.Errorf("persistent memory: internal error — wrong type %q", w.PersistentMemory.Type) + // Validate the mission memory block if present. + if w.Memory != nil { + if err := w.Memory.Validate(); err != nil { + return fmt.Errorf("memory: %w", err) } } - // Validate ephemeral mission memory if present - if w.EphemeralMemory != nil { - if err := w.EphemeralMemory.Validate(); err != nil { - return fmt.Errorf("ephemeral memory: %w", err) - } - if w.EphemeralMemory.Type != MemoryTypeEphemeral { - return fmt.Errorf("ephemeral memory: internal error — wrong type %q", w.EphemeralMemory.Type) + // Validate the mission scratchpad block if present. + if w.Scratchpad != nil { + if err := w.Scratchpad.Validate(); err != nil { + return fmt.Errorf("scratchpad: %w", err) } } diff --git a/docs/content/missions/_meta.js b/docs/content/missions/_meta.js index 3b5386f..5ed6486 100644 --- a/docs/content/missions/_meta.js +++ b/docs/content/missions/_meta.js @@ -5,7 +5,7 @@ export default { routing: 'Routing', datasets: 'Datasets', iteration: 'Iteration', - folders: 'Memory', + folders: 'Memory & Scratchpad', 'internal-tools': 'Internal Tools', budgets: 'Budgets', schedules: 'Schedules & Triggers', diff --git a/docs/content/missions/folders.mdx b/docs/content/missions/folders.mdx index ea03e29..d371a16 100644 --- a/docs/content/missions/folders.mdx +++ b/docs/content/missions/folders.mdx @@ -1,24 +1,20 @@ --- -title: Memory +title: Memory & Scratchpad --- -# Memory +# Memory & Scratchpad -Memory blocks are filesystem locations that agents can read from and write to. Squadron manages the storage paths for you — you declare what memory you need, and Squadron decides where on disk it lives, under `/memories/`. +Squadron gives missions two kinds of mission-scoped file storage, plus shared memory that can be referenced across missions: -Three kinds, each with a fixed location pattern: +| Kind | What it's for | Lifetime | Slot name agents use | +|------|---------------|----------|-----------------------| +| **Shared memory** | Reference or output data many missions touch | Persists | the HCL label | +| **Mission memory** | A mission's own long-lived state — archives, accumulated output | Persists | `"memory"` | +| **Mission scratchpad** | Working space for one mission run — intermediate files, drafts | One per run | `"scratchpad"` | -| Kind | HCL | Slot name agents use | On-disk path | -|------|-----|-----------------------|--------------| -| **Shared** | top-level `memory "name" { ... }` | the HCL label | `/memories/shared//` | -| **Persistent** (mission) | `memory { type = "persistent" }` inside a mission | `"mission"` | `/memories/mission//persistent/` | -| **Ephemeral** (per-run) | `memory { type = "ephemeral" }` inside a mission | `"run"` | `/memories/mission//run//` | +Agents reach all three through the same `file_list`, `file_read`, `file_create`, `file_delete`, `file_search`, and `file_grep` tools, naming the slot on each call via the `slot` parameter. -A mission may declare at most one persistent and one ephemeral memory. The names `"mission"` and `"run"` are reserved — a shared `memory "mission"` or `memory "run"` block is rejected. - -Agents reach memory through the `file_list`, `file_read`, `file_create`, `file_delete`, `file_search`, and `file_grep` tools, naming the slot on each call. - -> **No `path` attribute.** Squadron owns memory locations. You don't write `path = "./somewhere"` anywhere — paths are derived from the mission name (and, for ephemeral, the per-run instance ID). +> **No `path` attribute.** Squadron owns the on-disk layout — you declare what storage you need and Squadron picks the path. Everything lives under `/`. ## Shared Memory @@ -46,66 +42,64 @@ Missions opt in by listing them, then refer to them by name in tool calls: ```hcl mission "analyze" { memories = [memories.research, memories.reference] - # agents call e.g. file_read with memory = "research" + # agents call e.g. file_read with slot = "research" } ``` -## Persistent Mission Memory +The names `"memory"` and `"scratchpad"` are reserved — a shared `memory "memory"` or `memory "scratchpad"` block is rejected. + +## Mission Memory -The mission's own dedicated slot. Use it for things you want to keep around: a running log, an archive of finished reports, accumulated state across runs. +The mission's own dedicated long-lived slot. Use it for things you want to keep around: a running log, an archive of finished reports, accumulated state across runs. ```hcl mission "analyze" { memory { - # type defaults to "persistent" when omitted description = "Cumulative analysis output across every run" } } ``` -Agents reach it under the name `mission`: +Agents reach it under the name `memory`: ```json -{ "memory": "mission", "path": "2026-04-23/report.md", "content": "..." } +{ "slot": "memory", "path": "2026-04-23/report.md", "content": "..." } ``` | Attribute | Type | Description | |-----------|------|-------------| -| `type` | string | `"persistent"` (default) or `"ephemeral"` | | `description` | string | Shown to agents alongside the slot name | -## Ephemeral (Per-Run) Memory +## Mission Scratchpad A clean workspace for one mission run. Each invocation gets its own subdirectory, isolated from previous and concurrent runs — perfect for intermediate files, working drafts, or anything you don't want bleeding into the next run. ```hcl mission "analyze" { - memory { - type = "ephemeral" + scratchpad { description = "Scratch space for this run" } } ``` -By default, ephemeral memory is deleted 7 days after the run started. Set `cleanup` to override the window, or `cleanup = 0` to keep it forever. +By default, scratchpads are deleted 7 days after the run started. Set `cleanup` to override the window, or `cleanup = 0` to keep them forever. -Agents reach it under the name `run`: +Agents reach it under the name `scratchpad`: ```json -{ "memory": "run", "path": "notes.txt", "content": "..." } +{ "slot": "scratchpad", "path": "notes.txt", "content": "..." } ``` | Attribute | Type | Description | |-----------|------|-------------| -| `type` | string | Must be `"ephemeral"` | | `description` | string | Shown to agents alongside the slot name | | `cleanup` | integer | Delete the directory this many days after the run started. Defaults to `7`; set `0` to keep forever | -Ephemeral directories stick around after the mission ends — useful for inspecting what an agent produced, and required for `squadron mission --resume ` to find its workspace. `cleanup` is just a max age; nothing is deleted while a run is in flight. +Scratchpads stick around after the mission ends — useful for inspecting what an agent produced, and required for `squadron mission --resume ` to find its workspace. `cleanup` is just a max age; nothing is deleted while a run is in flight. ## Tool Reference -All six file tools take a required `memory` parameter — that's the slot name (a shared memory name, or the reserved `"mission"` / `"run"`): +All six file tools take a required `slot` parameter — that's the slot name (`"memory"`, `"scratchpad"`, or a shared memory's label): | Tool | Purpose | |------|---------| @@ -131,18 +125,16 @@ mission "research" { memories = [memories.reference] memory { - # persistent by default — survives across every run description = "Finished reports, one per run" } - memory { - type = "ephemeral" + scratchpad { description = "Working files for the in-flight run" cleanup = 14 # override the 7-day default } task "gather" { - objective = "Gather sources from the reference memory and save notes to the run memory" + objective = "Gather sources from the reference memory and save notes to the scratchpad" } task "publish" { diff --git a/docs/content/missions/overview.mdx b/docs/content/missions/overview.mdx index 4bbe4c4..4a60b4d 100644 --- a/docs/content/missions/overview.mdx +++ b/docs/content/missions/overview.mdx @@ -43,8 +43,9 @@ mission "data_pipeline" { | `input` | block | Mission input parameters (repeatable) | | `task` | block | Task definitions (repeatable) | | `dataset` | block | Dataset definitions (optional) | -| `memories` | list | Shared memory references, e.g. `[memories.data]` (see [Memory](/missions/folders)) | -| `memory` | block | Mission-scoped memory; `type = "persistent"` (default) registers as `"mission"`, `type = "ephemeral"` registers as `"run"`. At most one of each per mission. | +| `memories` | list | Shared memory references, e.g. `[memories.data]` (see [Memory & Scratchpad](/missions/folders)) | +| `memory` | block | Mission-scoped persistent memory (slot `"memory"`). At most one per mission. | +| `scratchpad` | block | Mission-scoped ephemeral per-run scratchpad (slot `"scratchpad"`). At most one per mission. | | `schedule` | block | Automatic run schedules (optional, repeatable) | | `trigger` | block | Webhook trigger (optional) | | `max_parallel` | number | Max concurrent instances (default: 3) | diff --git a/mission/memory_store.go b/mission/memory_store.go index 91353a1..2e190b7 100644 --- a/mission/memory_store.go +++ b/mission/memory_store.go @@ -14,16 +14,18 @@ import ( "squadron/internal/paths" ) -// memoriesSubdir is the directory under SquadronHome that holds every -// materialized memory slot. Layout: +// On-disk layout under SquadronHome: // -// /memories/shared// -// /memories/mission//persistent/ -// /memories/mission//run// -const memoriesSubdir = "memories" +// /memories/shared// — shared memory +// /memories/mission// — mission memory +// /scratchpads/// — mission scratchpad +const ( + memoriesSubdir = "memories" + scratchpadSubdir = "scratchpads" +) -// runMetadataFile is the sidecar written inside each materialized ephemeral -// memory directory so the cleanup sweep can tell when it was created. +// runMetadataFile is the sidecar written inside each materialized scratchpad +// directory so the cleanup sweep can tell when it was created. const runMetadataFile = ".squadron-run.json" type runMetadata struct { @@ -34,8 +36,7 @@ type runMetadata struct { } // MemoriesRoot returns `/memories`, the parent of every -// materialized memory slot. Exported so callers like the cleanup loop can -// pivot off the same base. +// materialized memory slot (shared + per-mission). func MemoriesRoot() (string, error) { home, err := paths.SquadronHome() if err != nil { @@ -44,57 +45,55 @@ func MemoriesRoot() (string, error) { return filepath.Join(home, memoriesSubdir), nil } -// SharedMemoryPath returns the on-disk path for a top-level shared memory -// named `name`. -func SharedMemoryPath(name string) (string, error) { - root, err := MemoriesRoot() +// ScratchpadsRoot returns `/scratchpads`, the parent of every +// materialized per-run scratchpad. Used by the cleanup sweep. +func ScratchpadsRoot() (string, error) { + home, err := paths.SquadronHome() if err != nil { return "", err } - return filepath.Join(root, "shared", name), nil + return filepath.Join(home, scratchpadSubdir), nil } -// PersistentMemoryPath returns the on-disk path for a mission's persistent -// memory slot. Path is stable across runs of the same mission. -func PersistentMemoryPath(missionName string) (string, error) { +// SharedMemoryPath returns the on-disk path for a top-level shared memory +// named `name`. +func SharedMemoryPath(name string) (string, error) { root, err := MemoriesRoot() if err != nil { return "", err } - return filepath.Join(root, "mission", missionName, "persistent"), nil + return filepath.Join(root, "shared", name), nil } -// EphemeralMemoryPath returns the on-disk path for one run's ephemeral -// memory slot. Path is unique per mission run instance. -func EphemeralMemoryPath(missionName, missionInstanceID string) (string, error) { +// MissionMemoryPath returns the on-disk path for a mission's persistent +// memory slot. Stable across runs of the same mission. +func MissionMemoryPath(missionName string) (string, error) { root, err := MemoriesRoot() if err != nil { return "", err } - return filepath.Join(root, "mission", missionName, "run", missionInstanceID), nil + return filepath.Join(root, "mission", missionName), nil } -// MissionRunRoot returns `/mission//run`, the parent -// dir of every ephemeral memory directory for a given mission. Used by the -// cleanup sweep. -func MissionRunRoot(missionName string) (string, error) { - root, err := MemoriesRoot() +// MissionScratchpadPath returns the on-disk path for one run's scratchpad. +// Unique per mission run instance. +func MissionScratchpadPath(missionName, missionInstanceID string) (string, error) { + root, err := ScratchpadsRoot() if err != nil { return "", err } - return filepath.Join(root, "mission", missionName, "run"), nil + return filepath.Join(root, missionName, missionInstanceID), nil } -// resolvedEphemeralCleanup returns the cleanup window in days for an -// ephemeral mission memory. Reads the parsed pointer when set (Validate / -// the config parser fill in the default at config load time); falls back to -// the default for callers that hand-build a struct and skip Validate -// (notably tests). -func resolvedEphemeralCleanup(mm *config.MissionMemory) int { - if mm == nil || mm.Cleanup == nil { - return config.DefaultEphemeralCleanupDays +// resolvedScratchpadCleanup returns the cleanup window in days for a +// mission scratchpad. Reads the parsed pointer when set (the config parser +// fills in the default at load time); falls back to the default for callers +// that hand-build a struct and skip parsing (notably tests). +func resolvedScratchpadCleanup(ms *config.MissionScratchpad) int { + if ms == nil || ms.Cleanup == nil { + return config.DefaultScratchpadCleanupDays } - return *mm.Cleanup + return *ms.Cleanup } type missionMemoryStore struct { @@ -108,9 +107,9 @@ type memorySlot struct { } // buildMemoryStore creates an aitools.MemoryStore from the mission config and -// the declared top-level memories. missionInstanceID scopes the ephemeral -// memory path; it must be non-empty when mission.EphemeralMemory is set. -// Returns nil if no memory slots are configured. +// the declared top-level memories. missionInstanceID scopes the scratchpad +// path; it must be non-empty when mission.Scratchpad is set. Returns nil if +// no slots are configured. func buildMemoryStore(mission *config.Mission, memories []config.Memory, missionInstanceID string) (aitools.MemoryStore, error) { store := &missionMemoryStore{ slots: make(map[string]*memorySlot), @@ -122,8 +121,8 @@ func buildMemoryStore(mission *config.Mission, memories []config.Memory, mission } for _, name := range mission.Memories { - if name == config.PersistentSlotName || name == config.EphemeralSlotName { - return nil, fmt.Errorf("shared memory %q uses a reserved name", name) + if name == config.MemorySlotName || name == config.ScratchpadSlotName { + return nil, fmt.Errorf("shared memory %q uses a reserved slot name", name) } mem, ok := memByName[name] if !ok { @@ -147,38 +146,38 @@ func buildMemoryStore(mission *config.Mission, memories []config.Memory, mission } } - if mission.PersistentMemory != nil { - absPath, err := PersistentMemoryPath(mission.Name) + if mission.Memory != nil { + absPath, err := MissionMemoryPath(mission.Name) if err != nil { - return nil, fmt.Errorf("persistent memory: resolve path: %w", err) + return nil, fmt.Errorf("memory: resolve path: %w", err) } if err := os.MkdirAll(absPath, 0755); err != nil { - return nil, fmt.Errorf("persistent memory: create directory: %w", err) + return nil, fmt.Errorf("memory: create directory: %w", err) } - store.slots[config.PersistentSlotName] = &memorySlot{ + store.slots[config.MemorySlotName] = &memorySlot{ absPath: absPath, - description: mission.PersistentMemory.Description, + description: mission.Memory.Description, writable: true, } } - if mission.EphemeralMemory != nil { + if mission.Scratchpad != nil { if missionInstanceID == "" { - return nil, fmt.Errorf("ephemeral memory requires a mission instance ID") + return nil, fmt.Errorf("scratchpad requires a mission instance ID") } - absPath, err := EphemeralMemoryPath(mission.Name, missionInstanceID) + absPath, err := MissionScratchpadPath(mission.Name, missionInstanceID) if err != nil { - return nil, fmt.Errorf("ephemeral memory: resolve path: %w", err) + return nil, fmt.Errorf("scratchpad: resolve path: %w", err) } if err := os.MkdirAll(absPath, 0755); err != nil { - return nil, fmt.Errorf("ephemeral memory: create directory: %w", err) + return nil, fmt.Errorf("scratchpad: create directory: %w", err) } - if err := writeRunMetadata(absPath, mission.Name, missionInstanceID, resolvedEphemeralCleanup(mission.EphemeralMemory)); err != nil { - return nil, fmt.Errorf("ephemeral memory: write metadata: %w", err) + if err := writeRunMetadata(absPath, mission.Name, missionInstanceID, resolvedScratchpadCleanup(mission.Scratchpad)); err != nil { + return nil, fmt.Errorf("scratchpad: write metadata: %w", err) } - store.slots[config.EphemeralSlotName] = &memorySlot{ + store.slots[config.ScratchpadSlotName] = &memorySlot{ absPath: absPath, - description: mission.EphemeralMemory.Description, + description: mission.Scratchpad.Description, writable: true, } } @@ -190,10 +189,10 @@ func buildMemoryStore(mission *config.Mission, memories []config.Memory, mission return store, nil } -// writeRunMetadata records when the ephemeral memory directory was created -// so the sweep can decide when to delete it. Uses O_CREATE|O_EXCL so -// concurrent starts and resumes never clobber the original timestamp — -// exactly one writer wins, others observe EEXIST and skip. +// writeRunMetadata records when the scratchpad directory was created so the +// sweep can decide when to delete it. Uses O_CREATE|O_EXCL so concurrent +// starts and resumes never clobber the original timestamp — exactly one +// writer wins, others observe EEXIST and skip. func writeRunMetadata(dir, missionName, missionID string, cleanupDays int) error { path := filepath.Join(dir, runMetadataFile) f, err := os.OpenFile(path, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0644) @@ -218,14 +217,14 @@ func writeRunMetadata(dir, missionName, missionID string, cleanupDays int) error return err } -func (s *missionMemoryStore) ResolvePath(memoryName string, relPath string) (string, bool, error) { - if memoryName == "" { - return "", false, fmt.Errorf("memory name is required (available: %v)", s.availableNames()) +func (s *missionMemoryStore) ResolvePath(slotName string, relPath string) (string, bool, error) { + if slotName == "" { + return "", false, fmt.Errorf("slot name is required (available: %v)", s.availableNames()) } - entry, ok := s.slots[memoryName] + entry, ok := s.slots[slotName] if !ok { - return "", false, fmt.Errorf("memory %q not found. Available: %v", memoryName, s.availableNames()) + return "", false, fmt.Errorf("slot %q not found. Available: %v", slotName, s.availableNames()) } cleaned := filepath.Clean(relPath) @@ -235,7 +234,7 @@ func (s *missionMemoryStore) ResolvePath(memoryName string, relPath string) (str fullPath := filepath.Join(entry.absPath, cleaned) if !strings.HasPrefix(fullPath, entry.absPath) { - return "", false, fmt.Errorf("path escapes memory root") + return "", false, fmt.Errorf("path escapes slot root") } return fullPath, entry.writable, nil @@ -261,21 +260,20 @@ func (s *missionMemoryStore) MemoryInfos() []aitools.MemoryInfo { return infos } -// SweepExpiredEphemeralMemories deletes any per-run ephemeral memory -// directory whose sidecar (.squadron-run.json) records a created_at older -// than its cleanup_days. Directories without a sidecar, or with -// cleanup_days == 0, are left alone. +// SweepExpiredScratchpads deletes any per-run scratchpad directory whose +// sidecar (.squadron-run.json) records a created_at older than its +// cleanup_days. Directories without a sidecar, or with cleanup_days == 0, +// are left alone. // -// It walks `/mission/*/run/*` and considers every -// per-run directory it finds — there's no per-mission filtering, so callers -// don't need to know which missions exist. -func SweepExpiredEphemeralMemories() (removed []string, err error) { - root, err := MemoriesRoot() +// Walks `/scratchpads/*/*` and considers every per-run +// directory — no per-mission filtering, so callers don't need to know which +// missions exist. +func SweepExpiredScratchpads() (removed []string, err error) { + root, err := ScratchpadsRoot() if err != nil { return nil, err } - missionsBase := filepath.Join(root, "mission") - entries, err := os.ReadDir(missionsBase) + entries, err := os.ReadDir(root) if err != nil { if os.IsNotExist(err) { return nil, nil @@ -287,7 +285,7 @@ func SweepExpiredEphemeralMemories() (removed []string, err error) { if !missionEntry.IsDir() { continue } - runBase := filepath.Join(missionsBase, missionEntry.Name(), "run") + runBase := filepath.Join(root, missionEntry.Name()) runEntries, err := os.ReadDir(runBase) if err != nil { if os.IsNotExist(err) { diff --git a/mission/memory_store_test.go b/mission/memory_store_test.go index 03a38a6..5e606d1 100644 --- a/mission/memory_store_test.go +++ b/mission/memory_store_test.go @@ -25,7 +25,7 @@ func withTempHome(t *testing.T) string { return home } -func TestBuildMemoryStore_NoMemories(t *testing.T) { +func TestBuildMemoryStore_NoSlots(t *testing.T) { withTempHome(t) m := &config.Mission{Name: "m"} store, err := buildMemoryStore(m, nil, "mid-1") @@ -33,17 +33,16 @@ func TestBuildMemoryStore_NoMemories(t *testing.T) { t.Fatalf("unexpected error: %v", err) } if store != nil { - t.Fatalf("expected nil store when no memories configured, got %+v", store) + t.Fatalf("expected nil store when no slots configured, got %+v", store) } } -func TestBuildMemoryStore_PersistentMemory(t *testing.T) { +func TestBuildMemoryStore_MissionMemory(t *testing.T) { home := withTempHome(t) m := &config.Mission{ Name: "m", - PersistentMemory: &config.MissionMemory{ - Type: "persistent", + Memory: &config.MissionMemory{ Description: "persistent", }, } @@ -56,19 +55,19 @@ func TestBuildMemoryStore_PersistentMemory(t *testing.T) { t.Fatal("expected non-nil store") } - abs, writable, err := store.ResolvePath(aitools.PersistentMemoryName, ".") + abs, writable, err := store.ResolvePath(aitools.MemorySlotName, ".") if err != nil { t.Fatalf("ResolvePath: %v", err) } if !writable { - t.Fatal("persistent memory must be writable") + t.Fatal("mission memory must be writable") } - want := filepath.Join(home, "memories", "mission", "m", "persistent") + want := filepath.Join(home, "memories", "mission", "m") if abs != want { - t.Fatalf("persistent path: want %s, got %s", want, abs) + t.Fatalf("mission memory path: want %s, got %s", want, abs) } if info, err := os.Stat(abs); err != nil || !info.IsDir() { - t.Fatalf("persistent directory not created at %s: %v", abs, err) + t.Fatalf("mission memory directory not created at %s: %v", abs, err) } // The mission name is NOT a valid slot key — prevents regression. @@ -77,13 +76,12 @@ func TestBuildMemoryStore_PersistentMemory(t *testing.T) { } } -func TestBuildMemoryStore_EphemeralMemory_CreatesSidecar(t *testing.T) { +func TestBuildMemoryStore_Scratchpad_CreatesSidecar(t *testing.T) { home := withTempHome(t) cleanup := 7 m := &config.Mission{ Name: "m", - EphemeralMemory: &config.MissionMemory{ - Type: "ephemeral", + Scratchpad: &config.MissionScratchpad{ Cleanup: &cleanup, }, } @@ -93,16 +91,16 @@ func TestBuildMemoryStore_EphemeralMemory_CreatesSidecar(t *testing.T) { t.Fatalf("unexpected error: %v", err) } - abs, writable, err := store.ResolvePath(aitools.EphemeralMemoryName, ".") + abs, writable, err := store.ResolvePath(aitools.ScratchpadSlotName, ".") if err != nil { t.Fatalf("ResolvePath: %v", err) } if !writable { - t.Fatal("ephemeral memory must be writable") + t.Fatal("scratchpad must be writable") } - want := filepath.Join(home, "memories", "mission", "m", "run", "mid-abc") + want := filepath.Join(home, "scratchpads", "m", "mid-abc") if abs != want { - t.Fatalf("ephemeral path: want %s, got %s", want, abs) + t.Fatalf("scratchpad path: want %s, got %s", want, abs) } metaBytes, err := os.ReadFile(filepath.Join(abs, runMetadataFile)) @@ -124,20 +122,18 @@ func TestBuildMemoryStore_EphemeralMemory_CreatesSidecar(t *testing.T) { } } -func TestBuildMemoryStore_EphemeralMemory_SidecarPreservedOnResume(t *testing.T) { +func TestBuildMemoryStore_Scratchpad_SidecarPreservedOnResume(t *testing.T) { home := withTempHome(t) m := &config.Mission{ - Name: "m", - EphemeralMemory: &config.MissionMemory{ - Type: "ephemeral", - }, + Name: "m", + Scratchpad: &config.MissionScratchpad{}, } // First build: sidecar written if _, err := buildMemoryStore(m, nil, "mid-1"); err != nil { t.Fatalf("first build: %v", err) } - runDir := filepath.Join(home, "memories", "mission", "m", "run", "mid-1") + runDir := filepath.Join(home, "scratchpads", "m", "mid-1") firstMetaBytes, _ := os.ReadFile(filepath.Join(runDir, runMetadataFile)) var first runMetadata _ = json.Unmarshal(firstMetaBytes, &first) @@ -157,13 +153,11 @@ func TestBuildMemoryStore_EphemeralMemory_SidecarPreservedOnResume(t *testing.T) } } -func TestBuildMemoryStore_EphemeralMemory_RequiresMissionID(t *testing.T) { +func TestBuildMemoryStore_Scratchpad_RequiresMissionID(t *testing.T) { withTempHome(t) m := &config.Mission{ - Name: "m", - EphemeralMemory: &config.MissionMemory{ - Type: "ephemeral", - }, + Name: "m", + Scratchpad: &config.MissionScratchpad{}, } _, err := buildMemoryStore(m, nil, "") if err == nil { @@ -173,14 +167,12 @@ func TestBuildMemoryStore_EphemeralMemory_RequiresMissionID(t *testing.T) { func TestBuildMemoryStore_RejectsReservedSharedMemoryNames(t *testing.T) { withTempHome(t) - for _, reserved := range []string{"mission", "run"} { + for _, reserved := range []string{"memory", "scratchpad"} { m := &config.Mission{ Name: "m", Memories: []string{reserved}, } - mems := []config.Memory{ - {Name: reserved}, - } + mems := []config.Memory{{Name: reserved}} _, err := buildMemoryStore(m, mems, "mid-1") if err == nil { t.Fatalf("expected error for reserved shared memory name %q", reserved) @@ -188,26 +180,22 @@ func TestBuildMemoryStore_RejectsReservedSharedMemoryNames(t *testing.T) { } } -func TestBuildMemoryStore_BothPersistentAndEphemeral(t *testing.T) { +func TestBuildMemoryStore_BothMemoryAndScratchpad(t *testing.T) { withTempHome(t) m := &config.Mission{ - Name: "m", - PersistentMemory: &config.MissionMemory{ - Type: "persistent", - }, - EphemeralMemory: &config.MissionMemory{ - Type: "ephemeral", - }, + Name: "m", + Memory: &config.MissionMemory{}, + Scratchpad: &config.MissionScratchpad{}, } store, err := buildMemoryStore(m, nil, "mid-1") if err != nil { t.Fatalf("unexpected error: %v", err) } - if _, _, err := store.ResolvePath(aitools.PersistentMemoryName, "."); err != nil { - t.Fatalf("persistent should resolve: %v", err) + if _, _, err := store.ResolvePath(aitools.MemorySlotName, "."); err != nil { + t.Fatalf("memory slot should resolve: %v", err) } - if _, _, err := store.ResolvePath(aitools.EphemeralMemoryName, "."); err != nil { - t.Fatalf("ephemeral should resolve: %v", err) + if _, _, err := store.ResolvePath(aitools.ScratchpadSlotName, "."); err != nil { + t.Fatalf("scratchpad slot should resolve: %v", err) } } @@ -265,59 +253,59 @@ func TestBuildMemoryStore_SharedMemory_ReadOnly(t *testing.T) { } } -func TestResolvePath_EmptyMemoryNameRejected(t *testing.T) { +func TestResolvePath_EmptySlotNameRejected(t *testing.T) { withTempHome(t) m := &config.Mission{ - Name: "m", - PersistentMemory: &config.MissionMemory{Type: "persistent"}, + Name: "m", + Memory: &config.MissionMemory{}, } store, err := buildMemoryStore(m, nil, "mid-1") if err != nil { t.Fatalf("build: %v", err) } if _, _, err := store.ResolvePath("", "."); err == nil { - t.Fatal("expected error when memory name is empty") + t.Fatal("expected error when slot name is empty") } } func TestResolvePath_RejectsPathEscape(t *testing.T) { withTempHome(t) m := &config.Mission{ - Name: "m", - PersistentMemory: &config.MissionMemory{Type: "persistent"}, + Name: "m", + Memory: &config.MissionMemory{}, } store, err := buildMemoryStore(m, nil, "mid-1") if err != nil { t.Fatalf("build: %v", err) } - if _, _, err := store.ResolvePath("mission", "../outside"); err == nil { + if _, _, err := store.ResolvePath("memory", "../outside"); err == nil { t.Fatal("expected path-escape error") } } -func TestResolvePath_UnknownMemory(t *testing.T) { +func TestResolvePath_UnknownSlot(t *testing.T) { withTempHome(t) m := &config.Mission{ - Name: "m", - PersistentMemory: &config.MissionMemory{Type: "persistent"}, + Name: "m", + Memory: &config.MissionMemory{}, } store, err := buildMemoryStore(m, nil, "mid-1") if err != nil { t.Fatalf("build: %v", err) } if _, _, err := store.ResolvePath("does_not_exist", "."); err == nil { - t.Fatal("expected error for unknown memory") + t.Fatal("expected error for unknown slot") } } -// --- SweepExpiredEphemeralMemories ---------------------------------------- +// --- SweepExpiredScratchpads ---------------------------------------------- -// writeEphemeral builds a fake per-run ephemeral memory directory under -// /memories/mission//run/, with a sidecar recording -// the given created_at and cleanup_days. -func writeEphemeral(t *testing.T, home, missionName, runID string, createdAt time.Time, cleanupDays int) string { +// writeScratchpad builds a fake per-run scratchpad directory under +// /scratchpads//, with a sidecar recording the given +// created_at and cleanup_days. +func writeScratchpad(t *testing.T, home, missionName, runID string, createdAt time.Time, cleanupDays int) string { t.Helper() - dir := filepath.Join(home, "memories", "mission", missionName, "run", runID) + dir := filepath.Join(home, "scratchpads", missionName, runID) if err := os.MkdirAll(dir, 0755); err != nil { t.Fatal(err) } @@ -336,9 +324,9 @@ func writeEphemeral(t *testing.T, home, missionName, runID string, createdAt tim func TestSweep_DeletesExpired(t *testing.T) { home := withTempHome(t) - expired := writeEphemeral(t, home, "m", "old", time.Now().Add(-8*24*time.Hour), 7) + expired := writeScratchpad(t, home, "m", "old", time.Now().Add(-8*24*time.Hour), 7) - removed, err := SweepExpiredEphemeralMemories() + removed, err := SweepExpiredScratchpads() if err != nil { t.Fatalf("sweep: %v", err) } @@ -352,9 +340,9 @@ func TestSweep_DeletesExpired(t *testing.T) { func TestSweep_KeepsUnexpired(t *testing.T) { home := withTempHome(t) - fresh := writeEphemeral(t, home, "m", "new", time.Now().Add(-2*24*time.Hour), 7) + fresh := writeScratchpad(t, home, "m", "new", time.Now().Add(-2*24*time.Hour), 7) - removed, err := SweepExpiredEphemeralMemories() + removed, err := SweepExpiredScratchpads() if err != nil { t.Fatalf("sweep: %v", err) } @@ -368,9 +356,9 @@ func TestSweep_KeepsUnexpired(t *testing.T) { func TestSweep_IgnoresZeroCleanup(t *testing.T) { home := withTempHome(t) - keep := writeEphemeral(t, home, "m", "forever", time.Now().Add(-365*24*time.Hour), 0) + keep := writeScratchpad(t, home, "m", "forever", time.Now().Add(-365*24*time.Hour), 0) - removed, err := SweepExpiredEphemeralMemories() + removed, err := SweepExpiredScratchpads() if err != nil { t.Fatalf("sweep: %v", err) } @@ -384,12 +372,12 @@ func TestSweep_IgnoresZeroCleanup(t *testing.T) { func TestSweep_IgnoresDirectoriesWithoutSidecar(t *testing.T) { home := withTempHome(t) - manual := filepath.Join(home, "memories", "mission", "m", "run", "hand_made") + manual := filepath.Join(home, "scratchpads", "m", "hand_made") if err := os.MkdirAll(manual, 0755); err != nil { t.Fatal(err) } - removed, err := SweepExpiredEphemeralMemories() + removed, err := SweepExpiredScratchpads() if err != nil { t.Fatalf("sweep: %v", err) } @@ -402,8 +390,8 @@ func TestSweep_IgnoresDirectoriesWithoutSidecar(t *testing.T) { } func TestSweep_MissingRootIsNotAnError(t *testing.T) { - withTempHome(t) // home is set, but memories/ subtree doesn't exist yet - removed, err := SweepExpiredEphemeralMemories() + withTempHome(t) // home is set, but scratchpads/ subtree doesn't exist yet + removed, err := SweepExpiredScratchpads() if err != nil { t.Fatalf("sweep should tolerate missing root: %v", err) } @@ -414,12 +402,11 @@ func TestSweep_MissingRootIsNotAnError(t *testing.T) { func TestSweep_WalksAcrossMissions(t *testing.T) { home := withTempHome(t) - // Two missions, one expired run each - a := writeEphemeral(t, home, "alpha", "run1", time.Now().Add(-10*24*time.Hour), 2) - b := writeEphemeral(t, home, "beta", "run1", time.Now().Add(-10*24*time.Hour), 2) - keep := writeEphemeral(t, home, "alpha", "run2", time.Now().Add(-1*24*time.Hour), 2) + a := writeScratchpad(t, home, "alpha", "run1", time.Now().Add(-10*24*time.Hour), 2) + b := writeScratchpad(t, home, "beta", "run1", time.Now().Add(-10*24*time.Hour), 2) + keep := writeScratchpad(t, home, "alpha", "run2", time.Now().Add(-1*24*time.Hour), 2) - removed, err := SweepExpiredEphemeralMemories() + removed, err := SweepExpiredScratchpads() if err != nil { t.Fatalf("sweep: %v", err) } @@ -436,26 +423,25 @@ func TestSweep_WalksAcrossMissions(t *testing.T) { } } -// TestSweepThenRebuildRoundTrip mirrors the real flow: an old ephemeral -// memory exists with a sidecar backdated past its cleanup window, the sweep -// deletes it, then a new buildMemoryStore (different missionID) creates a -// fresh ephemeral memory with a current sidecar — and the old one is gone. +// TestSweepThenRebuildRoundTrip mirrors the real flow: an old scratchpad +// exists with a sidecar backdated past its cleanup window, the sweep deletes +// it, then a new buildMemoryStore (different missionID) creates a fresh +// scratchpad with a current sidecar — and the old one is gone. func TestSweepThenRebuildRoundTrip(t *testing.T) { home := withTempHome(t) - stale := writeEphemeral(t, home, "demo", "old-run", time.Now().Add(-5*24*time.Hour), 2) + stale := writeScratchpad(t, home, "demo", "old-run", time.Now().Add(-5*24*time.Hour), 2) - if _, err := SweepExpiredEphemeralMemories(); err != nil { + if _, err := SweepExpiredScratchpads(); err != nil { t.Fatalf("sweep: %v", err) } if _, err := os.Stat(stale); !os.IsNotExist(err) { - t.Fatalf("stale ephemeral should have been deleted: %v", err) + t.Fatalf("stale scratchpad should have been deleted: %v", err) } cleanup := 2 m := &config.Mission{ Name: "demo", - EphemeralMemory: &config.MissionMemory{ - Type: "ephemeral", + Scratchpad: &config.MissionScratchpad{ Cleanup: &cleanup, }, } @@ -463,11 +449,11 @@ func TestSweepThenRebuildRoundTrip(t *testing.T) { if err != nil { t.Fatalf("build: %v", err) } - fresh, _, err := store.ResolvePath(aitools.EphemeralMemoryName, ".") + fresh, _, err := store.ResolvePath(aitools.ScratchpadSlotName, ".") if err != nil { t.Fatalf("resolve: %v", err) } - want := filepath.Join(home, "memories", "mission", "demo", "run", "new-run") + want := filepath.Join(home, "scratchpads", "demo", "new-run") if fresh != want { t.Fatalf("fresh path: want %s, got %s", want, fresh) } diff --git a/mission/runner.go b/mission/runner.go index 23b2d14..863292c 100644 --- a/mission/runner.go +++ b/mission/runner.go @@ -289,9 +289,9 @@ func NewRunner(cfg *config.Config, configPath string, missionName string, inputs } // Memory store is built later in Run() once missionID is known — the - // ephemeral memory path depends on it. Shared + persistent memory alone - // would let us build here, but deferring is simpler than branching on - // mission.EphemeralMemory. + // scratchpad path depends on it. Shared + mission memory alone would let + // us build here, but deferring is simpler than branching on + // mission.Scratchpad. return r, nil } @@ -568,12 +568,11 @@ func (r *Runner) Run(ctx context.Context, streamer streamers.MissionHandler) err r.resolvedDatasets = nil } - // Memory store depends on missionID (for ephemeral memory path), so build - // it here rather than in NewRunner. Sweep expired ephemeral memories - // async — the result doesn't affect this run's correctness, only disk - // usage. - if r.mission.EphemeralMemory != nil { - go func() { _, _ = SweepExpiredEphemeralMemories() }() + // Memory store depends on missionID (for the scratchpad path), so build + // it here rather than in NewRunner. Sweep expired scratchpads async — + // the result doesn't affect this run's correctness, only disk usage. + if r.mission.Scratchpad != nil { + go func() { _, _ = SweepExpiredScratchpads() }() } memoryStore, err := buildMemoryStore(r.mission, r.cfg.Memories, missionID) if err != nil { diff --git a/wsbridge/memory_browser.go b/wsbridge/memory_browser.go index 33a53f0..42f7132 100644 --- a/wsbridge/memory_browser.go +++ b/wsbridge/memory_browser.go @@ -48,12 +48,12 @@ func (c *Client) resolveMemoryPath(memoryName, relPath string) (*resolvedMemory, } } - // Check mission persistent memory (keyed by mission name). + // Check the mission's persistent memory (keyed by mission name). for _, m := range cfg.Missions { - if m.PersistentMemory != nil && m.Name == memoryName { - absPath, err := mission.PersistentMemoryPath(m.Name) + if m.Memory != nil && m.Name == memoryName { + absPath, err := mission.MissionMemoryPath(m.Name) if err != nil { - return nil, "", fmt.Errorf("resolve persistent memory for %q: %w", m.Name, err) + return nil, "", fmt.Errorf("resolve mission memory for %q: %w", m.Name, err) } rm := &resolvedMemory{name: m.Name, path: absPath, editable: true} path, err := c.resolveSafePath(absPath, relPath) @@ -135,18 +135,18 @@ func collectMemoryInfos(cfg *config.Config) ([]protocol.SharedFolderInfo, error) } for _, m := range cfg.Missions { - if m.PersistentMemory == nil { + if m.Memory == nil { continue } - path, err := mission.PersistentMemoryPath(m.Name) + path, err := mission.MissionMemoryPath(m.Name) if err != nil { - return nil, fmt.Errorf("persistent memory for %q: %w", m.Name, err) + return nil, fmt.Errorf("mission memory for %q: %w", m.Name, err) } folders = append(folders, protocol.SharedFolderInfo{ Name: m.Name, Path: path, Label: m.Name, - Description: m.PersistentMemory.Description, + Description: m.Memory.Description, Editable: true, IsShared: false, Missions: []string{m.Name}, From 016375ec0b15138aaac803f56e25142285f826bb Mon Sep 17 00:00:00 2001 From: Max Lund Date: Tue, 26 May 2026 22:59:29 -0500 Subject: [PATCH 6/8] Simplify: scratchpad becomes a bool; all memories editable; description required MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit DSL changes: - scratchpad is now a boolean attribute on the mission: mission "x" { scratchpad = true } No block, no fields. Auto-deletes 7 days after the run starts; cleanup window is fixed (config.ScratchpadCleanupDays = 7). - Both the top-level `memory "name" { ... }` and the mission-scoped `memory { ... }` blocks now take the same single field: a required `description`. The `label` and `editable` attributes are gone — all memories are writable. - A mission with no `memory { ... }` block has no "memory" slot; agents only see the shared memories it lists (plus the scratchpad, if enabled). Internal changes that follow: - aitools.MemoryStore.ResolvePath drops the writable bool return: ResolvePath(slot, relPath) (string, error) aitools.MemoryInfo loses the Writable field. The "slot is read-only" error paths in file_create and file_delete are gone, along with the same check in wsbridge's file-browser writer. - config.MissionScratchpad type is removed. Mission.Scratchpad is now `bool` instead of `*MissionScratchpad`. - config.DefaultScratchpadCleanupDays renamed to config.ScratchpadCleanupDays (no longer "default" — it's the only value). - config.Memory: drops Label and Editable; Description is required by the HCL parser. - config.MissionMemory: same shape, just a required Description. Tests rewritten for the simpler surface. Docs (CLAUDE.md, the missions/folders.mdx page, the missions overview table, README) updated to describe the new design. `go test ./...` is green across all packages. --- CLAUDE.md | 64 ++++++++------- README.md | 2 +- agent/internal/prompts/prompts.go | 6 +- aitools/memory_tools.go | 37 ++++----- config/config.go | 54 ++++++------- config/memory.go | 65 ++++++--------- config/memory_test.go | 123 +++++++++++++---------------- config/mission.go | 13 +-- docs/content/missions/folders.mdx | 46 ++++------- docs/content/missions/overview.mdx | 4 +- mission/memory_store.go | 45 +++-------- mission/memory_store_test.go | 111 +++++++------------------- mission/runner.go | 2 +- wsbridge/memory_browser.go | 32 +++----- 14 files changed, 223 insertions(+), 381 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 01727c9..c09d0ef 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -284,30 +284,30 @@ The scheduler lives in `scheduler/` but its lifecycle (creation, config updates, ### Memory + Scratchpad -There are two kinds of mission-scoped file storage, plus a top-level shared -kind: +Two kinds of file storage: - **Memory** — persistent storage that survives across runs. Declared with - a top-level `memory "name" { ... }` (shared, multiple) or a mission-scoped - `memory { ... }` (private to the mission, at most one). -- **Scratchpad** — ephemeral per-run working space. Declared with a - mission-scoped `scratchpad { ... }` block (at most one per mission). A - fresh directory is created for each mission run and the sweep deletes ones - past their `cleanup` window. + a top-level `memory "name" { description = "..." }` (shared, multiple) + or a mission-scoped `memory { description = "..." }` (private to the + mission, at most one). The two block forms take the same fields. Every + memory is writable. +- **Scratchpad** — ephemeral per-run working space. A mission opts in + with `scratchpad = true`; auto-deleted 7 days after the run starts. + Nothing to configure. Agents access both kinds via the same `file_list`, `file_read`, `file_create`, `file_delete`, `file_search`, `file_grep` tools. Each call takes a required `slot` parameter naming which slot to operate in — there is no implicit -default. +default. If a mission declares no `memory { ... }` and no `scratchpad = true`, +agents only see the shared memories listed in `memories = [...]`. -Squadron owns the on-disk paths; the HCL never accepts a `path` attribute. -The three slot kinds map to three fixed location patterns: +Squadron owns the on-disk paths; the HCL never accepts a `path` attribute: | Kind | HCL | `slot` agents pass | On-disk path | |------|-----|---------------------|--------------| | Shared memory | top-level `memory "name" { ... }` | the HCL label | `/memories/shared//` | | Mission memory | `memory { ... }` inside a mission | literal `"memory"` | `/memories/mission//` | -| Mission scratchpad | `scratchpad { ... }` inside a mission | literal `"scratchpad"` | `/scratchpads///` | +| Mission scratchpad | `scratchpad = true` inside a mission | literal `"scratchpad"` | `/scratchpads///` | The slot names `"memory"` and `"scratchpad"` are reserved — a top-level `memory "memory"` or `memory "scratchpad"` block is rejected. @@ -315,12 +315,11 @@ The slot names `"memory"` and `"scratchpad"` are reserved — a top-level The old DSL surfaces — `shared_folder` blocks, `folder` / `run_folder` blocks, the `folders = ...` attribute — are **not** accepted. The parser surfaces a clear error pointing at the new syntax. Path attributes anywhere -on a memory/scratchpad block are rejected too. +on a memory block are rejected too. ```hcl memory "reference" { description = "Shared reference materials" - editable = false # default read-only } mission "analyze" { @@ -330,40 +329,39 @@ mission "analyze" { description = "Cumulative reports — persists across runs" } - scratchpad { - description = "Per-run working space" - cleanup = 7 # optional; defaults to 7, set 0 to keep forever - } + scratchpad = true } ``` **Implementation:** -- `config.Memory` (top-level shared), `config.MissionMemory` (mission's - persistent), and `config.MissionScratchpad` (mission's ephemeral) all - live in [config/memory.go](config/memory.go). On a parsed mission they - land as `Memories []string`, `Memory *MissionMemory`, and - `Scratchpad *MissionScratchpad` (`config.Mission` in - [config/mission.go](config/mission.go)). +- `config.Memory` (top-level shared) and `config.MissionMemory` (mission's + persistent) both live in [config/memory.go](config/memory.go); both have + a required `Description` field. On a parsed mission they land as + `Memories []string`, `Memory *MissionMemory`, and `Scratchpad bool` + (`config.Mission` in [config/mission.go](config/mission.go)). - Top-level `memory "name"` blocks are parsed in Stage 1.5 (with `vars` - context); each mission's `memory { ... }` and `scratchpad { ... }` blocks - are parsed inside the mission block in Stage 5. The `memories.NAME` HCL - namespace exposes the shared-memory labels to mission attributes. + context); each mission's `memory { ... }` block and `scratchpad = bool` + attribute are parsed inside the mission block in Stage 5. The + `memories.NAME` HCL namespace exposes the shared-memory labels to + mission attributes. - [mission/memory_store.go](mission/memory_store.go) is the authoritative runtime resolver. Paths are derived from `paths.SquadronHome()`: `SharedMemoryPath(name)`, `MissionMemoryPath(missionName)`, and `MissionScratchpadPath(missionName, missionInstanceID)`. `buildMemoryStore(mission, memories, missionInstanceID)` must be called **after** the mission instance ID is assigned in `Runner.Run()` because - the scratchpad path depends on it. + the scratchpad path depends on it. `MemoryStore.ResolvePath` returns + just `(string, error)` — there is no read-only mode at this layer. - Each scratchpad directory gets a sidecar `.squadron-run.json` recording - `created_at` + `cleanup_days` so the sweep can find expired ones. + `created_at` + `cleanup_days` (always `config.ScratchpadCleanupDays = 7`) + so the sweep can find expired ones. - `mission.SweepExpiredScratchpads()` walks `/scratchpads/*/*`, deleting any per-run directory whose - sidecar is past its cleanup deadline. It runs opportunistically at the - start of every `Runner.Run()` (for missions with a scratchpad) and on an - hourly ticker in `cmd/engage.go`. No config lookup needed — the filesystem - layout is self-describing. + sidecar is past its 7-day deadline. It runs opportunistically at the + start of every `Runner.Run()` (for missions with a scratchpad) and on + an hourly ticker in `cmd/engage.go`. No config lookup needed — the + filesystem layout is self-describing. ### Mission-Scoped Agents diff --git a/README.md b/README.md index bf6e6ed..63a8b25 100644 --- a/README.md +++ b/README.md @@ -111,7 +111,7 @@ Honest tradeoffs on each comparison page. - **[MCP host](https://docs.squadron.sh/config/mcp_host)** — run Squadron as an MCP server for Claude Desktop, Claude Code, Cursor, and other MCP clients. - **[Schedules & triggers](https://docs.squadron.sh/missions/schedules)** — cron-based or HTTP-triggered missions, with per-mission concurrency limits. - **[Budgets](https://docs.squadron.sh/missions/budgets)** — token + dollar caps per mission or per task. -- **[Memory & Scratchpad](https://docs.squadron.sh/missions/folders)** — sandboxed filesystem locations agents can read/write. Persistent memory (shared with `memory "name"` or per-mission with `memory { }`) plus per-run `scratchpad { }`. Squadron owns the paths. +- **[Memory & Scratchpad](https://docs.squadron.sh/missions/folders)** — sandboxed filesystem locations agents can read/write. Persistent memory (shared with `memory "name"` or per-mission with `memory { }`) plus per-run `scratchpad = true` opt-in. Squadron owns the paths; scratchpads auto-clean after 7 days. - **[Gateways](https://docs.squadron.sh/config/gateways)** — managed subprocesses that bridge Squadron to Slack, Discord, Teams, or any custom system via the Gateway SDK. - **Encrypted vault** — secrets stored at rest with AES-256-GCM + Argon2id; passphrase in the OS keychain. - **Single binary, no runtime deps.** Docker images on every release. diff --git a/agent/internal/prompts/prompts.go b/agent/internal/prompts/prompts.go index 68bfb3e..d24a2cd 100644 --- a/agent/internal/prompts/prompts.go +++ b/agent/internal/prompts/prompts.go @@ -247,10 +247,6 @@ func FormatMemoryContext(store aitools.MemoryStore) string { sb.WriteString("The `slot` parameter is required on every call — pick one of the slot names below.\n\n") for _, info := range infos { - access := "read-only" - if info.Writable { - access = "read/write" - } label := "" switch info.Name { case aitools.MemorySlotName: @@ -262,7 +258,7 @@ func FormatMemoryContext(store aitools.MemoryStore) string { if info.Description != "" { desc = " — " + info.Description } - sb.WriteString(fmt.Sprintf("- **%s**%s%s (%s)\n", info.Name, label, desc, access)) + sb.WriteString(fmt.Sprintf("- **%s**%s%s\n", info.Name, label, desc)) } return sb.String() diff --git a/aitools/memory_tools.go b/aitools/memory_tools.go index 388c749..5aae5cf 100644 --- a/aitools/memory_tools.go +++ b/aitools/memory_tools.go @@ -22,11 +22,11 @@ const ( // MemoryStore provides slot access for missions. Implementations resolve a // slot name (the mission's "memory" / "scratchpad", or a shared memory's -// label) to an absolute path and enforce read/write semantics. +// label) to an absolute path. Every slot is read+write — there is no +// read-only mode at this layer. type MemoryStore interface { // ResolvePath resolves a slot name + relative path to an absolute path. - // Returns: absolute path, writable flag, error. - ResolvePath(slotName string, relPath string) (string, bool, error) + ResolvePath(slotName string, relPath string) (string, error) // MemoryInfos returns info about all available slots. MemoryInfos() []MemoryInfo } @@ -35,7 +35,6 @@ type MemoryStore interface { type MemoryInfo struct { Name string Description string - Writable bool } // validateRelPath ensures a path is relative and doesn't escape the slot root. @@ -53,9 +52,9 @@ func validateRelPath(relPath string) error { // resolveSlotPath is a helper that resolves the slot and validates the // relative path. -func resolveSlotPath(store MemoryStore, name, relPath string) (string, bool, error) { +func resolveSlotPath(store MemoryStore, name, relPath string) (string, error) { if err := validateRelPath(relPath); err != nil { - return "", false, err + return "", err } return store.ResolvePath(name, relPath) } @@ -131,9 +130,9 @@ func (t *MemoryListTool) Call(ctx context.Context, params string) string { var absPath string var err error if p.Path == "" { - absPath, _, err = t.Store.ResolvePath(p.Slot, ".") + absPath, err = t.Store.ResolvePath(p.Slot, ".") } else { - absPath, _, err = resolveSlotPath(t.Store, p.Slot, p.Path) + absPath, err = resolveSlotPath(t.Store, p.Slot, p.Path) } if err != nil { return "Error: " + err.Error() @@ -312,7 +311,7 @@ func (t *MemoryReadTool) Call(ctx context.Context, params string) string { return "Error: path is required" } - absPath, _, err := resolveSlotPath(t.Store, p.Slot, p.Path) + absPath, err := resolveSlotPath(t.Store, p.Slot, p.Path) if err != nil { return "Error: " + err.Error() } @@ -423,15 +422,11 @@ func (t *MemoryCreateTool) Call(ctx context.Context, params string) string { return "Error: path is required" } - absPath, writable, err := resolveSlotPath(t.Store, p.Slot, p.Path) + absPath, err := resolveSlotPath(t.Store, p.Slot, p.Path) if err != nil { return "Error: " + err.Error() } - if !writable { - return "Error: slot is read-only" - } - if p.Append { // Append mode: file must exist f, err := os.OpenFile(absPath, os.O_APPEND|os.O_WRONLY, 0644) @@ -511,15 +506,11 @@ func (t *MemoryDeleteTool) Call(ctx context.Context, params string) string { return "Error: path is required" } - absPath, writable, err := resolveSlotPath(t.Store, p.Slot, p.Path) + absPath, err := resolveSlotPath(t.Store, p.Slot, p.Path) if err != nil { return "Error: " + err.Error() } - if !writable { - return "Error: slot is read-only" - } - info, err := os.Stat(absPath) if err != nil { return "Error: " + err.Error() @@ -610,9 +601,9 @@ func (t *MemorySearchTool) Call(ctx context.Context, params string) string { // Resolve search root var absPath string if p.Path == "" { - absPath, _, err = t.Store.ResolvePath(p.Slot, ".") + absPath, err = t.Store.ResolvePath(p.Slot, ".") } else { - absPath, _, err = resolveSlotPath(t.Store, p.Slot, p.Path) + absPath, err = resolveSlotPath(t.Store, p.Slot, p.Path) } if err != nil { return "Error: " + err.Error() @@ -764,9 +755,9 @@ func (t *MemoryGrepTool) Call(ctx context.Context, params string) string { // Resolve search root var absPath string if p.Path == "" { - absPath, _, err = t.Store.ResolvePath(p.Slot, ".") + absPath, err = t.Store.ResolvePath(p.Slot, ".") } else { - absPath, _, err = resolveSlotPath(t.Store, p.Slot, p.Path) + absPath, err = resolveSlotPath(t.Store, p.Slot, p.Path) } if err != nil { return "Error: " + err.Error() diff --git a/config/config.go b/config/config.go index 89af238..34d4399 100644 --- a/config/config.go +++ b/config/config.go @@ -1784,7 +1784,8 @@ func parseMissionBlock(block *hcl.Block, ctx *hcl.EvalContext) (*Mission, error) Attributes: []hcl.AttributeSchema{ {Name: "agents", Required: true}, {Name: "directive"}, - {Name: "memories"}, // shared memory references: memories = [memories.foo] + {Name: "memories"}, // shared memory references: memories = [memories.foo] + {Name: "scratchpad"}, // bool: opt the mission into a per-run scratchpad slot {Name: "max_parallel"}, {Name: "inputs"}, // shorthand: inputs = { field = string("desc", { default = "val" }) } // Detected so we can produce a nicer error than "unsupported argument". @@ -1797,8 +1798,7 @@ func parseMissionBlock(block *hcl.Block, ctx *hcl.EvalContext) (*Mission, error) {Type: "input", LabelNames: []string{"name"}}, // verbose input blocks still supported {Type: "dataset", LabelNames: []string{"name"}}, {Type: "secret", LabelNames: []string{"name"}}, - {Type: "memory"}, // mission-scoped persistent memory (slot "memory") - {Type: "scratchpad"}, // mission-scoped ephemeral scratchpad (slot "scratchpad") + {Type: "memory"}, // mission-scoped persistent memory (slot "memory") {Type: "schedule"}, {Type: "trigger"}, {Type: "budget"}, @@ -1982,35 +1982,31 @@ func parseMissionBlock(block *hcl.Block, ctx *hcl.EvalContext) (*Mission, error) } } - // Parse the optional `memory { ... }` block (persistent, one per mission) - // and `scratchpad { ... }` block (ephemeral, one per mission). + // Parse the optional `memory { ... }` block (persistent, one per mission). var missionMemory *MissionMemory - var missionScratchpad *MissionScratchpad for _, mb := range missionContent.Blocks { - switch mb.Type { - case "memory": - if missionMemory != nil { - return nil, fmt.Errorf("mission '%s': only one memory block allowed", missionName) - } - var mm MissionMemory - if diags := gohcl.DecodeBody(mb.Body, ctx, &mm); diags.HasErrors() { - return nil, fmt.Errorf("mission '%s' memory: %w", missionName, diags) - } - missionMemory = &mm - case "scratchpad": - if missionScratchpad != nil { - return nil, fmt.Errorf("mission '%s': only one scratchpad block allowed", missionName) - } - var ms MissionScratchpad - if diags := gohcl.DecodeBody(mb.Body, ctx, &ms); diags.HasErrors() { - return nil, fmt.Errorf("mission '%s' scratchpad: %w", missionName, diags) - } - if ms.Cleanup == nil { - v := DefaultScratchpadCleanupDays - ms.Cleanup = &v - } - missionScratchpad = &ms + if mb.Type != "memory" { + continue + } + if missionMemory != nil { + return nil, fmt.Errorf("mission '%s': only one memory block allowed", missionName) + } + var mm MissionMemory + if diags := gohcl.DecodeBody(mb.Body, ctx, &mm); diags.HasErrors() { + return nil, fmt.Errorf("mission '%s' memory: %w", missionName, diags) + } + missionMemory = &mm + } + + // Parse optional `scratchpad = true` attribute. Default false — agents + // only get a scratchpad slot when the mission explicitly opts in. + var missionScratchpad bool + if attr, ok := missionContent.Attributes["scratchpad"]; ok { + v, diags := attr.Expr.Value(ctx) + if diags.HasErrors() { + return nil, fmt.Errorf("mission '%s' scratchpad: %w", missionName, diags) } + missionScratchpad = v.True() } // Parse schedule blocks (optional, multiple allowed) diff --git a/config/memory.go b/config/memory.go index 332bec5..9fd3e1e 100644 --- a/config/memory.go +++ b/config/memory.go @@ -2,77 +2,60 @@ package config import "fmt" -// Reserved slot names for mission-scoped storage. Tool calls reference these -// via the `slot` parameter (e.g. `slot: "memory"` or `slot: "scratchpad"`). +// Reserved slot names for the mission-scoped storage slots. Agents address +// them via the `slot` parameter on the file tools, alongside any shared +// memory names declared at the top level. const ( MemorySlotName = "memory" ScratchpadSlotName = "scratchpad" ) -// DefaultScratchpadCleanupDays is the auto-delete window applied to a -// mission's scratchpad when `cleanup` is not set. -const DefaultScratchpadCleanupDays = 7 +// ScratchpadCleanupDays is the auto-delete window applied to every +// mission's scratchpad. Not user-configurable — scratchpads are deliberately +// short-lived. +const ScratchpadCleanupDays = 7 // Memory describes a top-level shared memory block: // // memory "research" { // description = "..." -// label = "..." -// editable = true // } // // The storage path is derived by the runtime — it lives at -// `/memories/shared//` — so no `path` is accepted from -// HCL. +// `/memories/shared//`. Description is required so +// agents can tell what each shared slot is for. All memories are writable +// by their agents. type Memory struct { Name string `hcl:"name,label"` - Description string `hcl:"description,optional"` - Label string `hcl:"label,optional"` - Editable bool `hcl:"editable,optional"` + Description string `hcl:"description"` } -// Validate enforces naming rules. The literal names "memory" and "scratchpad" -// are reserved for mission-scoped slots and must not be reused by a shared -// memory. +// Validate enforces naming rules + the required description. func (m *Memory) Validate() error { if m.Name == MemorySlotName || m.Name == ScratchpadSlotName { return fmt.Errorf("name %q is reserved for mission-scoped slots", m.Name) } + if m.Description == "" { + return fmt.Errorf("description is required") + } return nil } // MissionMemory describes the `memory { ... }` block inside a mission — // persistent storage that survives across runs. At most one per mission. +// Same surface as the top-level Memory block minus the label: a required +// description. // -// Path is derived by the runtime from the mission name, so no `path` is -// accepted from HCL. +// The storage path is derived by the runtime from the mission name, so no +// `path` is accepted from HCL. type MissionMemory struct { - Description string `hcl:"description,optional"` -} - -// Validate is a no-op today; kept so the type satisfies the same surface as -// MissionScratchpad and so future fields can grow into it. -func (mm *MissionMemory) Validate() error { return nil } - -// MissionScratchpad describes the `scratchpad { ... }` block inside a -// mission — ephemeral per-run working space. At most one per mission. A -// fresh directory is materialized for each mission instance and the cleanup -// sweep deletes ones older than `cleanup` days. -// -// Path is derived by the runtime from the mission name and mission instance -// ID, so no `path` is accepted from HCL. -// -// Cleanup is a pointer so we can distinguish "user did not set it" (apply -// the default) from "user set 0" (keep forever). -type MissionScratchpad struct { - Description string `hcl:"description,optional"` - Cleanup *int `hcl:"cleanup,optional"` // days before auto-delete; 0 = never + Description string `hcl:"description"` } -// Validate rejects negative cleanup values. -func (ms *MissionScratchpad) Validate() error { - if ms.Cleanup != nil && *ms.Cleanup < 0 { - return fmt.Errorf("cleanup must be >= 0 (days)") +// Validate enforces the required description. +func (mm *MissionMemory) Validate() error { + if mm.Description == "" { + return fmt.Errorf("description is required") } return nil } diff --git a/config/memory_test.go b/config/memory_test.go index ec3903e..e7398bc 100644 --- a/config/memory_test.go +++ b/config/memory_test.go @@ -14,8 +14,6 @@ var _ = Describe("Memory + Scratchpad", func() { hcl := fullBaseHCL() + ` memory "research" { description = "Research docs" - label = "Research" - editable = true } mission "m" { commander { model = models.anthropic.claude_sonnet_4 } @@ -29,24 +27,55 @@ mission "m" { Expect(err).NotTo(HaveOccurred()) Expect(cfg.Memories).To(HaveLen(1)) Expect(cfg.Memories[0].Name).To(Equal("research")) - Expect(cfg.Memories[0].Editable).To(BeTrue()) + Expect(cfg.Memories[0].Description).To(Equal("Research docs")) Expect(cfg.Missions[0].Memories).To(ConsistOf("research")) }) + It("requires a description", func() { + hcl := fullBaseHCL() + ` +memory "research" {} +mission "m" { + commander { model = models.anthropic.claude_sonnet_4 } + agents = [agents.test_agent] + task "t" { objective = "go" } +} +` + _, f := writeFixture("config.hcl", hcl) + _, err := config.LoadFile(f) + Expect(err).To(HaveOccurred()) + }) + It("rejects the reserved name 'memory'", func() { - m := config.Memory{Name: "memory"} + m := config.Memory{Name: "memory", Description: "x"} err := m.Validate() Expect(err).To(HaveOccurred()) Expect(err.Error()).To(ContainSubstring("reserved")) }) It("rejects the reserved name 'scratchpad'", func() { - m := config.Memory{Name: "scratchpad"} + m := config.Memory{Name: "scratchpad", Description: "x"} err := m.Validate() Expect(err).To(HaveOccurred()) Expect(err.Error()).To(ContainSubstring("reserved")) }) + It("rejects an `editable` attribute (all memories are editable now)", func() { + hcl := fullBaseHCL() + ` +memory "research" { + description = "x" + editable = true +} +mission "m" { + commander { model = models.anthropic.claude_sonnet_4 } + agents = [agents.test_agent] + task "t" { objective = "go" } +} +` + _, f := writeFixture("config.hcl", hcl) + _, err := config.LoadFile(f) + Expect(err).To(HaveOccurred()) + }) + It("rejects the old shared_folder block with a pointer at the new syntax", func() { hcl := fullBaseHCL() + ` shared_folder "research" { @@ -68,7 +97,8 @@ mission "m" { It("rejects the old `path` attribute on a memory block", func() { hcl := fullBaseHCL() + ` memory "research" { - path = "./data" + description = "x" + path = "./data" } mission "m" { commander { model = models.anthropic.claude_sonnet_4 } @@ -83,7 +113,7 @@ mission "m" { }) Describe("mission memory block (persistent)", func() { - It("parses a memory block on a mission", func() { + It("parses a memory block with a description", func() { hcl := fullBaseHCL() + ` mission "m" { commander { model = models.anthropic.claude_sonnet_4 } @@ -99,45 +129,45 @@ mission "m" { Expect(err).NotTo(HaveOccurred()) Expect(cfg.Missions[0].Memory).NotTo(BeNil()) Expect(cfg.Missions[0].Memory.Description).To(Equal("Long-term notes")) - Expect(cfg.Missions[0].Scratchpad).To(BeNil()) + Expect(cfg.Missions[0].Scratchpad).To(BeFalse()) }) - It("rejects two memory blocks on the same mission", func() { + It("requires a description", func() { hcl := fullBaseHCL() + ` mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] - memory { description = "a" } - memory { description = "b" } + memory {} task "t" { objective = "go" } } ` _, f := writeFixture("config.hcl", hcl) _, err := config.LoadFile(f) Expect(err).To(HaveOccurred()) - Expect(err.Error()).To(ContainSubstring("only one memory block allowed")) }) - It("rejects a `type` attribute (was removed when the block split)", func() { + It("rejects two memory blocks on the same mission", func() { hcl := fullBaseHCL() + ` mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] - memory { type = "persistent" } + memory { description = "a" } + memory { description = "b" } task "t" { objective = "go" } } ` _, f := writeFixture("config.hcl", hcl) _, err := config.LoadFile(f) Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("only one memory block allowed")) }) - It("rejects a `path` attribute on a mission memory block", func() { + It("rejects a `path` attribute", func() { hcl := fullBaseHCL() + ` mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] - memory { path = "./x" } + memory { description = "x"; path = "./x" } task "t" { objective = "go" } } ` @@ -147,79 +177,36 @@ mission "m" { }) }) - Describe("mission scratchpad block (ephemeral)", func() { - It("parses a scratchpad block with default cleanup", func() { + Describe("mission scratchpad attribute", func() { + It("defaults to false when not set", func() { hcl := fullBaseHCL() + ` mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] - scratchpad { - description = "Scratch" - } task "t" { objective = "go" } } ` _, f := writeFixture("config.hcl", hcl) cfg, err := config.LoadFile(f) Expect(err).NotTo(HaveOccurred()) - Expect(cfg.Missions[0].Scratchpad).NotTo(BeNil()) - Expect(cfg.Missions[0].Scratchpad.Description).To(Equal("Scratch")) - Expect(cfg.Missions[0].Scratchpad.Cleanup).NotTo(BeNil()) - Expect(*cfg.Missions[0].Scratchpad.Cleanup).To(Equal(config.DefaultScratchpadCleanupDays)) - Expect(cfg.Missions[0].Memory).To(BeNil()) + Expect(cfg.Missions[0].Scratchpad).To(BeFalse()) }) - It("preserves an explicit cleanup = 0 (keep forever)", func() { + It("accepts scratchpad = true", func() { hcl := fullBaseHCL() + ` mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] - scratchpad { cleanup = 0 } + scratchpad = true task "t" { objective = "go" } } ` _, f := writeFixture("config.hcl", hcl) cfg, err := config.LoadFile(f) Expect(err).NotTo(HaveOccurred()) - Expect(cfg.Missions[0].Scratchpad.Cleanup).NotTo(BeNil()) - Expect(*cfg.Missions[0].Scratchpad.Cleanup).To(Equal(0)) - }) - - It("rejects two scratchpad blocks", func() { - hcl := fullBaseHCL() + ` -mission "m" { - commander { model = models.anthropic.claude_sonnet_4 } - agents = [agents.test_agent] - scratchpad {} - scratchpad {} - task "t" { objective = "go" } -} -` - _, f := writeFixture("config.hcl", hcl) - _, err := config.LoadFile(f) - Expect(err).To(HaveOccurred()) - Expect(err.Error()).To(ContainSubstring("only one scratchpad block allowed")) - }) - - It("rejects a negative cleanup", func() { - neg := -1 - ms := &config.MissionScratchpad{Cleanup: &neg} - Expect(ms.Validate()).To(MatchError(ContainSubstring("cleanup must be >= 0"))) + Expect(cfg.Missions[0].Scratchpad).To(BeTrue()) }) - It("rejects a `path` attribute", func() { - hcl := fullBaseHCL() + ` -mission "m" { - commander { model = models.anthropic.claude_sonnet_4 } - agents = [agents.test_agent] - scratchpad { path = "./x" } - task "t" { objective = "go" } -} -` - _, f := writeFixture("config.hcl", hcl) - _, err := config.LoadFile(f) - Expect(err).To(HaveOccurred()) - }) }) Describe("memory + scratchpad on the same mission", func() { @@ -229,7 +216,7 @@ mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] memory { description = "long-term" } - scratchpad { description = "per-run" } + scratchpad = true task "t" { objective = "go" } } ` @@ -237,7 +224,7 @@ mission "m" { cfg, err := config.LoadFile(f) Expect(err).NotTo(HaveOccurred()) Expect(cfg.Missions[0].Memory).NotTo(BeNil()) - Expect(cfg.Missions[0].Scratchpad).NotTo(BeNil()) + Expect(cfg.Missions[0].Scratchpad).To(BeTrue()) }) }) @@ -274,7 +261,7 @@ mission "m" { It("rejects the old `folders = ...` attribute", func() { hcl := fullBaseHCL() + ` -memory "ref" {} +memory "ref" { description = "x" } mission "m" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.test_agent] diff --git a/config/mission.go b/config/mission.go index f556e62..a29ad9c 100644 --- a/config/mission.go +++ b/config/mission.go @@ -309,9 +309,9 @@ type Mission struct { Tasks []Task `hcl:"task,block"` Inputs []MissionInput // Parsed from input blocks Datasets []Dataset // Parsed from dataset blocks - Memories []string // Shared memory names referenced by this mission - Memory *MissionMemory // Optional persistent mission memory (reserved slot "memory") - Scratchpad *MissionScratchpad // Optional per-run ephemeral scratchpad (reserved slot "scratchpad") + Memories []string // Shared memory names referenced by this mission + Memory *MissionMemory // Optional persistent mission memory (slot "memory") + Scratchpad bool // If true, mission gets an ephemeral per-run scratchpad (slot "scratchpad") Schedules []Schedule `json:"schedules,omitempty"` Trigger *Trigger `json:"trigger,omitempty"` MaxParallel int `json:"maxParallel,omitempty"` // default 3 @@ -492,13 +492,6 @@ func (w *Mission) Validate(models []Model, agents []Agent, memories []Memory, al } } - // Validate the mission scratchpad block if present. - if w.Scratchpad != nil { - if err := w.Scratchpad.Validate(); err != nil { - return fmt.Errorf("scratchpad: %w", err) - } - } - // Validate each task for _, t := range w.Tasks { if err := t.Validate(taskNames, agentNames, datasetNames, w.Agents, allMissionNames); err != nil { diff --git a/docs/content/missions/folders.mdx b/docs/content/missions/folders.mdx index d371a16..7bab521 100644 --- a/docs/content/missions/folders.mdx +++ b/docs/content/missions/folders.mdx @@ -4,15 +4,15 @@ title: Memory & Scratchpad # Memory & Scratchpad -Squadron gives missions two kinds of mission-scoped file storage, plus shared memory that can be referenced across missions: +Squadron gives missions two kinds of file storage: | Kind | What it's for | Lifetime | Slot name agents use | |------|---------------|----------|-----------------------| | **Shared memory** | Reference or output data many missions touch | Persists | the HCL label | | **Mission memory** | A mission's own long-lived state — archives, accumulated output | Persists | `"memory"` | -| **Mission scratchpad** | Working space for one mission run — intermediate files, drafts | One per run | `"scratchpad"` | +| **Mission scratchpad** | Working space for one mission run — intermediate files, drafts | One per run, auto-deleted after 7 days | `"scratchpad"` | -Agents reach all three through the same `file_list`, `file_read`, `file_create`, `file_delete`, `file_search`, and `file_grep` tools, naming the slot on each call via the `slot` parameter. +Agents reach all three through the same `file_list`, `file_read`, `file_create`, `file_delete`, `file_search`, and `file_grep` tools, naming the slot on each call via the `slot` parameter. **Every slot is writable** — there is no read-only mode. > **No `path` attribute.** Squadron owns the on-disk layout — you declare what storage you need and Squadron picks the path. Everything lives under `/`. @@ -23,19 +23,16 @@ Declared at the top level so multiple missions can share the same data: ```hcl memory "research" { description = "Shared research documents" - editable = true } memory "reference" { - # no editable flag → read-only + description = "Reference materials" } ``` | Attribute | Type | Description | |-----------|------|-------------| -| `description` | string | Human-readable description shown to agents | -| `label` | string | Optional label (falls back to description) | -| `editable` | bool | If `true`, agents may write. Default `false` (read-only) | +| `description` | string (required) | Human-readable description shown to agents | Missions opt in by listing them, then refer to them by name in tool calls: @@ -60,7 +57,7 @@ mission "analyze" { } ``` -Agents reach it under the name `memory`: +Same shape as the top-level memory block (just a required `description`) — no label because the slot name is always `"memory"`. Agents reach it under that name: ```json { "slot": "memory", "path": "2026-04-23/report.md", "content": "..." } @@ -68,21 +65,23 @@ Agents reach it under the name `memory`: | Attribute | Type | Description | |-----------|------|-------------| -| `description` | string | Shown to agents alongside the slot name | +| `description` | string (required) | What this mission's memory is for | + +A mission with no `memory { ... }` block has no `"memory"` slot — agents only see whatever shared memories the mission lists (plus the scratchpad, if enabled). ## Mission Scratchpad A clean workspace for one mission run. Each invocation gets its own subdirectory, isolated from previous and concurrent runs — perfect for intermediate files, working drafts, or anything you don't want bleeding into the next run. +Opt in with a single attribute: + ```hcl mission "analyze" { - scratchpad { - description = "Scratch space for this run" - } + scratchpad = true } ``` -By default, scratchpads are deleted 7 days after the run started. Set `cleanup` to override the window, or `cleanup = 0` to keep them forever. +That's the whole configuration. Scratchpads are auto-deleted 7 days after the run started — long enough to inspect what an agent produced and to resume a mission with `squadron mission --resume `, short enough to keep disk usage in check. Agents reach it under the name `scratchpad`: @@ -90,13 +89,6 @@ Agents reach it under the name `scratchpad`: { "slot": "scratchpad", "path": "notes.txt", "content": "..." } ``` -| Attribute | Type | Description | -|-----------|------|-------------| -| `description` | string | Shown to agents alongside the slot name | -| `cleanup` | integer | Delete the directory this many days after the run started. Defaults to `7`; set `0` to keep forever | - -Scratchpads stick around after the mission ends — useful for inspecting what an agent produced, and required for `squadron mission --resume ` to find its workspace. `cleanup` is just a max age; nothing is deleted while a run is in flight. - ## Tool Reference All six file tools take a required `slot` parameter — that's the slot name (`"memory"`, `"scratchpad"`, or a shared memory's label): @@ -122,16 +114,10 @@ memory "reference" { mission "research" { commander { model = models.anthropic.claude_sonnet_4 } agents = [agents.researcher] - memories = [memories.reference] - memory { - description = "Finished reports, one per run" - } - - scratchpad { - description = "Working files for the in-flight run" - cleanup = 14 # override the 7-day default - } + memories = [memories.reference] + memory { description = "Finished reports, one per run" } + scratchpad = true task "gather" { objective = "Gather sources from the reference memory and save notes to the scratchpad" diff --git a/docs/content/missions/overview.mdx b/docs/content/missions/overview.mdx index 4a60b4d..597c360 100644 --- a/docs/content/missions/overview.mdx +++ b/docs/content/missions/overview.mdx @@ -44,8 +44,8 @@ mission "data_pipeline" { | `task` | block | Task definitions (repeatable) | | `dataset` | block | Dataset definitions (optional) | | `memories` | list | Shared memory references, e.g. `[memories.data]` (see [Memory & Scratchpad](/missions/folders)) | -| `memory` | block | Mission-scoped persistent memory (slot `"memory"`). At most one per mission. | -| `scratchpad` | block | Mission-scoped ephemeral per-run scratchpad (slot `"scratchpad"`). At most one per mission. | +| `memory` | block | Mission-scoped persistent memory (slot `"memory"`). Required `description`. At most one per mission. | +| `scratchpad` | bool | If `true`, the mission gets an ephemeral per-run scratchpad (slot `"scratchpad"`); auto-deleted after 7 days. | | `schedule` | block | Automatic run schedules (optional, repeatable) | | `trigger` | block | Webhook trigger (optional) | | `max_parallel` | number | Max concurrent instances (default: 3) | diff --git a/mission/memory_store.go b/mission/memory_store.go index 2e190b7..510517f 100644 --- a/mission/memory_store.go +++ b/mission/memory_store.go @@ -85,17 +85,6 @@ func MissionScratchpadPath(missionName, missionInstanceID string) (string, error return filepath.Join(root, missionName, missionInstanceID), nil } -// resolvedScratchpadCleanup returns the cleanup window in days for a -// mission scratchpad. Reads the parsed pointer when set (the config parser -// fills in the default at load time); falls back to the default for callers -// that hand-build a struct and skip parsing (notably tests). -func resolvedScratchpadCleanup(ms *config.MissionScratchpad) int { - if ms == nil || ms.Cleanup == nil { - return config.DefaultScratchpadCleanupDays - } - return *ms.Cleanup -} - type missionMemoryStore struct { slots map[string]*memorySlot } @@ -103,12 +92,11 @@ type missionMemoryStore struct { type memorySlot struct { absPath string description string - writable bool } // buildMemoryStore creates an aitools.MemoryStore from the mission config and // the declared top-level memories. missionInstanceID scopes the scratchpad -// path; it must be non-empty when mission.Scratchpad is set. Returns nil if +// path; it must be non-empty when mission.Scratchpad is true. Returns nil if // no slots are configured. func buildMemoryStore(mission *config.Mission, memories []config.Memory, missionInstanceID string) (aitools.MemoryStore, error) { store := &missionMemoryStore{ @@ -135,14 +123,9 @@ func buildMemoryStore(mission *config.Mission, memories []config.Memory, mission if err := os.MkdirAll(absPath, 0755); err != nil { return nil, fmt.Errorf("shared memory %q: create directory: %w", name, err) } - desc := mem.Description - if desc == "" { - desc = mem.Label - } store.slots[name] = &memorySlot{ absPath: absPath, - description: desc, - writable: mem.Editable, + description: mem.Description, } } @@ -157,11 +140,10 @@ func buildMemoryStore(mission *config.Mission, memories []config.Memory, mission store.slots[config.MemorySlotName] = &memorySlot{ absPath: absPath, description: mission.Memory.Description, - writable: true, } } - if mission.Scratchpad != nil { + if mission.Scratchpad { if missionInstanceID == "" { return nil, fmt.Errorf("scratchpad requires a mission instance ID") } @@ -172,13 +154,13 @@ func buildMemoryStore(mission *config.Mission, memories []config.Memory, mission if err := os.MkdirAll(absPath, 0755); err != nil { return nil, fmt.Errorf("scratchpad: create directory: %w", err) } - if err := writeRunMetadata(absPath, mission.Name, missionInstanceID, resolvedScratchpadCleanup(mission.Scratchpad)); err != nil { + if err := writeRunMetadata(absPath, mission.Name, missionInstanceID, config.ScratchpadCleanupDays); err != nil { return nil, fmt.Errorf("scratchpad: write metadata: %w", err) } store.slots[config.ScratchpadSlotName] = &memorySlot{ - absPath: absPath, - description: mission.Scratchpad.Description, - writable: true, + absPath: absPath, + // No user-supplied description — the agent prompt explains + // what the scratchpad is for. } } @@ -217,27 +199,27 @@ func writeRunMetadata(dir, missionName, missionID string, cleanupDays int) error return err } -func (s *missionMemoryStore) ResolvePath(slotName string, relPath string) (string, bool, error) { +func (s *missionMemoryStore) ResolvePath(slotName string, relPath string) (string, error) { if slotName == "" { - return "", false, fmt.Errorf("slot name is required (available: %v)", s.availableNames()) + return "", fmt.Errorf("slot name is required (available: %v)", s.availableNames()) } entry, ok := s.slots[slotName] if !ok { - return "", false, fmt.Errorf("slot %q not found. Available: %v", slotName, s.availableNames()) + return "", fmt.Errorf("slot %q not found. Available: %v", slotName, s.availableNames()) } cleaned := filepath.Clean(relPath) if cleaned == "." { - return entry.absPath, entry.writable, nil + return entry.absPath, nil } fullPath := filepath.Join(entry.absPath, cleaned) if !strings.HasPrefix(fullPath, entry.absPath) { - return "", false, fmt.Errorf("path escapes slot root") + return "", fmt.Errorf("path escapes slot root") } - return fullPath, entry.writable, nil + return fullPath, nil } func (s *missionMemoryStore) availableNames() []string { @@ -254,7 +236,6 @@ func (s *missionMemoryStore) MemoryInfos() []aitools.MemoryInfo { infos = append(infos, aitools.MemoryInfo{ Name: name, Description: entry.description, - Writable: entry.writable, }) } return infos diff --git a/mission/memory_store_test.go b/mission/memory_store_test.go index 5e606d1..7609d15 100644 --- a/mission/memory_store_test.go +++ b/mission/memory_store_test.go @@ -55,13 +55,10 @@ func TestBuildMemoryStore_MissionMemory(t *testing.T) { t.Fatal("expected non-nil store") } - abs, writable, err := store.ResolvePath(aitools.MemorySlotName, ".") + abs, err := store.ResolvePath(aitools.MemorySlotName, ".") if err != nil { t.Fatalf("ResolvePath: %v", err) } - if !writable { - t.Fatal("mission memory must be writable") - } want := filepath.Join(home, "memories", "mission", "m") if abs != want { t.Fatalf("mission memory path: want %s, got %s", want, abs) @@ -71,19 +68,16 @@ func TestBuildMemoryStore_MissionMemory(t *testing.T) { } // The mission name is NOT a valid slot key — prevents regression. - if _, _, err := store.ResolvePath(m.Name, "."); err == nil { + if _, err := store.ResolvePath(m.Name, "."); err == nil { t.Fatal("expected error when resolving by mission name") } } func TestBuildMemoryStore_Scratchpad_CreatesSidecar(t *testing.T) { home := withTempHome(t) - cleanup := 7 m := &config.Mission{ - Name: "m", - Scratchpad: &config.MissionScratchpad{ - Cleanup: &cleanup, - }, + Name: "m", + Scratchpad: true, } store, err := buildMemoryStore(m, nil, "mid-abc") @@ -91,13 +85,10 @@ func TestBuildMemoryStore_Scratchpad_CreatesSidecar(t *testing.T) { t.Fatalf("unexpected error: %v", err) } - abs, writable, err := store.ResolvePath(aitools.ScratchpadSlotName, ".") + abs, err := store.ResolvePath(aitools.ScratchpadSlotName, ".") if err != nil { t.Fatalf("ResolvePath: %v", err) } - if !writable { - t.Fatal("scratchpad must be writable") - } want := filepath.Join(home, "scratchpads", "m", "mid-abc") if abs != want { t.Fatalf("scratchpad path: want %s, got %s", want, abs) @@ -111,8 +102,8 @@ func TestBuildMemoryStore_Scratchpad_CreatesSidecar(t *testing.T) { if err := json.Unmarshal(metaBytes, &meta); err != nil { t.Fatalf("sidecar not valid JSON: %v", err) } - if meta.CleanupDays != 7 { - t.Fatalf("CleanupDays: want 7, got %d", meta.CleanupDays) + if meta.CleanupDays != config.ScratchpadCleanupDays { + t.Fatalf("CleanupDays: want %d, got %d", config.ScratchpadCleanupDays, meta.CleanupDays) } if meta.MissionID != "mid-abc" { t.Fatalf("MissionID: want mid-abc, got %q", meta.MissionID) @@ -126,10 +117,9 @@ func TestBuildMemoryStore_Scratchpad_SidecarPreservedOnResume(t *testing.T) { home := withTempHome(t) m := &config.Mission{ Name: "m", - Scratchpad: &config.MissionScratchpad{}, + Scratchpad: true, } - // First build: sidecar written if _, err := buildMemoryStore(m, nil, "mid-1"); err != nil { t.Fatalf("first build: %v", err) } @@ -157,7 +147,7 @@ func TestBuildMemoryStore_Scratchpad_RequiresMissionID(t *testing.T) { withTempHome(t) m := &config.Mission{ Name: "m", - Scratchpad: &config.MissionScratchpad{}, + Scratchpad: true, } _, err := buildMemoryStore(m, nil, "") if err == nil { @@ -172,7 +162,7 @@ func TestBuildMemoryStore_RejectsReservedSharedMemoryNames(t *testing.T) { Name: "m", Memories: []string{reserved}, } - mems := []config.Memory{{Name: reserved}} + mems := []config.Memory{{Name: reserved, Description: "x"}} _, err := buildMemoryStore(m, mems, "mid-1") if err == nil { t.Fatalf("expected error for reserved shared memory name %q", reserved) @@ -184,17 +174,17 @@ func TestBuildMemoryStore_BothMemoryAndScratchpad(t *testing.T) { withTempHome(t) m := &config.Mission{ Name: "m", - Memory: &config.MissionMemory{}, - Scratchpad: &config.MissionScratchpad{}, + Memory: &config.MissionMemory{Description: "x"}, + Scratchpad: true, } store, err := buildMemoryStore(m, nil, "mid-1") if err != nil { t.Fatalf("unexpected error: %v", err) } - if _, _, err := store.ResolvePath(aitools.MemorySlotName, "."); err != nil { + if _, err := store.ResolvePath(aitools.MemorySlotName, "."); err != nil { t.Fatalf("memory slot should resolve: %v", err) } - if _, _, err := store.ResolvePath(aitools.ScratchpadSlotName, "."); err != nil { + if _, err := store.ResolvePath(aitools.ScratchpadSlotName, "."); err != nil { t.Fatalf("scratchpad slot should resolve: %v", err) } } @@ -206,7 +196,7 @@ func TestBuildMemoryStore_SharedMemory(t *testing.T) { Memories: []string{"research"}, } mems := []config.Memory{ - {Name: "research", Description: "Research notes", Editable: true}, + {Name: "research", Description: "Research notes"}, } store, err := buildMemoryStore(m, mems, "mid-1") @@ -214,13 +204,10 @@ func TestBuildMemoryStore_SharedMemory(t *testing.T) { t.Fatalf("build: %v", err) } - abs, writable, err := store.ResolvePath("research", ".") + abs, err := store.ResolvePath("research", ".") if err != nil { t.Fatalf("resolve: %v", err) } - if !writable { - t.Fatal("editable shared memory must resolve as writable") - } want := filepath.Join(home, "memories", "shared", "research") if abs != want { t.Fatalf("shared memory path: want %s, got %s", want, abs) @@ -230,40 +217,17 @@ func TestBuildMemoryStore_SharedMemory(t *testing.T) { } } -func TestBuildMemoryStore_SharedMemory_ReadOnly(t *testing.T) { - withTempHome(t) - m := &config.Mission{ - Name: "m", - Memories: []string{"reference"}, - } - mems := []config.Memory{ - {Name: "reference"}, // editable defaults to false - } - - store, err := buildMemoryStore(m, mems, "mid-1") - if err != nil { - t.Fatalf("build: %v", err) - } - _, writable, err := store.ResolvePath("reference", ".") - if err != nil { - t.Fatalf("resolve: %v", err) - } - if writable { - t.Fatal("default shared memory must be read-only") - } -} - func TestResolvePath_EmptySlotNameRejected(t *testing.T) { withTempHome(t) m := &config.Mission{ Name: "m", - Memory: &config.MissionMemory{}, + Memory: &config.MissionMemory{Description: "x"}, } store, err := buildMemoryStore(m, nil, "mid-1") if err != nil { t.Fatalf("build: %v", err) } - if _, _, err := store.ResolvePath("", "."); err == nil { + if _, err := store.ResolvePath("", "."); err == nil { t.Fatal("expected error when slot name is empty") } } @@ -272,13 +236,13 @@ func TestResolvePath_RejectsPathEscape(t *testing.T) { withTempHome(t) m := &config.Mission{ Name: "m", - Memory: &config.MissionMemory{}, + Memory: &config.MissionMemory{Description: "x"}, } store, err := buildMemoryStore(m, nil, "mid-1") if err != nil { t.Fatalf("build: %v", err) } - if _, _, err := store.ResolvePath("memory", "../outside"); err == nil { + if _, err := store.ResolvePath("memory", "../outside"); err == nil { t.Fatal("expected path-escape error") } } @@ -287,13 +251,13 @@ func TestResolvePath_UnknownSlot(t *testing.T) { withTempHome(t) m := &config.Mission{ Name: "m", - Memory: &config.MissionMemory{}, + Memory: &config.MissionMemory{Description: "x"}, } store, err := buildMemoryStore(m, nil, "mid-1") if err != nil { t.Fatalf("build: %v", err) } - if _, _, err := store.ResolvePath("does_not_exist", "."); err == nil { + if _, err := store.ResolvePath("does_not_exist", "."); err == nil { t.Fatal("expected error for unknown slot") } } @@ -324,7 +288,7 @@ func writeScratchpad(t *testing.T, home, missionName, runID string, createdAt ti func TestSweep_DeletesExpired(t *testing.T) { home := withTempHome(t) - expired := writeScratchpad(t, home, "m", "old", time.Now().Add(-8*24*time.Hour), 7) + expired := writeScratchpad(t, home, "m", "old", time.Now().Add(-8*24*time.Hour), config.ScratchpadCleanupDays) removed, err := SweepExpiredScratchpads() if err != nil { @@ -340,7 +304,7 @@ func TestSweep_DeletesExpired(t *testing.T) { func TestSweep_KeepsUnexpired(t *testing.T) { home := withTempHome(t) - fresh := writeScratchpad(t, home, "m", "new", time.Now().Add(-2*24*time.Hour), 7) + fresh := writeScratchpad(t, home, "m", "new", time.Now().Add(-2*24*time.Hour), config.ScratchpadCleanupDays) removed, err := SweepExpiredScratchpads() if err != nil { @@ -354,22 +318,6 @@ func TestSweep_KeepsUnexpired(t *testing.T) { } } -func TestSweep_IgnoresZeroCleanup(t *testing.T) { - home := withTempHome(t) - keep := writeScratchpad(t, home, "m", "forever", time.Now().Add(-365*24*time.Hour), 0) - - removed, err := SweepExpiredScratchpads() - if err != nil { - t.Fatalf("sweep: %v", err) - } - if len(removed) != 0 { - t.Fatalf("expected nothing removed when cleanup=0, got %v", removed) - } - if _, err := os.Stat(keep); err != nil { - t.Fatalf("directory with cleanup=0 should be preserved: %v", err) - } -} - func TestSweep_IgnoresDirectoriesWithoutSidecar(t *testing.T) { home := withTempHome(t) manual := filepath.Join(home, "scratchpads", "m", "hand_made") @@ -429,7 +377,7 @@ func TestSweep_WalksAcrossMissions(t *testing.T) { // scratchpad with a current sidecar — and the old one is gone. func TestSweepThenRebuildRoundTrip(t *testing.T) { home := withTempHome(t) - stale := writeScratchpad(t, home, "demo", "old-run", time.Now().Add(-5*24*time.Hour), 2) + stale := writeScratchpad(t, home, "demo", "old-run", time.Now().Add(-30*24*time.Hour), config.ScratchpadCleanupDays) if _, err := SweepExpiredScratchpads(); err != nil { t.Fatalf("sweep: %v", err) @@ -438,18 +386,15 @@ func TestSweepThenRebuildRoundTrip(t *testing.T) { t.Fatalf("stale scratchpad should have been deleted: %v", err) } - cleanup := 2 m := &config.Mission{ - Name: "demo", - Scratchpad: &config.MissionScratchpad{ - Cleanup: &cleanup, - }, + Name: "demo", + Scratchpad: true, } store, err := buildMemoryStore(m, nil, "new-run") if err != nil { t.Fatalf("build: %v", err) } - fresh, _, err := store.ResolvePath(aitools.ScratchpadSlotName, ".") + fresh, err := store.ResolvePath(aitools.ScratchpadSlotName, ".") if err != nil { t.Fatalf("resolve: %v", err) } diff --git a/mission/runner.go b/mission/runner.go index 863292c..5fccce2 100644 --- a/mission/runner.go +++ b/mission/runner.go @@ -571,7 +571,7 @@ func (r *Runner) Run(ctx context.Context, streamer streamers.MissionHandler) err // Memory store depends on missionID (for the scratchpad path), so build // it here rather than in NewRunner. Sweep expired scratchpads async — // the result doesn't affect this run's correctness, only disk usage. - if r.mission.Scratchpad != nil { + if r.mission.Scratchpad { go func() { _, _ = SweepExpiredScratchpads() }() } memoryStore, err := buildMemoryStore(r.mission, r.cfg.Memories, missionID) diff --git a/wsbridge/memory_browser.go b/wsbridge/memory_browser.go index 42f7132..95241b7 100644 --- a/wsbridge/memory_browser.go +++ b/wsbridge/memory_browser.go @@ -17,12 +17,11 @@ import ( ) // resolvedMemory describes one materialized memory slot for the UI: a -// human-friendly name, the absolute path it lives at, and whether agents are -// allowed to write to it. +// human-friendly name and the absolute path it lives at. Every slot is +// writable — there is no read-only mode. type resolvedMemory struct { - name string - path string - editable bool + name string + path string } // resolveMemoryPath looks up a memory slot by name (top-level shared @@ -38,11 +37,7 @@ func (c *Client) resolveMemoryPath(memoryName, relPath string) (*resolvedMemory, if err != nil { return nil, "", fmt.Errorf("resolve shared memory %q: %w", memoryName, err) } - rm := &resolvedMemory{ - name: cfg.Memories[i].Name, - path: absPath, - editable: cfg.Memories[i].Editable, - } + rm := &resolvedMemory{name: cfg.Memories[i].Name, path: absPath} path, err := c.resolveSafePath(absPath, relPath) return rm, path, err } @@ -55,7 +50,7 @@ func (c *Client) resolveMemoryPath(memoryName, relPath string) (*resolvedMemory, if err != nil { return nil, "", fmt.Errorf("resolve mission memory for %q: %w", m.Name, err) } - rm := &resolvedMemory{name: m.Name, path: absPath, editable: true} + rm := &resolvedMemory{name: m.Name, path: absPath} path, err := c.resolveSafePath(absPath, relPath) return rm, path, err } @@ -115,10 +110,6 @@ func collectMemoryInfos(cfg *config.Config) ([]protocol.SharedFolderInfo, error) var folders []protocol.SharedFolderInfo for _, mem := range cfg.Memories { - label := mem.Label - if label == "" { - label = mem.Name - } path, err := mission.SharedMemoryPath(mem.Name) if err != nil { return nil, fmt.Errorf("shared memory %q: %w", mem.Name, err) @@ -126,9 +117,9 @@ func collectMemoryInfos(cfg *config.Config) ([]protocol.SharedFolderInfo, error) folders = append(folders, protocol.SharedFolderInfo{ Name: mem.Name, Path: path, - Label: label, + Label: mem.Name, Description: mem.Description, - Editable: mem.Editable, + Editable: true, // every memory is writable IsShared: true, Missions: sharedMissions[mem.Name], }) @@ -259,17 +250,12 @@ func (c *Client) handleWriteBrowseFile(env *protocol.Envelope) (*protocol.Envelo return nil, fmt.Errorf("decode write_browse_file: %w", err) } - mem, fullPath, err := c.resolveMemoryPath(payload.BrowserName, payload.RelPath) + _, fullPath, err := c.resolveMemoryPath(payload.BrowserName, payload.RelPath) if err != nil { return protocol.NewResponse(env.RequestID, protocol.TypeWriteBrowseFileResult, &protocol.WriteBrowseFileResultPayload{Success: false, Error: err.Error()}) } - if !mem.editable { - return protocol.NewResponse(env.RequestID, protocol.TypeWriteBrowseFileResult, - &protocol.WriteBrowseFileResultPayload{Success: false, Error: "memory is read-only"}) - } - if err := os.WriteFile(fullPath, []byte(payload.Content), 0644); err != nil { return protocol.NewResponse(env.RequestID, protocol.TypeWriteBrowseFileResult, &protocol.WriteBrowseFileResultPayload{Success: false, Error: err.Error()}) From 87a08c7e2f01d73a218319839ef0cbed5c62982e Mon Sep 17 00:00:00 2001 From: Max Lund Date: Wed, 27 May 2026 22:07:54 -0500 Subject: [PATCH 7/8] Address code-review findings 1-4 + cross-name reject + clear missing-ref error MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Six fixes from the high-recall review of this PR: 1. Validate shared-memory + mission names against path-component abuse (config/memory.go validateSlotName, called from Memory.Validate and Mission.Validate). Rejects names containing `/`, `\`, `..`, or starting with `.` so `memory "../escape"` and `mission "../pwn"` no longer slip through filepath.Join + MkdirAll to create directories outside . The previous code accepted these because HCL labels are arbitrary quoted strings. 2. Reject shared-memory labels that collide with a mission name (config/config.go c.Validate). The wsbridge file-browser keys both under the same string-name namespace; a collision silently masked the mission's persistent memory in the UI. Now caught at load time with a clear "memory X: name conflicts with mission X" error. 3. Sort MemoryInfos() output (mission/memory_store.go). Go map iteration is randomized, and the result feeds the agent's system prompt via prompts.FormatMemoryContext. Without a sort, the prompt bytes change run-to-run and Anthropic prompt caching misses on otherwise-identical missions. 4. SweepExpiredScratchpads no longer aborts on the first per-mission ReadDir error. Previously a permission-denied or transient IO failure on one mission's scratchpads dir halted cleanup for every other mission in the same pass; now it's a `continue`. 5. Legacy-block error messages (config/config.go) used to tell users to migrate to `memory { type = "persistent" }` and `type = "ephemeral" }` — neither exists in the final DSL. Updated to point at the actual replacements: `memory { description = "..." }` and `scratchpad = true`. 6. Always register the `memories` HCL eval-context variable (even when empty). Previously a mission writing `memories = [memories.foo]` with no top-level memory block declared got the cryptic HCL error "Unknown variable; There is no variable named 'memories'"; now it gets "This object does not have an attribute named 'foo'" which at least names the bad reference. Tests added: - config/memory_test.go: path-component rejection, name collision rejection (via LoadAndValidate since c.Validate isn't called from LoadFile), missing-reference error contains the bad name, run_folder remediation mentions `scratchpad = true`. - mission/memory_store_test.go: MemoryInfos returns stable order across multiple calls, sweep doesn't halt when one mission's subdir is unreadable. Findings deliberately not addressed (per user decision): - Resume across PR boundary fails for in-flight sessions (JSON tag "folder" → "slot" change). Documented as breaking; users restart. - Scratchpads remain invisible in the command-center file browser (preserved behavior from the old run_folder; revisit later). --- config/config.go | 36 ++++++++++++++----- config/memory.go | 28 ++++++++++++++- config/memory_test.go | 63 +++++++++++++++++++++++++++++++++ config/mission.go | 4 +-- mission/memory_store.go | 16 ++++++--- mission/memory_store_test.go | 68 ++++++++++++++++++++++++++++++++++++ 6 files changed, 199 insertions(+), 16 deletions(-) diff --git a/config/config.go b/config/config.go index 34d4399..847bd9c 100644 --- a/config/config.go +++ b/config/config.go @@ -443,6 +443,20 @@ func (c *Config) Validate() error { } } + // A shared-memory label sharing a mission name silently masks that + // mission's persistent memory in the file-browser UI (resolveMemoryPath + // matches shared first by string-equal name). Reject the collision at + // load so users can't tie themselves in that knot. + missionNamesForMemoryCheck := make(map[string]bool, len(c.Missions)) + for _, mn := range c.Missions { + missionNamesForMemoryCheck[mn.Name] = true + } + for _, m := range c.Memories { + if missionNamesForMemoryCheck[m.Name] { + return fmt.Errorf("memory '%s': name conflicts with mission '%s' — both are exposed under the same name in the file browser", m.Name, m.Name) + } + } + // Validate plugins for _, p := range c.Plugins { if err := p.Validate(); err != nil { @@ -1157,13 +1171,19 @@ func loadFromFiles(files []string) (*Config, error) { agentsCtx := buildAgentsContext(skillsCtx, allAgents) // Add `memories` namespace for mission references: `memories.NAME` resolves - // to the memory's name as a string. - if len(allMemories) > 0 { - memMap := make(map[string]cty.Value) - for _, m := range allMemories { - memMap[m.Name] = cty.StringVal(m.Name) - } + // to the memory's name as a string. Register even when empty so that a + // reference to an unknown shared memory produces "object has no attribute + // NAME" (pointing at the bad reference) rather than the generic HCL + // "unknown variable memories" (which is mystifying when the user simply + // forgot to declare the shared memory). + memMap := make(map[string]cty.Value, len(allMemories)) + for _, m := range allMemories { + memMap[m.Name] = cty.StringVal(m.Name) + } + if len(memMap) > 0 { agentsCtx.Variables["memories"] = cty.ObjectVal(memMap) + } else { + agentsCtx.Variables["memories"] = cty.EmptyObjectVal } // Stage 5: Load missions (with vars + models + tools + agents + memories context) @@ -1976,9 +1996,9 @@ func parseMissionBlock(block *hcl.Block, ctx *hcl.EvalContext) (*Mission, error) for _, b := range missionContent.Blocks { switch b.Type { case "folder": - return nil, fmt.Errorf("mission '%s': the `folder { ... }` block is no longer supported — use `memory { type = \"persistent\" }` instead (path is now derived automatically)", missionName) + return nil, fmt.Errorf("mission '%s': the `folder { ... }` block is no longer supported — use `memory { description = \"...\" }` instead (path is now derived automatically)", missionName) case "run_folder": - return nil, fmt.Errorf("mission '%s': the `run_folder { ... }` block is no longer supported — use `memory { type = \"ephemeral\" }` instead", missionName) + return nil, fmt.Errorf("mission '%s': the `run_folder { ... }` block is no longer supported — use `scratchpad = true` on the mission instead (auto-cleaned after 7 days)", missionName) } } diff --git a/config/memory.go b/config/memory.go index 9fd3e1e..5a8deca 100644 --- a/config/memory.go +++ b/config/memory.go @@ -1,6 +1,9 @@ package config -import "fmt" +import ( + "fmt" + "strings" +) // Reserved slot names for the mission-scoped storage slots. Agents address // them via the `slot` parameter on the file tools, alongside any shared @@ -35,12 +38,35 @@ func (m *Memory) Validate() error { if m.Name == MemorySlotName || m.Name == ScratchpadSlotName { return fmt.Errorf("name %q is reserved for mission-scoped slots", m.Name) } + if err := validateSlotName(m.Name); err != nil { + return err + } if m.Description == "" { return fmt.Errorf("description is required") } return nil } +// validateSlotName rejects HCL labels that would break filesystem layout: +// path separators, parent-dir traversal, or leading dot. Used by both shared +// memory labels and mission names (since both become directory names under +// ). +func validateSlotName(name string) error { + if name == "" { + return fmt.Errorf("name is required") + } + if strings.ContainsAny(name, `/\`) { + return fmt.Errorf("name %q must not contain path separators", name) + } + if name == "." || name == ".." || strings.HasPrefix(name, ".") { + return fmt.Errorf("name %q must not start with '.'", name) + } + if strings.Contains(name, "..") { + return fmt.Errorf("name %q must not contain '..'", name) + } + return nil +} + // MissionMemory describes the `memory { ... }` block inside a mission — // persistent storage that survives across runs. At most one per mission. // Same surface as the top-level Memory block minus the label: a required diff --git a/config/memory_test.go b/config/memory_test.go index e7398bc..109a3b3 100644 --- a/config/memory_test.go +++ b/config/memory_test.go @@ -59,6 +59,51 @@ mission "m" { Expect(err.Error()).To(ContainSubstring("reserved")) }) + It("rejects names with path separators or '..' (path-traversal guard)", func() { + for _, bad := range []string{"../escape", "foo/bar", "foo\\bar", "..", ".", ".hidden", "a..b"} { + m := config.Memory{Name: bad, Description: "x"} + err := m.Validate() + Expect(err).To(HaveOccurred(), "expected %q to be rejected", bad) + } + }) + + It("rejects a shared memory whose label collides with a mission name", func() { + hcl := fullBaseHCL() + ` +memory "analyze" { + description = "shared" +} +mission "analyze" { + commander { model = models.anthropic.claude_sonnet_4 } + agents = [agents.test_agent] + task "t" { objective = "go" } +} +` + _, f := writeFixture("config.hcl", hcl) + // Cross-name collision is caught by Config.Validate (the wsbridge + // browser keys both under the same string). + _, err := config.LoadAndValidate(f) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("conflicts with mission")) + }) + + It("gives a clear error when a mission references an unknown shared memory", func() { + hcl := fullBaseHCL() + ` +mission "m" { + commander { model = models.anthropic.claude_sonnet_4 } + agents = [agents.test_agent] + memories = [memories.does_not_exist] + task "t" { objective = "go" } +} +` + _, f := writeFixture("config.hcl", hcl) + _, err := config.LoadFile(f) + Expect(err).To(HaveOccurred()) + // Must reference the missing name — not be a generic 'unknown + // variable memories' (which is what happens without the always- + // register-empty-namespace fix). + Expect(err.Error()).To(ContainSubstring("does_not_exist")) + }) + It("rejects an `editable` attribute (all memories are editable now)", func() { hcl := fullBaseHCL() + ` memory "research" { @@ -257,6 +302,24 @@ mission "m" { _, err := config.LoadFile(f) Expect(err).To(HaveOccurred()) Expect(err.Error()).To(ContainSubstring("`run_folder { ... }` block is no longer supported")) + // Remediation must point at the actual new DSL, not a defunct + // intermediate from earlier in this PR. + Expect(err.Error()).To(ContainSubstring("scratchpad = true")) + }) + + It("points at the new memory DSL when rejecting the old `folder { ... }` block", func() { + hcl := fullBaseHCL() + ` +mission "m" { + commander { model = models.anthropic.claude_sonnet_4 } + agents = [agents.test_agent] + folder { path = "./x" } + task "t" { objective = "go" } +} +` + _, f := writeFixture("config.hcl", hcl) + _, err := config.LoadFile(f) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring(`memory { description`)) }) It("rejects the old `folders = ...` attribute", func() { diff --git a/config/mission.go b/config/mission.go index a29ad9c..e7324b9 100644 --- a/config/mission.go +++ b/config/mission.go @@ -356,8 +356,8 @@ type TaskRoute struct { // Validate checks that the mission configuration is valid func (w *Mission) Validate(models []Model, agents []Agent, memories []Memory, allMissionNames map[string]bool) error { - if w.Name == "" { - return fmt.Errorf("mission name is required") + if err := validateSlotName(w.Name); err != nil { + return fmt.Errorf("mission name: %w", err) } if w.Commander == nil || w.Commander.Model == "" { diff --git a/mission/memory_store.go b/mission/memory_store.go index 510517f..0f0e377 100644 --- a/mission/memory_store.go +++ b/mission/memory_store.go @@ -6,6 +6,7 @@ import ( "fmt" "os" "path/filepath" + "sort" "strings" "time" @@ -231,13 +232,18 @@ func (s *missionMemoryStore) availableNames() []string { } func (s *missionMemoryStore) MemoryInfos() []aitools.MemoryInfo { - var infos []aitools.MemoryInfo + infos := make([]aitools.MemoryInfo, 0, len(s.slots)) for name, entry := range s.slots { infos = append(infos, aitools.MemoryInfo{ Name: name, Description: entry.description, }) } + // Stable order — the result feeds the agent's system prompt + // (prompts.FormatMemoryContext). Go map iteration is randomized, so + // without this sort the prompt bytes change run-to-run and Anthropic + // prompt caching misses on otherwise-identical missions. + sort.Slice(infos, func(i, j int) bool { return infos[i].Name < infos[j].Name }) return infos } @@ -269,10 +275,10 @@ func SweepExpiredScratchpads() (removed []string, err error) { runBase := filepath.Join(root, missionEntry.Name()) runEntries, err := os.ReadDir(runBase) if err != nil { - if os.IsNotExist(err) { - continue - } - return removed, err + // Skip this mission on any IO error (permission denied, transient + // stat failure, etc.) so one bad subdir doesn't halt the sweep + // for every other mission. NotExist is just the empty case. + continue } for _, e := range runEntries { if !e.IsDir() { diff --git a/mission/memory_store_test.go b/mission/memory_store_test.go index 7609d15..2127b12 100644 --- a/mission/memory_store_test.go +++ b/mission/memory_store_test.go @@ -170,6 +170,45 @@ func TestBuildMemoryStore_RejectsReservedSharedMemoryNames(t *testing.T) { } } +func TestMemoryInfos_IsAlphabeticallySorted(t *testing.T) { + // MemoryInfos feeds the agent's system prompt; map-iteration order would + // bust prompt caching on every run, so the order must be deterministic. + withTempHome(t) + m := &config.Mission{ + Name: "m", + Memories: []string{"zeta", "alpha", "mu"}, + Memory: &config.MissionMemory{Description: "x"}, + Scratchpad: true, + } + mems := []config.Memory{ + {Name: "zeta", Description: "z"}, + {Name: "alpha", Description: "a"}, + {Name: "mu", Description: "m"}, + } + store, err := buildMemoryStore(m, mems, "mid-1") + if err != nil { + t.Fatalf("build: %v", err) + } + + // Run a few times — random map seed should not change the output order. + want := []string{"alpha", "memory", "mu", "scratchpad", "zeta"} + for i := 0; i < 5; i++ { + infos := store.MemoryInfos() + got := make([]string, len(infos)) + for j, info := range infos { + got[j] = info.Name + } + if len(got) != len(want) { + t.Fatalf("iter %d: want %d slots, got %d: %v", i, len(want), len(got), got) + } + for j := range got { + if got[j] != want[j] { + t.Fatalf("iter %d: order: want %v, got %v", i, want, got) + } + } + } +} + func TestBuildMemoryStore_BothMemoryAndScratchpad(t *testing.T) { withTempHome(t) m := &config.Mission{ @@ -371,6 +410,35 @@ func TestSweep_WalksAcrossMissions(t *testing.T) { } } +func TestSweep_OneBadSubdirDoesNotHaltOthers(t *testing.T) { + home := withTempHome(t) + expired := writeScratchpad(t, home, "alpha", "old", time.Now().Add(-30*24*time.Hour), config.ScratchpadCleanupDays) + + // Create a sibling mission dir whose run subdir is unreadable. Use chmod + // 000 on the parent so os.ReadDir on `/broken/` fails with EACCES. + broken := filepath.Join(home, "scratchpads", "broken") + if err := os.MkdirAll(broken, 0755); err != nil { + t.Fatal(err) + } + // Put a file under it that ReadDir-on-a-file would error on. + if err := os.WriteFile(filepath.Join(broken, "not_a_dir"), []byte{}, 0644); err != nil { + t.Fatal(err) + } + if err := os.Chmod(broken, 0000); err != nil { + t.Skipf("can't chmod 0000 in this environment: %v", err) + } + t.Cleanup(func() { _ = os.Chmod(broken, 0755) }) + + removed, err := SweepExpiredScratchpads() + if err != nil { + t.Fatalf("sweep should not error when a sibling mission's dir is unreadable: %v", err) + } + // The expired alpha dir must still get cleaned up. + if _, statErr := os.Stat(expired); !os.IsNotExist(statErr) { + t.Fatalf("expired alpha scratchpad should be gone (got stat err %v, removed=%v)", statErr, removed) + } +} + // TestSweepThenRebuildRoundTrip mirrors the real flow: an old scratchpad // exists with a sidecar backdated past its cleanup window, the sweep deletes // it, then a new buildMemoryStore (different missionID) creates a fresh From 3091d6f87eca085ae87eef5d990ed53681b1ad17 Mon Sep 17 00:00:00 2001 From: Max Lund Date: Wed, 27 May 2026 22:49:41 -0500 Subject: [PATCH 8/8] Docs: flesh out scratchpad section + document file_* tools as internal MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two doc gaps surfaced after testing the new DSL end-to-end: 1. docs/content/missions/folders.mdx — the Mission Scratchpad section explained the *what* but not the *when*. Added a "When to use a scratchpad" subsection with four concrete use cases (multi-step pipelines, intermediate compute, concurrency isolation, resume material) and a Memory-vs-scratchpad decision table. 2. docs/content/missions/internal-tools.mdx — the page listed every commander tool, every dataset tool, but nothing about the six file_* tools that get auto-attached when a mission declares any slot. Added a File Tools subsection mirroring the Dataset Tools layout, calling out: - the auto-attach rule (file_* appear only when a slot is declared) - the slot parameter and its three sources ("memory", "scratchpad", or a shared memory's label) - the rejection of absolute paths and `..` escapes - a link back to /missions/folders for the storage layout --- docs/content/missions/folders.mdx | 22 ++++++++++++++++++++++ docs/content/missions/internal-tools.mdx | 23 +++++++++++++++++++++++ 2 files changed, 45 insertions(+) diff --git a/docs/content/missions/folders.mdx b/docs/content/missions/folders.mdx index 7bab521..09dacea 100644 --- a/docs/content/missions/folders.mdx +++ b/docs/content/missions/folders.mdx @@ -89,6 +89,28 @@ Agents reach it under the name `scratchpad`: { "slot": "scratchpad", "path": "notes.txt", "content": "..." } ``` +### When to use a scratchpad + +Reach for the scratchpad when one run needs working storage that the *next* run shouldn't see: + +- **Multi-step or multi-agent pipelines.** Agent A scrapes 50 pages and writes them to `scratchpad/raw/`; agent B reads, filters, and summarizes; agent C writes only the final summary to `memory/`. Each run starts clean — no stale `raw/` files leaking between runs. +- **Intermediate computation you don't want to ship.** Partial JSON, debug dumps, retry attempts, half-rendered output — anything the LLM produces while figuring out the right answer but that shouldn't accumulate forever. +- **Per-run isolation under concurrency.** Two simultaneous runs of the same mission can both write to `scratchpad` without colliding; each gets its own `//` directory. +- **Resume material.** If a long-running mission crashes mid-flight, its scratchpad stays on disk so `squadron mission --resume ` can pick up where it left off. + +### Memory vs scratchpad + +If you're not sure which to use, this is the rule: + +| Question | Use | +|----------|-----| +| Should the *next* run of this mission see this file? | `memory { }` | +| Is this a working draft, intermediate result, or temp file? | `scratchpad = true` | +| Will multiple missions need to read this? | top-level `memory "name" { ... }` | +| Does this need to survive across runs *and* be writable by agents? | `memory { }` (every memory is writable) | + +A mission can declare both — the persistent `memory { }` for things worth keeping, and `scratchpad = true` for everything else. + ## Tool Reference All six file tools take a required `slot` parameter — that's the slot name (`"memory"`, `"scratchpad"`, or a shared memory's label): diff --git a/docs/content/missions/internal-tools.mdx b/docs/content/missions/internal-tools.mdx index 9d02abe..ac61064 100644 --- a/docs/content/missions/internal-tools.mdx +++ b/docs/content/missions/internal-tools.mdx @@ -336,3 +336,26 @@ When running inside a mission, agents automatically get these dataset tools: | `result_to_dataset` | Convert a large intercepted result into a dataset for iteration | See [Datasets](/missions/datasets) for details and examples. + +### File Tools + +When a mission declares at least one storage slot — a top-level `memory "name"` referenced in `memories = [...]`, a mission-scoped `memory { }`, or `scratchpad = true` — every agent in that mission automatically gets these six file tools: + +| Tool | Description | +|------|-------------| +| `file_list` | List files and directories in a slot, with optional recursion and pagination | +| `file_read` | Read a file's contents (optionally capped by lines or bytes) | +| `file_create` | Create, overwrite, or append to a file | +| `file_delete` | Delete a file (directories are not allowed) | +| `file_search` | Recursively search for files by filename regex | +| `file_grep` | Search file contents by regex | + +Every call takes a required `slot` parameter naming which slot to operate in: + +- `"memory"` — the mission's persistent memory (only available if the mission declares `memory { }`) +- `"scratchpad"` — the mission's ephemeral per-run scratchpad (only available if the mission sets `scratchpad = true`; auto-cleaned after 7 days) +- a shared memory's HCL label — for any memory the mission lists in `memories = [...]` + +Paths are always relative to the slot's root. Absolute paths and `..` escapes are rejected. + +A mission with no `memories =`, no `memory { }`, and no `scratchpad = true` does NOT get the file tools — they appear only when there's somewhere to put files. See [Memory & Scratchpad](/missions/folders) for the full slot model and the storage paths Squadron picks.