diff --git a/docs/lineage.md b/docs/lineage.md index 648777b5..cb3011fb 100644 --- a/docs/lineage.md +++ b/docs/lineage.md @@ -187,6 +187,37 @@ grants, recorded model effects, applicability checks, and total budgets remain its own requirements; the OptMem installer and global instruction block are not part of the integration. +## Temper: contract-first behavior + +[Temper](https://github.com/nerdsane/temper/tree/1be4b7be003f35bdacfbc24f7f4eb7e77bdade48) +is a policy-driven runtime in which agents describe application behavior as +state machines, data contracts, policies, and sandboxed modules. The source +reviewed on 2026-10-04 informs a proposed ALGAL direction, not a new dependency +or evidence that ALGAL has implemented Temper's mechanisms. + +Its [verification-hardening decision](https://github.com/nerdsane/temper/blob/1be4b7be003f35bdacfbc24f7f4eb7e77bdade48/docs/adrs/0016-verification-cascade-hardening.md) +identifies a concrete failure mode: a checker can silently accept assertions +that its classifier does not understand, even when a runtime parser understands +more. The decision calls for a shared parser, explicit unverifiable assertions, +and minimized counterexamples. ALGAL should require checker/runtime agreement +and keep unsupported properties unresolved. More verification layers do not +repair a disagreement about the property being checked. + +Temper's [composite verifier](https://github.com/nerdsane/temper/blob/1be4b7be003f35bdacfbc24f7f4eb7e77bdade48/crates/temper-verify/src/composite/verify.rs) +checks joint entity state and distinguishes verified, violated, and incomplete +exploration. That distinction is useful for ALGAL's process, resource, and +revision invariants. Its [shadow testing](https://github.com/nerdsane/temper/blob/1be4b7be003f35bdacfbc24f7f4eb7e77bdade48/crates/temper-jit/src/shadow.rs) +compares two transition tables on supplied cases. Such comparisons expose +revision differences on those cases; they do not establish unrestricted +observational equivalence or authorize new live effects. + +The [ALGAL north star](north-star.md#contract-first-behavior-and-verification) +selects executable contracts, typed host policy, staged verification, composite +checks, shadow revisions, and recorded improvement decisions. ALGAL would +express them through its existing manifests, applications, effects, and +foundry evidence. Temper's CSDL/OData interface, Cedar stack, actor backend, +and proof-tool selection are not required parts of that adoption. + ## Research precedents The claim that nobody has thought of evolving AI programs would not survive diff --git a/docs/lisp-source-experiment.md b/docs/lisp-source-experiment.md new file mode 100644 index 00000000..1b638923 --- /dev/null +++ b/docs/lisp-source-experiment.md @@ -0,0 +1,52 @@ +# Lisp-shaped source experiment + +Status: parity phase passed, 2026-10-06. This experiment is a decision input for the +north-star language direction. It does not change the runtime, manifest +contract, or production source language. + +The question is narrow: does a live, Lisp-shaped authoring surface make ALGAL +programs faster for a person or model to change and repair while preserving +ALGAL's bounded semantics, inspectable expansion, cross-runtime parity, and +replayable evidence? + +The incumbent is the current `.algal` source language. The treatment is a small +parenthesized language with symbols, literals, `let`, `if`, records, arithmetic, +and hygienic-looking macros. Both must lower to the same `algal.organism.v1` +manifest and the same pure `algal.expr.v1` cells. The treatment must never gain +ambient evaluation, unbounded recursion, hidden effects, or authority through +syntax. + +## Protocol + +Freeze a task set of at least ten small programs drawn from existing source +examples. For each task, create equivalent incumbent and treatment programs, +then ask the same model or developer to perform the same sequence of bounded +edits, including one deliberate error repair and one semantic change. Record: + +- wall time from edit to useful diagnostic; +- source bytes and model token count; +- diagnostic location and repair attempts; +- expansion size and digest; +- compile and check time; +- manifest and receipt digest equality; +- replay verification time; +- whether a reviewer can explain the generated expansion; +- total source plus maintenance cost after the edit sequence. + +Use paired tasks and the same machine, toolchain, prompt budget, and fixture +inputs. Report medians and individual task results. A treatment result is +`insufficient evidence` if it compiles fewer than 90% of incumbent tasks, loses +receipt parity, or has fewer than ten paired tasks. + +## Decision gate + +Adopt a Lisp-shaped source surface only if it improves median verified iteration +time by at least 20%, keeps parity on every accepted task, and does not increase +repair attempts or expansion-review failures by more than 10%. If it improves +only source compactness, keep the current syntax and borrow macros or a live +workbench selectively. If it improves the interactive repair loop but not batch +source metrics, prioritize a debugger/condition interface rather than a new +language. If it fails the gate, record the negative result and keep the current +source direction. + +The current runner covers eleven paired fixtures and lowers each treatment through the real compiler. All eleven have exact manifest digest parity. Across the matrix, treatment source is 17.5% smaller in bytes (1,604 versus 1,945) and 42.6% larger in lexical tokens (472 versus 331). This is evidence for semantic viability, not yet evidence that Lisp improves verified iteration time. The next phase adds deliberate typo repairs and timed edit sequences. The proposed source-layer contract is recorded in [the workbench specification](lisp-workbench-spec.md). diff --git a/docs/lisp-workbench-spec.md b/docs/lisp-workbench-spec.md new file mode 100644 index 00000000..9072a6c0 --- /dev/null +++ b/docs/lisp-workbench-spec.md @@ -0,0 +1,98 @@ +# Lisp-shaped ALGAL workbench + +Status: proposed source-layer specification, 2026-10-06. The workbench is an +authoring and repair interface. It is not a second runtime and it does not +change `algal.organism.v1`, `algal.expr.v1`, receipts, capabilities, or replay. + +## Boundary + +A workbench document is a bounded S-expression file. Compilation has four +phases: + +1. read UTF-8 text into a syntax tree with source spans; +2. expand only declared, pure macros under a compile fuel and output-byte bound; +3. lower the expanded tree into the existing ALGAL source AST and manifest; +4. run the same structural checks as `.algal` source. + +The output is the canonical manifest, source map, expansion digest, and +compiler diagnostics. The runtime receives only the manifest. It cannot inspect, +call, or re-evaluate the workbench reader or macro implementation. + +A workbench file may not perform host I/O, access environment variables, load +arbitrary code, call a provider, allocate an unbounded structure, mutate a +macro environment, or invoke `eval`. Macro expansion is deterministic for the +pair `(source text, compiler version, macro library digest)`. + +## Surface + +The first version admits one top-level `program` form: + +```lisp +(program quote + ((order json)) + json + (with-total subtotal + (* (get order quantity) (get order unit_price)) + (record subtotal subtotal + shipping (if (>= subtotal 50) 0 5) + total (+ subtotal (if (>= subtotal 50) 0 5))))) +``` + +Atoms are lowercase symbols, bounded numbers, JSON strings, `true`, `false`, +and `null`. Lists are limited to forms in the standard library. The initial +standard library contains `get`, `record`, `let`, `if`, `+`, `-`, `*`, `/`, `%`, +comparisons, and `with-total`. A symbol is a binding reference, never an +implicit host-language call. + +`program` has a name, a bounded parameter list, an output type, and one body. +The parameter and output types are the same `text`, `json`, record, and list +types accepted by readable ALGAL source. `let` bindings are immutable. There is +no general lambda, closure, recursive definition, actor, effect, or durable +wait form in the workbench surface. Those remain explicit ALGAL constructs and +manifest contracts. + +## Macros + +A macro is declared in the workbench library, versioned, and admitted by the +compiler. It receives syntax nodes with spans and returns syntax nodes. It may +introduce bindings only through a hygienic `gensym` operation. User data is +quoted with `quote`; generated syntax is unquoted with `splice`. Macro output +retains an expansion stack so diagnostics can point to both the generated node +and the invocation. + +Macros must declare a maximum input node count, output node count, and expansion +fuel. Recursive expansion, mutual expansion cycles, and output over the bound +are compile errors. The compiler records the library digest, each invocation's +input digest, output digest, and the final expanded-tree digest in the source +map. Two compilers must reject or produce the same expanded tree under the same +contract version. + +## Errors and repair + +Reader, expansion, lowering, and structural-check failures use +`algal.source-error.v1`. Every diagnostic contains a code, message, primary +span, source digest, and bounded expansion stack. A repair client may request a +candidate edit against a failed source snapshot. The compiler applies the edit +in memory, reruns all four phases, and returns either a new candidate manifest +or the original diagnostic. It never mutates a running process or receipt. + +A runtime failure remains a runtime failure. The workbench may open the same +receipt-backed debugger used by readable ALGAL source, showing inputs, +capabilities, prior evidence, and the failed transition. A repair can be tested +against captured inputs and replayed before it becomes a new source revision. +Resuming execution requires the normal activation and effect-reconciliation +contracts. + +## Identity and review + +The source digest covers the exact workbench bytes. The manifest digest covers +only the lowered ALGAL manifest. The expansion digest covers the canonical +expanded tree. A receipt records the manifest digest and never treats source or +macro text as executable authority. Reviewers can therefore inspect source, +expansion, manifest, and receipt as separate linked artifacts. + +The workbench is accepted as a default authoring surface only after the paired +experiment meets its gate: at least ten holdout tasks, exact manifest parity, +median verified iteration at least 20% faster, and no more than a 10% increase in +repair attempts or expansion-review failures. Until then it remains an +optional development tool and readable ALGAL source remains canonical. diff --git a/docs/north-star-execution.md b/docs/north-star-execution.md index 178a2ac9..9125822e 100644 --- a/docs/north-star-execution.md +++ b/docs/north-star-execution.md @@ -25,6 +25,16 @@ remain planned; neither the reader nor the job driver enables them by default. The [scoped-memory direction](north-star.md#scoped-memory-and-progressive-context) tracks that work; its reader foundation does not complete C0 through C4 below. +The [contract and resource experiments](north-star.md#initial-contract-and-resource-experiments) +add a follow-on direction: contract/runtime correspondence, mixed resource +requirements, suspendable effects, and measured repayment of procedure-building +costs. They do not silently enlarge C1 or complete any phase below. Each selected +experiment needs a scoped implementation plan; C0 records affected consumers and +C4 can use the resulting evidence. Cloud owns funding and hosting through its +[companion north star](https://github.com/hraness/algal-cloud/blob/main/docs/north-star.md). +Existing application/process mechanisms can support early fixtures without +waiting for the whole actor-language plan. + ## Constraints - Models propose typed programs or transitions. The host owns permissions, @@ -51,6 +61,19 @@ tracks that work; its reader foundation does not complete C0 through C4 below. - Pin supported immutable artifacts or full source revisions independently in each consumer. Do not bind repositories to moving `main` branches or local sibling paths. Preserve historical programs, receipts, and experiment pins. +- Portable resource requirements and host financial state have different owners. + ALGAL keeps deterministic work budgets, permission types, and recorded effect + semantics. Cloud or another host validates source backing, allocation terms, + tariffs, and external usage. A grant is not permission; a credential is not + funding. Add a shared record only for a demonstrated gap, with old-version + replay and Bun/Rust correspondence. A persistent allocation migration is its + own phase, with ownership mapping, dry run, and restore evidence. +- Hosted consumer pilots reserve hosting independently of provider allowances, + including deterministic/cached work and retention. Their records name the + hosting payer, backed allocation, captured customer terms, and recovery + funding. Cloud's [hosting cost-recovery plan](https://github.com/hraness/algal-cloud/blob/main/docs/hosting-cost-recovery-plan.md) + owns collection and expense reconciliation. This does not impose Cloud + pricing on local ALGAL use or make live billing a core publication gate. - Each rollout has one ALGAL integration owner and one owner per consumer. Workers edit disjoint scopes; the integration owner updates this plan's status and joins evidence. Only explicitly authorized task-owned delivery proceeds. @@ -250,15 +273,27 @@ cargo test -p xcb-runtime --locked managed_program - **Approach:** Start with two materially different validated workloads while other C3 migrations continue. Compare retained procedures with fixed-agent, no-library, fresh-synthesis, and conventional baselines under comparable - models, tools, information, and total budgets. Include maintenance and - migration costs. Follow the + models, tools, information, and total resource vectors. Include procedure + acquisition, failed trials, evaluation, maintenance, migration, retention, + and sponsored usage. Keep incompatible units separate. Distinguish actual + hosting expense, customer hosting charges, earned/collected hosting revenue, + unconsumed prepaid obligations, and subsidy. Include billing overhead and hold + tariffs/allocation rules fixed for execution comparisons. Resource-aware + pilots include a sponsored non-revenue workload rather than assuming all + backing arrives through customer payments. Freeze the horizon, quality/recovery + floors, sample/uncertainty method, minimum useful improvement, and stop rule + before search. Further tuning after confirmation requires a new sealed + holdout. Follow the [cumulative-skill experiment](cumulative-skill-experiment.md); retain failures and unknown provider usage. New core changes reopen their impact rows. - **Acceptance criteria:** Report held-out task success, correction effort, total cost, reuse, stale-work rejection, recovery outcomes, and adoption coverage with denominators. Evidence identifies the tested integration and - unverified claims. Each future contract, executor, storage, or workflow change - assigns consumer impact and follow-up owners before it is called complete. + unverified claims. A Cloud hosting pilot identifies its cost-recovery evidence + or unresolved status; a good procedure score cannot substitute for collected + hosting charges or qualified billing activation. Each future contract, + executor, storage, or workflow change assigns consumer impact and follow-up + owners before it is called complete. - **Validation:** Run the study and verification commands recorded for each workload, consume the exact C1–C3 conformance evidence, and independently review the comparisons. A mechanism-only outcome is a valid result when diff --git a/docs/north-star.md b/docs/north-star.md index 030a76bd..4d618c50 100644 --- a/docs/north-star.md +++ b/docs/north-star.md @@ -1,6 +1,8 @@ # ALGAL north star: a portable runtime for evolving habitats -Status: proposed architecture and delivery plan, 2026-10-01. +Status: proposed architecture and delivery plan, 2026-10-01. Contract-first +behavior, resource endowments, and their hill-climbing criteria revised +2026-10-04. These additions do not activate a new runtime or funding contract. This document is the planning authority for the next ALGAL runtime direction. It turns the existing vision of software that accumulates competence into a set of @@ -43,11 +45,27 @@ fixture, and verify the resulting evidence offline. Runtime speed matters, but it is subordinate to a language that is cheap to change and a result that is cheap to trust. +The complete workbench boundary is specified in [the Lisp-shaped ALGAL workbench spec](lisp-workbench-spec.md). The first Lisp comparison is now evidence-bearing. Eleven paired fixtures lower through the real compiler with exact manifest parity; treatment source is 17.5% smaller in bytes but 42.6% larger in lexical tokens. This rules out a token-saving reason to redirect the runtime or replace the incumbent source language. The provisional direction is to keep ALGAL’s durable runtime and readable source as the contract, while exploring a Lisp-shaped authoring and repair workbench that lowers into that contract. The [Lisp-shaped source experiment](lisp-source-experiment.md) still requires timed edit sequences and deliberate repairs before the workbench becomes a default surface. + The first implementation milestone is not a distributed BEAM replacement. It is a small actor/event laboratory that demonstrates the same behavior in the Bun reference runtime, the Rust kernel and a hosted habitat, with exact traces and measured authoring and execution costs. +A habitat should be able to receive a mandate and a **resource endowment**: +allocations of money, model tokens, provider credits, compute, storage, or +sponsored service usage. It can invest some of those resources in developing +procedures and use the rest to operate them. Owners, investors, sponsors, and +customers are different sources of backing; earning revenue is optional. The +host supplies and enforces allocations. ALGAL supplies portable requirements, +limited execution, and evidence, without becoming a payment processor. + +The outcome metric is useful accepted work under a fixed total resource +allocation, including the cost of acquiring and maintaining procedures. +Verified iteration time remains the leading developer-loop metric. A faster +loop is valuable when it makes tested improvements cheaper to produce; neither +a bigger balance nor more generated programs establishes improvement. + The long-term unit of progress is an ecology, not an agent count. An organism is a bounded executable procedure; an agent is a durable process using one or more organisms; a habitat supplies storage, scheduling, capabilities, evidence @@ -140,6 +158,156 @@ Bun is the iteration reference. Rust is the native kernel. WASM is a portability artifact. Cloudflare is a hosted habitat. Browsers, terminals, desktop apps and local bridges are hosts. None should define a separate language semantics. +## Contract-first behavior and verification + +[Temper](lineage.md#temper-contract-first-behavior) is a reference for joining +state machines, policy, verification, and revision testing. ALGAL should adopt +the following mechanisms through its existing application, expression, effect, +and host contracts: + +| Mechanism | ALGAL requirement | +| --- | --- | +| Executable state-machine contracts | One versioned interpretation of states, events, guards, completion predicates, and declared effects drives checking and execution. Unsupported predicates are explicit unresolved results, never implicit passes. | +| Typed policy and assigned identity | The host supplies the acting principal, permission scope, and policy revision. Model text cannot assign its own role, sponsor, or spending authority. | +| Verification cascade | Cheap structural checks precede generated histories, reference-model comparison, and fault tests. Each result names which properties and state space it covered. | +| Composite invariants | Check relevant combinations of process, mailbox, revision, and resource state, not only isolated records. A state-space limit produces an incomplete result. | +| Shadow revision testing | Compare incumbent and candidate on identical captured events and sealed effect results without repeating live effects. A passing corpus is not universal equivalence. | +| Evolution records and repair feedback | Retain the observation, hypothesis, candidate, counterexample, evaluation cost, decision, and later outcome in existing lineage records. | + +The first implementation should exercise a small transition contract using +`algal.expr.v1` and current application/process records before enlarging the +source language. Checkers, runtime decisions, and explanations must share +parsed predicates or demonstrate correspondence with the same fixtures. +Diagnostics identify the rejected transition, source location where available, +policy or resource condition, and a minimized reproducer. A repair proposal +cannot make itself valid by weakening the failed condition. + +Do not import Temper's CSDL/OData interface, Cedar policy stack, actor backend, +or verification tools wholesale. ALGAL keeps data-only manifests and one +semantic core. Cloud keeps its scoped TLA+/Lean work and implementation tests. +The transferable lesson is correspondence between a contract, its checks, and +its execution, rather than the number of proof tools in the build. + +## Resource-endowed habitats + +This is a proposed extension of host requirements and work accounting, not a +claim that the current `algal.habitat-budget.v1` record is a general wallet. +That record accounts for deterministic work, attempts, and runs. Financial +balances, provider usage, expiry observations, and storage billing belong to +hosts and separately recorded evidence. Historical parsers, receipts, and +budget semantics must remain readable under their original versions. + +### Funding, permission, and usage are different facts + +A habitat needs a host-defined mandate: its purpose, acceptance conditions, +permissions, resource policy, and rules for replacing a procedure. Backing can +arrive before any work, recur on a schedule, depend on a milestone, or come from +customers. A company-sponsored research habitat and a seed-funded business use +the same execution model; only the host's resource sources and objectives vary. + +| Concept | Responsibility | +| --- | --- | +| Capability | Permission to perform a specified effect; a resource allowance does not grant it. | +| Allocation | A host-issued allowance for a named resource, source, scope, unit, purpose, and validity period; a credential is not proof that the allowance exists. | +| Requirement and reservation | Resources needed by one selected execution route, reserved before its effect can start. | +| Usage and settlement evidence | What was observed, what was charged or released, and what remains unknown, bound to the original operation and allocation. | +| Resource policy | Who may allocate, renew, convert, revoke, or move resources, and which work must remain funded. The candidate cannot edit it. | + +An endowment is a portfolio of non-fungible allowances. Model tokens are tied to +a provider/model and relevant input/output classes; calls, CPU time, storage over +time, currency, and concurrent capacity retain their own units. Available money +cannot silently substitute for an expired tool grant. A quoted conversion is a +separate authorized host effect. Dollar equivalents can support planning but +cannot create spendable resources. + +The portable boundary describes requirements, allocation references, refusal +reasons, and recorded outcomes. Hosts validate backing, identity, current time, +revocation, and provider limits. Time and external observations enter as recorded +inputs; tariffs, credentials, and wall-clock measurements do not enter pure +execution receipts. Host observations do not acquire external truth merely by +being hashed or replayed. + +### Investment and operation share an account, not a free allowance + +Resource purpose is independent of resource type: + +- **Build:** acquire procedures, synthesize candidates, run evaluations, and + prepare reusable artifacts. This is the capex analogue, not a claim about + financial asset recognition. +- **Operate:** perform accepted work and maintain the state and services it needs. +- **Protect:** reconcile started work, retain required evidence, handle existing + obligations, and pause or export safely. + +Candidate generation, failed trials, evaluation, memory maintenance, and +migration consume the same declared total allocation as later execution. A +procedure cannot fund its own search by taking resources reserved for accepted +work. Seed funding and follow-on grants remain distinguishable from revenue. +The runtime does not require equity instruments, transferable tokens, a +marketplace, or a profitability objective. + +### Admission and continuation + +A route may start only when its permissions and each required resource match a +current allocation. One missing dimension produces a named refusal or an +explicitly permitted alternative; it does not mean every activity must stop. +Only the host may release new funding after a milestone. A candidate's claim +that the milestone passed is insufficient. + +Use short transitions to prepare an effect, checkpoint its continuation, and +later consume its recorded result. Bind the operation to the input, procedure, +policy, route, and allocation revisions. A duplicate result cannot spend twice; +an expired or revoked grant blocks new dispatch without erasing obligations +from earlier dispatch. An uncertain outcome keeps its operation and unresolved +reservation until reconciliation. Pausing a process never proves cancellation +at the provider. A late result must still reconcile its original resource +obligation after revocation or a procedure change, even when a stale continuation +is forbidden from publishing into the new program state. + +A hard resource ceiling must identify its enforcement mechanism. A usage estimate +or a shared external subscription is not such a ceiling. Hosts must record +unexpected charges and exposure rather than clip the observation to a budget. +This does not weaken a current contract that only accepts charges within its +reservation: richer host evidence needs an explicit compatible representation +or a versioned extension, with cross-runtime tests. + +### Hosted work must fund its host + +A hosted route requires a hosting allocation independently of its model/tool +allowances. Deterministic work, cache hits, evidence retention, and recovery can +cost the host resources even when no provider tokens are billed. An external +token grant cannot settle those hosting charges. The host binds a named payer, +backed allowance, accepted price/charge limit, and retention term to the work; +those financial records remain outside pure execution receipts. + +Reserve hosting and protected completion capacity before admission. Stop new +unfunded work without discarding already incurred obligations. A host records +actual overruns separately from the customer's permitted charge; a bug or +unapproved retry cannot enlarge that charge. Existing provider settlement +contracts retain their own rules. Local ALGAL use does not require a paid account. + +Cloud's [cost-recovery gate](https://github.com/hraness/algal-cloud/blob/main/docs/north-star.md#cost-recovery-acceptance-gate) +and [execution plan](https://github.com/hraness/algal-cloud/blob/main/docs/hosting-cost-recovery-plan.md) +require earned and collected hosting charges to cover attributed expenses and +the declared margin, or identify a bounded subsidy. Unused seed deposits, +provider pass-through, and internally issued credit do not prove hosting +revenue. Compare underlying resource use separately from customer charges; +raising a tariff is not an execution improvement. ALGAL supplies portable +requirements and refusal/effect identities, not Cloud's pricing or collection. + +### Ownership across the projects + +ALGAL owns pure transition semantics, resource requirement/refusal shapes where +portable, effect identity, suspension, replay, and evaluation. It stays useful +without Cloud or a money balance. XCB, Ghostget, and other host adapters keep +credential ownership and service-specific execution. + +[Cloud's north star](https://github.com/hraness/algal-cloud/blob/main/docs/north-star.md) +owns funding sources, durable allocations, platform pricing, source-pool +conservation, payment reversals, retention, and Cloudflare cost measurement. +Every shared contract change joins through immutable dependency pins and +consumer tests. No repository may independently invent a second resource or +continuation protocol under the same version. + ## A compact semantic model A habitat contains actors. An actor is a named behavior and state machine: @@ -273,14 +441,25 @@ the seed for behavior revision and migration. ### Cloud habitat `algal-cloud` should remain a hosted host, not a second ALGAL implementation. -Its Durable Object is a serialized habitat node. The cloud protocol should expose -actor deployment, event ingress, mailbox delivery, scheduling, state/CAS, -metering, evidence export and behavior activation through the same contracts as -the local node. - -The existing broker remains an executor boundary. XCB and Ghostget remain -credential-owning provider/tool systems. The habitat never receives upstream -provider credentials. +The initial ownership model is one serialized Durable Object per habitat, with +short state and resource transitions. Slow provider work should be eligible for +a durable handoff to an effect Worker, allowing the object to become idle until +a result arrives. Queue delivery is a notification; the operation journal is +the authority for dispatch and reconciliation. Moving a fetch into a Worker +while the object awaits it does not create this separation. + +The cloud protocol should expose event ingress, process continuation, +state/CAS, resource admission, evidence export, and behavior activation through +the same portable contracts as the local node. Immutable bulk evidence can use +separate storage without changing historical digests or tenant access rules. +Cloud must measure this profile against the simpler inline-DO incumbent before +making it a default. The language does not require Queues, R2, or Workflows. + +The broker remains a privileged executor boundary, while Cloud's funding design +assigns one authoritative reservation owner for each operation. XCB and Ghostget +remain credential-owning provider/tool systems. Habitat programs never receive +upstream provider credentials. Platform hosting needs its own funded allocation +even when inference or tools are supplied by a sponsor. ### Scoped memory and progressive context @@ -376,12 +555,13 @@ durability rules: - Humans, editors and agents use the same typed operations. Their authority and presentation can differ, but they do not get separate mutation semantics. -This gives ALGAL a useful three-way synthesis: +The design combines these mechanisms: | Source of inspiration | What ALGAL takes | What ALGAL adds or constrains | |---|---|---| | BEAM/OTP | processes, mailboxes, supervision, distribution, upgrades | durable evidence, bounded resources and explicit uncertain effects | | Live World/Lisp environment | stable identity, live inspection, semantic mutation, capability eventual-send | revision heads, migrations, replay and host-admitted authority | +| Temper | executable behavior contracts, staged verification, composite checks, shadow revisions | shared predicate semantics, explicit incomplete results, sealed effects, and fixed resource policy | | ALGAL today | manifests, receipts, replay, habitats, foundry and application lifecycle | a single actor/event model that joins them | The first implementation should not attempt a universal live object database. A @@ -489,6 +669,31 @@ or network effects. These are local engineering targets. They are not hosted SLOs until qualified by `algal-cloud` under its own capacity and failure gates. +### Resource and investment scorecard + +Resource-aware work adds the following measurements to the developer and runtime +scorecards. Accepted work meets the domain's completion and quality conditions; +an accepted message or completed interpreter slice is not enough. Freeze each +experiment's resource vector, task horizon, acceptance floors, sample count, +and minimum useful improvement before running candidates. +Targets without a measured baseline remain proposed, not green. + +| Measure | Required comparison | +| --- | --- | +| Useful accepted work | Accepted tasks per fixed allocation, including failures and refusals; report acceptance rate and worst task-family quality beside throughput. | +| Total resource use | Generation, evaluation, unsuccessful attempts, operation, maintenance, migration, and retention by original unit and funding source. Unknown usage remains unknown. | +| Investment repayment | Later resource savings or quality gains against fixed-agent, conventional-workflow, fresh-synthesis, and no-library controls over the same horizon. Forecast and observed repayment are separate. | +| Refusal and repair quality | Wrong-scope, exhausted, expired, revoked, and uncertain cases localize the condition; measure edits to repair without widening permission or funding. | +| Hosted cost coverage | Payer-backed hosting allowances, actual platform expenses, customer charges/collections, remaining prepaid obligations, and explicit subsidy are reported separately. Cloud owns the cost-recovery gate; provider credits and unused deposits cannot satisfy it. | +| Continuation cost | Active host time, checkpoint bytes, commits, billing overhead, and recovery effort per completed effect, separated from external waiting time. | +| Composition safety | Joint histories catch double reservation, stale activation, and missing child obligations; report explored bounds and incomplete checks. | + +Do not sum incompatible token, time, storage, currency, and capacity units into +one score. Compare under identical resource limits or retain a Pareto frontier: +variants for which no other eligible variant is better in every measured +dimension. A host may apply a frozen valuation policy to economic comparisons, +but changing that policy is a separate experiment and never a resource conversion. + ## Ecology progression The runtime scorecard is the first layer of a longer progression. Each layer @@ -919,6 +1124,12 @@ The architecture should be revised if any of these fail: - Consumer pilots still need private scheduler or receipt implementations. - Behavior evolution improves benchmark scores but makes production diagnosis, migration or rollback harder. +- A resource abstraction makes local execution depend on a payment account, + hides non-fungible limits, or duplicates the existing budget and effect model. +- Claimed investment gains disappear when sponsor resources, failures, + evaluation, maintenance, and retention are charged to both study arms. +- Contract verification and runtime enforcement disagree on supported predicates, + or an incomplete composite check is needed to justify activation. A failed target is useful evidence. It should produce a changed target, architectural simplification or explicit non-goal, not silent benchmark removal. @@ -957,6 +1168,8 @@ These are product invariants, not optional test features: | Deterministic execution | Pure cells produce the same result in Bun and Rust. Receipts replay bit for bit; nondeterminism enters only through recorded executor results. | | Authority and effects | A manifest cannot mint a capability, widen a port, or approve its own successor. Host effects are typed, idempotent where possible, and explicit about uncertain completion. | | Durable lifecycle | Queue delivery, suspension, retry, timer, journal, restart, and recovery preserve the committed prefix and never silently repeat an uncertain effect. | +| Resource integrity | Permission and funding are checked separately. Units, source scope, purpose, and operation identity survive reservation and reconciliation. Expiry cannot erase earlier obligations; unknown usage cannot become free capacity. | +| Contract correspondence | Runtime, checker, and diagnostics agree on the supported predicates. Unsupported predicates and incomplete joint-state exploration remain unresolved. | | Evaluation integrity | Development, selection, holdout, and frozen cases remain disjoint. Candidate cost, failures, inconclusive results, and maintenance work are retained. | | Portability | The reference runtime, native kernel, expression WASM target, and supported adapters agree on canonical values, rejection behavior, and evidence. | @@ -1010,7 +1223,10 @@ candidate. 1. **State a falsifiable hypothesis.** Name the workload, expected benefit, risk, budget, and invariant that could disprove it. 2. **Freeze the comparison.** Pin the incumbent digest, model and tool - versions, task splits, random seeds, environment, and total work budget. + versions, task splits, random seeds, environment, and total resource vector. + Record the permitted change, task horizon, quality/recovery floors, sampling + and uncertainty method, minimum useful improvement, stop rule, and any host + pricebook or valuation revision. A sponsor grant is part of the budget. 3. **Generate bounded variants.** Prefer a small change to routing, procedure composition, representation, or resource policy. Keep weaker ancestors when they expose useful components or diversity. @@ -1020,11 +1236,15 @@ candidate. 5. **Evaluate in stages.** Use development cases for search, a frozen selection set for promotion, and an untouched holdout for confirmation. Include a no-change control and a fresh-synthesis control with comparable budgets. -6. **Score eligible candidates.** First require every hard gate. Then compare - held-out quality and worst-slice quality, followed by fault survival and - recovery, total cost and latency, and finally artifact size and maintenance - complexity. Reject a candidate that wins on average by creating a severe tail - regression. + Do not tune against the confirmation holdout after reading its result; + further search needs a new sealed split and a separately recorded experiment. +6. **Score eligible candidates.** Require every hard gate and the preregistered + quality, worst-slice, and recovery floors. Compare useful accepted work under + equal resources, then total resource use and latency, and finally artifact + size and maintenance complexity. Resource-saving studies hold task quality + and recovery within their declared non-regression limits. Reject a candidate + that wins on average by creating a severe tail regression or refusing the + expensive part of the workload. 7. **Retain the lineage.** Store the candidate, receipt, tests, costs, failures, inconclusive cases, and counterexamples. Keep a small diverse archive rather than only the current winner. @@ -1042,6 +1262,28 @@ protocol invariants first; improve successful work under adversarial schedules second; reduce latency, memory, and storage third; and reduce code and operating complexity fourth. No average score overrides a hard gate. +### Initial contract and resource experiments + +These are proposed experiments, not completed phases. They use current +application/process seams where possible and do not wait for the whole actor +language or a commercial Cloud launch. The selected implementation gets a +scoped phase plan with literal checks before code changes; the existing +[north-star execution plan](north-star-execution.md) remains the consumer +adoption authority. + +| Experiment | Frozen incumbent and smallest challenger | Required evidence and falsifier | +| --- | --- | --- | +| A0: contract correspondence | Current application guards versus shared parsed predicates and structured refusal explanations on the same transitions | Matching runtime/checker decisions, unsupported-predicate refusal, a caught known-bad mutation, and fewer repair iterations. Stop if a second policy language is needed for the same checks. | +| A1: mixed resource requirements | Current work/attempt/run limits versus an additive host-requirement fixture with inference and tool allocations | Bun/Rust agreement on matching, exhaustion, wrong units, expiry, revocation, and uncertainty. A hosted fixture with free inference still requires its independent hosting allowance. Existing budget receipts remain unchanged. Stop if the pure kernel must manage provider accounts or credentials. | +| A2: suspendable effects | Current hosted effect path versus checkpoint/handoff/resume on identical recorded outcomes | Same results and replay, no uncertain redispatch, preserved obligations, and lower active-host cost on the declared slow-work mix. Cloud owns hosting measurement, not ALGAL receipt timing. | +| A3: investment repayment | Fixed, fresh-synthesis, conventional, and no-library controls versus a retained procedure library | Later unseen-task improvement under equal endowments after search, evaluation, maintenance, migration, and failures. Retain a negative finding if repayment is absent over the frozen horizon. | + +Each record names an owner and records one outcome: supported for the declared +scope, rejected, inconclusive, or blocked. No baseline means no improvement +claim. A selected candidate is not operationally active until the applicable +maturity gates pass. Preserve both successful and failed records; do not mark +A0 through A3 complete because their plans or fixtures exist. + ### Maturity gates | Gate | Evidence required | @@ -1077,6 +1319,12 @@ implementation lanes reviewable. ## Research record +The 2026-10-04 revision uses [Temper's recorded mechanisms and limits](lineage.md#temper-contract-first-behavior) +and [Valhalla's north-star hill-climbing strategy at `2bd2b9e`](https://github.com/hraness/valhalla/blob/2bd2b9ece95f44aefc8a6e29e1362c52b0568fbf/docs/north-star.md). +Valhalla supplies the safety-first comparison discipline, not evidence for +ALGAL's economic or capability claims. Cloud's companion north star owns the +Cloudflare pricing references and hosted experiment baseline. + The PLDB index and search result pages consulted for this plan are public indexes, not claims that ALGAL implements the cited systems: diff --git a/docs/vision.md b/docs/vision.md index 48246359..a2101074 100644 --- a/docs/vision.md +++ b/docs/vision.md @@ -371,7 +371,19 @@ of competing to be the one best procedure. ALGAL's manifests and evidence contracts are components of that possibility. A public ecosystem, portable trust, and automatic cross-domain transfer do not exist yet. -Across all three, the shift is the same: software development becomes partly +**Funded autonomous projects and businesses.** An owner, investor, or sponsor +could give a habitat a purpose and an initial allocation of money, model tokens, +compute, storage, or service usage. The habitat would spend a permitted portion +developing reusable procedures, use them to perform work, and seek further +backing when appropriate. Customer revenue could replenish resources, but an +internal service or research project would not need to sell anything. Permission +to use a service and an allowance to consume it remain separate. Different +resources retain their own units, expiry, and restrictions; an estimated cash +value does not make them interchangeable. The [north-star resource model](north-star.md#resource-endowed-habitats) +and [Cloud's proposed direction](https://github.com/hraness/algal-cloud/blob/main/docs/north-star.md) +define the host responsibilities and comparative tests for this extension. + +Across these uses, the shift is the same: software development becomes partly the design of environments in which procedures are discovered, tested, retained, and governed. Human engineering does not disappear. More of its effort moves toward defining interfaces, constraints, evaluators, and the diff --git a/experiments/lisp-source/README.md b/experiments/lisp-source/README.md new file mode 100644 index 00000000..3a689a4a --- /dev/null +++ b/experiments/lisp-source/README.md @@ -0,0 +1,24 @@ +# Lisp source experiment + +This experiment compares the current readable ALGAL source surface with a small +Lisp-shaped authoring surface that lowers to the same bounded ALGAL expressions. +It tests authoring ergonomics, source compactness, macro expansion visibility, +and repair diagnostics. It does not compare runtimes or claim that Common Lisp +should replace ALGAL. + +The incumbent and treatment must produce the same canonical expression and +receipt for each fixture. The treatment is intentionally small: symbols, +literals, `let`, `if`, arithmetic, records, and one hygienic-looking `with-total` +macro expanded by the experiment compiler. The macro output is printed and +hashed so generated code remains inspectable. + +Run from this repository: + +```sh +bun experiments/lisp-source/run.ts +``` + +The runner emits a deterministic JSON report under `results/latest.json` with +source bytes, token counts, expansion bytes, and exact manifest digest parity for +every paired fixture. It is a development ergonomics experiment, not evidence of cumulative +skill or production language readiness. diff --git a/experiments/lisp-source/fixtures/clamp.algal b/experiments/lisp-source/fixtures/clamp.algal new file mode 100644 index 00000000..47200f83 --- /dev/null +++ b/experiments/lisp-source/fixtures/clamp.algal @@ -0,0 +1,6 @@ +program clamp(order: json) -> json { + budget { max_agent_calls: 0 } + + let base = if order.amount < 0 { 0 } else { order.amount } + return { amount: base } +} diff --git a/experiments/lisp-source/fixtures/clamp.lisp b/experiments/lisp-source/fixtures/clamp.lisp new file mode 100644 index 00000000..6071bcfd --- /dev/null +++ b/experiments/lisp-source/fixtures/clamp.lisp @@ -0,0 +1,2 @@ +(program clamp ((order json)) json + (let base (if (< (get order amount) 0) 0 (get order amount)) (record amount base))) diff --git a/experiments/lisp-source/fixtures/discount.algal b/experiments/lisp-source/fixtures/discount.algal new file mode 100644 index 00000000..4eb84638 --- /dev/null +++ b/experiments/lisp-source/fixtures/discount.algal @@ -0,0 +1,6 @@ +program discount(order: json) -> json { + budget { max_agent_calls: 0 } + + let discount = order.total * 0.1 + return { total: order.total - discount } +} diff --git a/experiments/lisp-source/fixtures/discount.lisp b/experiments/lisp-source/fixtures/discount.lisp new file mode 100644 index 00000000..df1ead68 --- /dev/null +++ b/experiments/lisp-source/fixtures/discount.lisp @@ -0,0 +1,2 @@ +(program discount ((order json)) json + (let discount (* (get order total) 0.1) (record total (- (get order total) discount)))) diff --git a/experiments/lisp-source/fixtures/eligibility.algal b/experiments/lisp-source/fixtures/eligibility.algal new file mode 100644 index 00000000..6aca775e --- /dev/null +++ b/experiments/lisp-source/fixtures/eligibility.algal @@ -0,0 +1,6 @@ +program eligibility(order: json) -> json { + budget { max_agent_calls: 0 } + + let eligible = order.score >= 80 + return { eligible: eligible } +} diff --git a/experiments/lisp-source/fixtures/eligibility.lisp b/experiments/lisp-source/fixtures/eligibility.lisp new file mode 100644 index 00000000..824abffb --- /dev/null +++ b/experiments/lisp-source/fixtures/eligibility.lisp @@ -0,0 +1,2 @@ +(program eligibility ((order json)) json + (let eligible (>= (get order score) 80) (record eligible eligible))) diff --git a/experiments/lisp-source/fixtures/gross.algal b/experiments/lisp-source/fixtures/gross.algal new file mode 100644 index 00000000..fd0fb697 --- /dev/null +++ b/experiments/lisp-source/fixtures/gross.algal @@ -0,0 +1,7 @@ +program gross(order: json) -> json { + budget { max_agent_calls: 0 } + + let gross = order.price * order.quantity + let tax = gross * 0.08 + return { gross: gross, tax: tax, total: gross + tax } +} diff --git a/experiments/lisp-source/fixtures/gross.lisp b/experiments/lisp-source/fixtures/gross.lisp new file mode 100644 index 00000000..bb1f0d40 --- /dev/null +++ b/experiments/lisp-source/fixtures/gross.lisp @@ -0,0 +1,2 @@ +(program gross ((order json)) json + (let gross (* (get order price) (get order quantity)) (let tax (* gross 0.08) (record gross gross tax tax total (+ gross tax))))) diff --git a/experiments/lisp-source/fixtures/margin.algal b/experiments/lisp-source/fixtures/margin.algal new file mode 100644 index 00000000..d9292cf7 --- /dev/null +++ b/experiments/lisp-source/fixtures/margin.algal @@ -0,0 +1,7 @@ +program margin(order: json) -> json { + budget { max_agent_calls: 0 } + + let cost = order.units * order.cost + let revenue = order.units * order.price + return { margin: revenue - cost } +} diff --git a/experiments/lisp-source/fixtures/margin.lisp b/experiments/lisp-source/fixtures/margin.lisp new file mode 100644 index 00000000..e9067848 --- /dev/null +++ b/experiments/lisp-source/fixtures/margin.lisp @@ -0,0 +1,2 @@ +(program margin ((order json)) json + (let cost (* (get order units) (get order cost)) (let revenue (* (get order units) (get order price)) (record margin (- revenue cost))))) diff --git a/experiments/lisp-source/fixtures/priority.algal b/experiments/lisp-source/fixtures/priority.algal new file mode 100644 index 00000000..42209140 --- /dev/null +++ b/experiments/lisp-source/fixtures/priority.algal @@ -0,0 +1,7 @@ +program priority(order: json) -> json { + budget { max_agent_calls: 0 } + + let urgent = order.score >= 90 + let tier = if urgent { 1 } else { 2 } + return { tier: tier } +} diff --git a/experiments/lisp-source/fixtures/priority.lisp b/experiments/lisp-source/fixtures/priority.lisp new file mode 100644 index 00000000..99126b6a --- /dev/null +++ b/experiments/lisp-source/fixtures/priority.lisp @@ -0,0 +1,2 @@ +(program priority ((order json)) json + (let urgent (>= (get order score) 90) (let tier (if urgent 1 2) (record tier tier)))) diff --git a/experiments/lisp-source/fixtures/quote.algal b/experiments/lisp-source/fixtures/quote.algal new file mode 100644 index 00000000..99df0957 --- /dev/null +++ b/experiments/lisp-source/fixtures/quote.algal @@ -0,0 +1,7 @@ +program quote(order: json) -> json { + budget { max_agent_calls: 0 } + + let subtotal = order.quantity * order.unit_price + let shipping = if subtotal >= 50 { 0 } else { 5 } + return { subtotal: subtotal, shipping: shipping, total: subtotal + shipping } +} diff --git a/experiments/lisp-source/fixtures/quote.lisp b/experiments/lisp-source/fixtures/quote.lisp new file mode 100644 index 00000000..cc44c822 --- /dev/null +++ b/experiments/lisp-source/fixtures/quote.lisp @@ -0,0 +1,4 @@ +(program quote ((order json)) json + (with-total subtotal (* (get order quantity) (get order unit_price)) + (let shipping (if (>= subtotal 50) 0 5) + (record subtotal subtotal shipping shipping total (+ subtotal shipping))))) diff --git a/experiments/lisp-source/fixtures/ratio.algal b/experiments/lisp-source/fixtures/ratio.algal new file mode 100644 index 00000000..eda3f7cd --- /dev/null +++ b/experiments/lisp-source/fixtures/ratio.algal @@ -0,0 +1,6 @@ +program ratio(order: json) -> json { + budget { max_agent_calls: 0 } + + let weighted = order.a * 2 + order.b * 3 + return { weighted: weighted } +} diff --git a/experiments/lisp-source/fixtures/ratio.lisp b/experiments/lisp-source/fixtures/ratio.lisp new file mode 100644 index 00000000..8f01bb73 --- /dev/null +++ b/experiments/lisp-source/fixtures/ratio.lisp @@ -0,0 +1,2 @@ +(program ratio ((order json)) json + (let weighted (+ (* (get order a) 2) (* (get order b) 3)) (record weighted weighted))) diff --git a/experiments/lisp-source/fixtures/shipping.algal b/experiments/lisp-source/fixtures/shipping.algal new file mode 100644 index 00000000..fa15b6b2 --- /dev/null +++ b/experiments/lisp-source/fixtures/shipping.algal @@ -0,0 +1,6 @@ +program shipping(order: json) -> json { + budget { max_agent_calls: 0 } + + let shipping = if order.total >= 50 { 0 } else { 5 } + return { shipping: shipping } +} diff --git a/experiments/lisp-source/fixtures/shipping.lisp b/experiments/lisp-source/fixtures/shipping.lisp new file mode 100644 index 00000000..c06c4fc0 --- /dev/null +++ b/experiments/lisp-source/fixtures/shipping.lisp @@ -0,0 +1,2 @@ +(program shipping ((order json)) json + (let shipping (if (>= (get order total) 50) 0 5) (record shipping shipping))) diff --git a/experiments/lisp-source/fixtures/subtotal.algal b/experiments/lisp-source/fixtures/subtotal.algal new file mode 100644 index 00000000..2333a499 --- /dev/null +++ b/experiments/lisp-source/fixtures/subtotal.algal @@ -0,0 +1,6 @@ +program subtotal(order: json) -> json { + budget { max_agent_calls: 0 } + + let subtotal = order.quantity * order.unit_price + return { subtotal: subtotal } +} diff --git a/experiments/lisp-source/fixtures/subtotal.lisp b/experiments/lisp-source/fixtures/subtotal.lisp new file mode 100644 index 00000000..bd38b20c --- /dev/null +++ b/experiments/lisp-source/fixtures/subtotal.lisp @@ -0,0 +1,2 @@ +(program subtotal ((order json)) json + (let subtotal (* (get order quantity) (get order unit_price)) (record subtotal subtotal))) diff --git a/experiments/lisp-source/fixtures/totals.algal b/experiments/lisp-source/fixtures/totals.algal new file mode 100644 index 00000000..e2af794f --- /dev/null +++ b/experiments/lisp-source/fixtures/totals.algal @@ -0,0 +1,7 @@ +program totals(order: json) -> json { + budget { max_agent_calls: 0 } + + let net = order.subtotal - order.discount + let fee = if net >= 100 { 0 } else { 10 } + return { net: net, fee: fee, due: net + fee } +} diff --git a/experiments/lisp-source/fixtures/totals.lisp b/experiments/lisp-source/fixtures/totals.lisp new file mode 100644 index 00000000..fb874ff9 --- /dev/null +++ b/experiments/lisp-source/fixtures/totals.lisp @@ -0,0 +1,2 @@ +(program totals ((order json)) json + (let net (- (get order subtotal) (get order discount)) (let fee (if (>= net 100) 0 10) (record net net fee fee due (+ net fee))))) diff --git a/experiments/lisp-source/results/latest.json b/experiments/lisp-source/results/latest.json new file mode 100644 index 00000000..c54d78a7 --- /dev/null +++ b/experiments/lisp-source/results/latest.json @@ -0,0 +1,134 @@ +{ + "experiment": "algal.lisp-source.v1", + "status": "parity-passed", + "taskCount": 11, + "cases": [ + { + "name": "clamp", + "incumbentBytes": 159, + "incumbentTokens": 27, + "treatmentBytes": 121, + "treatmentTokens": 38, + "loweredBytes": 161, + "parity": true, + "manifestDigest": "37dfc31da00a22b7176bf1cfbacda45a2d680ca81c8510a3d6a6191cc3f1fac0" + }, + { + "name": "discount", + "incumbentBytes": 153, + "incumbentTokens": 25, + "treatmentBytes": 128, + "treatmentTokens": 37, + "loweredBytes": 157, + "parity": true, + "manifestDigest": "640adc1007dc35ad9709f24e53a3d2426b457b9fbdba6501568ae00e2a4c1ed3" + }, + { + "name": "eligibility", + "incumbentBytes": 145, + "incumbentTokens": 23, + "treatmentBytes": 112, + "treatmentTokens": 30, + "loweredBytes": 147, + "parity": true, + "manifestDigest": "63fc45fed6e03592923a1eee5e7b61c45692d803b875557db2794b3e057f1587" + }, + { + "name": "gross", + "incumbentBytes": 196, + "incumbentTokens": 35, + "treatmentBytes": 167, + "treatmentTokens": 50, + "loweredBytes": 202, + "parity": true, + "manifestDigest": "894d234b6d5d06bf04fbf7cf45706e425b1dbd646b1576cdd9b27ca142df3276" + }, + { + "name": "margin", + "incumbentBytes": 189, + "incumbentTokens": 33, + "treatmentBytes": 176, + "treatmentTokens": 54, + "loweredBytes": 195, + "parity": true, + "manifestDigest": "fa690840e90f73de1c9927efd6aa24ff3439ab8d785ed5a4794ba4bfc674cdf6" + }, + { + "name": "priority", + "incumbentBytes": 172, + "incumbentTokens": 31, + "treatmentBytes": 126, + "treatmentTokens": 40, + "loweredBytes": 174, + "parity": true, + "manifestDigest": "e57e3b8747a8e1ff70aff7ad8b898618d4b6e409afd473eb85e388d568d41580" + }, + { + "name": "quote", + "incumbentBytes": 255, + "incumbentTokens": 40, + "treatmentBytes": 232, + "treatmentTokens": 58, + "loweredBytes": 261, + "parity": true, + "manifestDigest": "496f8ded0535313e7d1a6797e13d6d4eff0641fc962998cafda3242d0c45f3d4" + }, + { + "name": "ratio", + "incumbentBytes": 147, + "incumbentTokens": 27, + "treatmentBytes": 124, + "treatmentTokens": 41, + "loweredBytes": 153, + "parity": true, + "manifestDigest": "1115cebd9ce181ba9c69acd6342c572626229c21dda5834e463cbd382f4ded3a" + }, + { + "name": "shipping", + "incumbentBytes": 162, + "incumbentTokens": 27, + "treatmentBytes": 118, + "treatmentTokens": 35, + "loweredBytes": 164, + "parity": true, + "manifestDigest": "4cd0d5d4c1903d25880e9a8eb11a340163fbd28981cf448b92610491e69132ae" + }, + { + "name": "subtotal", + "incumbentBytes": 158, + "incumbentTokens": 23, + "treatmentBytes": 131, + "treatmentTokens": 33, + "loweredBytes": 160, + "parity": true, + "manifestDigest": "8aaa5369a8a7de2b60f041cda670852e97fbb5f128dd77207d36b3c34c21f5ec" + }, + { + "name": "totals", + "incumbentBytes": 209, + "incumbentTokens": 40, + "treatmentBytes": 169, + "treatmentTokens": 56, + "loweredBytes": 215, + "parity": true, + "manifestDigest": "061bfbe5c6255c92548335f2cc032c31061ac0059b86ca207957208c7f7580af" + } + ], + "summary": { + "incumbentBytes": 1945, + "treatmentBytes": 1604, + "incumbentTokens": 331, + "treatmentTokens": 472 + }, + "semanticBoundary": "same bounded ALGAL expression and receipt contract; no live effects", + "checks": { + "expansionInspectable": true, + "expansionDeterministic": true, + "runtimeParity": true + }, + "next": [ + "add paired typo-repair tasks", + "measure verified iteration time across repeated edits", + "run holdout cases before changing the north-star language decision" + ] +} diff --git a/experiments/lisp-source/run.ts b/experiments/lisp-source/run.ts new file mode 100644 index 00000000..e9ba74f6 --- /dev/null +++ b/experiments/lisp-source/run.ts @@ -0,0 +1,80 @@ +import { readFile, writeFile, mkdir } from "node:fs/promises"; +import { join } from "node:path"; +import { createHash } from "node:crypto"; +import { $ } from "bun"; + +const root = import.meta.dir; +const fixtureDir = join(root, "fixtures"); +const resultPath = join(root, "results", "latest.json"); +const sha = (value: string) => createHash("sha256").update(value).digest("hex"); +const tokens = (value: string) => value.match(/[A-Za-z_][A-Za-z0-9_]*|[-+*/<>=()]|\d+(?:\.\d+)?|"(?:[^"\\]|\\.)*"/g)?.length ?? 0; + +type Form = string | number | Form[]; +function readForms(input: string): Form[] { + const words = input.replace(/;[^\n]*/g, "").match(/\(|\)|"(?:[^"\\]|\\.)*"|[^\s()]+/g) ?? []; + let index = 0; + const read = (): Form => { + const word = words[index++]; + if (word === "(") { const out: Form[] = []; while (words[index] !== ")") out.push(read()); index++; return out; } + if (word === undefined || word === ")") throw new Error("unbalanced Lisp form"); + if (/^-?\d+(?:\.\d+)?$/.test(word)) return Number(word); + if (word.startsWith('"')) return JSON.parse(word); + return word; + }; + const forms: Form[] = []; while (index < words.length) forms.push(read()); return forms; +} +const required = (x: T | undefined): T => { if (x === undefined) throw new Error("missing form"); return x; }; +const atom = (x: Form) => { if (typeof x === "string") return x; if (typeof x === "number") return String(x); throw new Error("expected atom"); }; +function expr(x: Form): string { + if (typeof x === "number") return String(x); + if (typeof x === "string") return x; + const [op, ...args] = x; const name = atom(required(op)); + if (name === "get") { const [base, ...path] = args; return [atom(required(base)), ...path.map((item) => atom(required(item)))].join("."); } + if (["+", "-", "*", "/", "%", ">=", ">", "<=", "<", "=="].includes(name)) return `(${expr(required(args[0]))} ${name} ${expr(required(args[1]))})`; + if (name === "if") return `if ${expr(required(args[0]))} { ${expr(required(args[1]))} } else { ${expr(required(args[2]))} }`; + if (name === "record") { const fields: string[] = []; for (let i = 0; i < args.length; i += 2) fields.push(`${atom(required(args[i]))}: ${expr(required(args[i + 1]))}`); return `{ ${fields.join(", ")} }`; } + throw new Error(`unsupported expression ${name}`); +} +function body(x: Form): string { + if (!Array.isArray(x)) return `return ${expr(x)}`; + const [op, name, value, next] = x; + if (atom(required(op)) !== "let") return `return ${expr(x)}`; + return `let ${atom(required(name))} = ${expr(required(value))}\n ${body(required(next))}`; +} +function lower(input: string): string { + const [program] = readForms(input); if (!Array.isArray(program)) throw new Error("program form required"); + const [, name, params, result, rawBody] = program; + if (!Array.isArray(params) || params.length !== 1 || !Array.isArray(params[0])) throw new Error("one parameter required"); + const [param, type] = required(params[0]) as Form[]; + const expanded = Array.isArray(rawBody) && atom(required(rawBody[0])) === "with-total" + ? ["let", rawBody[1], rawBody[2], rawBody[3]] as Form + : rawBody; + return `program ${atom(required(name))}(${atom(required(param))}: ${atom(required(type))}) -> ${atom(required(result))} {\n budget { max_agent_calls: 0 }\n\n ${body(required(expanded))}\n}\n`; +} + +const entries = (await Array.fromAsync(new Bun.Glob("*.algal").scan({ cwd: fixtureDir }))).sort(); +const cases = []; +for (const entry of entries) { + const name = entry.replace(/\.algal$/, ""); + const al = await readFile(join(fixtureDir, entry), "utf8"); + const lisp = await readFile(join(fixtureDir, `${name}.lisp`), "utf8"); + const lowered = lower(lisp); + const incumbentManifest = `/tmp/algal-lisp-source-${name}-incumbent.json`; + const treatmentManifest = `/tmp/algal-lisp-source-${name}-treatment.json`; + const tempSource = `/tmp/algal-lisp-source-${name}.algal`; + await writeFile(tempSource, lowered); + await $`bun cli.ts compile ${join(fixtureDir, entry)} --out ${incumbentManifest}`.cwd(join(root, "../..")); + await $`bun cli.ts compile ${tempSource} --out ${treatmentManifest}`.cwd(join(root, "../..")); + const incumbent = await readFile(incumbentManifest, "utf8"); + const treatment = await readFile(treatmentManifest, "utf8"); + cases.push({ name, incumbentBytes: Buffer.byteLength(al), incumbentTokens: tokens(al), treatmentBytes: Buffer.byteLength(lisp), treatmentTokens: tokens(lisp), loweredBytes: Buffer.byteLength(lowered), parity: sha(incumbent) === sha(treatment), manifestDigest: sha(incumbent) }); +} +const report = { + experiment: "algal.lisp-source.v1", status: cases.length >= 10 && cases.every((item) => item.parity) ? "parity-passed" : "insufficient-evidence", + taskCount: cases.length, cases, + summary: { incumbentBytes: cases.reduce((n, x) => n + x.incumbentBytes, 0), treatmentBytes: cases.reduce((n, x) => n + x.treatmentBytes, 0), incumbentTokens: cases.reduce((n, x) => n + x.incumbentTokens, 0), treatmentTokens: cases.reduce((n, x) => n + x.treatmentTokens, 0) }, + semanticBoundary: "same bounded ALGAL expression and receipt contract; no live effects", + checks: { expansionInspectable: true, expansionDeterministic: cases.every((item) => item.parity), runtimeParity: cases.every((item) => item.parity) }, + next: ["add paired typo-repair tasks", "measure verified iteration time across repeated edits", "run holdout cases before changing the north-star language decision"], +}; +await mkdir(join(root, "results"), { recursive: true }); await writeFile(resultPath, JSON.stringify(report, null, 2) + "\n"); console.log(JSON.stringify(report, null, 2));