diff --git a/.github/workflows/codspeed.yml b/.github/workflows/codspeed.yml index 431c086..6c07838 100644 --- a/.github/workflows/codspeed.yml +++ b/.github/workflows/codspeed.yml @@ -101,6 +101,9 @@ jobs: ./target/release/ryo build benchmarks/string_slicing/string_slicing.ryo ./target/release/ryo build benchmarks/mandelbrot/mandelbrot.ryo ./target/release/ryo build benchmarks/collatz/collatz.ryo + ./target/release/ryo build benchmarks/struct_records/struct_records.ryo + ./target/release/ryo build benchmarks/struct_records_reuse/struct_records_reuse.ryo + ./target/release/ryo build benchmarks/struct_records_inout/struct_records_inout.ryo - name: Install cargo-codspeed uses: taiki-e/install-action@v2 with: @@ -134,6 +137,9 @@ jobs: ./target/release/ryo build benchmarks/string_slicing/string_slicing.ryo ./target/release/ryo build benchmarks/mandelbrot/mandelbrot.ryo ./target/release/ryo build benchmarks/collatz/collatz.ryo + ./target/release/ryo build benchmarks/struct_records/struct_records.ryo + ./target/release/ryo build benchmarks/struct_records_reuse/struct_records_reuse.ryo + ./target/release/ryo build benchmarks/struct_records_inout/struct_records_inout.ryo - name: Install cargo-codspeed uses: taiki-e/install-action@v2 with: diff --git a/ISSUES.md b/ISSUES.md index 8ceb579..44b1125 100644 --- a/ISSUES.md +++ b/ISSUES.md @@ -167,6 +167,12 @@ Resolved entries are **removed** from this file. Language-visible decisions behi **Summary:** tir.rs re-defines near-identical `extra`-layout modules with different layouts: `call_extra` appends a modes tail; `var_decl_extra` drops the `TY` slot (`LEN: 3` vs uir's `4`). Same names, same constants, different meanings — a footgun when editing one side. `ExtraRange` itself is also byte-duplicated (`uir.rs:107-118` vs `tir.rs:87-98`), and `IfStmt` has no layout doc module at all in tir.rs (:677-715). **Resolution:** Unify the shared pieces (`ExtraRange` at minimum) in one module; rename or document the layout differences explicitly; add the missing `if_stmt_extra` doc module. +### I-173 — Parse-error statements vanish in astgen, cascading a spurious `MissingReturn` (E0036) + +**Files:** `ryo-frontend/src/astgen.rs` (`StmtKind::Error` empty arms :167, :602), `ryo-core/src/ast.rs` (`StmtKind::Error` :319), `ryo-core/src/tir.rs` (`block_definitely_returns` and the `Unreachable` suppression :1604-1612) +**Summary:** The parser recovers at statement boundaries by emitting `StmtKind::Error` placeholders (R10), and sema's return-flow analysis already suppresses cascading `MissingReturn` diagnostics for *sema-level* errors via the TIR `Unreachable` sentinel. The parse-error path leaks between those two mechanisms: astgen lowers `StmtKind::Error` to *nothing*, so a function whose only `return` failed to parse reaches sema with a body that genuinely ends without returning, and the user gets a bogus E0036 stacked on the real parse diagnostic (reproduced 2026-09-11: a typo'd `return Person{name=p.name, age=.age + 1}` produced E0100 at the typo *and* E0036 "missing return" on the function signature, pointing the user at the wrong place). +**Resolution:** Lower `StmtKind::Error` to a UIR error/unreachable sentinel (or have sema treat it as one) so the existing TIR `Unreachable` rule suppresses `MissingReturn` for parse-broken bodies, matching the cascade suppression sema tests already enforce for sema-internal errors. Regression test: a function whose only return statement fails to parse yields exactly the parse diagnostic, no E0036. + --- ## 🟢 Cleanup @@ -351,6 +357,30 @@ Resolved entries are **removed** from this file. Language-visible decisions behi **Summary:** The repo convention is lowercase with underscores for docs (special files like `README.md` excepted). The eight `ryo-*-*.md` files under `docs/dev/` use hyphens instead. `NOTES.md` was renamed to `notes.md` as the cheap half of this cleanup; the hyphenated set was scoped out because each rename must also update every inbound link (`CLAUDE.md`, `ISSUES.md`, the roadmap, and the docs/dev README index at minimum). **Resolution:** One sweep: `git mv` each `ryo-*.md` to its underscore form, then repo-wide grep for each old basename to update links. Verify no residual references with a final grep for `ryo-.*\.md` across tracked markdown. +### I-171 — No small-string optimization: runtime-created `str` values heap-allocate, even ≤ 15-byte ones + +**Files:** `runtime/src/lib.rs` (`RyoStrFat`, `ryo_str_alloc` / `ryo_str_concat` / `ryo_int_to_str` / `ryo_str_free`), `ryo-backend/src/codegen/mod.rs` (`STR_SLOT_SIZE` 24-byte slot layout), `ryo-backend/src/codegen/` (every site emitting str alloc/free/concat calls, including struct field drop glue) +**Summary:** `str` is a 24-byte fat pointer (ptr, len, cap). Static literals are the exception — they point into `.rodata` with cap == 0 (non-heap-owned, never freed) and empty strings avoid allocation entirely — but every runtime-created string (`int_to_str`, concatenation, anything built at runtime) heap-allocates, no matter how short. Swift's `String` inlines up to 15 bytes on 64-bit, so on string-churn workloads names like `user499999` never touch the heap there. That allocation difference is a likely contributor to Swift's win over both Rust and Ryo in `benchmarks/struct_records/` (~1.7x over Ryo AOT, ~2.1x over Rust), alongside the struct-return, field-copy, and drop-glue traffic the benchmark exercises (and to its edge in `many_small_strings`), per the 2026-09-11 checkpoint. Short strings dominate real programs (identifiers, keys, small messages), so this is the largest known structural gap in Ryo's string runtime, and it compounds with M9 structs: every short runtime-created `str` field embedded in an aggregate pays a heap alloc + free per copy. +**Resolution:** Repurpose the 24-byte slot as a tagged union: a discriminator (e.g. in the cap word or the final byte) selects between the current heap fat pointer and an inline representation holding up to ~22 bytes plus length in the slot itself. Runtime entry points (`alloc`, `concat`, `free`, `len`, comparison) branch on the tag; inline strings make `free` a no-op and `concat` fall back to heap only past the inline capacity. This is a breaking change to `RyoStrFat` and every codegen site that materializes or drops a `str` (including struct field drops and sret paths), so it should land as one coordinated runtime+codegen change with ASan coverage; re-checkpoint `many_small_strings` and `struct_records` to measure the win. + +### I-172 — Consuming struct update has no ergonomic form: move + mutate + return dance, no update sugar, no clone + +**Files:** `ryo-frontend/src/ownership/structs.rs` (`check_field_move_out`, E0043), `docs/specification.md` (§5.1 ownership rules; §4.5 struct literals; the operator-uniqueness rule reserving `..` for type bounds), `benchmarks/struct_records/` (the motivating measurement) +**Summary:** Updating one field of an owned struct into a new value — the everyday "record update" — currently takes three statements: `mut q = p; q.age = q.age + 1; return q` (with a `move` parameter, since parameters borrow by default). Moving a single field out is rejected (E0043: fields move only with the whole struct), and there is no clone builtin, so the shorter `Person{name=p.name, age=p.age+1}` shape is inexpressible. The struct_records benchmark quantified the trap: the natural transliteration that re-derives the field costs ~1.8x walltime versus the move+mutate form (36.9 ms → 20.1 ms, commit `cb462b1`), so users who don't know the idiom write measurably slow code. Two candidate fixes exist but both are deferred design decisions: (a) struct-update syntax sugar — note Rust's `..p` spelling collides with the spec's operator-uniqueness rule (`..` is reserved for type bounds), so a different spelling would be needed; (b) same-type duplication via the `Clone` trait already designed for the v0.2/v0.3 trait milestone (see `docs/dev/ryo-view-materialization.md` §2), which would also cover the harder duplicate-and-modify case where the original must survive. +**Resolution:** Revisit at the trait milestone, with the design space already mapped by the Python comparison (2026-09-11). Python's `Person(p.name, p.age + 1)` one-liner works because reference semantics + refcounting make the field read a retain, and `str` immutability makes the aliasing safe; Ryo's uniquely-owned `str` can express the same surface syntax only as a move (unsound from a borrow — double free) or a hidden clone (O(n) + alloc). That splits the problem cleanly on the *liveness of the source*, not the syntax: (1) **Consuming case** — `move` parameter (or any binding) that is dead after the struct literal: moving the field out is sound, the struct is being consumed field-by-field. Relaxing E0043 for dead-parent field reads (the last-use analysis eager destruction already runs) gives Python's ergonomics at zero runtime cost and matches the M8.1 roadmap's prediction that field-by-field move tracking would follow from the same dataflow. (2) **Duplicate-and-modify case** — borrowed parameter or live source: a borrow guarantees the parent stays alive, so no liveness relaxation can ever apply; the field read must be a retain or a clone. Auto-clone here is rejected as a hidden-cost footgun — the struct_records trap measured exactly this work at ~1.8x walltime, and making it the default meaning of innocent syntax would recreate the trap everywhere; auto-share is impossible without changing the field's declared representation to `shared[T]`. The explicit options are the right ones: `.clone()` via the planned `Clone` trait (v0.2/v0.3, see `docs/dev/ryo-view-materialization.md` §2, visible cost) or a `shared[T]` field (retain, visible in the type, spec §5.6). The target end-state is therefore: same syntax, semantics selected by ownership context — borrowed source requires `.clone()`, moved-from dying source compiles as a field move — with any update sugar (spelling must avoid the `..` collision) as optional ergonomics on top of (1). Until then the move + mutate + return pattern is the documented idiom — make sure the struct documentation states it explicitly so users don't rediscover the slow path. The keep-original case has a dedicated tracking benchmark, `benchmarks/struct_records_reuse/` (added 2026-09-11): Ryo clones by hand (`p.name + ""`) and lands with Rust, while Swift/Go/Python share cheaply — the gap the `Clone` trait, `shared[T]`, or a small-string optimization would close. + +### I-174 — Benchmark runner mechanism is copy-pasted across 11 suites; centralize into a shared framework + +**Files:** `benchmarks/*/run_benchmarks.sh` (11 copies, ~1,030 lines total), `benchmarks/README.md` (idiomatic/checkpoint conventions), `codspeed.yml`, `.github/workflows/codspeed.yml` +**Summary:** Every benchmark suite carries its own `run_benchmarks.sh`, and they are literally copies: the struct_records_reuse and struct_records_inout scripts were created by `sed`-substituting the suite name into the struct_records one. Each copy re-implements the same mechanism — prerequisite checks, `cargo build --release`, per-language compile lines, the compiler-version banner, the macOS/Linux `measure_mem` switch, the hyperfine invocation — with the suite-specific part (which arms exist, build commands, run commands) interleaved rather than declared. Drift is already visible: only some suites have Go or Python arms, the version-banner formats differ subtly, the table format and Version-column convention live only in prose in the global README, and adding a suite means another 100-line fork (two were added on 2026-09-11). The same duplication extends to registration: a new suite must be added to the root `codspeed.yml` exec list *and* both AOT build lists in `.github/workflows/codspeed.yml` by hand. +**Resolution:** One shared runner (a single script, or a small `xtask`-style tool) where each suite declares its arms — name, source file, build command, run command — in one manifest (e.g. a TOML/YAML per suite or one central file), and the framework does everything else: prereq checks, builds, correctness run (assert checksum) before timing, version capture, RSS measurement, hyperfine, and emitting the README results table (Version column included) in the canonical format. Suite registration for CodSpeed should be generated from the same manifest so `codspeed.yml` and the workflow lists can't drift from the suites. Migrate the existing 11 suites and delete the per-suite scripts. + +### I-175 — Consuming `str` concat always allocates a fresh exact-size buffer; no in-place append on a provably-unique lhs + +**Files:** `runtime/src/lib.rs` (`ryo_str_concat`, `__ryo_str_push`, `RyoStrFat`), `ryo-backend/src/codegen/` (concat call sites), `ryo-frontend/src/ownership/` (the reassign/dead-binding analysis that already proves uniqueness), `benchmarks/string_building/` (the tracking measure) +**Summary:** `s = s + suffix` compiles to `ryo_str_concat`, which allocates a fresh exact-size buffer (`cap == len`), copies both operands, and frees the old buffer at the reassign — so a 50,000-iteration append loop is O(n²) (~1.25 GB copied, the entire ~11.8x gap to Rust in `benchmarks/string_building/`). Rust proves this is unnecessary: its `impl Add<&str> for String` consumes the lhs and reuses its buffer (documented behavior), so the identical source `s = s + "x"` is amortized O(n) — uniqueness comes from ownership, not refcounts. Ryo's ownership pass already proves the same fact statically: at a reassign concat the old binding is dead, and a reassignable `s` provably has no live views, so in-place append is sound without COW refcounts or runtime uniqueness checks. What is missing is purely allocation policy: Ryo buffers carry no growth headroom (`cap == len` always), and concat never attempts to extend the lhs allocation even when it could. +**Resolution:** Make `s = s + suffix` compile (or lower) to the amortized path when the ownership pass proves the lhs binding is consumed by the concat — i.e. route it through `__ryo_str_push`-style growth (realloc-or-extend, copy the suffix only) instead of fresh-buffer `ryo_str_concat`. Two substrate changes: (1) string buffers must be allowed `cap > len` headroom from concat/push paths (the fat-pointer layout already carries `cap`; only allocation policy changes), and (2) codegen/sema must select the push path only when the lhs is a plain local binding that dies at the concat — field reads, shared results, and any borrowed lhs keep the allocating path. Interacts with the small-string work (I-171), which redesigns the same slot layout; land them in coordination. Re-checkpoint `string_building` — the gap should collapse toward Rust parity with no source change — and `doubling_concat`. + --- ## Cross-References diff --git a/benchmarks/README.md b/benchmarks/README.md index 2166a6a..3b63747 100644 --- a/benchmarks/README.md +++ b/benchmarks/README.md @@ -6,7 +6,9 @@ This directory contains various benchmarks used to measure, validate, and compar We target both execution speed and memory efficiency (specifically focusing on Ryo's Ahead-Of-Time AOT compiler via Cranelift and JIT execution). -**Checkpoint convention:** run the full suite before each release and after merging any change that touches generated-code shape (`ryo-backend/src/codegen/`, the Cranelift pin, ownership sidecar consumption); record results in each benchmark's README so the trend is visible in git history. +**Idiomatic convention:** every benchmark is written the way a developer would naturally write it in that language. Never adapt another language's implementation to work around something Ryo doesn't support yet (e.g. moving a field out of a struct) — the cost of Ryo's current limitations is part of what the suite measures. Where that makes workloads diverge, checksums are per-language and the benchmark's README must say so explicitly. + +**Checkpoint convention:** run the full suite before each release and after merging any change that touches generated-code shape (`ryo-backend/src/codegen/`, the Cranelift pin, ownership sidecar consumption); record results in each benchmark's README so the trend is visible in git history. Every results table must include a **Version** column capturing each language toolchain's version at measurement time (`rustc --version`, `swiftc --version`, `python3 --version`, `ryo --version`) — timings without versions are not reproducible. --- @@ -15,6 +17,7 @@ We target both execution speed and memory efficiency (specifically focusing on R We maintain self-contained, reproducible benchmarks in separate subdirectories: ### 1. [Fibonacci Benchmark](./fibonacci/) + * **Focus:** Deep function recursion, standard integer arithmetic, and basic execution overhead. * **Languages compared:** Rust, Go, Swift, Kotlin, Bun (TypeScript), Julia, Elixir, Python, Ruby, and Ryo. * **Highlights:** Ryo AOT achieves the **lightest memory usage of all languages tested** (1.34 MB max resident size). On execution speed, Ryo currently runs at ~1.42Ɨ Rust's time on `fib(40)` — see the note below for why. @@ -25,40 +28,62 @@ The 1.00Ɨ Rust baseline is compiled in release mode, where integer overflow **w The fair like-for-like is **Swift**, which also traps on overflow and sits at ~1.26Ɨ Rust. Ryo's remaining margin over Swift is not semantic but mechanical: Cranelift 0.135.1 lowers each surviving overflow check to `cset` + `tst` + `b.ne` (~3 extra instructions per op; verified by disassembly) instead of a single branch on the CPU overflow flag. Closing that gap is tracked as compiler work, not accepted as a language cost: -- Value-range guard elision **landed** (commit `d6aee06`, checkpoint 2026-08-26 in [`fibonacci/README.md`](./fibonacci/README.md#checkpoint-value-range-guard-elision-2026-08-26)): `if n <= 1: return n` proves `n - 1` and `n - 2` cannot overflow, so those guards are no longer emitted. Measured walltime change was ~zero — the elided branches were perfectly predicted — and the residual gap is structural (middle-end transforms), not the guards. -- The one surviving guard on the outer `fibonacci(n - 1) + fibonacci(n - 2)` addition lowers to an unfused `cset` + `tst` + `b.ne`; the flag-fusing fix is tracked as **I-165** (`ISSUES.md`). Cranelift itself is pinned and upgraded regularly (0.135.1 at the time of writing). +* Value-range guard elision **landed** (commit `d6aee06`, checkpoint 2026-08-26 in [`fibonacci/README.md`](./fibonacci/README.md#checkpoint-value-range-guard-elision-2026-08-26)): `if n <= 1: return n` proves `n - 1` and `n - 2` cannot overflow, so those guards are no longer emitted. Measured walltime change was ~zero — the elided branches were perfectly predicted — and the residual gap is structural (middle-end transforms), not the guards. +* The one surviving guard on the outer `fibonacci(n - 1) + fibonacci(n - 2)` addition lowers to an unfused `cset` + `tst` + `b.ne`; the flag-fusing fix is tracked as **I-165** (`ISSUES.md`). Cranelift itself is pinned and upgraded regularly (0.135.1 at the time of writing). JIT and AOT land within noise of each other (~1.42–1.43Ɨ) because both share the same Cranelift codegen (both at `opt_level=speed`). ### 2. [Eager Destruction Benchmark](./eager_destruction/) + * **Focus:** Eager memory deallocation at last use (Eager Destruction / ASAP Destruction) vs. scope-based (RAII) destruction under deep recursion. * **Languages compared:** Rust (Scope-Based vs. Manual Drop) and Ryo. * **Highlights:** Ryo AOT uses nearly **3x less heap memory** than standard Rust and is completely immune to stack overflows under deep recursion because deallocations are automatically and eagerly scheduled *before* nested recursive calls. ### 3. [String Building Benchmark](./string_building/) + * **Focus:** Runtime string ABI + eager destruction — concat over 50,000 iterations; the direct before/after measure for the packed-`u128` runtime ABI. * **Languages compared:** Rust, Swift, and Ryo (AOT vs JIT). ### 4. [String Slicing Benchmark](./string_slicing/) + * **Focus:** Zero-copy views — scan a 688 KiB in-program-generated string counting substring matches through `strview` slices, copying and storing nothing. * **Languages compared:** Rust, Swift, and Ryo (AOT vs JIT). ### 5. [Mandelbrot Benchmark](./mandelbrot/) + * **Focus:** Float codegen — 401Ɨ501 grid, max 80 iterations per pixel; no overflow guards in play, the cleanest Cranelift readout. * **Languages compared:** Rust, Swift, and Ryo (AOT vs JIT). ### 6. [Collatz Benchmark](./collatz/) + * **Focus:** Integer loop/branch — total stopping time for seeds 1..1,000,000; a hot flat loop complementing fibonacci's recursion profile. * **Languages compared:** Rust, Swift, and Ryo (AOT vs JIT). ### 7. [Doubling Concat Benchmark](./doubling_concat/) + * **Focus:** Runtime allocation strategy — `s = s + s` exponential growth to 16 MiB, stressing `ryo_str_alloc` / `ryo_str_concat`. * **Languages compared:** Rust, Swift, and Ryo (AOT vs JIT). ### 8. [Many Small Strings Benchmark](./many_small_strings/) + * **Focus:** Flat-loop alloc/free churn — 500,000 short strings built and dropped, complementing eager_destruction's recursion angle. * **Languages compared:** Rust, Swift, and Ryo (AOT vs JIT). +### 9. [Struct Records Benchmark](./struct_records/) + +* **Focus:** Aggregate ABI traffic — 500,000 rounds of build → update → score on a `str + int` record, idiomatic per language; stresses struct returns, field-wise copies, and drop glue across a heap field. Ryo AOT currently beats Rust and Go here; only Swift's small-string optimization keeps it ahead. +* **Languages compared:** Rust, Swift, Go, Python, and Ryo (AOT vs JIT). + +### 10. [Struct Records Reuse Benchmark](./struct_records_reuse/) + +* **Focus:** The keep-original record update — same record, but the caller uses `p` again after `birthday`, so the update cannot consume it. Rust and Ryo pay an explicit clone, Swift/Go/Python share cheaply; tracking measure for the record-update ergonomics gap (I-172) and what `shared[T]` or a small-string optimization would buy. +* **Languages compared:** Rust, Swift, Go, Python, and Ryo (AOT vs JIT). + +### 11. [Struct Records Inout Benchmark](./struct_records_inout/) + +* **Focus:** Imperative update-in-place through a mutable borrow (`inout` / `&mut` / pointer / attribute store) — no new record, no clone, no sret. Verifies that choosing between inout and the consuming move+return form costs nothing, so the idiom choice can be driven by intent. +* **Languages compared:** Rust, Swift, Go, Python, and Ryo (AOT vs JIT). + --- ## General Prerequisites @@ -84,6 +109,7 @@ cd benchmarks/eager_destruction To deeply inspect the performance characteristics of Ryo's execution via flamegraphs, we recommend using [samply](https://github.com/mstange/samply). First, install `samply`: + ```bash cargo install samply ``` @@ -93,11 +119,13 @@ Then, navigate to any benchmark directory, build the benchmark, and profile eith ### Example: Profiling Fibonacci **Profile the standalone AOT binary:** + ```bash samply record ./fib ``` **Profile the JIT compiler executing a file:** + ```bash samply record ../../target/release/ryo run fib.ryo ``` diff --git a/benchmarks/collatz/README.md b/benchmarks/collatz/README.md index 15186de..cc4ba6d 100644 --- a/benchmarks/collatz/README.md +++ b/benchmarks/collatz/README.md @@ -6,14 +6,14 @@ ## Benchmarks & Performance Results -Measured on **macOS 26.6.2 on a MacBook Pro (Apple M3 Pro, 18 GB RAM)**, 2026-08-26. Hyperfine `--warmup 3 --shell=none`; peak RSS via `/usr/bin/time -l` (macOS) or `%M` (Linux). - -| Candidate | Mean time | vs fastest | Max RSS | -|---|---|---|---| -| **Rust** | 103.7 ms ± 0.3 ms | 1.00x | 1.44 MB | -| **Swift** | 166.8 ms ± 5.5 ms | 1.61x slower | 5.55 MB | -| **Ryo (AOT)** | 219.3 ms ± 5.6 ms | 2.11x slower | 1.34 MB | -| **Ryo (JIT)** | 220.1 ms ± 4.6 ms | 2.12x slower | 5.12 MB | +Measured on **macOS 26.6.2 on a MacBook Pro (Apple M3 Pro, 18 GB RAM)**, 2026-09-11. Hyperfine `--warmup 3 --shell=none`; peak RSS via `/usr/bin/time -l` (macOS) or `%M` (Linux). + +| Candidate | Version | Mean time | vs fastest | Max RSS | +|---|---|---|---|---| +| **Rust** | 1.98.0 | 104.3 ms ± 1.2 ms | 1.00x | 1.44 MB | +| **Swift** | 6.3.3 | 164.4 ms ± 1.0 ms | 1.58x slower | 5.55 MB | +| **Ryo (AOT)** | 0.1.0-dev.20260911+b3b7d25 | 216.0 ms ± 1.3 ms | 2.07x slower | 1.36 MB | +| **Ryo (JIT)** | 0.1.0-dev.20260911+b3b7d25 | 220.8 ms ± 2.3 ms | 2.12x slower | 5.09 MB | ## How to Run diff --git a/benchmarks/doubling_concat/README.md b/benchmarks/doubling_concat/README.md index 0f8fb65..bd697df 100644 --- a/benchmarks/doubling_concat/README.md +++ b/benchmarks/doubling_concat/README.md @@ -6,14 +6,14 @@ ## Benchmarks & Performance Results -Measured on **macOS 26.6.2 on a MacBook Pro (Apple M3 Pro, 18 GB RAM)**, 2026-08-26. Hyperfine `--warmup 3 --shell=none`; peak RSS via `/usr/bin/time -l` (macOS) or `%M` (Linux). - -| Candidate | Mean time | vs fastest | Max RSS | -|---|---|---|---| -| **Rust** | 4.0 ms ± 0.2 ms | 1.11x slower | 35.64 MB | -| **Swift** | 4.1 ms ± 0.3 ms | 1.14x slower | 34.05 MB | -| **Ryo (AOT)** | 3.6 ms ± 0.2 ms | 1.00x | 33.42 MB | -| **Ryo (JIT)** | 4.6 ms ± 0.2 ms | 1.28x slower | 36.92 MB | +Measured on **macOS 26.6.2 on a MacBook Pro (Apple M3 Pro, 18 GB RAM)**, 2026-09-11. Hyperfine `--warmup 3 --shell=none`; peak RSS via `/usr/bin/time -l` (macOS) or `%M` (Linux). + +| Candidate | Version | Mean time | vs fastest | Max RSS | +|---|---|---|---|---| +| **Ryo (AOT)** | 0.1.0-dev.20260911+b3b7d25 | 3.5 ms ± 0.1 ms | 1.00x | 33.42 MB | +| **Rust** | 1.98.0 | 3.7 ms ± 0.2 ms | 1.06x slower | 35.64 MB | +| **Swift** | 6.3.3 | 4.0 ms ± 0.1 ms | 1.13x slower | 34.03 MB | +| **Ryo (JIT)** | 0.1.0-dev.20260911+b3b7d25 | 4.7 ms ± 0.6 ms | 1.34x slower | 37.03 MB | ## How to Run diff --git a/benchmarks/mandelbrot/README.md b/benchmarks/mandelbrot/README.md index cdb1279..b881328 100644 --- a/benchmarks/mandelbrot/README.md +++ b/benchmarks/mandelbrot/README.md @@ -6,14 +6,14 @@ ## Benchmarks & Performance Results -Measured on **macOS 26.6.2 on a MacBook Pro (Apple M3 Pro, 18 GB RAM)**, 2026-08-26. Hyperfine `--warmup 3 --shell=none`; peak RSS via `/usr/bin/time -l` (macOS) or `%M` (Linux). - -| Candidate | Mean time | vs fastest | Max RSS | -|---|---|---|---| -| **Rust** | 13.4 ms ± 0.3 ms | 1.00x | 1.44 MB | -| **Swift** | 14.3 ms ± 1.1 ms | 1.07x slower | 5.55 MB | -| **Ryo (AOT)** | 15.2 ms ± 0.6 ms | 1.13x slower | 1.36 MB | -| **Ryo (JIT)** | 16.1 ms ± 0.1 ms | 1.20x slower | 5.20 MB | +Measured on **macOS 26.6.2 on a MacBook Pro (Apple M3 Pro, 18 GB RAM)**, 2026-09-11. Hyperfine `--warmup 3 --shell=none`; peak RSS via `/usr/bin/time -l` (macOS) or `%M` (Linux). + +| Candidate | Version | Mean time | vs fastest | Max RSS | +|---|---|---|---|---| +| **Rust** | 1.98.0 | 13.2 ms ± 0.1 ms | 1.00x | 1.44 MB | +| **Swift** | 6.3.3 | 14.0 ms ± 0.3 ms | 1.06x slower | 5.55 MB | +| **Ryo (AOT)** | 0.1.0-dev.20260911+b3b7d25 | 14.6 ms ± 0.2 ms | 1.11x slower | 1.36 MB | +| **Ryo (JIT)** | 0.1.0-dev.20260911+b3b7d25 | 15.8 ms ± 0.2 ms | 1.20x slower | 5.22 MB | ## How to Run diff --git a/benchmarks/many_small_strings/README.md b/benchmarks/many_small_strings/README.md index 58b91a8..1bbce88 100644 --- a/benchmarks/many_small_strings/README.md +++ b/benchmarks/many_small_strings/README.md @@ -6,14 +6,14 @@ ## Benchmarks & Performance Results -Measured on **macOS 26.6.2 on a MacBook Pro (Apple M3 Pro, 18 GB RAM)**, 2026-08-26. Hyperfine `--warmup 3 --shell=none`; peak RSS via `/usr/bin/time -l` (macOS) or `%M` (Linux). - -| Candidate | Mean time | vs fastest | Max RSS | -|---|---|---|---| -| **Rust** | 9.8 ms ± 0.2 ms | 1.00x | 1.50 MB | -| **Swift** | 10.1 ms ± 0.4 ms | 1.03x slower | 1.58 MB | -| **Ryo (AOT)** | 19.1 ms ± 0.8 ms | 1.95x slower | 1.39 MB | -| **Ryo (JIT)** | 21.6 ms ± 0.7 ms | 2.20x slower | 4.97 MB | +Measured on **macOS 26.6.2 on a MacBook Pro (Apple M3 Pro, 18 GB RAM)**, 2026-09-11. Hyperfine `--warmup 3 --shell=none`; peak RSS via `/usr/bin/time -l` (macOS) or `%M` (Linux). + +| Candidate | Version | Mean time | vs fastest | Max RSS | +|---|---|---|---|---| +| **Rust** | 1.98.0 | 10.3 ms ± 0.3 ms | 1.00x | 1.50 MB | +| **Swift** | 6.3.3 | 10.5 ms ± 0.5 ms | 1.03x slower | 1.58 MB | +| **Ryo (AOT)** | 0.1.0-dev.20260911+b3b7d25 | 19.0 ms ± 0.5 ms | 1.86x slower | 1.38 MB | +| **Ryo (JIT)** | 0.1.0-dev.20260911+b3b7d25 | 20.9 ms ± 0.7 ms | 2.04x slower | 4.88 MB | ## How to Run diff --git a/benchmarks/string_building/README.md b/benchmarks/string_building/README.md index c6133e2..a4e74cc 100644 --- a/benchmarks/string_building/README.md +++ b/benchmarks/string_building/README.md @@ -1,28 +1,28 @@ # String Building Benchmark -**Focus:** Runtime string ABI + eager destruction. Concat over 50,000 iterations (`s = s + "x"`): every iteration allocates a fresh buffer through `ryo_str_concat` and eagerly frees the previous one at the reassign. This is the direct before/after measure for the packed-`u128` string runtime ABI (commit `7d0a047`, return-by-value replacing the per-call-site out-pointer stack slot) — the ABI decision and its rationale are recorded on `pack_pair` in `runtime/src/lib.rs` and pinned by the `clif_string_ops_use_packed_return_no_stack_slots` integration test. +**Focus:** Runtime string ABI + eager destruction. Concat over 50,000 iterations (`s = s + "x"` — now spelled identically in Rust and Ryo): every iteration allocates a fresh buffer through `ryo_str_concat` and eagerly frees the previous one at the reassign. This is the direct before/after measure for the packed-`u128` string runtime ABI (commit `7d0a047`, return-by-value replacing the per-call-site out-pointer stack slot) — the ABI decision and its rationale are recorded on `pack_pair` in `runtime/src/lib.rs` and pinned by the `clif_string_ops_use_packed_return_no_stack_slots` integration test. **Languages compared:** Rust, Swift, Ryo (AOT vs JIT), and Python. -## Why Ryo trails here: value-semantic concat (and the planned fix) +## Why Ryo trails here: same source, different allocation policy -The two programs do different work. Rust's `s.push_str("x")` appends with amortized growth — capacity doubling means ~17 reallocs total. Ryo's `s = s + "x"` is value-semantic: `ryo_str_concat` constructs a **fresh exact-size buffer every iteration**, copies the whole current string into it, and eager destruction frees the old buffer at the reassign. Iteration *i* copies *i* bytes, so the loop copies ~1.25 GB in total — that O(n²) churn is the entire ~12.5Ɨ gap, not codegen quality. +The Rust and Ryo arms are now the *identical* program — both are `s = s + "x"` in a loop — so the ~12x gap is entirely runtime semantics, not algorithm choice. Rust's `impl Add<&str> for String` **consumes the left-hand side and reuses its buffer** (documented std behavior): ownership moves into the operator, uniqueness is proven by the type system, and the append happens in place with amortized capacity growth (~17 reallocs total, O(n)). Ryo's `s = s + "x"` calls `ryo_str_concat`, which constructs a **fresh exact-size buffer every iteration**, copies the whole current string into it, and eager destruction frees the old buffer at the reassign. Iteration *i* copies *i* bytes, so the loop copies ~1.25 GB in total — that O(n²) churn is the entire gap, not codegen quality. -This is deliberately **not** filed as a compiler issue: nothing is miscompiled — the copying is the honest cost of asking for a fresh value per iteration. The amortized fast path already exists as `str_push(&s, "x")` (capacity growth via `__ryo_str_push`, `runtime/src/lib.rs:382`); this benchmark intentionally measures the concat + eager-free path (the ABI / eager-destruction measure), not the fastest way to build a string in Ryo. +The sharper learning (2026-09-11): Ryo doesn't need COW refcounts to close this. The ownership pass already proves statically what Rust's type system proves — at a reassign concat the old binding is dead, and a reassignable `s` provably has no live views — so in-place append is sound for exactly this pattern. What is missing is purely allocation policy: Ryo buffers are always exact-size (`cap == len`, no growth headroom), and concat never attempts to extend the lhs buffer. This is filed as tracked work in `ISSUES.md` (the consuming-concat in-place-append entry, complementing the small-string entry that redesigns the same slot layout): route a provably-consuming `s = s + suffix` through the `__ryo_str_push`-style growth path — realloc-or-extend, copy the suffix only — turning this loop amortized O(n) with no source change. The SSO/COW roadmap work (`docs/dev/implementation_roadmap.md` → *Standard Library Allocation Optimizations*, `docs/dev/stdlib_optimizations.md`) then generalizes the win beyond the consuming case. This benchmark is the tracking measure: the gap should collapse when the entries land. -The planned fix lives in the roadmap's SSO/COW work (see `docs/dev/implementation_roadmap.md` → *Standard Library Allocation Optimizations*, and `docs/dev/stdlib_optimizations.md`): once `str` carries COW refcounts and growth capacity, `s = s + suffix` on a uniquely-referenced buffer becomes an in-place append — realloc-or-extend plus copying the suffix only — turning this loop amortized O(n), at `push_str` parity, without changing the language. The ownership pass already proves the old binding dead at the concat, and a reassignable `s` provably has no live views, so the uniqueness side of the check is compiler-known; what is missing is the allocation policy (today every buffer is exact-size, `cap == len`). This benchmark is the tracking measure for that win: the gap should collapse when the SSO/COW entry lands. +The amortized fast path also already exists explicitly as `str_push(&s, "x")` (capacity growth via `__ryo_str_push`, `runtime/src/lib.rs:382`); this benchmark intentionally measures the concat + eager-free path (the ABI / eager-destruction measure), not the fastest way to build a string in Ryo. ## Benchmarks & Performance Results -Measured on **macOS 26.6.2 on a MacBook Pro (Apple M3 Pro, 18 GB RAM)**, 2026-09-01. Hyperfine `--warmup 3 --shell=none`; peak RSS via `/usr/bin/time -l` (macOS) or `%M` (Linux). +Measured on **macOS 26.6.2 on a MacBook Pro (Apple M3 Pro, 18 GB RAM)**, 2026-09-11. Hyperfine `--warmup 3 --shell=none`; peak RSS via `/usr/bin/time -l` (macOS) or `%M` (Linux). -| Candidate | Mean time | vs fastest | Max RSS | -|---|---|---|---| -| **Rust** | 1.5 ms ± 0.1 ms | 1.00x | 1.61 MB | -| **Swift** | 2.4 ms ± 0.1 ms | 1.61x slower | 1.81 MB | -| **Ryo (AOT)** | 18.3 ms ± 0.3 ms | 12.45x slower | 2.25 MB | -| **Ryo (JIT)** | 19.7 ms ± 0.3 ms | 13.41x slower | 5.77 MB | -| **Python** | 36.6 ms ± 0.5 ms | 24.87x slower | 14.72 MB | +| Candidate | Version | Mean time | vs fastest | Max RSS | +|---|---|---|---|---| +| **Rust** | 1.98.0 | 1.4 ms ± 0.0 ms | 1.00x | 1.61 MB | +| **Swift** | 6.3.3 | 2.4 ms ± 0.1 ms | 1.64x slower | 1.81 MB | +| **Ryo (AOT)** | 0.1.0-dev.20260911+f25e95a | 17.7 ms ± 1.5 ms | 12.33x slower | 2.25 MB | +| **Ryo (JIT)** | 0.1.0-dev.20260911+f25e95a | 18.6 ms ± 0.4 ms | 12.95x slower | 5.73 MB | +| **Python** | 3.14.7 | 36.1 ms ± 1.4 ms | 25.11x slower | 14.75 MB | Python (CPython 3.14.7) runs the same `s += "x"` loop interpreted; its ~25x gap over Rust is interpreter overhead, and its ~2x gap over Ryo shows the interpreted baseline is slower than Ryo's compiled O(n²) concat even before any allocation-policy fix lands. diff --git a/benchmarks/string_building/string_building.rs b/benchmarks/string_building/string_building.rs index 632da6b..8297c15 100644 --- a/benchmarks/string_building/string_building.rs +++ b/benchmarks/string_building/string_building.rs @@ -1,7 +1,7 @@ fn main() { let mut s = String::new(); for _ in 0..50000 { - s.push_str("x"); + s = s + "x"; } assert!(s.len() == 50000, "string_building length check"); println!("assert passed, string_building is correct"); diff --git a/benchmarks/string_slicing/README.md b/benchmarks/string_slicing/README.md index 530591a..30fe6ef 100644 --- a/benchmarks/string_slicing/README.md +++ b/benchmarks/string_slicing/README.md @@ -23,16 +23,14 @@ A second, smaller asymmetry: hyperfine times whole processes, so every arm's in- ## Benchmarks & Performance Results -Measured on **macOS 26.6.2 on a MacBook Pro (Apple M3 Pro, 18 GB RAM)**, 2026-08-26 — all rows re-measured after literal hoisting + static-cap free elision landed (same day); the Swift arm's match loop now uses a zero-copy slice `elementsEqual` (same shape as Rust's `&text[i..i+3] == b"fox"`). Hyperfine `--warmup 3 --shell=none`; peak RSS via `/usr/bin/time -l` (macOS) or `%M` (Linux). - -| Candidate | Mean time | vs fastest | Max RSS | -|---|---|---|---| -| **Rust** | 1.8 ms ± 0.6 ms | 1.00x | 2.97 MB | -| **Swift** | 2.4 ms ± 0.1 ms | 1.39x slower | 7.09 MB | -| **Ryo (AOT)** | 4.9 ms ± 0.2 ms | 2.80x slower | 2.75 MB | -| **Ryo (JIT)** | 6.7 ms ± 0.7 ms | 3.84x slower | 6.58 MB | - -Note: the JIT regression from the packed-`u128` ABI (~6.6 ms → ~10 ms) is gone — the JIT is back to ~6.8 ms now that the per-iteration `ryo_str_from_literal` / `ryo_str_free` calls are eliminated, confirming those extern calls priced higher under the JIT than under AOT. +Measured on **macOS 26.6.2 on a MacBook Pro (Apple M3 Pro, 18 GB RAM)**, 2026-09-11 — all rows re-measured after the Rust and Swift arms were rewritten to be idiomatic per the repository convention: Rust now builds the string with `s.repeat(2)` and scans with `text.windows(3).filter(|w| *w == b"fox")`, and Swift hoists the needle array out of the scan loop (previously it transliterated Ryo's `s = s + s` as `s.clone() + &s` and rebuilt a magic `[102, 111, 120]` literal per comparison). Same checksums and semantics; only expression quality changed. Hyperfine `--warmup 3 --shell=none`; peak RSS via `/usr/bin/time -l` (macOS) or `%M` (Linux). + +| Candidate | Version | Mean time | vs fastest | Max RSS | +|---|---|---|---|---| +| **Rust** | 1.98.0 | 1.7 ms ± 0.2 ms | 1.00x | 2.88 MB | +| **Swift** | 6.3.3 | 2.6 ms ± 0.2 ms | 1.54x slower | 7.09 MB | +| **Ryo (AOT)** | 0.1.0-dev.20260911+490b10d | 4.9 ms ± 0.3 ms | 2.94x slower | 2.75 MB | +| **Ryo (JIT)** | 0.1.0-dev.20260911+490b10d | 6.5 ms ± 0.4 ms | 3.90x slower | 6.62 MB | ## How to Run diff --git a/benchmarks/string_slicing/string_slicing.rs b/benchmarks/string_slicing/string_slicing.rs index f6badba..7c94752 100644 --- a/benchmarks/string_slicing/string_slicing.rs +++ b/benchmarks/string_slicing/string_slicing.rs @@ -1,20 +1,11 @@ fn count_fox(text: &[u8]) -> usize { - let mut count = 0; - let mut i = 0; - let n = text.len(); - while i + 3 <= n { - if &text[i..i + 3] == b"fox" { - count += 1; - } - i += 1; - } - count + text.windows(3).filter(|w| *w == b"fox").count() } fn main() { let mut s = String::from("the quick brown fox jumps over the lazy dog"); for _ in 0..14 { - s = s.clone() + &s; + s = s.repeat(2); } let count = count_fox(s.as_bytes()); let n = s.len(); diff --git a/benchmarks/string_slicing/string_slicing.swift b/benchmarks/string_slicing/string_slicing.swift index fcbff1d..3b9ec01 100644 --- a/benchmarks/string_slicing/string_slicing.swift +++ b/benchmarks/string_slicing/string_slicing.swift @@ -1,11 +1,13 @@ import Foundation +let fox = Array("fox".utf8) + func countFox(_ text: [UInt8]) -> Int { var count = 0 var i = 0 let n = text.count while i + 3 <= n { - if text[i..<(i + 3)].elementsEqual([102, 111, 120]) { + if text[i..<(i + 3)].elementsEqual(fox) { count += 1 } i += 1 diff --git a/benchmarks/struct_records/.gitignore b/benchmarks/struct_records/.gitignore new file mode 100644 index 0000000..e2efc6b --- /dev/null +++ b/benchmarks/struct_records/.gitignore @@ -0,0 +1,5 @@ +struct_records +struct_records_rs +struct_records_swift +struct_records_go +__pycache__/ diff --git a/benchmarks/struct_records/README.md b/benchmarks/struct_records/README.md new file mode 100644 index 0000000..27dd44e --- /dev/null +++ b/benchmarks/struct_records/README.md @@ -0,0 +1,32 @@ +# Struct Records Benchmark + +**Focus:** Aggregate ABI traffic, idiomatic per language. Runs 500,000 rounds of `make_person(i)` → `birthday(p)` → `score(q)` on a `Person{name: str, age: int}` record — the natural "update one field, keep the rest" struct update. Each language expresses it the way a developer actually would: Rust moves the `name` field out (partial move, free), Swift value-copies it (COW-backed `String`, cheap), Go shallow-copies the struct (string header shares immutable backing bytes; GC owns lifetime), Python shares the `str` reference (free), and Ryo takes the parameter by `move` and mutates the field in place (`mut q = p; q.age = q.age + 1`). Ryo rejects moving a *single field* out of a struct (E0043 — fields move only with the whole struct) and has no clone builtin, but whole-struct moves are the idiomatic answer here, so **all five languages do the same work and assert the same checksum (27,638,890)**. Note the `move` is not a Ryo-specific shortcut: it matches Rust's by-value parameter exactly — both consume the caller's record and transfer the name allocation with zero heap traffic; Swift/Go/Python keep the original usable and pay for that flexibility in ARC retains, GC headroom, or refcounting. Rust runs its drop glue, Swift its ARC retain/release traffic on the `String` field, Go its GC, Python its object model (`__slots__` class), and Ryo its eager-destruction scheduling. + +**Languages compared:** Rust, Swift, Go, Python, and Ryo (AOT vs JIT). + +## Evolution note: this benchmark must mutate to B + +This flat-loop record workload is the interim form. Once `list[T]` lands (M22, see `docs/dev/implementation_roadmap.md`), this benchmark **must** evolve into the AoS *particles* workload: a `list[Particle]` of structs with `x/y/z` float fields, integrated over N time steps — the array-of-structs layout comparison Ryo vs Rust vs Swift vs Python. That is the comparison that actually exercises contiguous aggregate storage, iteration over struct elements, and in-place field mutation at scale. When it does, this synthetic loop retires or survives as the flat-loop baseline leg. + +## Benchmarks & Performance Results + +Measured on **macOS 26.6.2 on a MacBook Pro (Apple M3 Pro, 18 GB RAM)**, 2026-09-11. Hyperfine `--warmup 3 --shell=none`; peak RSS via `/usr/bin/time -l` (macOS) or `%M` (Linux). + +| Candidate | Version | Mean time | vs fastest | Max RSS | +|---|---|---|---|---| +| **Swift** | 6.3.3 | 11.8 ms ± 0.4 ms | 1.00x | 1.56 MB | +| **Ryo (AOT)** | 0.1.0-dev.20260911+a151528 | 20.6 ms ± 0.6 ms | 1.74x slower | 1.39 MB | +| **Ryo (JIT)** | 0.1.0-dev.20260911+a151528 | 22.8 ms ± 0.6 ms | 1.94x slower | 5.36 MB | +| **Rust** | 1.98.0 | 24.4 ms ± 0.6 ms | 2.07x slower | 1.52 MB | +| **Go** | 1.27.1 | 25.6 ms ± 0.8 ms | 2.17x slower | 9.30 MB | +| **Python** | 3.14.7 | 132.2 ms ± 2.1 ms | 11.21x slower | 14.59 MB | + +Ryo AOT **beats both Rust and Go** here (20.6 ms vs 24.4 / 25.6 ms). The earlier Ryo arm re-derived the name per round because field-level moves are rejected (E0043); rewriting `birthday` to take the record by `move` and mutate the field in place — the idiomatic Ryo shape — removed the per-round `int_to_str` + concat + alloc/free entirely (36.9 ms → ~20 ms) and unified the checksum across all five languages. What remains is pure aggregate ABI traffic, where Ryo's eager-destruction scheduling and sret returns hold up well; Rust additionally pays `format!` machinery per round, and Go pays its GC twice over — in walltime (write barriers, allocation pacing) and most visibly in memory (9.30 MB RSS vs Ryo's 1.39 MB, the classic GC headroom tax). Swift still wins outright: its small-string optimization keeps every name inline (≤ 10 UTF-8 bytes) and its value copy is cheap — closing that is tracked as the small-string optimization work in `ISSUES.md`. Ryo AOT runs **6.4x faster than Python** with ~10x less memory, at the lightest RSS of the suite. + +## How to Run + +Prerequisites: `hyperfine`, `rustc`, `swiftc`, `go`, `python3`, plus a release build of the compiler (`cargo build --release` from the repository root — the script runs it for you). + +```bash +./run_benchmarks.sh +``` diff --git a/benchmarks/struct_records/run_benchmarks.sh b/benchmarks/struct_records/run_benchmarks.sh new file mode 100755 index 0000000..4019140 --- /dev/null +++ b/benchmarks/struct_records/run_benchmarks.sh @@ -0,0 +1,100 @@ +#!/bin/bash +set -e + +# Check for prerequisites +if ! command -v hyperfine &> /dev/null; then + echo "Error: 'hyperfine' is not installed or not in PATH. Please install it to run performance benchmarks." + exit 1 +fi + +if ! command -v rustc &> /dev/null; then + echo "Error: 'rustc' is not installed or not in PATH." + exit 1 +fi + +if ! command -v swiftc &> /dev/null; then + echo "Error: 'swiftc' is not installed or not in PATH." + exit 1 +fi + +if ! command -v go &> /dev/null; then + echo "Error: 'go' is not installed or not in PATH." + exit 1 +fi + +if ! command -v python3 &> /dev/null; then + echo "Error: 'python3' is not installed or not in PATH." + exit 1 +fi + +echo "Building benchmarks..." +(cd ../.. && cargo build --release > /dev/null) +rustc -O struct_records.rs -o struct_records_rs +swiftc -O struct_records.swift -o struct_records_swift +go build -o struct_records_go struct_records.go +ryo_bin="../../target/release/ryo" +$ryo_bin build struct_records.ryo > /dev/null + +echo "" +echo "-------------------" +echo "Compiler Version" +echo "-------------------" +echo "Rust: $(rustc --version | cut -d' ' -f2)" +echo "Swift: $(swiftc --version | head -1 | awk '{for (i = 1; i < NF; i++) if ($i == "Swift" && $(i+1) == "version") { print $(i+2); exit }}')" +echo "Go: $(go version | cut -d' ' -f3 | sed 's/^go//')" +echo "Python: $(python3 --version | cut -d' ' -f2)" +echo "Ryo: $($ryo_bin --version 2>&1 || echo 'dev')" + +echo "" +echo "-------------------" +echo "Memory Usage (Maximum Resident Set Size)" +echo "-------------------" +_OS="$(uname -s)" +measure_mem() { + local name=$1 + shift + + local mem_kb + local mem_out + case "$_OS" in + Darwin*) + # /usr/bin/time -l reports bytes on macOS; convert to KB + mem_kb=$( ( /usr/bin/time -l "$@" > /dev/null ) 2>&1 | awk '/maximum resident set size/ {printf "%d", $1 / 1024; exit}' ) + ;; + Linux*) + mem_kb=$( { /usr/bin/time -f "%M" "$@" > /dev/null; } 2>&1 | tail -n1 ) + ;; + *) + mem_kb="" + ;; + esac + + if [[ -n "$mem_kb" ]]; then + mem_out=$(awk -v kb="$mem_kb" 'BEGIN { printf "%.2f MB", kb / 1024 }') + else + mem_out="N/A" + fi + + printf "%-28s %s\n" "[$name]" "$mem_out" +} + +# Run once each to collect memory usage +measure_mem "Rust" ./struct_records_rs +measure_mem "Swift" ./struct_records_swift +measure_mem "Go" ./struct_records_go +measure_mem "Python" python3 struct_records.py +measure_mem "Ryo (AOT)" ./struct_records +measure_mem "Ryo (JIT)" $ryo_bin run struct_records.ryo + +echo "" +echo "-------------------" +echo "Running Benchmarks (500,000 record build-update-score rounds) using hyperfine" +echo "-------------------" + +hyperfine --warmup 3 --shell=none \ + './struct_records_rs' \ + './struct_records_swift' \ + './struct_records_go' \ + 'python3 struct_records.py' \ + './struct_records' \ + "$ryo_bin run struct_records.ryo" diff --git a/benchmarks/struct_records/struct_records.go b/benchmarks/struct_records/struct_records.go new file mode 100644 index 0000000..70ac966 --- /dev/null +++ b/benchmarks/struct_records/struct_records.go @@ -0,0 +1,33 @@ +package main + +import "fmt" + +type Person struct { + Name string + Age int +} + +func makePerson(i int) Person { + return Person{Name: fmt.Sprintf("user%d", i), Age: 20 + i%50} +} + +func birthday(p Person) Person { + return Person{Name: p.Name, Age: p.Age + 1} +} + +func score(p Person) int { + return len(p.Name) + p.Age +} + +func main() { + total := 0 + for i := range 500000 { + p := makePerson(i) + q := birthday(p) + total += score(q) + } + if total != 27638890 { + panic("struct_records checksum") + } + fmt.Println("assert passed, struct_records is correct") +} diff --git a/benchmarks/struct_records/struct_records.py b/benchmarks/struct_records/struct_records.py new file mode 100644 index 0000000..aab69e8 --- /dev/null +++ b/benchmarks/struct_records/struct_records.py @@ -0,0 +1,31 @@ +class Person: + __slots__ = ("age", "name") + + def __init__(self, name, age): + self.name = name + self.age = age + + +def make_person(i): + return Person("user" + str(i), 20 + i % 50) + + +def birthday(p): + return Person(p.name, p.age + 1) + + +def score(p): + return len(p.name) + p.age + + +def main(): + total = 0 + for i in range(500000): + p = make_person(i) + q = birthday(p) + total += score(q) + assert total == 27638890, "struct_records checksum" + print("assert passed, struct_records is correct") + + +main() diff --git a/benchmarks/struct_records/struct_records.rs b/benchmarks/struct_records/struct_records.rs new file mode 100644 index 0000000..d2e765f --- /dev/null +++ b/benchmarks/struct_records/struct_records.rs @@ -0,0 +1,33 @@ +struct Person { + name: String, + age: i64, +} + +fn make_person(i: i64) -> Person { + Person { + name: format!("user{}", i), + age: 20 + i % 50, + } +} + +fn birthday(p: Person) -> Person { + Person { + name: p.name, + age: p.age + 1, + } +} + +fn score(p: &Person) -> i64 { + p.name.len() as i64 + p.age +} + +fn main() { + let mut total = 0i64; + for i in 0..500000 { + let p = make_person(i); + let q = birthday(p); + total += score(&q); + } + assert!(total == 27638890, "struct_records checksum"); + println!("assert passed, struct_records is correct"); +} diff --git a/benchmarks/struct_records/struct_records.ryo b/benchmarks/struct_records/struct_records.ryo new file mode 100644 index 0000000..3ec156c --- /dev/null +++ b/benchmarks/struct_records/struct_records.ryo @@ -0,0 +1,23 @@ +struct Person: + name: str + age: int + +fn make_person(i: int) -> Person: + return Person{name="user" + int_to_str(i), age=20 + i % 50} + +fn birthday(move p: Person) -> Person: + mut q = p + q.age += 1 + return q + +fn score(p: Person) -> int: + return p.name.len() + p.age + +fn main(): + mut total = 0 + for i in range(0, 500000): + p = make_person(i) + q = birthday(p) + total += score(q) + assert(total == 27638890, "struct_records checksum") + print("assert passed, struct_records is correct\n") diff --git a/benchmarks/struct_records/struct_records.swift b/benchmarks/struct_records/struct_records.swift new file mode 100644 index 0000000..da6f44a --- /dev/null +++ b/benchmarks/struct_records/struct_records.swift @@ -0,0 +1,25 @@ +struct Person { + var name: String + var age: Int +} + +func makePerson(_ i: Int) -> Person { + Person(name: "user" + String(i), age: 20 + i % 50) +} + +func birthday(_ p: Person) -> Person { + Person(name: p.name, age: p.age + 1) +} + +func score(_ p: Person) -> Int { + p.name.utf8.count + p.age +} + +var total = 0 +for i in 0..<500000 { + let p = makePerson(i) + let q = birthday(p) + total += score(q) +} +precondition(total == 27638890, "struct_records checksum") +print("assert passed, struct_records is correct") diff --git a/benchmarks/struct_records_inout/.gitignore b/benchmarks/struct_records_inout/.gitignore new file mode 100644 index 0000000..fb15024 --- /dev/null +++ b/benchmarks/struct_records_inout/.gitignore @@ -0,0 +1,5 @@ +struct_records_inout +struct_records_inout_rs +struct_records_inout_swift +struct_records_inout_go +__pycache__/ diff --git a/benchmarks/struct_records_inout/README.md b/benchmarks/struct_records_inout/README.md new file mode 100644 index 0000000..aa7fcbc --- /dev/null +++ b/benchmarks/struct_records_inout/README.md @@ -0,0 +1,30 @@ +# Struct Records Inout Benchmark + +**Focus:** Imperative update-in-place through a mutable borrow — the third record-update idiom, alongside the consuming update ([`struct_records`](../struct_records/)) and the keep-original update ([`struct_records_reuse`](../struct_records_reuse/)). Runs 500,000 rounds of `make_person(i)` → `birthday(&p)` → `score(p)` on a `Person{name: str, age: int}` record, where `birthday` mutates the caller's record in place and returns nothing. Each language spells it its own way: Ryo `fn birthday(inout p: Person)` called as `birthday(&p)`, Rust `&mut Person`, Swift `inout Person` (also `&p` at the call site), Go `*Person`, and Python plain attribute mutation on the shared object reference. No new record is created, so no clone, move, retain, or sret return is involved — this isolates the mutable-borrow call itself. **All five languages do the same work and assert the same checksum (27,638,890).** + +**Languages compared:** Rust, Swift, Go, Python, and Ryo (AOT vs JIT). + +**Why this suite exists:** the inout idiom is Ryo's zero-ceremony answer to "change my record" — mutation is visible at the call site (`&p`), the binding must be `mut`, and Rule 7 rejects aliasing hazards at compile time. This benchmark verifies the performance half of that story: choosing between inout and the consuming move+return form should cost nothing, so users can pick by intent rather than by speed. + +## Benchmarks & Performance Results + +Measured on **macOS 26.6.2 on a MacBook Pro (Apple M3 Pro, 18 GB RAM)**, 2026-09-11, at Ryo revision `af2cae3`; later branch commits through `b3b7d25` touch only benchmark files and docs — no compiler code — so these numbers remain directly comparable with suites re-measured at `b3b7d25`. Hyperfine `--warmup 3 --shell=none`; peak RSS via `/usr/bin/time -l` (macOS) or `%M` (Linux). + +| Candidate | Version | Mean time | vs fastest | Max RSS | +|---|---|---|---|---| +| **Swift** | 6.3.3 | 11.7 ms ± 0.3 ms | 1.00x | 1.56 MB | +| **Ryo (AOT)** | 0.1.0-dev.20260911+af2cae3 | 20.2 ms ± 0.5 ms | 1.72x slower | 1.34 MB | +| **Ryo (JIT)** | 0.1.0-dev.20260911+af2cae3 | 22.7 ms ± 0.7 ms | 1.94x slower | 5.22 MB | +| **Rust** | 1.98.0 | 24.4 ms ± 1.7 ms | 2.08x slower | 1.53 MB | +| **Go** | 1.27.1 | 25.1 ms ± 0.6 ms | 2.15x slower | 9.92 MB | +| **Python** | 3.14.7 | 110.3 ms ± 1.5 ms | 9.42x slower | 14.52 MB | + +Read against the sibling suites: Ryo AOT matches its own consuming-update time (20.2 ms here vs 20.6 ms in `struct_records` — a wash, as designed) and again beats Rust and Go. The inout form also beats Ryo's keep-original time (28.7 ms in `struct_records_reuse`) by exactly the clone it avoids — the three suites together price Ryo's record-update vocabulary: in-place mutation ā‰ˆ consuming update < duplicate-and-modify. Python improves relative to the reuse suite (9.4x vs 12.4x) because in-place mutation skips its object allocation, though it remains an order of magnitude behind. Swift's lead is unchanged and remains the small-string optimization story (I-171). Ryo AOT runs **5.5x faster than Python** with ~11x less memory, at the lightest RSS of the suite. + +## How to Run + +Prerequisites: `hyperfine`, `rustc`, `swiftc`, `go`, `python3`, plus a release build of the compiler (`cargo build --release` from the repository root — the script runs it for you). + +```bash +./run_benchmarks.sh +``` diff --git a/benchmarks/struct_records_inout/run_benchmarks.sh b/benchmarks/struct_records_inout/run_benchmarks.sh new file mode 100755 index 0000000..c0f9626 --- /dev/null +++ b/benchmarks/struct_records_inout/run_benchmarks.sh @@ -0,0 +1,100 @@ +#!/bin/bash +set -e + +# Check for prerequisites +if ! command -v hyperfine &> /dev/null; then + echo "Error: 'hyperfine' is not installed or not in PATH. Please install it to run performance benchmarks." + exit 1 +fi + +if ! command -v rustc &> /dev/null; then + echo "Error: 'rustc' is not installed or not in PATH." + exit 1 +fi + +if ! command -v swiftc &> /dev/null; then + echo "Error: 'swiftc' is not installed or not in PATH." + exit 1 +fi + +if ! command -v go &> /dev/null; then + echo "Error: 'go' is not installed or not in PATH." + exit 1 +fi + +if ! command -v python3 &> /dev/null; then + echo "Error: 'python3' is not installed or not in PATH." + exit 1 +fi + +echo "Building benchmarks..." +(cd ../.. && cargo build --release > /dev/null) +rustc -O struct_records_inout.rs -o struct_records_inout_rs +swiftc -O struct_records_inout.swift -o struct_records_inout_swift +go build -o struct_records_inout_go struct_records_inout.go +ryo_bin="../../target/release/ryo" +$ryo_bin build struct_records_inout.ryo > /dev/null + +echo "" +echo "-------------------" +echo "Compiler Version" +echo "-------------------" +echo "Rust: $(rustc --version | cut -d' ' -f2)" +echo "Swift: $(swiftc --version | head -1 | awk '{for (i = 1; i < NF; i++) if ($i == "Swift" && $(i+1) == "version") { print $(i+2); exit }}')" +echo "Go: $(go version | cut -d' ' -f3 | sed 's/^go//')" +echo "Python: $(python3 --version | cut -d' ' -f2)" +echo "Ryo: $($ryo_bin --version 2>&1 || echo 'dev')" + +echo "" +echo "-------------------" +echo "Memory Usage (Maximum Resident Set Size)" +echo "-------------------" +_OS="$(uname -s)" +measure_mem() { + local name=$1 + shift + + local mem_kb + local mem_out + case "$_OS" in + Darwin*) + # /usr/bin/time -l reports bytes on macOS; convert to KB + mem_kb=$( ( /usr/bin/time -l "$@" > /dev/null ) 2>&1 | awk '/maximum resident set size/ {printf "%d", $1 / 1024; exit}' ) + ;; + Linux*) + mem_kb=$( { /usr/bin/time -f "%M" "$@" > /dev/null; } 2>&1 | tail -n1 ) + ;; + *) + mem_kb="" + ;; + esac + + if [[ -n "$mem_kb" ]]; then + mem_out=$(awk -v kb="$mem_kb" 'BEGIN { printf "%.2f MB", kb / 1024 }') + else + mem_out="N/A" + fi + + printf "%-28s %s\n" "[$name]" "$mem_out" +} + +# Run once each to collect memory usage +measure_mem "Rust" ./struct_records_inout_rs +measure_mem "Swift" ./struct_records_inout_swift +measure_mem "Go" ./struct_records_inout_go +measure_mem "Python" python3 struct_records_inout.py +measure_mem "Ryo (AOT)" ./struct_records_inout +measure_mem "Ryo (JIT)" $ryo_bin run struct_records_inout.ryo + +echo "" +echo "-------------------" +echo "Running Benchmarks (500,000 record build-mutate-score rounds) using hyperfine" +echo "-------------------" + +hyperfine --warmup 3 --shell=none \ + './struct_records_inout_rs' \ + './struct_records_inout_swift' \ + './struct_records_inout_go' \ + 'python3 struct_records_inout.py' \ + './struct_records_inout' \ + "$ryo_bin run struct_records_inout.ryo" diff --git a/benchmarks/struct_records_inout/struct_records_inout.go b/benchmarks/struct_records_inout/struct_records_inout.go new file mode 100644 index 0000000..3fbd712 --- /dev/null +++ b/benchmarks/struct_records_inout/struct_records_inout.go @@ -0,0 +1,33 @@ +package main + +import "fmt" + +type Person struct { + Name string + Age int +} + +func makePerson(i int) Person { + return Person{Name: fmt.Sprintf("user%d", i), Age: 20 + i%50} +} + +func birthday(p *Person) { + p.Age++ +} + +func score(p Person) int { + return len(p.Name) + p.Age +} + +func main() { + total := 0 + for i := range 500000 { + p := makePerson(i) + birthday(&p) + total += score(p) + } + if total != 27638890 { + panic("struct_records_inout checksum") + } + fmt.Println("assert passed, struct_records_inout is correct") +} diff --git a/benchmarks/struct_records_inout/struct_records_inout.py b/benchmarks/struct_records_inout/struct_records_inout.py new file mode 100644 index 0000000..5266230 --- /dev/null +++ b/benchmarks/struct_records_inout/struct_records_inout.py @@ -0,0 +1,31 @@ +class Person: + __slots__ = ("age", "name") + + def __init__(self, name, age): + self.name = name + self.age = age + + +def make_person(i): + return Person("user" + str(i), 20 + i % 50) + + +def birthday(p): + p.age += 1 + + +def score(p): + return len(p.name) + p.age + + +def main(): + total = 0 + for i in range(500000): + p = make_person(i) + birthday(p) + total += score(p) + assert total == 27638890, "struct_records_inout checksum" + print("assert passed, struct_records_inout is correct") + + +main() diff --git a/benchmarks/struct_records_inout/struct_records_inout.rs b/benchmarks/struct_records_inout/struct_records_inout.rs new file mode 100644 index 0000000..91821f4 --- /dev/null +++ b/benchmarks/struct_records_inout/struct_records_inout.rs @@ -0,0 +1,30 @@ +struct Person { + name: String, + age: i64, +} + +fn make_person(i: i64) -> Person { + Person { + name: format!("user{}", i), + age: 20 + i % 50, + } +} + +fn birthday(p: &mut Person) { + p.age += 1; +} + +fn score(p: &Person) -> i64 { + p.name.len() as i64 + p.age +} + +fn main() { + let mut total = 0i64; + for i in 0..500000 { + let mut p = make_person(i); + birthday(&mut p); + total += score(&p); + } + assert!(total == 27638890, "struct_records_inout checksum"); + println!("assert passed, struct_records_inout is correct"); +} diff --git a/benchmarks/struct_records_inout/struct_records_inout.ryo b/benchmarks/struct_records_inout/struct_records_inout.ryo new file mode 100644 index 0000000..631be7c --- /dev/null +++ b/benchmarks/struct_records_inout/struct_records_inout.ryo @@ -0,0 +1,21 @@ +struct Person: + name: str + age: int + +fn make_person(i: int) -> Person: + return Person{name="user" + int_to_str(i), age=20 + i % 50} + +fn birthday(inout p: Person): + p.age += 1 + +fn score(p: Person) -> int: + return p.name.len() + p.age + +fn main(): + mut total = 0 + for i in range(0, 500000): + mut p = make_person(i) + birthday(&p) + total += score(p) + assert(total == 27638890, "struct_records_inout checksum") + print("assert passed, struct_records_inout is correct\n") diff --git a/benchmarks/struct_records_inout/struct_records_inout.swift b/benchmarks/struct_records_inout/struct_records_inout.swift new file mode 100644 index 0000000..4456051 --- /dev/null +++ b/benchmarks/struct_records_inout/struct_records_inout.swift @@ -0,0 +1,25 @@ +struct Person { + var name: String + var age: Int +} + +func makePerson(_ i: Int) -> Person { + Person(name: "user" + String(i), age: 20 + i % 50) +} + +func birthday(_ p: inout Person) { + p.age += 1 +} + +func score(_ p: Person) -> Int { + p.name.utf8.count + p.age +} + +var total = 0 +for i in 0..<500000 { + var p = makePerson(i) + birthday(&p) + total += score(p) +} +precondition(total == 27638890, "struct_records_inout checksum") +print("assert passed, struct_records_inout is correct") diff --git a/benchmarks/struct_records_reuse/.gitignore b/benchmarks/struct_records_reuse/.gitignore new file mode 100644 index 0000000..ea36544 --- /dev/null +++ b/benchmarks/struct_records_reuse/.gitignore @@ -0,0 +1,5 @@ +struct_records_reuse +struct_records_reuse_rs +struct_records_reuse_swift +struct_records_reuse_go +__pycache__/ diff --git a/benchmarks/struct_records_reuse/README.md b/benchmarks/struct_records_reuse/README.md new file mode 100644 index 0000000..70bc10a --- /dev/null +++ b/benchmarks/struct_records_reuse/README.md @@ -0,0 +1,30 @@ +# Struct Records Reuse Benchmark + +**Focus:** The keep-original record update — what `struct_records` cannot measure. Runs 500,000 rounds of `make_person(i)` → `birthday(p)` → `score(p) + score(q)` on a `Person{name: str, age: int}` record, where the caller **uses `p` again after the update**, so `birthday` cannot consume it. This is the duplicate-and-modify case, and each language pays its own price for keeping both records alive: Rust clones the `String` explicitly (`p.name.clone()` — alloc + memcpy), Swift's value copy retains (SSO keeps the name inline, nearly free), Go shallow-copies the header and shares the immutable backing bytes (GC owns lifetime), Python bumps a refcount, and Ryo writes the clone by hand as `p.name + ""` (borrow the field, concat with empty string, fresh owned `str` — the only way to duplicate a `str` today, since E0043 rejects field moves, structs cannot hold borrows, and there is no `clone` builtin or `shared[T]` yet). **All five languages do the same work and assert the same checksum (54,777,780)** — the difference is purely what "keep the original" costs. + +**Languages compared:** Rust, Swift, Go, Python, and Ryo (AOT vs JIT). + +**Why this suite exists:** it is the tracking measure for the record-update ergonomics gap (`ISSUES.md` I-172). In the consuming case (`struct_records`) Ryo matches Rust's move semantics and beats it on walltime; here Ryo must clone like Rust, but through an ugly manual idiom, while the share-friendly languages get the update for free. When the `Clone` trait or `shared[T]` lands, this benchmark is where the win shows up. + +## Benchmarks & Performance Results + +Measured on **macOS 26.6.2 on a MacBook Pro (Apple M3 Pro, 18 GB RAM)**, 2026-09-11, at Ryo revision `e2db4c6`; later branch commits through `b3b7d25` touch only benchmark files and docs — no compiler code — so these numbers remain directly comparable with suites re-measured at `b3b7d25`. Hyperfine `--warmup 3 --shell=none`; peak RSS via `/usr/bin/time -l` (macOS) or `%M` (Linux). + +| Candidate | Version | Mean time | vs fastest | Max RSS | +|---|---|---|---|---| +| **Swift** | 6.3.3 | 12.0 ms ± 0.7 ms | 1.00x | 1.58 MB | +| **Go** | 1.27.1 | 24.9 ms ± 0.5 ms | 2.08x slower | 9.80 MB | +| **Ryo (AOT)** | 0.1.0-dev.20260911+e2db4c6 | 28.7 ms ± 0.7 ms | 2.40x slower | 1.36 MB | +| **Ryo (JIT)** | 0.1.0-dev.20260911+e2db4c6 | 31.3 ms ± 0.7 ms | 2.61x slower | 5.33 MB | +| **Rust** | 1.98.0 | 31.6 ms ± 0.8 ms | 2.64x slower | 1.52 MB | +| **Python** | 3.14.7 | 148.9 ms ± 2.2 ms | 12.44x slower | 14.61 MB | + +The ranking is the cost of sharing, read directly. Swift wins because SSO keeps every name inline — its "clone" never touches the heap. Go is second: no clone at all, just a shared header — paid for in GC headroom (9.80 MB RSS, 7x Ryo's). Ryo and Rust both pay a real per-round alloc + memcpy and land together, Ryo AOT ahead of Rust (28.7 vs 31.6 ms) on the strength of cheaper string formatting; Ryo's deficit here is ergonomic, not runtime — `p.name + ""` *is* the clone, and it performs like one. The gap to Swift/Go is what `shared[T]` (retain instead of copy) or a small-string optimization (I-171) would close. Ryo AOT runs **5.2x faster than Python** with ~11x less memory, again at the lightest RSS of the suite. + +## How to Run + +Prerequisites: `hyperfine`, `rustc`, `swiftc`, `go`, `python3`, plus a release build of the compiler (`cargo build --release` from the repository root — the script runs it for you). + +```bash +./run_benchmarks.sh +``` diff --git a/benchmarks/struct_records_reuse/run_benchmarks.sh b/benchmarks/struct_records_reuse/run_benchmarks.sh new file mode 100755 index 0000000..b1d89cf --- /dev/null +++ b/benchmarks/struct_records_reuse/run_benchmarks.sh @@ -0,0 +1,100 @@ +#!/bin/bash +set -e + +# Check for prerequisites +if ! command -v hyperfine &> /dev/null; then + echo "Error: 'hyperfine' is not installed or not in PATH. Please install it to run performance benchmarks." + exit 1 +fi + +if ! command -v rustc &> /dev/null; then + echo "Error: 'rustc' is not installed or not in PATH." + exit 1 +fi + +if ! command -v swiftc &> /dev/null; then + echo "Error: 'swiftc' is not installed or not in PATH." + exit 1 +fi + +if ! command -v go &> /dev/null; then + echo "Error: 'go' is not installed or not in PATH." + exit 1 +fi + +if ! command -v python3 &> /dev/null; then + echo "Error: 'python3' is not installed or not in PATH." + exit 1 +fi + +echo "Building benchmarks..." +(cd ../.. && cargo build --release > /dev/null) +rustc -O struct_records_reuse.rs -o struct_records_reuse_rs +swiftc -O struct_records_reuse.swift -o struct_records_reuse_swift +go build -o struct_records_reuse_go struct_records_reuse.go +ryo_bin="../../target/release/ryo" +$ryo_bin build struct_records_reuse.ryo > /dev/null + +echo "" +echo "-------------------" +echo "Compiler Version" +echo "-------------------" +echo "Rust: $(rustc --version | cut -d' ' -f2)" +echo "Swift: $(swiftc --version | head -1 | awk '{for (i = 1; i < NF; i++) if ($i == "Swift" && $(i+1) == "version") { print $(i+2); exit }}')" +echo "Go: $(go version | cut -d' ' -f3 | sed 's/^go//')" +echo "Python: $(python3 --version | cut -d' ' -f2)" +echo "Ryo: $($ryo_bin --version 2>&1 || echo 'dev')" + +echo "" +echo "-------------------" +echo "Memory Usage (Maximum Resident Set Size)" +echo "-------------------" +_OS="$(uname -s)" +measure_mem() { + local name=$1 + shift + + local mem_kb + local mem_out + case "$_OS" in + Darwin*) + # /usr/bin/time -l reports bytes on macOS; convert to KB + mem_kb=$( ( /usr/bin/time -l "$@" > /dev/null ) 2>&1 | awk '/maximum resident set size/ {printf "%d", $1 / 1024; exit}' ) + ;; + Linux*) + mem_kb=$( { /usr/bin/time -f "%M" "$@" > /dev/null; } 2>&1 | tail -n1 ) + ;; + *) + mem_kb="" + ;; + esac + + if [[ -n "$mem_kb" ]]; then + mem_out=$(awk -v kb="$mem_kb" 'BEGIN { printf "%.2f MB", kb / 1024 }') + else + mem_out="N/A" + fi + + printf "%-28s %s\n" "[$name]" "$mem_out" +} + +# Run once each to collect memory usage +measure_mem "Rust" ./struct_records_reuse_rs +measure_mem "Swift" ./struct_records_reuse_swift +measure_mem "Go" ./struct_records_reuse_go +measure_mem "Python" python3 struct_records_reuse.py +measure_mem "Ryo (AOT)" ./struct_records_reuse +measure_mem "Ryo (JIT)" $ryo_bin run struct_records_reuse.ryo + +echo "" +echo "-------------------" +echo "Running Benchmarks (500,000 record build-update-reuse rounds) using hyperfine" +echo "-------------------" + +hyperfine --warmup 3 --shell=none \ + './struct_records_reuse_rs' \ + './struct_records_reuse_swift' \ + './struct_records_reuse_go' \ + 'python3 struct_records_reuse.py' \ + './struct_records_reuse' \ + "$ryo_bin run struct_records_reuse.ryo" diff --git a/benchmarks/struct_records_reuse/struct_records_reuse.go b/benchmarks/struct_records_reuse/struct_records_reuse.go new file mode 100644 index 0000000..dfa9537 --- /dev/null +++ b/benchmarks/struct_records_reuse/struct_records_reuse.go @@ -0,0 +1,33 @@ +package main + +import "fmt" + +type Person struct { + Name string + Age int +} + +func makePerson(i int) Person { + return Person{Name: fmt.Sprintf("user%d", i), Age: 20 + i%50} +} + +func birthday(p Person) Person { + return Person{Name: p.Name, Age: p.Age + 1} +} + +func score(p Person) int { + return len(p.Name) + p.Age +} + +func main() { + total := 0 + for i := range 500000 { + p := makePerson(i) + q := birthday(p) + total += score(p) + score(q) + } + if total != 54777780 { + panic("struct_records_reuse checksum") + } + fmt.Println("assert passed, struct_records_reuse is correct") +} diff --git a/benchmarks/struct_records_reuse/struct_records_reuse.py b/benchmarks/struct_records_reuse/struct_records_reuse.py new file mode 100644 index 0000000..90d479c --- /dev/null +++ b/benchmarks/struct_records_reuse/struct_records_reuse.py @@ -0,0 +1,31 @@ +class Person: + __slots__ = ("age", "name") + + def __init__(self, name, age): + self.name = name + self.age = age + + +def make_person(i): + return Person("user" + str(i), 20 + i % 50) + + +def birthday(p): + return Person(p.name, p.age + 1) + + +def score(p): + return len(p.name) + p.age + + +def main(): + total = 0 + for i in range(500000): + p = make_person(i) + q = birthday(p) + total += score(p) + score(q) + assert total == 54777780, "struct_records_reuse checksum" + print("assert passed, struct_records_reuse is correct") + + +main() diff --git a/benchmarks/struct_records_reuse/struct_records_reuse.rs b/benchmarks/struct_records_reuse/struct_records_reuse.rs new file mode 100644 index 0000000..a75f5ce --- /dev/null +++ b/benchmarks/struct_records_reuse/struct_records_reuse.rs @@ -0,0 +1,33 @@ +struct Person { + name: String, + age: i64, +} + +fn make_person(i: i64) -> Person { + Person { + name: format!("user{}", i), + age: 20 + i % 50, + } +} + +fn birthday(p: &Person) -> Person { + Person { + name: p.name.clone(), + age: p.age + 1, + } +} + +fn score(p: &Person) -> i64 { + p.name.len() as i64 + p.age +} + +fn main() { + let mut total = 0i64; + for i in 0..500000 { + let p = make_person(i); + let q = birthday(&p); + total += score(&p) + score(&q); + } + assert!(total == 54777780, "struct_records_reuse checksum"); + println!("assert passed, struct_records_reuse is correct"); +} diff --git a/benchmarks/struct_records_reuse/struct_records_reuse.ryo b/benchmarks/struct_records_reuse/struct_records_reuse.ryo new file mode 100644 index 0000000..849ef18 --- /dev/null +++ b/benchmarks/struct_records_reuse/struct_records_reuse.ryo @@ -0,0 +1,21 @@ +struct Person: + name: str + age: int + +fn make_person(i: int) -> Person: + return Person{name="user" + int_to_str(i), age=20 + i % 50} + +fn birthday(p: Person) -> Person: + return Person{name=p.name + "", age=p.age + 1} + +fn score(p: Person) -> int: + return p.name.len() + p.age + +fn main(): + mut total = 0 + for i in range(0, 500000): + p = make_person(i) + q = birthday(p) + total += score(p) + score(q) + assert(total == 54777780, "struct_records_reuse checksum") + print("assert passed, struct_records_reuse is correct\n") diff --git a/benchmarks/struct_records_reuse/struct_records_reuse.swift b/benchmarks/struct_records_reuse/struct_records_reuse.swift new file mode 100644 index 0000000..eed4e4a --- /dev/null +++ b/benchmarks/struct_records_reuse/struct_records_reuse.swift @@ -0,0 +1,25 @@ +struct Person { + var name: String + var age: Int +} + +func makePerson(_ i: Int) -> Person { + Person(name: "user" + String(i), age: 20 + i % 50) +} + +func birthday(_ p: Person) -> Person { + Person(name: p.name, age: p.age + 1) +} + +func score(_ p: Person) -> Int { + p.name.utf8.count + p.age +} + +var total = 0 +for i in 0..<500000 { + let p = makePerson(i) + let q = birthday(p) + total += score(p) + score(q) +} +precondition(total == 54777780, "struct_records_reuse checksum") +print("assert passed, struct_records_reuse is correct") diff --git a/codspeed.yml b/codspeed.yml index a12a3ad..04882e8 100644 --- a/codspeed.yml +++ b/codspeed.yml @@ -15,3 +15,9 @@ benchmarks: exec: ./benchmarks/mandelbrot/mandelbrot - name: collatz-aot exec: ./benchmarks/collatz/collatz + - name: struct-records-aot + exec: ./benchmarks/struct_records/struct_records + - name: struct-records-reuse-aot + exec: ./benchmarks/struct_records_reuse/struct_records_reuse + - name: struct-records-inout-aot + exec: ./benchmarks/struct_records_inout/struct_records_inout