Document and implement runtime memory model pipeline updates

PLANS.md

@@ -6,6 +6,9 @@ This file tracks implementation work derived from specs that do not yet have a c
 - SPEC-000 Project Charter and Ethics
 - SPEC-010 Target Platform Baseline
 - SPEC-020 Inputs and Provenance
+- SPEC-045 Runtime Memory Model and Load/Store Lowering
+- SPEC-046 Runtime Memory Layout Configuration
+- SPEC-047 Memory Image Initialization
 - SPEC-090 Build, Packaging, and Distribution
 - SPEC-095 Build Manifest Integrity
 - SPEC-096 Bundle Manifest Integrity
@@ -59,6 +62,51 @@ Exit criteria (from SPEC-020)
 - The toolchain refuses to build without provenance metadata.
 - A format detector identifies NCA/ExeFS/NSO0/NRO0/NRR0 inputs and logs the chosen path.
 
+## SPEC-045: Runtime Memory Model and Load/Store Lowering
+Outcome
+- Block-based output can execute basic load/store instructions against a minimal runtime memory model.
+
+Work items
+- [x] Define a memory layout descriptor schema and emit it with outputs.
+- [x] Implement runtime memory regions with alignment, bounds, and permission checks.
+- [x] Lower ISA load/store ops to runtime memory helper calls.
+- [x] Add tests and sample blocks that validate load/store behavior and error handling.
+
+Exit criteria (from SPEC-045)
+- Block-based output executes a test block with loads and stores using runtime helpers.
+- Unaligned or out-of-bounds accesses return deterministic error codes.
+- A sample pipeline output includes a memory layout descriptor that matches runtime regions.
+
+## SPEC-046: Runtime Memory Layout Configuration
+Outcome
+- Runtime memory layout is configurable via `title.toml` while preserving a safe default.
+
+Work items
+- [ ] Extend `title.toml` schema to include `runtime.memory_layout` regions.
+- [ ] Validate region overlap, zero sizes, and overflow errors.
+- [ ] Emit configured memory layout in `manifest.json` and generated runtime init.
+- [ ] Add tests for default layout and custom layout parsing.
+
+Exit criteria (from SPEC-046)
+- Custom memory layout in `title.toml` is parsed and emitted in `manifest.json`.
+- Invalid layouts fail the pipeline with clear errors.
+- Default behavior remains unchanged when no layout is provided.
+
+## SPEC-047: Memory Image Initialization
+Outcome
+- Runtime memory is initialized from module segment metadata (code/rodata/data/bss).
+
+Work items
+- [ ] Define segment descriptor schema and carry it through pipeline output metadata.
+- [ ] Populate runtime memory regions with initial segment bytes and zeroed bss.
+- [ ] Validate init sizes and bounds during initialization.
+- [ ] Add tests covering initialized load/store behavior and error paths.
+
+Exit criteria (from SPEC-047)
+- A sample module with init bytes executes a load/store path against initialized memory.
+- BSS regions are zeroed deterministically.
+- Invalid init sizes or region mismatches fail with clear errors.
+
 ## SPEC-090: Build, Packaging, and Distribution
 Outcome
 - Produce a reproducible, policy-compliant bundle layout with a release checklist.

README.md

@@ -28,6 +28,10 @@ Legal and provenance policy:
 - The dev environment is managed with Nix + devenv.
 - See `docs/DEVELOPMENT.md` for commands and sample usage.
 
+## Samples and Flow Docs
+- `samples/memory-image/` shows the memory image initialization flow (segment blob + lifted module).
+- `docs/static-recompilation-flow.md` outlines a hypothetical macOS static recompilation flow and verification pipeline.
+
 ## Back Pressure Hooks
 These hooks add fast, consistent feedback to keep the repo autonomous and reduce review churn. Hooks are defined in `.pre-commit-config.yaml` and can be run with `prek` (preferred) or `pre-commit`.
 

crates/recomp-pipeline/src/config.rs

@@ -1,3 +1,4 @@
+use crate::memory::{MemoryLayoutDescriptor, MemoryPermissionsDescriptor, MemoryRegionDescriptor};
 use serde::Deserialize;
 use std::collections::BTreeMap;
 use std::str::FromStr;
@@ -40,10 +41,32 @@ impl FromStr for PerformanceMode {
     }
 }
 
-#[derive(Debug, Deserialize)]
+#[derive(Debug, Deserialize, Default)]
 struct RawRuntimeConfig {
     #[serde(default)]
     performance_mode: Option<String>,
+    #[serde(default)]
+    memory_layout: Option<RawMemoryLayoutConfig>,
+}
+
+#[derive(Debug, Deserialize)]
+struct RawMemoryLayoutConfig {
+    regions: Vec<RawMemoryRegionConfig>,
+}
+
+#[derive(Debug, Deserialize)]
+struct RawMemoryRegionConfig {
+    name: String,
+    base: u64,
+    size: u64,
+    permissions: RawMemoryPermissionsConfig,
+}
+
+#[derive(Debug, Deserialize)]
+struct RawMemoryPermissionsConfig {
+    read: bool,
+    write: bool,
+    execute: bool,
 }
 
 #[derive(Debug)]
@@ -69,6 +92,7 @@ pub struct TitleConfig {
     pub abi_version: String,
     pub stubs: BTreeMap<String, StubBehavior>,
     pub runtime: RuntimeConfig,
+    pub memory_layout: MemoryLayoutDescriptor,
 }
 
 impl TitleConfig {
@@ -80,17 +104,44 @@ impl TitleConfig {
             let parsed = StubBehavior::from_str(&behavior)?;
             stubs.insert(name, parsed);
         }
-        let runtime_mode = raw
-            .runtime
-            .and_then(|runtime| runtime.performance_mode)
+        let runtime = raw.runtime.unwrap_or_default();
+        let runtime_mode = runtime
+            .performance_mode
             .unwrap_or_else(|| "handheld".to_string());
         let performance_mode = PerformanceMode::from_str(&runtime_mode)?;
+        let memory_layout = match runtime.memory_layout {
+            Some(layout) => parse_memory_layout(layout)?,
+            None => MemoryLayoutDescriptor::minimal_default(),
+        };
         Ok(TitleConfig {
             title: raw.title,
             entry: raw.entry,
             abi_version: raw.abi_version,
             stubs,
             runtime: RuntimeConfig { performance_mode },
+            memory_layout,
         })
     }
 }
+
+fn parse_memory_layout(layout: RawMemoryLayoutConfig) -> Result<MemoryLayoutDescriptor, String> {
+    let regions = layout
+        .regions
+        .into_iter()
+        .map(|region| {
+            MemoryRegionDescriptor::new(
+                region.name,
+                region.base,
+                region.size,
+                MemoryPermissionsDescriptor::new(
+                    region.permissions.read,
+                    region.permissions.write,
+                    region.permissions.execute,
+                ),
+            )
+        })
+        .collect();
+    let descriptor = MemoryLayoutDescriptor { regions };
+    descriptor.validate()?;
+    Ok(descriptor)
+}

crates/recomp-pipeline/src/homebrew/lift.rs

@@ -99,6 +99,7 @@ fn lift_stub(
 
     let lifted = Module {
         arch: "aarch64".to_string(),
+        segments: Vec::new(),
         functions: vec![Function {
             name: options.entry_name.clone(),
             ops: vec![Op::Ret],
@@ -183,6 +184,7 @@ fn lift_decode(
 
     let lifted = Module {
         arch: "aarch64".to_string(),
+        segments: Vec::new(),
         functions,
     };
 

crates/recomp-pipeline/src/input.rs

@@ -3,9 +3,32 @@ use serde::{Deserialize, Serialize};
 #[derive(Debug, Deserialize, Serialize)]
 pub struct Module {
     pub arch: String,
+    #[serde(default)]
+    pub segments: Vec<ModuleSegment>,
     pub functions: Vec<Function>,
 }
 
+#[derive(Debug, Deserialize, Serialize)]
+pub struct ModuleSegment {
+    pub name: String,
+    pub base: u64,
+    pub size: u64,
+    pub permissions: ModuleSegmentPermissions,
+    #[serde(default)]
+    pub init_path: Option<String>,
+    #[serde(default)]
+    pub init_size: Option<u64>,
+    #[serde(default)]
+    pub zero_fill: bool,
+}
+
+#[derive(Debug, Deserialize, Serialize)]
+pub struct ModuleSegmentPermissions {
+    pub read: bool,
+    pub write: bool,
+    pub execute: bool,
+}
+
 #[derive(Debug, Deserialize, Serialize)]
 pub struct Function {
     pub name: String,

crates/recomp-pipeline/src/lib.rs

@@ -2,6 +2,7 @@ pub mod bundle;
 pub mod config;
 pub mod homebrew;
 pub mod input;
+pub mod memory;
 pub mod output;
 pub mod pipeline;
 pub mod provenance;

crates/recomp-pipeline/src/memory.rs

@@ -0,0 +1,149 @@
+use serde::Serialize;
+
+#[derive(Debug, Serialize, Clone)]
+pub struct MemoryLayoutDescriptor {
+    pub regions: Vec<MemoryRegionDescriptor>,
+}
+
+impl MemoryLayoutDescriptor {
+    pub fn minimal_default() -> Self {
+        Self {
+            regions: vec![
+                MemoryRegionDescriptor::new(
+                    "code",
+                    0x1000_0000,
+                    0x0001_0000,
+                    MemoryPermissionsDescriptor::new(true, false, true),
+                ),
+                MemoryRegionDescriptor::new(
+                    "rodata",
+                    0x1001_0000,
+                    0x0001_0000,
+                    MemoryPermissionsDescriptor::new(true, false, false),
+                ),
+                MemoryRegionDescriptor::new(
+                    "data",
+                    0x1002_0000,
+                    0x0001_0000,
+                    MemoryPermissionsDescriptor::new(true, true, false),
+                ),
+                MemoryRegionDescriptor::new(
+                    "heap",
+                    0x2000_0000,
+                    0x0004_0000,
+                    MemoryPermissionsDescriptor::new(true, true, false),
+                ),
+                MemoryRegionDescriptor::new(
+                    "stack",
+                    0x3000_0000,
+                    0x0004_0000,
+                    MemoryPermissionsDescriptor::new(true, true, false),
+                ),
+            ],
+        }
+    }
+
+    pub fn validate(&self) -> Result<(), String> {
+        if self.regions.is_empty() {
+            return Err("memory layout must define at least one region".to_string());
+        }
+
+        let mut ranges = Vec::with_capacity(self.regions.len());
+        for region in &self.regions {
+            if region.size == 0 {
+                return Err(format!("memory region {} has zero size", region.name));
+            }
+            let end = region
+                .base
+                .checked_add(region.size)
+                .ok_or_else(|| format!("memory region {} overflows address space", region.name))?;
+            ranges.push((region.name.as_str(), region.base, end));
+        }
+
+        ranges.sort_by(|a, b| a.1.cmp(&b.1));
+        for window in ranges.windows(2) {
+            let left = window[0];
+            let right = window[1];
+            if left.2 > right.1 {
+                return Err(format!("memory regions {} and {} overlap", left.0, right.0));
+            }
+        }
+
+        Ok(())
+    }
+}
+
+#[derive(Debug, Serialize, Clone)]
+pub struct MemoryRegionDescriptor {
+    pub name: String,
+    pub base: u64,
+    pub size: u64,
+    pub permissions: MemoryPermissionsDescriptor,
+}
+
+impl MemoryRegionDescriptor {
+    pub fn new(
+        name: impl Into<String>,
+        base: u64,
+        size: u64,
+        permissions: MemoryPermissionsDescriptor,
+    ) -> Self {
+        Self {
+            name: name.into(),
+            base,
+            size,
+            permissions,
+        }
+    }
+}
+
+#[derive(Debug, Serialize, Clone)]
+pub struct MemoryImageDescriptor {
+    pub init_segments: Vec<MemoryInitSegmentDescriptor>,
+    pub zero_segments: Vec<MemoryZeroSegmentDescriptor>,
+}
+
+impl MemoryImageDescriptor {
+    pub fn empty() -> Self {
+        Self {
+            init_segments: Vec::new(),
+            zero_segments: Vec::new(),
+        }
+    }
+
+    pub fn is_empty(&self) -> bool {
+        self.init_segments.is_empty() && self.zero_segments.is_empty()
+    }
+}
+
+#[derive(Debug, Serialize, Clone)]
+pub struct MemoryInitSegmentDescriptor {
+    pub name: String,
+    pub base: u64,
+    pub size: u64,
+    pub init_path: String,
+}
+
+#[derive(Debug, Serialize, Clone)]
+pub struct MemoryZeroSegmentDescriptor {
+    pub name: String,
+    pub base: u64,
+    pub size: u64,
+}
+
+#[derive(Debug, Serialize, Clone, Copy)]
+pub struct MemoryPermissionsDescriptor {
+    pub read: bool,
+    pub write: bool,
+    pub execute: bool,
+}
+
+impl MemoryPermissionsDescriptor {
+    pub fn new(read: bool, write: bool, execute: bool) -> Self {
+        Self {
+            read,
+            write,
+            execute,
+        }
+    }
+}