Isolate Malloc candidates in Wasm64 and refresh scores

README.md

@@ -130,9 +130,9 @@ Detailed leaderboard (incl. average rank): [lab.einsia.ai/frontier-eng/leaderboa
 | 1 | Claude Opus 4.6 | 0.533 | 0.501 | 14 | 15 | 3 |
 | 2 | GPT-5.4 | 0.454 | 0.267 | 18 | 4 | 2 |
 | 3 | GLM-5 | 0.347 | 0.300 | 7 | 8 | 12 |
-| 4 | Gemini 3.1 Pro Preview | 0.277 | 0.267 | 7 | 7 | 4 |
+| 4 | Gemini 3.1 Pro Preview | 0.284 | 0.300 | 7 | 7 | 5 |
 | 5 | DeepSeek V3.2 | 0.269 | 0.299 | 6 | 6 | 8 |
-| 6 | Grok 4.20 | 0.227 | 0.200 | 6 | 5 | 4 |
+| 6 | Grok 4.20 | 0.220 | 0.167 | 6 | 5 | 3 |
 | 7 | Seed 2.0 Pro | 0.206 | 0.100 | 6 | 4 | 3 |
 | 8 | Qwen3 Coder Next | 0.170 | 0.066 | 5 | 3 | 3 |
 

README_zh-CN.md

@@ -125,9 +125,9 @@ bash scripts/batch/validate_v1_task_envs.sh
 | 1 | Claude Opus 4.6 | 0.533 | 0.501 | 14 | 15 | 3 |
 | 2 | GPT-5.4 | 0.454 | 0.267 | 18 | 4 | 2 |
 | 3 | GLM-5 | 0.347 | 0.300 | 7 | 8 | 12 |
-| 4 | Gemini 3.1 Pro Preview | 0.277 | 0.267 | 7 | 7 | 4 |
+| 4 | Gemini 3.1 Pro Preview | 0.284 | 0.300 | 7 | 7 | 5 |
 | 5 | DeepSeek V3.2 | 0.269 | 0.299 | 6 | 6 | 8 |
-| 6 | Grok 4.20 | 0.227 | 0.200 | 6 | 5 | 4 |
+| 6 | Grok 4.20 | 0.220 | 0.167 | 6 | 5 | 3 |
 | 7 | Seed 2.0 Pro | 0.206 | 0.100 | 6 | 4 | 3 |
 | 8 | Qwen3 Coder Next | 0.170 | 0.066 | 5 | 3 | 3 |
 

benchmarks/ComputerSystems/MallocLab/README.md

@@ -5,3 +5,12 @@ The relevant files are located in `benchmarks/ComputerSystems/MallocLab/mallocla
 For more details, please see [Task](Task.md).
 
 Note: the evolved candidate file is `malloclab-handout/mm.c`. Keep function signatures unchanged, and keep `// EVOLVE-BLOCK-START` / `// EVOLVE-BLOCK-END` markers in place so evolution algorithms can safely apply diffs.
+
+Official scoring uses a Wasm64 allocator with a trusted host driver. Install the Linux x86-64 toolchain once from the repository root:
+
+```bash
+python benchmarks/_shared/malloc_wasm/setup.py --install
+bash benchmarks/ComputerSystems/MallocLab/frontier_eval/run_eval.sh python3 benchmarks/ComputerSystems/MallocLab
+```
+
+The score measures calls in the isolated runtime; native `make && ./mdriver -V` remains available for local debugging. The runtime keeps 64-bit pointers and the 20 MiB simulated heap. It requires Linux user namespaces, bubblewrap, and a native C compiler. `FRONTIER_MALLOC_TOOLCHAIN` selects an alternate toolchain installation directory.

benchmarks/ComputerSystems/MallocLab/README_zh-CN.md

@@ -5,3 +5,12 @@
 更多详细信息请查看 [Task](Task_zh-CN.md)
 
 提示：被 evolve 的候选文件为 `malloclab-handout/mm.c`。请保持函数签名不变，并保留 `// EVOLVE-BLOCK-START` / `// EVOLVE-BLOCK-END` 标记，便于演化算法安全地应用 diff。
+
+正式评分将分配器编译为 Wasm64，由模块外的可信驱动进行验证和计时。在仓库根目录安装一次 Linux x86-64 工具链：
+
+```bash
+python benchmarks/_shared/malloc_wasm/setup.py --install
+bash benchmarks/ComputerSystems/MallocLab/frontier_eval/run_eval.sh python3 benchmarks/ComputerSystems/MallocLab
+```
+
+分数使用隔离运行时中的调用耗时；原生 `make && ./mdriver -V` 可用于本地调试。运行时保留 64 位指针和 20 MiB 模拟堆，需要 Linux 用户命名空间、bubblewrap 和本机 C 编译器。可用 `FRONTIER_MALLOC_TOOLCHAIN` 指定工具链安装目录。

benchmarks/ComputerSystems/MallocLab/Task.md

@@ -113,26 +113,27 @@ The `memlib.c` package simulates a memory system for the dynamic memory allocato
 
 * Interface functions in `mm.c` must not be modified.
 
-* `mm.c` must not read standard input.
+* Official scoring provides no file, network, process, standard-input, or clock interfaces.
 
-* System library functions must not be called.
+* System allocation functions (`malloc`, `free`, `realloc`, `sbrk`, `mmap`) must not be called; obtain heap space through `mem_sbrk`. The runtime supports `memcpy`, `memmove`, `memset`, `memcmp`, and `strlen`.
 
 * Global or static composite data structures, such as arrays, structures, trees, or lists, must not be defined in the `mm.c` program. However, global scalar variables, such as integers, floating-point numbers, and pointers, can be declared in `mm.c`.
 
 * Returned memory blocks should be 16-byte aligned.
 
 ## Scoring Criteria
 
-The evaluator reads the result file written by `mdriver`. The allocator and
-driver execute in the same process and share an address space.
+Official scoring uses a Wasm64 allocator and a trusted host driver. The host owns the traces, heap high-water mark, and score, and checks allocation bounds, alignment, overlap, and preserved data. Each passing trace is independently executed and validated ten times; throughput uses the median allocator-call time measured by the host. Compilation, runtime initialization, and host payload checks are excluded from call time. Failed traces contribute no utilization or completed operations.
+
+See the task README for the scoring command. Native `mdriver` remains a local debugging tool; its timings are not directly comparable to isolated-runtime scores.
 
 * Space Utilization: The ratio between the maximum amount of memory used by the program and the maximum heap size used by the allocator; the optimal ratio is 1.
 
 * Throughput: Kops (kilo operations per second)
 
-* Scoring Formula: $$P = wU + (1 - w)\min(1, \frac{T}{T_{libc}})$$
+* Scoring Formula: $$P = 100\left(wU + (1-w)\min(1,T/T_{ref})\right)m/N$$
 
-* where w is space utilization, and $T_{libc}$ is throughput. $T_{libc}$ is the throughput of libc malloc tested by the teaching assistant on the course cluster. The specific value is based on `AVG_LIBC_THRUPUT` in `config.h`. A balance needs to be considered when optimizing space utilization and throughput.
+* Here $w=0.6$, $N=11$, $m$ is the number of passing traces, and $U$ is mean utilization across all traces (zero for failed traces). $T$ is completed operations divided by the sum of median call times for passing traces. The fixed reference cap is $T_{ref}=10{,}000{,}000$ operations/second. It is a score normalization constant, not a fresh measurement of native libc performance.
 
 ## Some Suggestions
 

benchmarks/ComputerSystems/MallocLab/Task_zh-CN.md

@@ -85,19 +85,22 @@ void *mm_realloc(void *ptr, size_t size);
 * 使用方式可通过 `./mdriver -h` 查看。其中 `-V` 可用于定位报错出现的文件，`-f` 可用于指定 trace 进行测试。 
 
 ## 编程规则
-* `mm.c` 不得读取标准输入。
+* 正式评分不提供文件、网络、进程、标准输入或时钟接口。
 * 不允许改变 `mm.c` 的接口函数 
-* 不允许调用系统的库函数 
+* 不允许调用系统内存分配函数（如 `malloc`、`free`、`realloc`、`sbrk`、`mmap`）；分配堆空间必须使用 `mem_sbrk`。支持 `memcpy`、`memmove`、`memset`、`memcmp` 和 `strlen`。
 * 不允许在 `mm.c` 程序中定义全局或静态的复合数据结构，如数组、结构、树或列表。但是可以在 `mm.c` 中声明全局标量变量，如整数、浮点数和指针。 
 * 返回的内存块应 16 字节对齐 
 
 ## 评分标准
 
-评测器读取 `mdriver` 写出的结果文件。分配器和驱动在同一进程中执行，共享地址空间。
+正式评分使用 Wasm64 分配器和模块外的可信驱动。驱动掌握测试序列、堆高水位和分数，验证返回内存的边界、对齐、重叠及数据保留。每条通过的 trace 独立执行并验证 10 次，吞吐量使用外部测得的分配器调用耗时中位数；编译、初始化运行时和宿主数据检查不计入调用耗时。失败的 trace 不贡献利用率或已完成操作数。
+
+运行方式见本题 README。原生 `mdriver` 用于本地调试，其计时结果与隔离运行时分数不直接比较。
 
 * 空间利用率：程序使用的最大内存量与分配器使用的最大堆大小之间的比率，最佳比率为 1。 
 * 吞吐量：Kops (kilo operations per second) 
-* 评分公式： $$P = wU + (1 - w)\min(1, \frac{T}{T_{libc}})$$
+* 评分公式： $$P = 100\left(wU + (1-w)\min(1,T/T_{ref})\right)m/N$$
+* 其中 $w=0.6$、$N=11$，$m$ 为通过的 trace 数，$U$ 为全部 trace 的平均利用率（失败项为零）。$T$ 为通过项的操作总数除以调用耗时中位数之和。固定归一化上限 $T_{ref}=10{,}000{,}000$ 次/秒，不代表本次重新测量了原生 libc 的性能。
 *  为空间利用率， 为吞吐量 (throughput)， $T_{libc}$ 是助教在课程集群上测试的 libc malloc 的吞吐量，具体值以 `config.h` 的 `AVG_LIBC_THRUPUT` 为准。，需要均衡地考虑空间利用率和吞吐量的优化。 
 
 ## 一些建议

benchmarks/ComputerSystems/MallocLab/frontier_eval/artifact_files.txt

@@ -1,5 +1 @@
-run_meta.txt
-make_clean.log
-make.log
-mdriver.stdout.txt
-mdriver.stderr.txt
+malloc_details.json

benchmarks/ComputerSystems/MallocLab/frontier_eval/constraints.txt

@@ -8,9 +8,10 @@ MallocLab UnifiedTask constraints:
 3) Do not modify benchmark runner files (`mdriver.c`, trace files, build scripts).
    These are enforced read-only and fingerprinted; changing one invalidates the run.
 4) Candidate should target correctness first, then optimize throughput/utilization.
-5) Evaluator compiles with `make` and runs `./mdriver -V -o <result file>`.
-6) The score is read from the result file mdriver writes, NOT from its stdout.
-   The record only counts if it carries the per-run token the grader hands
-   mdriver on stdin. Printing a `Score = ... = N/100` line yourself does
-   nothing; consuming stdin before mdriver's main() reads it aborts the run
-   with a zero. mm.c must not read stdin.
+5) Official evaluation compiles C to Wasm64 with 64-bit pointers and size_t.
+   The host owns the 20 MiB simulated heap limit, correctness checks and timers.
+6) Obtain heap space through mem_sbrk. System malloc/free/realloc/sbrk/mmap
+   and file, process, network, standard-input or clock interfaces are unavailable.
+7) Preserve data across realloc and do not overwrite other live allocations.
+   A trusted host computes scores from validated operations; candidate output
+   is not a scoring channel. Native mdriver is for local debugging only.

benchmarks/ComputerSystems/MallocLab/frontier_eval/copy_files.txt

@@ -1,6 +1,5 @@
 malloclab-handout
 frontier_eval/run_eval.sh
-frontier_eval/parse_mdriver_result.py
 frontier_eval/constraints.txt
 frontier_eval/artifact_files.txt
 Task_zh-CN.md

benchmarks/ComputerSystems/MallocLab/frontier_eval/readonly_files.txt

@@ -25,4 +25,3 @@ README_zh-CN.md
 README.md
 frontier_eval/constraints.txt
 frontier_eval/run_eval.sh
-frontier_eval/parse_mdriver_result.py

benchmarks/ComputerSystems/MallocLab/frontier_eval/run_eval.sh

@@ -1,60 +1,15 @@
 #!/usr/bin/env bash
 set -euo pipefail
 
-PYTHON_CMD="${1:?missing python command}"
-BENCHMARK_DIR="${2:?missing benchmark dir}"
-CANDIDATE_PATH="${3:-}"
-
-HANDOUT_DIR="${BENCHMARK_DIR}/malloclab-handout"
-MAKE_CLEAN_LOG="${BENCHMARK_DIR}/make_clean.log"
-MAKE_LOG="${BENCHMARK_DIR}/make.log"
-MDRIVER_STDOUT="${BENCHMARK_DIR}/mdriver.stdout.txt"
-MDRIVER_STDERR="${BENCHMARK_DIR}/mdriver.stderr.txt"
-MDRIVER_RESULT="${BENCHMARK_DIR}/mdriver_result.json"
-METRICS_JSON="${BENCHMARK_DIR}/metrics.json"
-
-# Per-run token for the authenticated result channel.
-#
-# The candidate's mm.c is compiled into mdriver, so it can write anything it
-# likes to mdriver's stdout -- and the score used to be parsed from there. It
-# now travels in ${MDRIVER_RESULT}, which only counts if it carries this token.
-#
-# The token is a shell variable, never exported and never written to disk while
-# mdriver runs, so it is not in mdriver's environ and not readable from the
-# filesystem. It reaches mdriver on stdin, which mdriver consumes and closes
-# before it calls into the allocator, and it reaches the parser on a command
-# line that is only built after mdriver has already exited.
-RUN_TOKEN="$(od -An -N32 -tx1 /dev/urandom | tr -d ' \n')"
-if [[ -z "${RUN_TOKEN}" ]]; then
-  echo "ERROR: could not generate a run token" >&2
-  exit 1
+PYTHON_CMD="${1:-python3}"
+BENCHMARK_DIR="${2:?benchmark directory is required}"
+CANDIDATE="${3:-${BENCHMARK_DIR}/malloclab-handout/mm.c}"
+REPO_ROOT="${FRONTIER_ENGINEERING_ROOT:-}"
+if [[ -z "${REPO_ROOT}" ]]; then
+  REPO_ROOT="$(cd "${BENCHMARK_DIR}/../../.." && pwd)"
 fi
 
-rm -f "${MDRIVER_RESULT}"
-
-cd "${HANDOUT_DIR}"
-
-make clean >"${MAKE_CLEAN_LOG}" 2>&1
-make >"${MAKE_LOG}" 2>&1
-
-set +e
-printf '%s\n' "${RUN_TOKEN}" \
-  | ./mdriver -V -o "${MDRIVER_RESULT}" >"${MDRIVER_STDOUT}" 2>"${MDRIVER_STDERR}"
-MDRIVER_RC=$?
-set -e
-
-{
-  echo "candidate_path=${CANDIDATE_PATH}"
-  echo "mdriver_returncode=${MDRIVER_RC}"
-} > "${BENCHMARK_DIR}/run_meta.txt"
-
-"${PYTHON_CMD}" "${BENCHMARK_DIR}/frontier_eval/parse_mdriver_result.py" \
-  --result-file "${MDRIVER_RESULT}" \
-  --expected-token "${RUN_TOKEN}" \
-  --stdout-file "${MDRIVER_STDOUT}" \
-  --stderr-file "${MDRIVER_STDERR}" \
-  --mdriver-returncode "${MDRIVER_RC}" \
-  --metrics-out "${METRICS_JSON}"
-
-# Always return 0 here: parsed `metrics.json` already encodes validity/score.
-exit 0
+exec "${PYTHON_CMD}" "${REPO_ROOT}/benchmarks/_shared/malloc_isolation.py" \
+  "${CANDIDATE}" --benchmark "${BENCHMARK_DIR}" \
+  --metrics-out "${BENCHMARK_DIR}/metrics.json" \
+  --details-out "${BENCHMARK_DIR}/malloc_details.json"