Add dynamic edge service replica placement benchmark

TASK_DETAILS.md

@@ -204,14 +204,18 @@ We welcome new engineering problem ideas — even without complete verification
       <td>Polarization-multiplexed holography</td>
     </tr>
     <tr>
-      <td rowspan="2"><b>ComputerSystems</b></td>
+      <td rowspan="3"><b>ComputerSystems</b></td>
       <td><code>MallocLab</code></td>
       <td>High-performance C memory allocator (utilization &amp; throughput)</td>
     </tr>
     <tr>
       <td><code>DuckDBWorkloadOptimization</code></td>
       <td>Index / materialized-view selection and query rewriting on official DuckDB workloads</td>
     </tr>
+    <tr>
+      <td><code>EdgeServiceReplicaPlacement</code></td>
+      <td>Dynamic edge-service replica placement and routing under workload bursts, failures, and link degradation</td>
+    </tr>
     <tr>
       <td><b>EngDesign</b></td>
       <td><code>CY_03, WJ_01, XY_05, AM_02, AM_03, YJ_02, YJ_03</code></td>

TASK_DETAILS_zh-CN.md

@@ -204,14 +204,18 @@ Frontier-Eng 目前已覆盖以下领域的任务。每个任务均配有可运
       <td>偏振复用全息</td>
     </tr>
     <tr>
-      <td rowspan="2"><b>ComputerSystems</b></td>
+      <td rowspan="3"><b>ComputerSystems</b></td>
       <td><code>MallocLab</code></td>
       <td>高性能 C 动态内存分配器（utilization &amp; throughput）</td>
     </tr>
     <tr>
       <td><code>DuckDBWorkloadOptimization</code></td>
       <td>基于 DuckDB 官方 workload 的索引 / 物化视图选择与查询改写</td>
     </tr>
+    <tr>
+      <td><code>EdgeServiceReplicaPlacement</code></td>
+      <td>面向流量突发、节点故障和链路退化的动态边缘服务副本放置与请求路由</td>
+    </tr>
     <tr>
       <td><b>EngDesign</b></td>
       <td><code>CY_03, WJ_01, XY_05, AM_02, AM_03, YJ_02, YJ_03</code></td>

benchmarks/ComputerSystems/EdgeServiceReplicaPlacement/README.md

@@ -0,0 +1,56 @@
+# EdgeServiceReplicaPlacement
+
+This is a CPU-only, deterministic benchmark for stateful edge-service replica
+placement and traffic routing. A candidate implements a policy, not a one-shot allocation:
+
+```python
+def decide(observation: dict) -> dict:
+    ...
+```
+
+The evaluator runs ten 24-period scenarios covering normal diurnal demand, regional
+bursts, node-failure-plus-burst, cross-region link degradation, and post-failure traffic
+migration. Scale-ups have a one-period cold start. The simulator independently computes
+availability, P95/P99 latency estimates, SLA violations, compute cost, cross-region
+traffic/cost, and recovery time.
+
+## Direct validation
+
+From this directory:
+
+```bash
+python verification/evaluator.py scripts/init.py \
+  --metrics-out metrics.json --artifacts-out artifacts.json
+python -m unittest discover -s verification -p "test_*.py" -v
+```
+
+## Unified zero-iteration validation
+
+From repository root:
+
+```bash
+python -m frontier_eval \
+  task=unified \
+  task.benchmark=ComputerSystems/EdgeServiceReplicaPlacement \
+  algorithm=openevolve \
+  algorithm.iterations=0
+```
+
+On Windows, set `PYTHONUTF8=1` and `PYTHONIOENCODING=utf-8` if the OpenEvolve dependency
+otherwise uses the system GBK codec. This is an environment workaround, not a benchmark
+requirement.
+
+## Editable boundary
+
+Only the EVOLVE-BLOCK in `scripts/init.py` is agent-editable. The worker receives JSON
+observations and returns JSON actions. It runs in a fresh temporary process per scenario,
+with bounded decision time and output size. This process boundary prevents ordinary
+shared-state and protocol coupling; it is not a substitute for an operating-system
+security sandbox.
+
+See [Task.md](Task.md) for the interface and [references/design_notes.md](references/design_notes.md)
+for model scope. Detailed review material is in
+[docs/parameter_assumptions.md](docs/parameter_assumptions.md),
+[docs/tiny_oracle.md](docs/tiny_oracle.md),
+[docs/evaluator-threat-model.md](docs/evaluator-threat-model.md), and
+[docs/scoring-calibration-report.md](docs/scoring-calibration-report.md).

benchmarks/ComputerSystems/EdgeServiceReplicaPlacement/README_zh-CN.md

@@ -0,0 +1,45 @@
+# EdgeServiceReplicaPlacement
+
+这是一个 CPU-only、确定性的 edge service 副本放置与流量路由 benchmark。候选实现
+动态策略，而不是一次性 allocation：
+
+```python
+def decide(observation: dict) -> dict:
+    ...
+```
+
+Evaluator 在 10 个、每个 24 时段的场景中运行策略，覆盖正常日内负载、区域突发、节点
+故障叠加突发、跨区链路降级和故障恢复后的流量迁移。扩容有一个时段 cold start。
+Simulator 独立计算 availability、P95/P99 估计、SLA 违规、计算成本、跨区流量/成本和
+恢复时长。
+
+## 直接验证
+
+在当前目录运行：
+
+```bash
+python verification/evaluator.py scripts/init.py \
+  --metrics-out metrics.json --artifacts-out artifacts.json
+python -m unittest discover -s verification -p "test_*.py" -v
+```
+
+## Unified 零迭代验证
+
+在仓库根目录运行：
+
+```bash
+python -m frontier_eval \
+  task=unified \
+  task.benchmark=ComputerSystems/EdgeServiceReplicaPlacement \
+  algorithm=openevolve \
+  algorithm.iterations=0
+```
+
+## 可编辑边界
+
+Agent 只能修改 `scripts/init.py` 中的 EVOLVE-BLOCK。Worker 接收 JSON observation 并返回
+JSON action；每个 scenario 使用新的临时进程，并限制决定时间和输出大小。该进程边界可
+避免一般的共享状态和协议耦合，但不能替代操作系统级安全沙箱。
+
+接口见 [Task_zh-CN.md](Task_zh-CN.md)，模型假设见
+[references/design_notes.md](references/design_notes.md)。

benchmarks/ComputerSystems/EdgeServiceReplicaPlacement/Task.md

@@ -0,0 +1,68 @@
+# Task: Dynamic edge service replica placement and routing
+
+Improve the policy inside the EVOLVE-BLOCK in `scripts/init.py`. At each five-minute
+period, choose desired service replicas and current request routes. The same policy is
+evaluated across hidden deterministic variants of five operating regimes. Future workload,
+failure, and link events are not visible.
+
+## Observation
+
+`decide(observation)` receives JSON-compatible data containing:
+
+- timestep and period duration;
+- nodes with region, failure domain, CPU capacity, and current alive status;
+- services with CPU/replica, service rate, base latency, P99 SLO, response size, and
+  reliability class;
+- current regional service demand and up to four historical periods;
+- active and pending replicas;
+- region RTTs and current cross-region bandwidth limits;
+- the prior action and coarse violation feedback.
+
+No future trace values or candidate-computed score components are supplied.
+
+## Action
+
+Return exactly:
+
+```json
+{
+  "replicas": [
+    {"service_id": "api", "node_id": "a-1", "count": 2}
+  ],
+  "routes": [
+    {"service_id": "api", "source_region": "edge-a", "node_id": "a-1", "fraction": 0.8}
+  ]
+}
+```
+
+`replicas` is the complete desired placement for the next state. New replicas are pending
+for one period. Removed replicas stop serving immediately. `routes` controls only the
+current period and may target active replicas that the same action retains. Fractions may
+sum to less than one; the remainder becomes unserved demand and receives a continuous
+performance penalty. Fractions above one are invalid.
+
+## Hard-invalid conditions
+
+- malformed schema, unknown or duplicate IDs;
+- booleans used as integers, negative/non-integer replica counts;
+- NaN/Infinity or route fractions outside `[0, 1]`;
+- placement above physical CPU capacity or on failed nodes;
+- routing to failed, pending, absent, or immediately removed replicas;
+- per-service/source route sum above one;
+- import/runtime error, timeout, or response above 64 KiB.
+
+Utilization, under-routing, oversubscription, SLA misses, cross-region traffic, temporary
+failure impact, slow recovery, and overprovisioning remain continuous performance effects.
+
+## Metrics and score
+
+The evaluator independently reports request availability, request-weighted P95/P99,
+P99-SLO violation rate, compute cost, cross-region GB/cost, and failure recovery steps.
+For valid policies, each scenario receives a bounded engineering loss. Reliability and SLA
+components carry 70% of the current weight; latency, compute, bandwidth, and recovery form
+the remainder. Tail latency is normalized continuously by the median configured service
+P99 SLO. The aggregate is 75% mean scenario score plus 25% 20th-percentile score.
+
+Raw metrics remain visible so engineering trade-offs are not hidden by the combined score.
+Normalization, caps, extreme-policy checks, and weight sensitivity are documented in
+`docs/scoring-calibration-report.md`.

benchmarks/ComputerSystems/EdgeServiceReplicaPlacement/Task_zh-CN.md

@@ -0,0 +1,63 @@
+# 任务：动态 Edge Service 副本放置与路由
+
+改进 `scripts/init.py` 的 EVOLVE-BLOCK。每个五分钟时段，策略需要决定目标服务副本和
+当前请求路由。同一个策略会在五类运行状态的隐藏确定性变体上评测，未来 workload、
+failure 和 link event 不可见。
+
+## Observation
+
+`decide(observation)` 接收 JSON-compatible 数据：
+
+- 当前 timestep 和时段长度；
+- 节点的 region、failure domain、CPU 容量和存活状态；
+- 服务的单副本 CPU、服务率、基础延迟、P99 SLO、响应大小和可靠性等级；
+- 当前各 region/service 的 demand，以及最多四个历史时段；
+- active 和 pending replicas；
+- region RTT 和当前跨区带宽上限；
+- 上一步动作与粗粒度违规反馈。
+
+Observation 不包含未来 trace，也不包含候选自行计算的得分分量。
+
+## Action
+
+必须准确返回：
+
+```json
+{
+  "replicas": [
+    {"service_id": "api", "node_id": "a-1", "count": 2}
+  ],
+  "routes": [
+    {"service_id": "api", "source_region": "edge-a", "node_id": "a-1", "fraction": 0.8}
+  ]
+}
+```
+
+`replicas` 是完整的 desired placement。新副本 pending 一个时段后才 active；被移除的副本
+立即停止服务。`routes` 只控制当前时段，且只能指向当前 active、同时被本动作保留的副本。
+Fraction 合计可以小于 1，剩余请求会成为未服务流量并得到连续惩罚；合计大于 1 为非法。
+
+## Hard-invalid 条件
+
+- schema 错误、未知或重复 ID；
+- bool 冒充 int、负数或非整数副本数；
+- NaN/Infinity 或 `[0, 1]` 外的 route fraction；
+- placement 超过物理 CPU，或放在故障节点；
+- 向故障、pending、不存在或被本动作移除的副本路由；
+- 每个 service/source 的 route sum 大于 1；
+- import/runtime error、超时或响应大于 64 KiB。
+
+高利用率、未完全路由、超载、SLA miss、临时故障影响、恢复慢和过度配置属于连续性能
+后果，不直接判 invalid。
+
+## Metrics 与评分
+
+Evaluator 独立报告 request availability、request-weighted P95/P99、P99 SLO violation、
+compute cost、cross-region GB/cost 和 failure recovery steps。
+
+有效策略的每个 scenario 先得到 bounded engineering loss。当前 reliability + SLA 权重为
+70%，其余由 latency、compute、bandwidth 和 recovery 构成；tail latency 按三个服务 P99
+SLO 的中位数连续归一化；跨 scenario 的聚合为 75% 平均分 + 25% P20 分位数。
+
+Raw metrics 始终可见，避免 combined score 隐藏工程权衡。Normalization、cap、极端策略
+和权重敏感性分析见 `docs/scoring-calibration-report.md`。

benchmarks/ComputerSystems/EdgeServiceReplicaPlacement/baseline/result_log.txt

@@ -0,0 +1,120 @@
+EdgeServiceReplicaPlacement benchmark calibration
+Date: 2026-09-07
+Environment:
+  direct/tests: Windows, Python 3.13.5, CPU-only
+  unified/wrapper: Git Bash 5.2.37 and Python 3.12.13 driver venv
+Scenarios: 10 total (five families, two deterministic variants each), 24 periods each
+
+Scoring correction in this validation round:
+  Before: tail latency used max(0, P99 / SLO - 1), so every below-SLO policy
+  received a zero tail-latency loss even though raw P99 values differed.
+  After: tail latency is normalized continuously against the median configured
+  service P99 SLO (180 ms). Compute and bandwidth caps were raised from 2 to 3
+  after an overprovision policy reached the cap and obtained free reliability gains.
+  Simulator behavior and raw metrics were not changed.
+
+Final calibration policies
+  weak.py
+    valid: true
+    combined_score: 74.0766
+
+  scripts/init.py (reasonable baseline)
+    valid: true
+    evaluated_scenarios: 10/10
+    combined_score: 73.9434
+    mean_request_availability: 0.9944750191
+    mean_request_weighted_p95_ms: 39.5209897096
+    mean_request_weighted_p99_ms: 47.4463666782
+    mean_p99_slo_violation_rate: 0.0294749761
+    mean_compute_cost: 4.36
+    mean_cross_region_gb: 1.1677365842
+    mean_failure_recovery_steps: 0.2
+
+  strong.py (calibration-only policy; not agent context)
+    valid: true
+    combined_score: 74.4628
+
+The labels are implementation descriptions, not an enforced score ordering.
+Final observed ordering: reasonable (73.9434) < weak (74.0766) < strong (74.4628).
+The small weak/reasonable reversal reflects a bandwidth-versus-reliability trade-off;
+all three remain valid and distinguishable.
+
+Extreme-policy check
+  sla-first overprovision: 73.1402
+  fixed full capacity: 70.2510
+  local routing only: 70.0369
+  ignore failures: 62.1667
+  zero replicas: 11.7809
+  cost-first underprovision: 11.1039
+  minimum replicas: 9.5307
+  aggressive cross-region routing: 7.6351
+  No obviously absurd policy exceeded the three calibration policies after the cap fix.
+
+Determinism check
+  Three complete reasonable-baseline runs produced byte-identical metrics JSON.
+  metrics SHA-256: E8A32CDF200775F4E038E022CA8F8CC04041D8C2300B5172545AC15F621C99B7
+  artifacts SHA-256: BBCF4D34CF15B345D8B39162ADAD4FDC00E8227EB5CA6820B2FF263E9F715B4A
+  Run wall times: 1.1619 s, 1.1417 s, 1.1981 s
+
+Adversarial/unit suite
+  command: python -m pytest benchmarks/ComputerSystems/EdgeServiceReplicaPlacement/verification -q
+  result: 36 passed in 11.44 s; exit 0; command wall time 12.56 s
+
+Multi-timestep tiny oracle
+  command: python benchmarks/ComputerSystems/EdgeServiceReplicaPlacement/verification/tiny_oracle.py
+  trajectories enumerated: 723
+  best raw metrics: availability=1.0, p95=32.0 ms, p99=42.0 ms,
+    SLO violation=0.0, compute cost=0.55, cross-region=0.0, recovery=0.0
+  normalized components: reliability=0.0, SLA=0.0, tail=0.2333333333,
+    compute=0.0916666667, bandwidth=0.0, recovery=0.0
+  combined_score: 96.62493678284117
+  independent oracle and production evaluator matched exactly
+  command wall time: 0.26 s; exit 0
+
+Direct evaluator
+  valid=true; 10/10 scenarios; score=73.9434
+  command wall time: 1.31 s; exit 0
+
+frontier_eval/run_eval.sh
+  valid=true; 10/10 scenarios; score=73.9434
+  command wall time: 1.68 s; exit 0
+
+Unified zero-iteration
+  valid=1; benchmark_returncode=0; score=73.9434
+  benchmark runtime_s=3.0958; complete command wall time=6.17 s; exit 0
+  Windows-specific invocation used Git Bash plus an explicit existing Python 3.12
+  executable. No machine-specific absolute path is stored in task metadata.
+
+Metadata/readonly audit
+  repository audit command exited 0 in standard mode. It printed warnings for existing
+  tasks elsewhere in the repository; EdgeServiceReplicaPlacement was not listed.
+  Strict mode additionally treats repository-wide optional recommendations as failures;
+  it is not used as the task pass criterion.
+
+Scoring sensitivity
+  Each component weight was varied independently by -5%, -2%, +2%, and +5%, with
+  the other weights proportionally renormalized. Strong remained top in every run.
+  Only the close fixed-full/local-only pair reversed under small reliability/compute
+  preference changes, consistent with their genuine cost/reliability trade-off.
+
+Agent optimization
+  command: python -m frontier_eval task=unified
+    task.benchmark=ComputerSystems/EdgeServiceReplicaPlacement
+    algorithm=openevolve algorithm.iterations=10 [Windows runtime overrides]
+  purpose: generative-optimization sanity check using an OpenAI-compatible LLM backend
+  parser success: 10/10 model responses
+  evaluator-valid candidates: 4/10
+  evaluator-invalid candidates: 6/10 due to candidate constraint violations
+  API timeouts: 0
+  initial score: 73.9434
+  handwritten strong calibration score: 74.4628
+  best evolved score: 74.6663
+  first and best improvement: iteration 6; remained best through iteration 10
+  best raw metrics: availability=0.995251, P99=47.237 ms,
+    SLO violation=0.027664, compute cost=4.342,
+    cross-region=1.027 GB, recovery=0.2 steps
+  The benchmark itself does not depend on the provider used for this local experiment.
+  This run is a sanity check, not evidence of general model performance.
+
+These values demonstrate a deterministic, executable benchmark and calibrated score.
+They do not claim production validity or comprehensive model performance.