A benchmark for surgical robots, built and measured by agents
This site reports a research benchmark that measures how well robot systems perform surgical tasks in simulation.
Software agents build the simulated tasks, run each one through an automatic quality gate, and record every result in a receipt file that ships with the site.
No number on this site is hand-entered: hover any value to see the file it was computed from.
Three pillars
Owner framing: we face the future — build the bench for where surgical robotics.
Robot AI
Multi-arm policies are a first-class bench object: VLA/LLM baselines (LLM planner over the arms, VLA, ACT/scripted) are evaluated on the whole bench with receipts, and 2–8 cooperating arms (handover, retract + dissect, clip-and-cut with 3–4 arms, 5–8-arm cells) are a scored axis, not an afterthought — target ≥ 15 such tasks.
High-quality surgery
≥ 100 grounded procedures — task, step order, safety checks and success criteria cited to the surgical references through the procedure library (proclib) — each counted only at realism ≥ 8/10 with real anatomy (CT/atlas at life scale) and cited/calibrated physics. Targets: ≥ 100 envs and ≥ 1,000 variants by the RSS package date — floors, not caps: more grounded envs at the realism bar is better (D-S20).
Agent-integrated
An agent env factory builds, gates and registers new environments with no human step; an agent reads the guideline, plans the steps, allocates the arms and drives the procedure — deterministic tools execute and per-step success checks are measured in simulation, so the agent never asserts its own success.
The bench in numbers
Where we stand: measured vs target
| Axis | Measured today | Target |
|---|---|---|
| Counted environments | 0 (counted bar: green env AND realism receipt glm_score >= 8) | PLAN >= 1,000 COUNTED envs by 2026-11-30 (paper number freeze); >= 10,000 for the next version (RSS final 2027-04-16 / bench v2) (/home/robot/workspace/agent-next/task-runs/20261104-surgery-multiarm/BENCH-PLAN-RSS-20261204.md) |
| Best judged realism score | 5/10 (i4h Slang base render, receipt) | PLAN an env counts only at realism >= 8/10 with real anatomy and cited/calibrated physics |
| Green environments | 53 | PLAN ≥ 100 high-quality surgical procedure/task envs, each past the env gate and citing proclib; ≥ 1,000 env variants (task × patient anatomy × arm count × port layout × difficulty). |
| Simulation throughput (i4h Slang renderer) | 48.8 (measured receipt R2: `artifacts/verify/i4h-slang-render-20261106.md`) | PLAN (not found - gap) no numeric throughput target is stated in the plan documents |
Data provenance
Generated at 2026-11-06T19:03:03Z from bench/registry.yaml (sha256 b3d63438982b) plus the procedure library, robot catalog, gallery receipts and repository documents; 149 receipt and media copies ship with the site under receipts/ and gallery/.
Start with the gallery (what the envs look like, with honest defect labels) or the environment registry (every env, its gate status and its receipt).