Contribute to the bench

README.md docs/surgery/contribute/README.md, 10725 bytes (full)

Contribute to the surgery bench

You do not need any background in this repository to contribute. This page is the single entry point: it explains what the surgery bench is, what you can contribute, and walks you through your first contribution in about 30 minutes. Every contribution kind has its own page with the exact commands and the exact rules an automated gate checks.

What this is, in plain words

hecaton is a robotics repository. Inside it, the surgery bench is a large collection of simulated surgical training environments ("envs"): a virtual patient, surgical instruments, and a robot — plus a machine-checkable definition of what "doing the task right" means. Each env is grounded in real surgical practice: its steps, safety checks and success criteria must cite a real guideline, textbook or paper.

The bench is built from contributed pieces. You do not need to write simulation code to help — most contributions are structured text files (YAML) or small data/module files, validated by automated gates before a human ever looks at them.

flowchart LR
  A[You: clone the repo] --> B[surgbench new<br/>scaffolds your directory]
  B --> C[You fill it in<br/>with cited content]
  C --> D[surgbench check<br/>CPU gate + receipt]
  D -->|fail| C
  D -->|pass| E[surgbench submit<br/>writes the branch + PR plan]
  E --> F[you push + open the PR<br/>CI re-runs the check]
  F --> G[Agent first review<br/>+ cross-family reviewer]
  G --> H[Maintainer merges]
  H --> I[You are credited in<br/>CONTRIBUTORS.md + CITATION.cff]

What you can contribute — pick your kind

If none of these fit, open an issue on the forge (see below) describing what you have; a maintainer will route it.

Your first contribution in 30 minutes

This walkthrough adds one procedure — the fastest kind (one YAML file, no GPU, no simulation). The same three commands (new, check, submit) are used for every kind; only the files you fill in change.

Setup, minutes 0–5. Prerequisites: Python 3.10 and pip installed, network access, and an account on the forge — git.agent-next.com is this organization's own Git server and the clone asks for your forge username and password/token (it is not a public anonymous repo). Then:

git clone http://git.agent-next.com/agent-next/hecaton.git
cd hecaton
python3.10 -m venv .venv
.venv/bin/pip install -q PyYAML==6.0.2   # one wheel; reads/writes the YAML files

If git clone asks for credentials and fails, your account is not provisioned — ask a maintainer to add you on the forge first (account creation is maintainer-only; there is no self-service signup URL).

Scaffold, minutes 5–10. From the repository root:

.venv/bin/python scripts/surgery/surgbench new procedure lap_appendectomy

creates contributions/procedure/lap_appendectomy/ containing:

• procedure_template_drill.yaml — the contribution itself: a **valid
• NOTES.md — the three rules that matter for this kind;
• contribution.yaml — the kind marker the tooling reads (do not edit);
• receipts/ — where every check's evidence lands.

starter procedure (a two-arm suturing drill) whose REPLACE markers you edit with your real content; Fill it in, minutes 10–25. Open procedure_template_drill.yaml and replace the placeholders with real content: rename the row's id (lowercase letters, digits and underscores only — lap_appendectomy is a good id), fill the anatomy/instruments/arms fields, and write the phases and steps — every step needs a citation (a guideline URL starting http, or a source id with a known prefix: TB: textbook, GD: guideline, DS: dataset, CU: curriculum) and a measurable** success criterion ("needle exits within 2 mm of the mark" is measurable; "done carefully" is not). The field-by-field schema and a fully filled example: procedure.md (same directory as this file: docs/surgery/contribute/procedure.md). A surgeon typically writes this from a guideline they already know.

One note while the procedure library's merge lives outside this repo: the check shells out to the real proclib validator. If your clone does not carry one, point the check at a proclib checkout once — a maintainer gives you the path, or see the lane task board:

check for it with ls bench/proclib/validate_proclib.py — when it is not there:

export SURGBENCH_PROCLIB_DIR=/path/to/a/proclib-checkout   # dir containing validate_proclib.py

(Once bench/proclib lands in-repo this line disappears — the check finds the validator next to the repo copy.)

Check and submit, minutes 25–30.

.venv/bin/python scripts/surgery/surgbench check contributions/procedure/lap_appendectomy
.venv/bin/python scripts/surgery/surgbench submit contributions/procedure/lap_appendectomy

check prints each stage as PASS / FAIL / PENDING (CPU-only; the receipt JSON lands in receipts/). Fix anything FAILed and re-run. When only real PENDINGs remain (GPU or lane stages — the tool prints the exact command someone with that hardware runs; a procedure contribution has none), submit writes submit-plan.md next to your file: the exact branch/commit/push commands and the forge compare URL — it never pushes for you. Run those commands, open the pull request, and the review flow takes it from here (diagram above).

Where contribution files live

Everything a contributor adds lives under one root, contributions/, one directory per contribution so the tooling knows its kind:

contributions/
  procedure/<name>/   procedure_template_drill.yaml (a proclib part file: a YAML list of rows)
  physics/<name>/     template_backend.py (a dsofa Backend module; rename it)
  robot/<name>/       robots.yaml rows + assets/<robot>.urdf
  anatomy/<name>/     mesh.stl + mesh.reference.stl + meta.yaml
  baseline/<name>/    template_adapter.py (an env-gate baseline adapter)
  review/<name>/      review.yaml + the clip it rates

Each directory also gets NOTES.md, contribution.yaml and a receipts/ dir — the evidence bundle that travels to CI.

How credit works

Every contributed row can carry an attribution field — the name you want to be credited under, optionally with ORCID and affiliation:

attribution:
  name: 'Jane Surgeon'
  orcid: '0000-0002-1825-0097'      # optional
  affiliation: 'Mass General Hospital'  # optional

After your contribution merges, python scripts/surgery/contrib/gen_credit.py regenerates the repo's CONTRIBUTORS.md and appends you to CITATION.cff from the merged rows, so credit is derived from the data, never hand-maintained. Nothing to do on your side beyond filling the field.

What happens after you open the pull request

1. CI re-checks your contribution on the PR: the contrib workflow runs
2. An agent reviewer reads your diff first (correctness and the stated
3. A maintainer merges. Contributors never merge their own PRs.
4. If anything is unclear, the PR discussion is the place — every check that

surgbench check on each changed directory under contributions/ and posts the receipt summary as a PR comment. Stages that need a GPU or the lane's own tools do not run in this job — the receipt prints the exact command instead (see gpu-runner-proposal.md for the plan to run them on a labelled runner). requirements only), then a reviewer from a different model family double-checks it. failed comes with the exact rule and the offending value.

Glossary

anatomy.md docs/surgery/contribute/anatomy.md, 5589 bytes (full)

Contribute anatomy

Who this is for: medical imaging labs, anatomists, anyone with segmented CT/MRI data or atlas-derived organ meshes and the right to share them. You contribute the mesh plus a reference; automated gates verify it is watertight and numerically faithful before it can be used by any env.

Status: the gate constants and metric are the bench's own, merged on main (multiarm/surgery/patient.py, forge PR #50) — its first virtual patient passed them on all 10 organs. The check runs CPU-only with a stdlib STL parser; with the complete patient metric dependencies it uses the exact point-to-triangle metric instead (SURGBENCH_ANATOMY_LOADER=trimesh|stdlib forces one).

What you are adding, in plain words

Every surgical env needs a virtual patient: 3-D organ surfaces generated from real imaging. A mesh is useful only if it is *closed* (no holes — "watertight", otherwise collision and cutting do not work) and *accurate* (close to the true organ surface — measured as HD95, the 95th-percentile surface distance, in millimetres). The bench enforces both with numbers, not eyeballing: a contributed mesh must be watertight and within 2.0 mm HD95 of its reference.

flowchart TD
  A[Your contribution dir<br/>contributions/anatomy/&lt;name&gt;/] --> B{meta.yaml:<br/>organ + cited source present}
  B -->|missing| X[FAIL: field named]
  B -->|ok| C{mesh.stl watertight?<br/>every edge in exactly 2 triangles}
  C -->|no| X[FAIL: edge + Euler numbers reported]
  C -->|yes| D{reference mesh watertight<br/>AND HD95 <= 2.0 mm vs it}
  D -->|no| X[FAIL: measured distance reported]
  D -->|yes| E[organ registered for<br/>env authoring]

Step by step

1. Scaffold:

   .venv/bin/python scripts/surgery/surgbench new anatomy kidney_right_ct

creates contributions/anatomy/kidney_right_ct/ with:

• mesh.stl — a tiny generated tetrahedron (a valid stand-in so the
• mesh.reference.stl — the reference the gate measures against;
• meta.yaml — {organ, source, reference, units} with REPLACE

1. Replace both meshes with your data: mesh.stl is your organ surface

1. Fill in meta.yaml: organ (the structure name), source (where

1. Check and submit:

gate shows green before your data lands); markers. as a binary STL in millimetres (convert other formats first — the gate checks .stl); mesh.reference.stl is the ground truth it is measured against (the original segmentation surface or a published atlas mesh). A mesh without a reference cannot be scored and is not accepted. the mesh comes from, cited — atlas or CT source with URL), reference (the reference file name), units (mm). Patient-derived data must be de-identified and from a source that permits resharing.

   .venv/bin/python scripts/surgery/surgbench check contributions/anatomy/kidney_right_ct
   .venv/bin/python scripts/surgery/surgbench submit contributions/anatomy/kidney_right_ct

What the gate checks

These are the same constants the bench's own patient build enforces — its first virtual patient measured 0.358–0.498 mm HD95 across 10 organs, so the gate is achievable with careful segmentation. A missing reference is an error (the gate cannot run). Without the surface metric dependencies, different triangle surfaces fail instead of passing on a vertex proxy.

Broken example (rejected): a raw marching-cubes mesh straight out of a segmentation — full of holes — fails watertight with the edge and Euler numbers in the receipt; a heavily decimated mesh fails hausdorff95 with the measured distance printed.

Where the full-body path lives

If you are contributing a whole CT (not one organ), the existing CLI scripts/surgery/patient_from_ct.py --ct <nifti> --out <dir> runs TotalSegmentator, derives all organ surfaces, prints the per-organ metrics table and exits 1 when any organ misses a target — the same gates, end to end. That path needs the segmentation stack installed; the contribution gate itself does not.

After your anatomy merges

Your organ becomes a selectable anatomy variant in env authoring — the bench's variant axis "task × patient anatomy × arm count × port layout × difficulty" draws from contributed anatomy, and realism scores credit real anatomy explicitly (life-scale, real-patient-derived geometry is part of the ≥ 8/10 bar).

Key terms

watertight, HD95, env — see the glossary.

The stdlib fallback reports vertex distance only as a diagnostic. It passes HD95 only for identical triangle surfaces (exact distance zero, ignoring face order and winding). Other meshes fail with surface Hausdorff95 unverified; install the patient metric dependencies and rerun the check with SURGBENCH_ANATOMY_LOADER=trimesh. Shared vertices alone cannot pass.

baseline.md docs/surgery/contribute/baseline.md, 6359 bytes (full)

Contribute an agent baseline

Who this is for: AI and robot-learning labs, or anyone with a policy (VLA, LLM planner, ACT/scripted, RL) or a cited human-expert result that can run on the bench's registered envs. You contribute a baseline adapter — a small Python module the bench's env gate uses to roll your agent out; the scores come from the runs, never from hand-entered numbers.

Status: the adapter contract is the bench env gate's (branch bench-w1-20261106, env_gate.py); the local check is an AST lint — it never imports your module. The actual rollout over registered envs is a CI/NRP stage: surgbench check prints the exact env_gate.py command as PENDING and never runs it.

What you are adding, in plain words

A benchmark without baselines is a leaderboard nobody can read. The bench scores agent runs on registered envs and reports the agent / human-expert ratio per task family — "1.4× the published expert time" is a number other labs can compare against. Your adapter is how your agent becomes one of the reference points: the gate drives it with fixed seeds, hashes the resulting state twice (determinism), runs the env's oracle, and checks the oracle's known-fail control — the same green bar every registered env passes.

flowchart LR
  A[Your adapter:<br/>contributions/baseline/&lt;name&gt;/] --> B{AST lint:<br/>adapter contract,<br/>oracle honesty, lazy imports}
  B -->|rule miss| X[FAIL: rule + fix named]
  B -->|ok| C[env_gate.py rollout<br/>CI/NRP stage, exact command<br/>printed as PENDING]
  C --> D[run receipts: hashes,<br/>oracle verdict, Hz]
  D --> E[scorecard vs cited<br/>human-expert baselines]

The adapter contract, exactly

One module (template_adapter.py from the scaffold, renamed to your baseline) exposing:

• DETERMINISM_MODE — 'fresh_process' (two fresh subprocesses compare
• run(spec, steps, warmup, seed) -> dict — roll the baseline out on
• the returned dict carries at least the keys the gate's green predicate
• a success()/oracle that uses its argument, actually compares, and is
• backend/engine imports only inside run()/run_twice() — the child

hashes; you provide run(...)) or 'same_process' (you provide run_twice(...)); spec['env'] for steps steps after warmup; reads: reset_hash, steps_hash, hz, oracle_name, oracle_executed, success_on_rolled_out_state, known_fail_control, known_fail_control_passed; not constant-true — plus a known_fail_control() entry point that is False on a must-fail state; runs in the env's own venv, so nothing heavy may load at module level.

Step by step

1. Scaffold:

   .venv/bin/python scripts/surgery/surgbench new baseline my_vla

creates contributions/baseline/my_vla/template_adapter.py — a working stub adapter with the full contract present and CHANGE_ME markers where your policy plugs in.

1. Wire your agent into run() — load your policy lazily, step the env

1. Check and submit:

with its actions, hash the observed state, and let the env's oracle (or your own honest one) decide success from the rolled-out state.

   .venv/bin/python scripts/surgery/surgbench check contributions/baseline/my_vla
   .venv/bin/python scripts/surgery/surgbench submit contributions/baseline/my_vla

metrics-style lint stages run locally; the rollout stage is PENDING with its exact command (python scripts/surgery/bench/gate/env_gate.py --env <registry env id> --registry registry.yaml --receipts-dir receipts), run by the CI/NRP tier where the envs and their venvs live.

What the gate checks

The local AST lint (fail-closed, every rule names its fix):

Broken example (rejected): a success() that returns True without looking at its argument — oracle_not_always_true fails; a module-level import torch — lazy_backend_imports fails; a missing known_fail_control() — rejected, because an oracle that cannot fail is not a measurement.

How the human side of the ratio works

Human-expert numbers come only from the baselines library (human-baselines.yaml), where every measured entry quotes its source — e.g. the FLS proficiency benchmark's published expert task times. Entries without a citable measurement stay target_only and are never divided against. If you know published expert metrics for a procedure you work on, adding them there is a procedure-adjacent contribution — the format is the one on the procedure page.

Key terms

env, oracle, receipt, determinism — see the glossary.

CPU known-fail oracle check

The CPU gate interprets the pure success/oracle function and the known_fail_control/control function on a known-fail input. Both must return Boolean False. The default state is {'steps': -1}. For another state schema, declare a module-level literal KNOWN_FAIL_STATE. The control accepts zero arguments or the same state argument.

The supported pure subset is assignments, if, return, literals, state indexing and dictionary .get, comparisons, Boolean operators, numeric addition/subtraction, calls to pure local helpers, bool/int/float/len, isinstance(..., dict), all/any, and math.isfinite. A single generator may traverse at most 64 list/tuple items; evaluation allows 1024 AST visits and the state literal at most 128 AST nodes. Unsupported code, exceptions or exhaustion fail with a named diagnostic. The checker never imports or executes the submitted module. This bounded negative control does not prove an arbitrary oracle correct; the real rollout gate remains pending.

codeowners-proposal.md docs/surgery/contribute/codeowners-proposal.md, 2208 bytes (full)

Proposal: CODEOWNERS additions for the surgery-contribution area

Status: proposal — not applied. The live CODEOWNERS is unchanged; CODEOWNERS affects review routing (a permission-adjacent change), so it needs a lead decision. This file records the proposed diff in one read.

Proposed additions

Append to CODEOWNERS:

# Surgery contribution flow: docs, the surgbench tool, the credit generator
# and every contributed payload. Review routing per docs/OWNERSHIP.md.
docs/surgery/contribute/ @agent-next/maintainers
scripts/surgery/contrib/ @agent-next/maintainers
scripts/surgery/surgbench @agent-next/maintainers
multiarm/surgery/contrib/ @agent-next/maintainers
contributions/ @agent-next/maintainers

Rationale

• contributions/ is contributor-facing input: it is merged only after
• scripts/surgery/contrib/, scripts/surgery/surgbench and
• Agents are not forge identities (existing CODEOWNERS comment), so the

surgbench check + agent review + cross-family review + maintainer merge (the flow on README.md); CODEOWNERS makes the maintainer team the required reviewer for the tree that flow writes into. multiarm/surgery/contrib/ are the gate tooling itself — a bug here silently weakens every contribution check, so they keep maintainer ownership. per-kind agent lane stays documented in docs/OWNERSHIP.md rather than in this file; the proposal adds no agent handles.

Optional finer split (decide once there are two+ owners)

If the bench later splits human ownership per kind (e.g. a clinician owner for procedures, a sim owner for physics), the rows become:

contributions/procedure/ @agent-next/maintainers
contributions/physics/ @agent-next/maintainers
...

with real handles replacing the team when they exist on the forge. Not proposed now — there is exactly one owning team today.

Enforcement note

Per the existing CODEOWNERS header: server-side enforcement depends on the forge's branch protection being configured to use the file (unverified). The review flow itself does not depend on enforcement — surgbench submit + the PR checklist already route every contribution PR through the same reviews.

gpu-runner-proposal.md docs/surgery/contribute/gpu-runner-proposal.md, 2929 bytes (full)

Proposal: GPU-tier contribution checks on a labelled runner

Status: proposal — nothing below is applied. No CI permissions, runner labels or concurrency budgets are changed by the contrib-flow lane; this file records the design so the lead can approve or amend it in one read.

Problem

surgbench check is CPU-only by rule: GPU-dependent stages (the physics gate on sofa/i4h backends, env compiles, renders) print the exact command into the receipt instead of running, and the PR comment says "deferred". That keeps the PR gate cheap, deterministic and runnable on the existing one-shot CI runner — but it means a physics contribution's real gate never runs in CI today; the contributor runs it on their own machine (or it does not run at all).

Proposed design

Add a second, opt-in job to .github/workflows/contrib.yml:

  check-gpu:
    if: github.event_name == 'pull_request' &&
        contains(github.event.pull_request.labels.*.name, 'surg-gpu')
    runs-on: [self-hosted, agent-next-ci-gpu]   # new label, see below
    timeout-minutes: 30
    concurrency:
      group: contrib-gpu-${{ github.ref }}
      cancel-in-progress: true

• Label-gated (surg-gpu): a maintainer adds the label when a PR
• New runner label agent-next-ci-gpu: the existing shared 5090 host
• Scope: the job runs only the receipt's deferred commands verbatim —
• Permissions: identical to the CPU job (contents: read +

actually contains a physics/anatomy contribution whose deferred commands need a GPU. Nothing changes for procedure/robot/baseline/review PRs. already runs production work with priority (PersonaRobotBench); the label must point at a runner that coexists with it. The standing host rule applies unchanged inside the job: gate every GPU step with the yield script (gpu_yield.sh <MiB> [max_wait_s]; exit 3 = busy ⇒ the step retries later, never preempts), ≤ 6 GB per job, one GPU job per lane. the same commands the CPU job printed — and posts their receipts as a follow-up PR comment. No new checks are invented on the GPU tier. pull-requests: write for the comment). Label-add is a maintainer action, not a workflow permission.

What needs a lead decision

Cost of doing nothing

Physics contributions are gated only on the contributor's own machine; the receipt bundle in the PR is evidence but not CI-reproducible. That is acceptable for the current contribution volume and is the status quo this proposal would lift.

physics.md docs/surgery/contribute/physics.md, 7705 bytes (full)

Contribute a physics module

Who this is for: simulation engineers — soft-tissue FEM (SOFA, warp), rigid-body/game physics (Isaac/PhysX, MuJoCo), or your own solver. You write one Python module behind a small fixed interface; the bench lints it locally, gates the real simulation on a GPU host, and registers the envs it can produce.

Status: the interface below is multiarm/surgery/phys (the dsofa lane, branch dsofa-20261106); it lands in this repo unchanged. The local check is an AST lint — it never imports your module, so it runs on any CPU-only machine. The real gate (python -m multiarm.surgery.phys.gate) is a GPU stage: surgbench check prints its exact command as PENDING and never runs it.

What you are adding, in plain words

A physics backend simulates one surgical scene: tissue that deforms when grasped, instruments that grip, fluid that bleeds. The bench does not care which solver you use; it cares that your module obeys one contract:

• reset(seed) regenerates the scene and returns the first observation;
• step(cmd) advances exactly one timestep for one robot command;
• observe() returns the current state without advancing;
• close() releases everything (idempotent).

Commands and observations are fixed dataclasses (below), so any backend is interchangeable — that is what lets the bench compare physics engines on the same task and pick the most realistic one per tissue type.

sequenceDiagram
  participant G as gate / runner (GPU tier)
  participant B as your Backend
  participant R as run artifacts
  G->>B: reset(seed)
  B-->>G: Obs (tracked points, tip pose, gripped ids)
  loop each step
    G->>B: step(Cmd: psm_q + gripper)
    B-->>G: Obs
  end
  G->>R: dump trajectory + hashes
  G->>G: M1 smoke, M2 determinism (2 runs byte-exact), M3 success, controls must FAIL

The contract, exactly

Cmd (what you receive): time_s, psm_q — six joints in the fixed order yaw, pitch, insertion, roll, wrist_pitch, wrist_yaw (radians; insertion in metres) — and gripper in [0, 1].

Obs (what you return): time_s, tracked_pos — an (L, 3) float64 array of tracked tissue-point positions in metres (this is what determinism hashes), tip_pose — a 4×4 tool-tip pose, gripped_ids — indices of gripped tissue points, and a free-form info dict.

Backend constructor signature is frozen: Backend(task, out_dir=None, seed=0, control=None, device=None, overrides=None). Additional rules:

• step() after close() raises RuntimeError; close() may be called
• Engine imports live only inside methods — a module-level `import
• Register with a BACKENDS row in your module — a module-level
• Your module also carries the two honesty hooks the gate requires: an

twice. sofa/isaac/torch is rejected by the lint (the same rule the bench's own loader follows: importing a backend must never import an engine). literal mapping one backend name to your class (BACKENDS = {'my_tissue': '<module>:MyTissueBackend'}); the lint checks the row resolves to the class defined in your file. Never monkey-patch a loader. events() entry point deriving its judged events **from the task** (the realism judge scores one still per event), and a known_fail_control()` entry point returning False on a must-fail state.

Step by step

1. Scaffold:

   .venv/bin/python scripts/surgery/surgbench new physics my_tissue

creates contributions/physics/my_tissue/template_backend.py — a working deterministic CPU stub with every contract element present and CHANGE_ME markers where your engine goes. Rename the file (and the class) to your backend's name.

1. Replace the stub bodies with your engine calls (engine imports stay

1. Check locally, CPU-only:

inside methods). The stub's reset() derives state from seed — keep that property; the determinism gate hashes two runs with the same seed and requires byte-equal results.

   .venv/bin/python scripts/surgery/surgbench check contributions/physics/my_tissue

1. Submit — surgbench submit writes the plan; the GPU gate runs later

(by you, on a machine with your engine, or on the labelled CI runner — its exact command is in the receipt, e.g. PYTHONPATH=<phys root> python -m multiarm.surgery.phys.gate --backend sofa --task <task.yaml> --runs-root <dir> --receipt gate-sofa.json --python-sofa $SOFA_PY).

What the gate checks — and why an honest oracle matters

The local lint (AST only, fail-closed, every rule named in the output):

The oracle rules are the honesty check: an oracle that can never fail (one that returns True regardless of state) is rejected here, before any GPU time is spent. On the GPU tier the gate additionally runs M1 smoke (finite, under 600 s), M2 determinism (two fresh subprocesses, byte-exact hash_tracked/hash_tip) and the two must-fail control runs.

Broken example (rejected): a success() that only returns True — oracle_not_always_true fails with the rule named; a module-level import torch — lazy_engine_imports fails; reset() ignoring its seed — reset_uses_seed fails.

After your module merges

Your backend name joins the registry; task envs can request it, and the bake-off tables (same task, every backend, cross-RMSE + realism) get a new column. Cross-engine comparison (--cross receiptA receiptB) and the one-line-diff swap proof (--swap-proof) run automatically over your receipts.

Key terms

oracle, determinism, receipt, registry — see the glossary.

CPU known-fail oracle check

The CPU gate interprets the pure success/oracle function and the known_fail_control/control function on a known-fail input. Both must return Boolean False. The default state is {'tracked_flat': [0.0, 0.0, 0.0], 'retraction_max_mm': 0.0}. For another state schema, declare a module-level literal KNOWN_FAIL_STATE. The control accepts zero arguments or the same state argument.

The supported pure subset is assignments, if, return, literals, state indexing and dictionary .get, comparisons, Boolean operators, numeric addition/subtraction, calls to pure local helpers, bool/int/float/len, isinstance(..., dict), all/any, and math.isfinite. A single generator may traverse at most 64 list/tuple items; evaluation allows 1024 AST visits and the state literal at most 128 AST nodes. Unsupported code, exceptions or exhaustion fail with a named diagnostic. The checker never imports or executes the submitted module. This bounded negative control does not prove an arbitrary oracle correct; the real rollout gate remains pending.

procedure.md docs/surgery/contribute/procedure.md, 6525 bytes (full)

Contribute a procedure

Who this is for: surgeons, surgical residents, medical students, guideline authors — anyone who knows how an operation is actually performed and where the authoritative description is published. You need no programming and no simulation background; the contribution is one structured text (YAML) file.

Status: the file format and every rule below are the bench proclib's (validate_proclib.py, the proclib-1.0 schema). surgbench check runs that real validator as a subprocess — set SURGBENCH_PROCLIB_DIR to a proclib checkout if your clone does not carry one yet (see the walkthrough).

What you are adding, in plain words

The procedure library (proclib) is the bench's surgical ground truth: every simulated env is built on top of a procedure definition. A definition says, for one operation: which anatomy is involved, which instruments, how many robot arms and why more than one, what can go wrong (complications), what must be checked to stay safe, and the phases and steps of the operation itself — where every step carries a citation to a real guideline, textbook or paper and a measurable success criterion ("cystic duct clipped twice" is measurable; "done carefully" is not).

Your contribution is one part file — a YAML list of procedure rows — that merges into the library. It is validated by machine first, then read by a surgeon-reviewer, then used to author simulation envs.

flowchart TD
  A[Your part file<br/>contributions/procedure/&lt;name&gt;/] --> B{validate_proclib.py<br/>schema + citation rules}
  B -->|any error| X[FAIL with the exact<br/>rule and value]
  B -->|ok| C[local citation gate +<br/>id-collision check]
  C -->|uncited / duplicate id| X
  C -->|ok| D[merged into the proclib<br/>procedures.yaml]
  D --> E[env factory authors<br/>simulation envs from it]
  E --> F[registered env with your<br/>procedure as its ground truth]

Prerequisites

• The repo cloned and the light venv from
• One authoritative source you can cite — a society guideline page, a

README.md (5 minutes). surgical atlas, or a peer-reviewed paper, with a URL.

Step by step

1. Scaffold the directory:

   .venv/bin/python scripts/surgery/surgbench new procedure lap_appendectomy

creates contributions/procedure/lap_appendectomy/procedure_template_drill.yaml — a valid starter drill whose REPLACE markers you edit. The file is a YAML list of rows (or {"procedures": [...]}); you can add more rows to the same file.

1. Fill in the top-level fields of each row:

1. Write the steps. Each step needs all of:

1. Enough steps: a procedure needs at least 6 steps total, a drill

1. Check, iterate, submit:

at least 3.

   .venv/bin/python scripts/surgery/surgbench check contributions/procedure/lap_appendectomy
   .venv/bin/python scripts/surgery/surgbench submit contributions/procedure/lap_appendectomy

What the gate checks (and why it rejects)

surgbench check runs these stages (each shows as PASS/FAIL with the offending value; the receipt JSON lands in receipts/):

The design intent: a step without a citation is folklore, not ground truth, so it never enters the library.

A broken example (rejected):

- step: Dissect the triangle carefully
  actions: [dissect]
  citation: from memory
  success_criterion: done well

Rejected, each rule named: citation has no URL and no known source-id prefix (validator + local citation gate); success_criterion is too short to be measurable. Fix: cite the actual guideline page and state what is observable ("cystic artery identified and clipped before division — GD:sages ...").

After your procedure merges

Your procedure becomes an id other contributions reference: env authoring, the procedure-to-env map, and human-expert baselines all key off it. If you also know published expert metrics for it (task times, error criteria), the baseline page explains how those are added — they are what "agent vs human" numbers are divided by.

Key terms

proclib, oracle, receipt — see the glossary.

review.md docs/surgery/contribute/review.md, 4775 bytes (full)

Contribute a realism review

Who this is for: clinicians and trained raters — anyone who can look at rendered surgical footage and say, with reasons, where it stops looking like real surgery — and the review tooling that automates the first pass. A review submission is one review.yaml: what you rated, the two model families that judged it, and the verdicts it produced.

Status: surgbench check validates the review's structure and recounts its score locally; the actual two-family judging runs where the gateway credentials and rendered stills live — it prints the judge call as PENDING and never runs here.

What you are adding, in plain words

An env only counts toward the bench's quality goals at realism ≥ 8/10 (D-S16c). That score is never one person's gut feeling:

1. an automated two-family judge scores the stills — two reviewers that
2. a human frame read opens the actual frames and confirms or
3. the verdict plus the evidence it was made on go into the review record,

must come from *different* model families, so one model's blind spots do not pass unnoticed; overturns it — a reviewer who did not open the frames has not reviewed; in a format that can be recounted from the raw verdicts. Your review.yaml declares one such review and carries its verdicts.

flowchart TD
  A[Your submission<br/>contributions/review/&lt;name&gt;/] --> B{structure lint:<br/>env_id, clip present,<br/>2 families, events,<br/>verdicts recount}
  B -->|rule miss| X[FAIL: field named]
  B -->|ok| C[two-family judge:<br/>runs on the gateway lane,<br/>exact call printed PENDING]
  C --> D[verdict + evidence<br/>into the review record]
  D --> E[realism table updated;<br/>env counted only at >= 8/10]

Step by step

1. Scaffold:

   .venv/bin/python scripts/surgery/surgbench new review chole_clip_0142

creates contributions/review/chole_clip_0142/review.yaml.

1. Add the clip into the same directory (default clip.mp4) — the

1. Fill in review.yaml — the fields the check enforces:

review records the media it was made from, and the lint checks it exists.

1. Check and submit:

   .venv/bin/python scripts/surgery/surgbench check contributions/review/chole_clip_0142
   .venv/bin/python scripts/surgery/surgbench submit contributions/review/chole_clip_0142

The judge call (from multiarm.surgery.factory import judge; judge.judge_env(env_id, events, stills, runs_dir, clip=...)) is PENDING — it runs on the gateway lane where credentials and rendered stills exist, and its verdicts fill your verdicts list.

What the gate checks

• Structure — every rule in the table above; errors name the field
• Recountability — min_realism must equal the minimum of the family

(families must be exactly 2 DIFFERENT real model families, clip 'clip.mp4' not found, verdicts[1].family must name the judging family, ...). scores; each declared family must have exactly one verdict, and every verdict must cover each declared event id exactly once with visible: yes or visible: no (quote these strings in YAML). Declared event ids must be unique strings. A review whose summary disagrees with its own verdicts is rejected — the format is the audit trail. Broken example (rejected): families naming the same family twice, or one still carrying a placeholder string — two_families fails; min_realism: 9 with family scores of 8 and 7 — the recount fails with both numbers shown.

Why this role matters

Realism is the bench's quality axis: it decides which envs count, which physics backend wins a bake-off, and what "beyond human" claims can be made. All of that rests on reviews that named their evidence — the fields above exist so a verdict can always be traced back to the frames and the two families that produced it.

Key terms

realism score, receipt, env, registry — see the glossary.

robot.md docs/surgery/contribute/robot.md, 5286 bytes (full)

Contribute a robot

Who this is for: robotics engineers and robot companies. You know a surgical robot's architecture, degrees of freedom and public specifications, and can point to (or produce) a URDF description of it. No simulation experience needed — the contribution is one catalog row plus a URDF asset, both machine-checked.

Status: the row shape below follows the robot-catalog brief (proclib-v2 PART A2, ≥ 40 robots, every row sourced); the catalog lands in the repo as the bench tree merges. Your contribution is a part file under contributions/robot/ that merges into it.

What you are adding, in plain words

Every env in the bench needs to know which robot it uses: how many arms, what kind of kinematics, which instruments it holds, and where a simulation asset for it can be found. That is one row of the robot catalog. A robot that is only a press release cannot be simulated — so each field must carry its source (datasheet, paper, regulatory filing), and the URDF must pass a mechanical sanity check before the row is accepted.

flowchart TD
  A[Your contribution dir<br/>contributions/robot/&lt;name&gt;/] --> B{rows stage:<br/>schema + every source cited}
  B -->|schema / citation error| X[FAIL: row + field named]
  B -->|ok| C{URDF stages:<br/>referenced, present, parses,<br/>links + unique names,<br/>joint limits sane}
  C -->|missing / malformed| X
  C -->|ok| D[merged into the robot catalog;<br/>full FK check runs once the<br/>robot is wired into a task]

Step by step

1. Scaffold:

   .venv/bin/python scripts/surgery/surgbench new robot dvrk_xi

creates contributions/robot/dvrk_xi/robots.yaml (one starter row you fill in) and assets/template_psm.urdf (a minimal valid URDF you replace).

1. Fill in the row. robots.yaml is a YAML list of rows; the fields

the check enforces:

Rule: every quantitative claim carries a source. A spec you cannot cite gets dropped, not guessed.

1. Replace the URDF with the robot's real kinematic chain (keep it next

1. Check and submit:

to the yaml under assets/, license file alongside).

   .venv/bin/python scripts/surgery/surgbench check contributions/robot/dvrk_xi
   .venv/bin/python scripts/surgery/surgbench submit contributions/robot/dvrk_xi

What the gate checks

All stages are CPU-only and fail-closed, each naming the row and field:

The deeper kinematics check — joint values inside the asset's limits and the forward-kinematics tip reaching stated targets — runs where tasks are loaded (the same joint-limit + FK pattern every task file passes), once your robot is wired into a task.

Broken example (rejected): a source of "maker website" with no URL — citation fails with the row named; a URDF whose revolute joint has no <limit> element — the joint sanity rules reject it (a URDF without joint limits breaks every downstream check).

After your robot merges

The row joins the robot catalog; env authoring can reference your robot id, and multi-arm tasks (the bench's scored axis: 2–8 cooperating arms) can mix your arms with others. If you also want your robot driven in sim, the physics and baseline pages are the natural next contributions.

Key terms

URDF, RCM, registry, attribution — see the glossary.