cablepilot

A 28,401-parameter world model of a cable robot, a sampling planner that imagines a few hundred futures with it every control step, and task cards: plain YAML files that say what a job is, which anyone (or their agent) can write, improve, score and share.

No demonstrations. No GPU. A new task is a card plus a few numbers, not a dataset and a training run.

Agents: read AGENTS.md. It is written for you: install, check the setup, score a card, contribute.

model	28k parameters, 124 KB, trained on task-free simulated motion in 12,000 random rooms, 11 min on a laptop CPU
one plan	128 futures x 2 s imagined in ~25 ms on a laptop CPU (5 plans per second leaves ~85 % of the budget free)
any room	the room enters only as its 4 calibrated mount points: a room it never saw is just 4 new numbers
clean the room (sim)	29/36 toys into the box over 3 episodes in a real bedroom imported from a Stringman recording, 0 furniture contacts, 62 s per toy
real robot	not yet run on hardware. The Stringman adapter is written, not tested: see "Status"

Why a small world model

The usual route to a robot that does chores is to record people driving it, then train a large policy on the recordings. On Stringman the public record of that route is 97 hours of teleoperation and 56 models (52 M to 4.1 B parameters), on rented H100s / H200s, and most of those models cannot decide fast enough on a laptop.

This repository takes the other route. It learns the machine (how the gantry, the lines and the swinging pole respond to commands), once, from simulation, without any task in it. Then each task is a cost on imagined futures (near the toy, low over it, not swinging, lines not too tight, clear of the bed) plus a few events on what the robot measures (the jaw stopped on something, the lines weigh 30 g more). The planner (MPPI, "the vortex": the purple futures in the videos) picks the commands whose futures cost least, five times a second.

So a new task costs a person an afternoon of thinking, not a week of teleoperation, and anyone's improvement to a task is a small text diff that the next person starts from.

Try it (3 minutes, no robot)

git clone https://huggingface.co/MacroSwan/cablepilot && cd cablepilot
pip install -e ".[sim]"                       # numpy, torch, pyyaml + mujoco for the simulator
cablepilot doctor                             # what this machine can do (threads, plan time)
cablepilot fetch clean_the_room               # the task folder from the hub: card + best tuning + results
cablepilot check pilot-tasks/tasks/clean_the_room/card.yaml
cablepilot run pilot-tasks/tasks/clean_the_room/card.yaml --world bedroom --video

The run writes runs/<...>/result.json (the score, with the hashes of what ran) and a film: the room with the plan in purple | the overhead camera with what it found | the gripper camera.

The three files of a task

Tasks live in the dataset MacroSwan/pilot-tasks, one folder each, shared by every pilot (cable robot, arm, drone, ...):

tasks/clean_the_room/
  card.yaml             WHAT: stages, skills, fallbacks. Body-free: the same file runs on an arm or a cable robot.
  tuning/cable.yaml     HOW MUCH, for this body: the cost weights and thresholds (every number lives here)
  results/*.json        what each run scored, where, with which card and tuning (by hash), by whom

A card reads like the job: observe the floor; approach the stray thing; grasp it; bring it over the box, still holding it; let go. If you lose sight of it, look again; if it slips, approach again. Each stage names a skill from a shared vocabulary (see, pre_grasp, grasp, pre_inside, release, avoid, ...). A card may only use skills the body has; cablepilot check lists the ones it is missing, and that list is the best place to contribute.

Cards carry their lineage (parent, by, why), so a task grows as a tree: clean the room -> clean the room, sorted -> put things back where they live; pen in cup -> pen in box. Two cards with the same signature() are the same task with different names.

Your setup

Everything that differs between two cable robots goes in a small rig.yaml (the agent fills it with you):

rig: living-room
robot: stringman          # stringman | sim
gripper: stringman        # picks the world model trained for that gripper
mounts: robot             # read the calibrated anchor points from the robot, or give (4, 3) metres
cameras: {overhead: ortho, gripper: true}
compute: {device: auto, threads: auto, plan_hz: 5}
places: {toy_box: hamper} # a card's site -> one of the robot's named places

cablepilot doctor --rig rig.yaml measures this machine against it and writes doctor.json beside it.

Vision. cablepilot does not bring its own heavy vision model. It uses the cameras your robot already has, in the cheapest way that works. The overhead view compares the floor with the clean floor and masks out the robot's own pixels (about 9 ms per look). The gripper camera then finds the target exactly as the hand gets close. Both feed one belief per object, which the task reads. On Stringman, the overhead view is the robot's own stitched floor image and the gripper view is its gripper camera.

Inside

	file
world model (JEPA latent + a hand-written cable/pendulum backbone it corrects)	cablepilot/model/learned.py, weights/stringman_g1.pt
the vortex (MPPI: correlated noise, temperature by effective sample size, warm start)	cablepilot/planner/mppi.py
task cards: schema, validation, signature, lineage	cablepilot/core/cards.py
the cable body's skills and how a card runs on them	cablepilot/cardrun.py, cablepilot/tasks/pick_place.py
perception: overhead differencing, gripper camera, one belief per object	cablepilot/perception/
MuJoCo twin of Stringman (4 lines, spools, pole, gripper), every number with its source	cablepilot/sim/, configs/stringman.yaml
the real robot (Neufangled's Python API, as a Maneuver)	cablepilot/stringman.py

State (14): gantry position and velocity, pole swing and its rate, wrist angle and rate, finger closure and rate. Command (5): gantry velocity (what move_direction_speed takes), wrist rate, closure target. Model step 0.1 s, 1 s of history.

Train it yourself

The model was trained on the public dataset MacroSwan/cablepilot-motion: 12,000 trajectories, about 100 hours of task-free simulated motion, each in its own random room. It can be regenerated in about 15 minutes on 4 cores (scripts/cdpr_datagen.py) and retrained on a laptop CPU without a GPU (scripts/train_cdpr_wm.py). scripts/import_room.py turns a Stringman LeRobot recording into a simulated room. AGENTS.md, Step 7, gives the commands and the bar a new model has to beat.

Status

• Simulation: proven. Model test on 500 held-out windows in 310 unseen rooms: grasp point 19.6 mm median error at 2 s, the same as MuJoCo with nominal physics (21.0 mm). clean_the_room in the bedroom imported from a Stringman recording: 29/36 over three episodes (this release, MuJoCo 3.15.0). Scores move by about one toy in twelve between library versions (MuJoCo 4.0 gives 28/36 with the same code), so compare on the reference simulator and several episodes.
• Real robot: not yet run. cablepilot/stringman.py maps the model's state and commands onto Neufangled's published Python API. It starts in dry_run mode, which plans and logs but sends nothing. The first real test is hover(). The places where it guesses at the API are marked ASSUME. This is the first thing to do together with a Stringman owner.
• Gripper: the released model is for Stringman's stock gripper. An SO-101 pinch model exists and is not released yet.

Limits

The twin's masses, line stretch, spool acceleration and swing damping are estimates until measured on a real robot. Furniture, the floor and the objects are not in the model (the task's costs handle them). Payloads up to 0.5 kg.

Credits and license

Apache-2.0. The two real rooms are imported from Neufangled's public Stringman recordings naavox/bedroom-sep6 and naavox/playroom-sep29-2 (Apache-2.0). Stringman is Neufangled's robot (neufangled.com); this project is independent of it. By Edgar Moreau (huggingface.co/Ethgar).