AI for Industry Challenge: Autonomous Cable Insertion

Overview

The AI for Industry Challenge (AIC) is a robotics manipulation competition hosted by Intrinsic (Alphabet's robotics company). The task: train an AI model to autonomously plug fiber-optic cables into network devices — SFP slots on NIC cards and SC ports on optical patch panels — using a UR5e robot arm equipped with wrist cameras and a 6-axis force/torque sensor. The total prize pool across the top five finalists is $180,000.

I competed as a solo participant (team egeliler) against teams from academia and industry. The challenge ran from March to September 2026, with three phases: qualification in simulation, Phase 1 with access to Intrinsic's Flowstate IDE, and Phase 2 deploying on physical hardware at Intrinsic HQ.

The Challenge

Why It's Hard

Cable insertion is a deceptively difficult robotics problem:

• Cables are flexible and deform under contact forces in ways that are hard to model
• Plugs require sub-millimetre alignment to mate cleanly with their ports
• The sim-to-real gap is real — Phase 2 moves from Gazebo simulation to a physical workcell
• No ground-truth port pose is available at evaluation time — the policy must perceive the port from cameras alone

Hardware

The simulated workcell centers on a Universal Robots UR5e (6-DOF, ±0.03 mm repeatability) with a Robotiq Hand-E parallel-jaw gripper. An ATI AXIA80-M20 6-axis force/torque sensor sits at the wrist, and three Basler acA2440-20gc cameras (1152×1024 px @ 20 FPS) are mounted on the wrist in a left/center/right configuration.

Scoring

Each trial is bounded by a 30-second wall-clock budget. Three trials per submission, scored up to 100 points each across three tiers:

• Tier 1 (Model Validity): Does the model load and produce valid commands? Binary gate.
• Tier 2 (Performance): Trajectory smoothness, task duration, path efficiency, with force and contact penalties.
• Tier 3 (Task Success): Full insertion (+75), partial insertion (+38–50), proximity (+0–25), or wrong port (−12).

Maximum possible: 300 points across three trials.

My Approach

Architecture: Hybrid Learned + Classical

After initial experiments with pure behavior cloning (BC) hit a structural ceiling, I designed a hybrid architecture that plays to the strengths of both learned and classical methods:

1. Learned Approach (BC trunk): A Diffusion Policy Transformer with frozen DINOv2 vision encoding handles the visual approach — moving the gripper from anywhere in the workspace to within ~5 cm of the target port. Trained on 1,200 human demonstrations collected via a CheatCode reference policy.

2. Classical Insertion: A 5-phase state machine (APPROACH_REFINE → ALIGN → PROBE → INSERT → VERIFY) handles the last few millimetres using impedance control and an Archimedean spiral search. This is fully deterministic and auditable.

3. RL Residual Refinement: A ResiP-style residual MLP running at 100 Hz on top of the frozen BC trunk, producing small Cartesian corrections conditioned on live force/torque history. Fine-tuned via PPO (Diffusion Policy Policy Optimization) to close the sim-to-sim gap.

4. Safety Supervisor: A background thread enforcing an 18 N force cap, jerk limits, and singularity guards — 2 N below the controller's hard 20 N cap for clean abort margins.

All components live inside a single ROS 2 Lifecycle node — no IPC hops, no multi-process complexity.

Tech Stack

Data Pipeline

Instead of recording ROS bags (which would produce ~1.3 TB per 50 demos at 200 MB/s), I built a direct-to-LeRobot recorder (demo_recorder.py) that writes the HuggingFace dataset format at capture time — downsampled 288×256 images, AV1-encoded video, parquet state/action columns. Result: 274 MiB per 50 demos, a 4,700× reduction.

The final dataset reached 1,200 episodes collected via an autonomous loop (collect_demos_host.sh) that drove the CheatCode reference policy through perturbed trial configurations (±2 cm XY, ±5° yaw) to build robustness.

Submission Pipeline

The submission is a single Docker image built via a multi-stage Dockerfile:

• Stage A (build): ros:kilted-ros-core → install Pixi → copy source + pixi.toml → pixi install → pip install LeRobot + PyTorch → bake model weights
• Stage B (runtime): Strip build cache → copy .pixi/envs/default → entrypoint with RMW pin → CMD with policy class parameter

Pushed to AWS ECR under aic-team/egeliler:submission-<milestone>-<sha> and submitted via the AIC portal.

Engineering Outcomes

Before the honest failure analysis below, the concrete engineering this campaign produced — all solo, in 4–5 weeks:

• A complete challenge pipeline, built from scratch: ROS 2 Kilted + Zenoh workspace, Gazebo evaluation harness, multi-stage Docker builds, AWS ECR submission flow — the full path from source code to a scored portal run.
• A compact LeRobot data recorder that writes the HuggingFace dataset format directly at capture time — 274 MiB per 50 demos instead of ~1.3 TB of rosbags, a 4,700× reduction — and a 1,200-episode dataset collected with it.
• A validated hybrid control architecture: BC visual approach → classical impedance insertion → RL residual refinement → safety supervisor, all inside a single ROS 2 Lifecycle node, exercised end-to-end across 7+ smoke runs.
• A rock-solid classical insertion module: the 5-phase impedance + spiral-search state machine reliably completed insertion whenever the approach policy delivered the gripper in range.
• A measurable RL result: residual PPO refinement moved the score from −21 to +53 by correcting exactly the failure mode it was designed for.
• A precisely identified blocker: the runtime-vs-training tensorization mismatch in the BC inference path — sub-millimetre offline accuracy against ~17 cm of closed-loop drift — pinned down as the single diagnostic that gates all future work.

What I Learned

The BC Ceiling

Across 9 behavior cloning training runs — varying architecture (ACT, Diffusion Policy), dataset size (40 → 1,200 demos), conditioning (socket pose, task one-hot), loss functions (smoothness, trajectory consistency), and oversampling strategies — no BC variant ever achieved insertion. The submittable score plateaued at +3 (Tier-1 model validity only, no T2/T3 points).

Key finding: validation loss does not predict closed-loop smoke score. Offline action accuracy was sub-millimetre, but cumulative runtime drift reached ~17 cm in X. The BC inference path had a tensorization mismatch between training and runtime that was never fully diagnosed.

What Worked

• Reference baselines confirmed the pipeline was correct: ReplayPolicy (parquet replay) scored +190, CheatCode (ground-truth) scored +276. The dataset, engine, scoring, and insertion mechanism were all sound.
• The hybrid architecture itself was validated end-to-end across 7+ smoke runs — lifecycle, ACT inference, classical handoff, and safety watchdog all behaved correctly.
• RL residual refinement moved the score from −21 (M2 portal, force violations) to +53 (local paired smoke, G1-clean) — a meaningful improvement that showed the RL layer was addressing the right failure mode.
• Classical insertion (5-phase impedance + spiral) was rock-solid once the approach policy landed within range. The spiral search reliably found ports within the 8-second probe budget.

What Didn't Work

• BC alone cannot solve contact-rich insertion — this is a structural ceiling, not a hyperparameter issue. The missing primitive is contact-reactive correction, which BC (open-loop chunk execution) cannot provide.
• Sim-to-portal variance — the same Docker image scored +3 locally and −21 on the portal across different runs. No deterministic seed. The user-imposed "multi-seed local evidence ≥ 3 consecutive ≥ 70 before submit" bar was the right call.
• Cloud budget ran out — a promising vision spatial fix (Perceiver-style attention pooling on DINOv2 patches) was extracted but never trained because the RunPod credit hit zero mid-session. ~$17 total cloud burn.

Lessons for Next Time

1. Write the runtime-vs-train diff diagnostic first. If offline accuracy is sub-mm but runtime drifts 17 cm, the bug is in the inference tensorization — not the training. This should have been the first diagnostic, not the last.
2. RL refinement is the biggest single lever. Published results (ResiP: 5% → 99% on 0.2 mm peg-in-hole) suggest residual RL on top of any frozen BC backbone is the path to breaking the imitation ceiling.
3. Validation loss is not a task-performance metric for closed-loop diffusion BC. Trust paired smoke evaluations with statistical rigor (rliable IQM, bootstrap CIs).
4. Budget cloud spend carefully. $17 bought a lot of learning but ran out at the worst possible moment. Reserve 30% of budget for the final push.

Solo Dev Constraints

This project was executed entirely solo under tight constraints:

• 4-5 week runway to the qualification deadline
• 6 GB VRAM on a local RTX 4050 laptop (batch size capped at 4–8)
• ~$17 cloud budget (RunPod spot A100s)
• No team — architecture, implementation, training, DevOps, and submission all by one person
• Full-stack ownership: ROS 2 packages, PyTorch training loops, Docker multi-stage builds, AWS ECR, CI-like evaluation harness, reward function design, and PPO implementation

The project timeline compressed what a team would do in months into weeks. Every architectural decision (22 total, documented in ARCHITECTURE.md) was made with runway in mind — the kill-switch criterion for the ACT approach, the single-node constraint (G3), and the plain-Python state machine (G2) were all runway-driven choices.

Project Status

My campaign concluded with the qualification window. The qualification phase closed with a best submittable score of +3 (Tier-1 model validity); the architecture, dataset, pipeline, and findings above are the deliverables of that campaign. If the work resumes, the first step is already scoped: the diff_bc_train_vs_runtime.py diagnostic — everything else (DPPO, computer vision port detection, DAgger) presupposes the BC inference path is faithful.

The unconstrained strategy (with cloud GPU and unlimited time) would be a fundamentally different stack: HIL-SERL or ResiP-style residual RL on top of force-fused Diffusion Policy, targeting 90–100 points per trial.