2026.03.22

Notes on GR00T N1.6 fine-tuning on Blackwell.

A working log on dataset shaping, training bottlenecks, and what changed once the hardware constraints eased.

ManipulationGR00TImitation Learning

GR00T N1.6 is NVIDIA's foundation model for humanoid manipulation. We fine-tuned it on 105 teleoperated episodes of pick-and-place for the Unitree G1's dexterous gripper. These are notes from the training runs, not a polished report.

Dataset: what we started with

105 episodes, ~45 seconds each, 30 Hz recording
Objects: foam blocks (4 colors), cylindrical cups, flat cards
Teleoperation via HTC Vive wrist trackers + G1 SDK
Roughly 30% of episodes had gripper calibration drift — filtered manually

Dataset structure (LeRobot format)

dataset/
  meta_data/
    stats.json         # normalization stats
    episode_index.json
  data/
    episode_000/
      observation.images.head.mp4
      observation.images.wrist_left.mp4
      observation.state.npy   # joint positions
      action.npy              # end-effector delta

Blackwell changed the training calculus. On our previous Ampere setup, a 50-episode dataset took ~6 hours per fine-tuning run. On Blackwell, the same run finishes in under 90 minutes — enough to iterate on reward shaping within a workday.

What moved the needle

Wrist camera: adding a left-wrist view cut pick failures by ~40% (the head camera has occlusion at close range)
Action chunking: chunk size 8 worked better than 16 for the small gripper workspace
Data augmentation: random brightness/contrast on images improved sim-to-real transfer noticeably
Filtering: removing the 30% of bad episodes improved convergence more than adding 20 more raw episodes

The model still fails on cards (flat objects at the gripper's limit of dexterity). That's a hardware constraint, not a model one — the G1 gripper wasn't designed for thin-object grasps. Ongoing.