⚠ All standard safety protocols apply during data collection. Spotter present, e-stop in hand, 3 m × 3 m clear area, 30-minute operator rotation for VR sessions.

Camera Setup

A minimum of two cameras is recommended for whole-body manipulation demonstrations:

  • Head-mounted egocentric camera — captures the arm workspace from the robot's perspective. Mount on the K1's head. Intel RealSense or USB webcam at 1280×720 @ 30 fps.
  • External fixed camera — captures the full body and task scene. Mount 2–3 m away on a tripod at approximately waist height. 1920×1080 @ 30 fps.
  • Wrist cameras (optional) — add left and right wrist cameras for fine manipulation tasks where end-effector visibility is critical.

Starting a Recording Session with k1_agent.py

python k1_agent.py \
  --robot-ip 192.168.10.102 \
  --platform-url https://fearless-backend-533466225971.us-central1.run.app \
  --record \
  --task "pick up the red block from the table" \
  --cameras head_cam,external \
  --episode-timeout 60 \
  --output-dir ./recordings/session_001/

The agent streams joint telemetry to the platform in real time and saves local copies. Use the keyboard shortcut (default: Space) to start and stop each episode.

Dataset Format

Each episode is saved as a directory:

recordings/session_001/episode_000001/
  joint_states.npy   # [T, 44] — positions and velocities for 22 joints
  imu.npy            # [T, 6]  — accel + gyro
  head_pose.npy      # [T, 2]  — yaw and pitch
  head_cam.mp4       # 1280x720 @ 30 fps
  external.mp4       # 1920x1080 @ 30 fps
  timestamps.npy     # [T] unix timestamps
  metadata.json      # {task, success, duration_s, operator}

Quality Filtering Protocol

After each session, review and filter episodes before adding to your training dataset:

  1. Watch each episode in the platform review panel (or with python review_episode.py --dir episode_000001).
  2. Mark episodes as success=true only if the task was completed end-to-end.
  3. Discard any episode where the robot showed instability, a near-fall, or the operator intervened.
  4. Verify camera streams are complete — no dropped frames or occlusions of the task area.
  5. Aim for 50 successful episodes before proceeding to Unit 5 training.

Session Safety Protocol

  • Maximum 60-second episode duration. Short episodes are easier to filter and lower risk.
  • Rotate operators every 30 minutes in VR sessions.
  • Check joint temperature every 20 episodes: python check_joint_temp.py --robot-ip 192.168.10.102. If any joint exceeds 60°C, pause for 10 minutes.
  • Log all incidents (falls, near-falls, aborted episodes) in your session notes.

Unit 4 Complete When...

You have 50 or more episodes labeled success=true in your dataset. Camera streams are complete for all 50. Joint state timestamps are continuous (no gaps). You have reviewed all episodes and discarded any that failed the quality checklist. Your dataset directory is backed up to at least one additional location.