UFACTORY xArm7 · LeRobot (GELLO / Manual Drag)

中文版本

UFACTORY xArm integration with the LeRobot framework, focused on two data-collection workflows:

• GELLO — joint-space teleoperation with a Dynamixel leader arm
• Manual drag — record demonstrations by freely moving the arm in xArm teach mode

Collected data is stored in the standard LeRobot dataset format and can be used for imitation learning (ACT / Diffusion Policy, etc.) and real-time policy inference.

Features

• 🤖 UFACTORY xArm7 control
• 🎮 GELLO joint-space teleoperation (Dynamixel leader arm)
• ✋ Manual drag recording via xArm teach mode
• 📷 Intel RealSense camera observation (D435 / D435i)
• 📊 LeRobot-compatible dataset recording & management
• 🧠 Imitation learning training and policy inference
• ▶️ Episode replay for recorded manual demonstrations

Requirements

• Ubuntu 22.04 / 24.04
• Python >= 3.10
• CUDA >= 12.0 (recommended for GPU training)
• UFACTORY xArm7 and its controller
• GELLO arm (FTDI USB serial)
• Intel RealSense D435 / D435i (for camera observations)

Installation

git clone https://git.weiyantech.cn/wangshuxun/Xarm-DataCollection.git lerobot_xarm7
cd lerobot_xarm7

uv venv --python 3.10
uv sync --extra gello

The base dependencies include lerobot==0.4.3 (with Intel RealSense support), xarm-python-sdk, numpy, pyyaml, and opencv-python. The gello extra adds the GELLO software and Dynamixel SDK.

Serial port permission

The GELLO arm connects over a serial port, so add your user to the dialout group (re-login afterwards):

sudo usermod -aG dialout $USER

Find the GELLO serial port path (used as teleop.port in the configs):

ls /dev/serial/by-id/

Configuration

Predefined configs are provided under config/:

Workflow	Config
GELLO · xArm7	config/gello/xarm7_gello_record_config.yaml
Manual drag · xArm7	config/manual_mode/xarm7_manual_record_config.yaml

GELLO config

• robot.robot_ip — xArm controller IP (e.g. 192.168.1.245)
• robot.robot_dof — 7
• robot.gripper_type — 1 for the xArm gripper
• teleop.port — GELLO serial port (/dev/serial/by-id/...)
• teleop.joint_ids / teleop.joint_signs — per-arm servo mapping and direction
• teleop.start_joints — GELLO calibration reference, should match the xArm SDK initial point (degrees)
• teleop.gripper_id — GELLO gripper servo ID (8; -1 disables it)
• teleop.realtime_control_fps — independent GELLO-to-xArm command loop rate; it is separate from dataset.fps
• dataset.root / dataset.repo_id — where the dataset is stored
• dataset.single_task — task description saved with each frame
• dataset.fps / episode_time_s / reset_time_s — recording timing

The xArm7 config already contains the correct joint mapping; only edit the port, IP, and dataset fields for your setup.

Manual-drag config

• robot.manual_mode: true — enable xArm teach mode (joint free-drive)
• robot.teach_sensitivity — teaching sensitivity, valid range 1–5
• robot.manual_gripper_speed — gripper velocity in normalized position per second (default 0.5)
• robot.observe_joint_vel — record joint velocities in observations (false by default)
• robot.enable_logs — enable optional per-cycle timing/diagnostic logs (false by default)
• robot.cameras.camera — Intel RealSense camera (serial_number_or_name, resolution, fps)
• dataset.root / dataset.repo_id / single_task / fps / episode_time_s / reset_time_s / num_episodes — dataset settings

Camera configuration

When adding a camera to the robot config, use the template in config/manual_mode/xarm7_manual_record_config.yaml:

robot:
  cameras:
    camera:
      type: intelrealsense        # RealSense type, NOT opencv
      serial_number_or_name: "148522072685"
      width: 640
      height: 480
      fps: 30

• type must be intelrealsense (RealSense), not opencv (generic USB camera).
• serial_number_or_name must be filled with the actual RealSense serial number obtained beforehand, otherwise connecting/recording fails. Get it with:

uv run uf-camera-view -l -T realsense     # prints each camera's serial number

or with the librealsense tool rs-enumerate-devices.

Usage

1. GELLO teleop test

Test the GELLO → robot control loop without recording:

uv run uf-robot-teleop --config_path config/gello/xarm7_gello_record_config.yaml
uv run uf-robot-teleop --config_path config/gello/xarm7_gello_record_config.yaml --fps 60  # optional loop rate

Space reset & start, ← reset, Esc exit.

Guard latency experiment

This command runs for 60 seconds with the min_tcp_z_mm guard enabled and records the loop period, GELLO read, safety guard, ServoJ, and complete send_action latency:

uv run uf-robot-teleop \
  --config_path config/gello/xarm7_gello_record_config.yaml \
  --robot.enable_logs=true \
  --fps 60 \
  --guard_latency_experiment=true \
  --experiment_duration_s 60

Press Space to reset and start. While staying safe, include motion both far from and near the configured height floor. With tcp_z_guard_backend: local_projection, the CSV guard_path column marks local_safe, local_projected, local_hold, or model_fault. The terminal prints per-path summaries. Results are written to logs/gello_guard_latency_<time>.csv.

Configure the GELLO TCP height floor

Stop other robot control processes, move the TCP to its lowest safe pose, then read its height:

uv run uf-read-tcp-z \
  --config-path config/gello/xarm7_gello_record_config.yaml \
  --margin-mm 5

The command does not move the arm. Put the hard floor in min_tcp_z_mm; the configured tcp_z_soft_margin_mm is added above it for CPU-local projection. The xArm7 GELLO joint path preserves all seven joint targets and projects only the component that would cross the soft TCP-height plane. Controller Safety Boundary remains enabled at the hard floor as a final stop.

This protects the TCP against crossing a horizontal plane. It does not detect collisions involving links, the elbow, or the gripper body, and it does not replace the emergency stop. Measure again after changing the tool, TCP offset, robot base, or table position.

2. GELLO data collection

# Record a new dataset
uv run record --config_path config/gello/xarm7_gello_record_config.yaml

# Continue recording on an existing dataset
uv run record --config_path config/gello/xarm7_gello_record_config.yaml -r

# Optional: save episodes in the background
uv run record --config_path config/gello/xarm7_gello_record_config.yaml -a

Controls: Space start the episode, → save it, ← discard and re-record it, Esc stop recording. The arm resets to its initial point between episodes.

The xArm7 GELLO example also enables a lightweight browser camera preview at http://127.0.0.1:8765/ (or http://<recorder-ip>:8765/ from another machine). It reuses frames already captured by the recorder: JPEG encoding and HTTP streaming run on background threads, the preview is capped at 8 FPS, and stale preview frames are dropped rather than delaying dataset collection. Disable it or tune it in the top-level web_preview section of the YAML configuration. When robot.enable_logs is enabled for diagnostics, one logs/gello_record_sync_*.csv file is written per episode. The preview_clients, record_period_ms, frame_overrun_ms, action_age_ms, and preview_publish_ms columns can be used to measure preview timing impact.

During collection the relative position between the robot arm and the camera must not change, and the camera setup at inference time must match the one used during collection. If the arm or camera moves, previously collected data becomes invalid.

If the dataset root already exists and -r is not given, the script asks whether to overwrite it, resume, or cancel.

3. Manual drag recording

./start_manual_record.sh
./start_manual_record.sh -r   # force resume; fails if the dataset directory does not exist

The record command reads dataset.root from the config and checks the path before recording, then:

• Directory does not exist → records a new dataset.
• Directory already exists (valid LeRobot dataset) and no -r was given → asks interactively:
  • o overwrite: delete the existing dataset and record a new one
  • r resume: keep existing episodes and continue recording
  • c cancel
• Directory exists but is incomplete (missing required metadata or data parquet files) → asks to overwrite it or cancel; non-interactive runs error out instead.

Pass -r to resume directly without asking. Note that ./start_manual_record.sh keeps its original launcher behavior — it automatically resumes an existing valid dataset (equivalent to -r), so run uv run record --config_path config/manual_mode/xarm7_manual_record_config.yaml directly if you want to see the overwrite/resume prompt.

During recording the arm is in teach mode: the actual joint state is written as both the observation and the action. Hold C to slowly close the gripper and O to slowly open it. Controls: Space start, → save, ← discard & re-record, Esc stop. Reset the arm manually between episodes.

IMPORTANT: At the start of every episode, wait until the arm has finished resetting and then wait another 5 seconds before operating it, or wait until the console prints Start Recording before operating it. This prevents reset commands from conflicting with operation commands and causing errors.

4. Policy training

uv run lerobot-train --policy act --dataset ufactory/xarm7_gello_datas

Example with explicit training parameters (checkpoints are saved every save_freq steps into output_dir):

uv run lerobot-train \
  --dataset.root=/home/<user>/lerobot_datas/record/ufactory/xarm7_gello_datas \
  --dataset.repo_id=ufactory/xarm7_gello_datas \
  --policy.type=act \
  --policy.device=cuda \
  --policy.repo_id=ufactory/xarm7_gello_datas \
  --output_dir=/home/<user>/lerobot_datas/train/xarm7_gello_datas \
  --job_name=xarm7_gello_datas \
  --steps=800000 \
  --batch_size=8 \
  --save_freq=20000

5. Policy inference

uv run uf-lerobot-eval \
  --config_path config/gello/xarm7_gello_record_config.yaml \
  --policy.path /path/to/train/output/checkpoints/last/pretrained_model/

← / → reset, Esc stop.

6. Replay recorded episodes

Replay the absolute joint states (observation.state) of a manual-drag episode on an xArm7. States are sent as absolute targets at the dataset FPS (default 30), so the motion matches the recording:

uv run replay \
  --dataset-root /path/to/xarm7_manual_datas \
  --robot-ip 192.168.1.245

# Skip the interactive confirmation (non-interactive use)
uv run replay --dataset-root /path/to/xarm7_manual_datas --robot-ip 192.168.1.245 --yes

# Replay another episode
uv run replay --dataset-root /path/to/xarm7_manual_datas --robot-ip 192.168.1.245 --episode-index 3

The robot first moves to the xArm SDK initial point, then replays the episode and stays at the last state. Make sure the workspace is clear and the recorded initial pose matches the current arm setup.

IMPORTANT: Robot Power-On and Power-Off

Power-on

1. Connect the computer to the robot controller with an Ethernet cable.
2. Configure the computer's Ethernet interface to the same IP subnet as the robot IP shown on the controller (for example, 192.168.1.xxx).
3. Open http://192.168.1.245:18333/ in a browser. The controller console should be displayed.
4. Release the emergency-stop button before operating the robot.

Power-off

1. Use the web console to return the arm to its initial position.
2. Press the emergency-stop button.
3. Turn off the controller power.

Tools

Camera viewer

uv run uf-camera-view -l                # list cameras
uv run uf-camera-view -T realsense      # view RealSense cameras

LeRobot dataset tools

# View episode 17
uv run lerobot-dataset-viz \
  --root=/path/to/record/ufactory/xarm7_manual_datas \
  --repo-id ufactory/xarm7_manual_datas \
  --display-compressed-images true \
  --episode-index 17

# Delete episodes 18 and 19
uv run lerobot-edit-dataset \
  --root=/path/to/record/ufactory/xarm7_manual_datas \
  --repo_id ufactory/xarm7_manual_datas \
  --new_repo_id ../xarm7_manual_datas_new \
  --operation.type delete_episodes \
  --operation.episode_indices "[18, 19]"

# Merge datasets
uv run lerobot-edit-dataset \
  --root=/path/to/record \
  --repo_id ufactory/xarm7_datas_merge \
  --operation.type merge \
  --operation.repo_ids "['ufactory/xarm7_datas_1', 'ufactory/xarm7_datas_2']"

Project structure

lerobot_xarm7/
├── config/
│   ├── gello/                     # xArm7 GELLO record config
│   └── manual_mode/               # xArm7 manual-drag record config
├── src/lerobot_robot_ufactory/
│   ├── robots/
│   │   └── uf_robot/              # xArm control (joint/cartesian, teach mode)
│   ├── teleoperators/
│   │   ├── base_teleop/           # shared teleop base class
│   │   └── gello_teleop/          # GELLO (Dynamixel leader arm)
│   ├── scripts/
│   │   ├── uf_robot_teleop.py     # teleop test loop
│   │   ├── uf_lerobot_record.py   # data collection (incl. manual mode)
│   │   ├── uf_lerobot_eval.py     # policy inference
│   │   ├── uf_lerobot_replay.py   # episode replay
│   │   └── uf_camera_view.py      # camera viewer
│   └── configs/parser.py          # config loading / CLI overrides
├── start_manual_record.sh         # manual-drag launcher
├── pyproject.toml
├── README.md
└── README_ZH.md

Important notes

• The provided configs are examples: edit IPs, serial ports, camera serials, dataset paths, and task descriptions to match your hardware.
• For GELLO data, keep the robot–camera relative pose identical between collection and inference.
• The LeRobot default parameters for diffusion policies are mostly designed for simulation and are not optimized for real robots — tune them for your task.
• Check the workspace for obstacles and keep the arm's initial pose consistent with the recorded data before replay or inference.

License

This project is released under the Apache License 2.0. See LICENSE.