wangshuxun/Xarm-DataCollection
1
0
Fork 1
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(project): 新增配置、udev 规则并完善文档结构

Browse Source 
添加 GELLO/Pika/UMI/SpaceMouse 遥操作配置，
新增 udev 设备规则，更新 README 项目结构，
优化键盘提示格式。
This commit is contained in:
Vinman 2026-06-16 13:13:21 +08:00
parent 6f520b5fef
commit e01fec8b91
27 changed files with 996 additions and 146 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
.gitignore vendored
View File

@ -83,4 +83,5 @@ models/
*.onnx
*.engine
*.trt
third_party/
*.xvcd
ufactory_usage/

236
LICENSE
View File

@ -1,27 +1,217 @@
Copyright (c) 2026, UFACTORY Inc.
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
All rights reserved.
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
1. Definitions.
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of the copyright holder nor the names of its contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2026 UFACTORY Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
--------------------------------------------------------------------
This project includes code derived from the following third-party projects:
- LeRobot (https://github.com/huggingface/lerobot)
Copyright 2024 The HuggingFace Inc. team
Licensed under the Apache License 2.0
- GELLO (https://github.com/wuphilipp/gello_software)
Copyright Philipp Wu
Licensed under the MIT License
- xArm-Python-SDK (https://github.com/xArm-Developer/xArm-Python-SDK)
Copyright UFACTORY Inc.
Licensed under the BSD 3-Clause License

198
README.md
View File

@ -1,54 +1,202 @@
# uFactory LeRobot
# UFACTORY LeRobot
uFactory 机械臂与 LeRobot 框架集成项目，用于机器人学习、数据采集和策略部署。
UFACTORY 机械臂与 LeRobot 框架集成项目，支持多种遥操作方式的数据采集、策略训练和部署推理。
## 功能特性
- 🤖 uFactory 机械臂控制接口
- 📷 多摄像头数据采集
- 🧠 模仿学习（行为克隆、ACT、扩散策略等）
- 🚀 策略部署与实时推理
- 📊 训练过程可视化和日志记录
- 🤖 UFACTORY 机械臂控制（xArm 系列）
- 🎮 多种遥操作方式：GELLO / Pika / UMI / SpaceMouse
- 📷 多摄像头数据采集（RealSense / UMI 相机）
- 📊 数据集录制与管理（兼容 LeRobot 格式）
- 🧠 模仿学习训练（ACT / Diffusion Policy 等）
- 🚀 策略评估与实时推理
- 🔧 Mock 机器人模拟（只用遥操作设备采集数据）
## 环境要求
- Ubuntu 22.04 / 24.04
- Python >= 3.10
- CUDA >= 12.0（GPU 训练推荐）
- uFactory 机械臂（xArm 系列）
- UFACTORY 机械臂（xArm 系列，可选）
## 安装
### 基础项目安装
```bash
# 克隆仓库
git clone <repo-url>
git clone https://github.com/xArm-Developer/ufactory_lerobot.git
cd ufactory_lerobot
# 创建虚拟环境
python -m venv .venv
source .venv/bin/activate # Linux/macOS
# .venv\Scripts\activate # Windows
# 创建 conda 环境
conda create -n uf_lerobot python=3.10 -y
conda activate uf_lerobot
# 安装依赖
pip install -e ".[dev]"
# 安装项目
pip install -e .
```
## 快速开始
包含：`lerobot==0.4.3`、`xarm-python-sdk`、`numpy`、`pyyaml`（lerobot 已自动携带 torch、opencv、wandb 等训练相关依赖）。
```python
# TODO: 添加示例代码
### 外设模块安装
外设依赖以可选模块形式提供，通过 `[模块名]` 安装。
#### GELLO 遥操作
适用于 GELLO 示教臂（Dynamixel 舵机方案），控制空间为关节空间。
```bash
# 1. 安装 GELLO 模块
pip install -e ".[gello]"
# 2. 添加串口权限（重新登录后生效）
sudo usermod -aG dialout $USER
```
#### Pika 遥操作
适用于 Pika Sense 手持示教器 + Vive Tracker，控制空间为笛卡尔空间。
```bash
# 1. 安装外设依赖（不需要它们的间接依赖）
pip install pysurvive agx-pypika --no-deps
# 2. 安装 udev 规则（重新插拔设备后生效）
sudo cp src/rules/*.rules /etc/udev/rules.d/
sudo udevadm control --reload-rules && sudo udevadm trigger
```
> Vive Tracker 首次使用前需校准：`uf-vive-calibrate`
#### UMI 遥操作
适用于 UMI（Universal Manipulation Interface）方案，含 Vive Tracker 追踪，支持双机械臂。
```bash
# 1. 安装 XVSDK（系统级依赖，仅支持 Ubuntu Focal）
sudo dpkg -i src/xvsdk/XVSDK_focal_amd64.deb
sudo apt install -y --fix-broken
# 2. 安装外设依赖
pip install pysurvive --no-deps
# 3. 安装 udev 规则（重新插拔设备后生效）
sudo cp src/rules/*.rules /etc/udev/rules.d/
sudo udevadm control --reload-rules && sudo udevadm trigger
```
> Vive Tracker 首次使用前需校准：`uf-vive-calibrate`
**多 UMI 设备配置**（使用两台及以上时）：
```bash
# 增加 USB 缓冲区大小
sudo sed -i '/GRUB_CMDLINE_LINUX_DEFAULT/s/quiet splash/quiet splash usbcore.usbfs_memory_mb=128/' /etc/default/grub
sync
sudo update-grub
sudo reboot
```
#### SpaceMouse 遥操作
适用于 3Dconnexion SpaceMouse / SpaceNavigator。
```bash
# 1. 安装 SpaceMouse 模块
pip install -e ".[spacemouse]"
# 2. 安装 udev 规则（重新插拔设备后生效）
sudo cp src/rules/*.rules /etc/udev/rules.d/
sudo udevadm control --reload-rules && sudo udevadm trigger
```
## 使用
### 1. 遥操作测试
测试遥操作设备与机械臂的联动，不录制数据。
```bash
uf-robot-teleop -c path/to/config.yaml
uf-robot-teleop -c path/to/config.yaml -f 60 # 指定频率
```
### 2. 数据采集
通过遥操作录制数据集。
```bash
uf-lerobot-record -c path/to/record_config.yaml
uf-lerobot-record -c path/to/config.yaml --resume # 续录
```
### 3. 策略训练
采集数据后，使用 LeRobot 训练管道进行模仿学习训练。
```bash
lerobot-train --policy act --dataset your_dataset_name
```
### 4. 策略评估
```bash
uf-lerobot-eval -c path/to/eval_config.yaml
```
### Mock 模式（无实体机械臂）
```yaml
type: "uf::mock_robot" # 单臂模拟
type: "uf::multiple_mock_robot" # 双臂模拟
```
## 遥操作方式对比
| 特性 | GELLO | Pika | UMI | SpaceMouse |
|------|-------|------|-----|------------|
| 控制空间 | 关节空间 | 任务空间 | 任务空间 | 任务空间 |
| 跟踪方式 | Dynamixel 舵机 | Vive Tracker | UMI SLAM / Vive | 3D 鼠标 |
| 双臂支持 | ❌ | ❌ | ✅ | ❌ |
| 系统依赖 | dialout 组 | — | XVSDK deb | — |
## 项目结构
```
ufactory_lerobot/
├── src/ # 源代码
├── configs/ # 配置文件
├── scripts/ # 脚本
├── tests/ # 测试
└── docs/ # 文档
├── src/
│ ├── ufactory_lerobot/
│ │ ├── robots/ # 机器人控制
│ │ │ ├── uf_robot/ # xArm 实体机器人
│ │ │ ├── uf_mock_robot/ # 仿真 Mock 机器人
│ │ ├── teleoperators/ # 遥操作器
│ │ │ ├── gello_teleop/ # GELLO (Dynamixel 示教臂)
│ │ │ ├── pika_teleop/ # Pika Sense (手持示教器 + Vive)
│ │ │ ├── umi_teleop/ # UMI (含双机械臂)
│ │ │ ├── space_mouse/ # SpaceMouse (3D 鼠标)
│ │ ├── cameras/ # 摄像头模块
│ │ │ └── umi_camera/ # UMI 相机
│ │ ├── devices/ # 外部设备驱动
│ │ │ ├── pika/ # Pika 串口驱动
│ │ │ └── umi/ # XVLib / Vive Tracker
│ │ ├── scripts/ # 执行脚本
│ │ │ ├── uf_robot_teleop.py # 遥操作测试
│ │ │ ├── uf_lerobot_record.py # 数据采集
│ │ │ ├── uf_lerobot_eval.py # 策略评估
│ │ │ └── vive_calibrate.py # Vive Tracker 校准
│ │ └── utils/ # 工具函数
│ ├── rules/ # udev 设备规则
│ └── xvsdk/ # XVSDK 系统依赖
├── config/ # YAML 配置文件
│ ├── gello/
│ ├── pika/
│ ├── umi/
│ └── spacemouse/
├── pyproject.toml
└── README.md
```
## 许可证
本项目基于 MIT 许可证发布。详见 [LICENSE](LICENSE) 文件。
本项目基于 Apache License 2.0 发布，详见 [LICENSE](LICENSE) 文件。

30
config/gello/xarm5_gello_record_config.yaml Normal file
View File

@ -0,0 +1,30 @@
RobotConfig:
_target_: ufactory_lerobot.robots.uf_robot.UFRobotConfig
id: "uf_robot"
robot_dof: 5
control_space: "joint"
robot_ip: "192.168.1.75"
gripper_type: 1
start_joints: [0, 0, -90, 90, 0]
DatasetRecordConfig:
_target_: lerobot.scripts.lerobot_record.DatasetRecordConfig
# root of local repo: /home/<user_name>/.cache/huggingface/lerobot (default)
root: "../lerobot_datas/record/ufactory/xarm5_gello_datas"
repo_id: "ufactory/xarm5_gello_datas"
single_task: "Pick up the purple grape and drop into the box on the left."
fps: 30
episode_time_s: 60 # max duration for one episode
reset_time_s: 20 # (invalid) time for resetting env between episodes
push_to_hub: False
# make sure to edit with your correct configurations!
TeleoperatorConfig:
_target_: ufactory_lerobot.teleoperators.gello_teleop.GelloTeleopConfig
port: "/dev/serial/by-id/usb-FTDI_USB__-__Serial_Converter_FTAJZYC7-if00-port0"
joint_ids: [1, 2, 4, 6, 7]
joint_signs: [1, 1, -1, 1, 1]
start_joints: [0, 0, -90, 90, 0]
gripper_id: 8
torque_joint_ids: [3, 5]

30
config/gello/xarm6_gello_record_config.yaml Normal file
View File

@ -0,0 +1,30 @@
RobotConfig:
_target_: ufactory_lerobot.robots.uf_robot.UFRobotConfig
id: "uf_robot"
robot_dof: 6
control_space: "joint"
robot_ip: "192.168.1.68"
gripper_type: 1
start_joints: [0, 0, -90, 0, 90, 0]
DatasetRecordConfig:
_target_: lerobot.scripts.lerobot_record.DatasetRecordConfig
# root of local repo: /home/<user_name>/.cache/huggingface/lerobot (default)
root: "../lerobot_datas/record/ufactory/xarm6_gello_datas"
repo_id: "ufactory/xarm6_gello_datas"
single_task: "Pick up the purple grape and drop into the box on the left."
fps: 30
episode_time_s: 60 # max duration for one episode
reset_time_s: 20 # (invalid) time for resetting env between episodes
push_to_hub: False
# make sure to edit with your correct configurations!
TeleoperatorConfig:
_target_: ufactory_lerobot.teleoperators.gello_teleop.GelloTeleopConfig
port: "/dev/serial/by-id/usb-FTDI_USB__-__Serial_Converter_FTAJZYC7-if00-port0"
joint_ids: [1, 2, 4, 5, 6, 7]
joint_signs: [1, 1, -1, 1, 1, 1]
start_joints: [0, 0, -90, 0, 90, 0]
gripper_id: 8
torque_joint_ids: [3]

27
config/gello/xarm7_gello_record_config.yaml Normal file
View File

@ -0,0 +1,27 @@
RobotConfig:
_target_: ufactory_lerobot.robots.uf_robot.UFRobotConfig
id: "uf_robot"
robot_dof: 7
control_space: "joint"
robot_ip: "192.168.1.85"
gripper_type: 1
start_joints: [0, 0, 0, 90, 0, 90, 0]
DatasetRecordConfig:
_target_: lerobot.scripts.lerobot_record.DatasetRecordConfig
# root of local repo: /home/<user_name>/.cache/huggingface/lerobot (default)
root: "../lerobot_datas/record/ufactory/xarm6_gello_datas"
repo_id: "ufactory/xarm6_gello_datas"
single_task: "Pick up the purple grape and drop into the box on the left."
fps: 30
episode_time_s: 60 # max duration for one episode
reset_time_s: 20 # (invalid) time for resetting env between episodes
push_to_hub: False
# make sure to edit with your correct configurations!
TeleoperatorConfig:
_target_: ufactory_lerobot.teleoperators.gello_teleop.GelloTeleopConfig
id: "gello_teleop"
port: "/dev/serial/by-id/usb-FTDI_USB__-__Serial_Converter_FTAJZYC7-if00-port0"
start_joints: [0, 0, 0, 90, 0, 90, 0]

34
config/pika/pika_record_config.yaml Normal file
View File

@ -0,0 +1,34 @@
RobotConfig:
_target_: ufactory_lerobot.robots.uf_mock_robot.UFMockRobotConfig
id: "uf_mock_robot"
control_space: "cartesian"
gripper_type: 2
state_offset_action: 3
cameras:
fisheye:
_target_: lerobot.cameras.opencv.configuration_opencv.OpenCVCameraConfig
index_or_path: "/dev/v4l/by-id/usb-DECXIN_CAMERA_DECXIN_CAMERA_01.00.00-video-index0"
width: 640
height: 480
fps: 30
fourcc: "MJPG"
DatasetRecordConfig:
_target_: lerobot.scripts.lerobot_record.DatasetRecordConfig
# root of local repo: /home/<user_name>/.cache/huggingface/lerobot (default)
root: "../lerobot_datas/record/ufactory/pika_datas"
repo_id: "ufactory/pika_datas"
single_task: "Pick up the purple grape and drop into the box on the left."
fps: 30
episode_time_s: 60 # max duration for one episode
reset_time_s: 20 # (invalid) time for resetting env between episodes
push_to_hub: False
# make sure to edit with your correct configurations!
TeleoperatorConfig:
_target_: ufactory_lerobot.teleoperators.pika_teleop.PikaTeleopConfig
# port: "/dev/ttyUSB0"
scale_xyz: 1.5
tracker_to_robot_eef: [0, 0, 0, 180, -90, 0]
robot_base_pose: [400, 0, 400, 180, 0, 0]

38
config/pika/xarm7_pika_record_config.yaml Normal file
View File

@ -0,0 +1,38 @@
RobotConfig:
_target_: ufactory_lerobot.robots.uf_robot.UFRobotConfig
id: "uf_robot"
robot_dof: 7
control_space: "cartesian"
robot_ip: "192.168.1.85"
gripper_type: 10
max_linear_velocity: 200
start_joints: [-0.2, -13.9, -0.3, 52.6, 0.9, 66.6, 0.2]
start_tcp_pose: [400, 0, 400, 180, 0, 0]
cameras:
fisheye:
_target_: lerobot.cameras.opencv.configuration_opencv.OpenCVCameraConfig
index_or_path: "/dev/v4l/by-id/usb-DECXIN_CAMERA_DECXIN_CAMERA_01.00.00-video-index0"
width: 640
height: 480
fps: 30
fourcc: "MJPG"
DatasetRecordConfig:
_target_: lerobot.scripts.lerobot_record.DatasetRecordConfig
# root of local repo: /home/<user_name>/.cache/huggingface/lerobot (default)
root: "../lerobot_datas/record/ufactory/xarm7_pika_datas"
repo_id: "ufactory/xarm7_pika_datas"
single_task: "Pick up the purple grape and drop into the box on the left."
fps: 30
episode_time_s: 60 # max duration for one episode
reset_time_s: 20 # (invalid) time for resetting env between episodes
push_to_hub: False
# make sure to edit with your correct configurations!
TeleoperatorConfig:
_target_: ufactory_lerobot.teleoperators.pika_teleop.PikaTeleopConfig
# port: "/dev/ttyUSB0"
scale_xyz: 1.5
tracker_to_robot_eef: [0, 0, 0, 180, -90, 0]
robot_base_pose: [400, 0, 400, 180, 0, 0]

38
config/spacemouse/xarm7_spacemouse_record_config.yaml Normal file
View File

@ -0,0 +1,38 @@
RobotConfig:
_target_: ufactory_lerobot.robots.uf_robot.UFRobotConfig
id: "uf_robot"
robot_dof: 7
control_space: "cartesian"
robot_ip: "192.168.1.127"
gripper_type: 0
cameras:
overhead:
_target_: lerobot.cameras.realsense.configuration_realsense.RealSenseCameraConfig
serial_number_or_name: "Intel RealSense D435I"
width: 640
height: 480
fps: 30
tool:
_target_: lerobot.cameras.realsense.configuration_realsense.RealSenseCameraConfig
serial_number_or_name: "Intel RealSense D435"
width: 640
height: 480
fps: 30
DatasetRecordConfig:
_target_: lerobot.scripts.lerobot_record.DatasetRecordConfig
# root: /home/<user_name>/.cache/huggingface/lerobot (default)
root: "../lerobot_datas/record/ufactory/xarm7_pushT"
repo_id: "ufactory/xarm7_pushT"
single_task: "Push the T-block into the specified zone then go to reset position."
fps: 10
episode_time_s: 60 # max duration for one episode
reset_time_s: 20 # (invalid) time for resetting env between episodes
push_to_hub: False
# space mouse
TeleoperatorConfig:
_target_: ufactory_lerobot.teleoperators.space_mouse.SpaceMouseTeleopConfig
max_value: 300
use_gripper: False

68
config/umi/dual_umi_record_config.yaml Normal file
View File

@ -0,0 +1,68 @@
RobotConfig:
_target_: ufactory_lerobot.robots.uf_mock_robot.MultipleUFMockRobotConfig
id: "multiple_uf_mock_robots"
robots:
left:
_target_: ufactory_lerobot.robots.uf_mock_robot.UFMockRobotConfig
id: "left_uf_mock_robot"
control_space: "cartesian"
gripper_type: 1
state_offset_action: 3
cameras:
fisheye:
_target_: ufactory_lerobot.cameras.umi_camera.configuration_umi.UmiCameraConfig
serial_number: "250801DR48FP26001318"
width: 1280
height: 1280
fps: 30
right:
_target_: ufactory_lerobot.robots.uf_mock_robot.UFMockRobotConfig
id: "right_uf_mock_robot"
control_space: "cartesian"
gripper_type: 1
state_offset_action: 3
cameras:
fisheye:
_target_: ufactory_lerobot.cameras.umi_camera.configuration_umi.UmiCameraConfig
serial_number: "250801DR48FP26001295"
width: 1280
height: 1280
fps: 30
# make sure to edit with your correct configurations!
TeleoperatorConfig:
_target_: ufactory_lerobot.teleoperators.umi_teleop.MultipleUmiTeleopConfig
id: "multiple_teleops"
teleops:
left:
_target_: ufactory_lerobot.teleoperators.umi_teleop.UmiTeleopConfig
id: "left_teleop"
serial_number: "250801DR48FP26001318"
use_vive_tracker: False
vive_tracker_id: "LHR-555DC7BF"
use_gripper: True
tracker_to_robot_eef: [0, 0, 0, 0, 0, -90]
robot_base_pose: [400, 0, 150, -90, 0, 0]
right:
_target_: ufactory_lerobot.teleoperators.umi_teleop.UmiTeleopConfig
id: "right_teleop"
serial_number: "250801DR48FP26001295"
use_vive_tracker: False
vive_tracker_id: "LHR-2425BAD3"
use_gripper: True
tracker_to_robot_eef: [0, 0, 0, 0, 0, -90]
robot_base_pose: [400, 0, 150, 90, 0, 0]
DatasetRecordConfig:
_target_: lerobot.scripts.lerobot_record.DatasetRecordConfig
# root of local repo: /home/<user_name>/.cache/huggingface/lerobot (default)
root: "../lerobot_datas/record/ufactory/dual_xarm6_umi_datas"
repo_id: "ufactory/dual_xarm6_umi_datas"
single_task: "Pick up the purple grape and drop into the box on the left."
fps: 60
episode_time_s: 60 # max duration for one episode
reset_time_s: 20 # (invalid) time for resetting env between episodes
num_episodes: 100
push_to_hub: False

81
config/umi/dual_xarm6_umi_record_config.yaml Normal file
View File

@ -0,0 +1,81 @@
RobotConfig:
_target_: ufactory_lerobot.robots.uf_robot.MultipleUFRobotConfig
id: "multiple_uf_robots"
async_connect: True
async_configure: True
async_action: False
robots:
left:
_target_: ufactory_lerobot.robots.uf_robot.UFRobotConfig
id: "left_uf_robot"
robot_dof: 6
control_space: "cartesian"
robot_ip: "192.168.1.29"
gripper_type: 2
max_linear_velocity: 200
start_joints: [-14.7, 25.6, -49.6, -282.5, 95.9, -65.9]
start_tcp_pose: [400, 0, 150, -90, 0, 0]
cameras:
fisheye:
_target_: lerobot.cameras.opencv.configuration_opencv.OpenCVCameraConfig
index_or_path: "/dev/v4l/by-id/usb-XVisio_Technology_XVisio_vSLAM_250801DR48FB26001379-video-index0"
width: 1280
height: 1280
fps: 100
fourcc: "YUYV"
right:
_target_: ufactory_lerobot.robots.uf_robot.UFRobotConfig
id: "right_uf_robot"
robot_dof: 6
control_space: "cartesian"
robot_ip: "192.168.1.83"
gripper_type: 2
max_linear_velocity: 200
start_joints: [14.5, 25.5, -49.6, -76.8, 95.9, 66.6]
start_tcp_pose: [400, 0, 150, 90, 0, 0]
cameras:
fisheye:
_target_: lerobot.cameras.opencv.configuration_opencv.OpenCVCameraConfig
index_or_path: "/dev/v4l/by-id/usb-XVisio_Technology_XVisio_vSLAM_250801DR48FB26001396-video-index0"
width: 1280
height: 1280
fps: 100
fourcc: "YUYV"
# make sure to edit with your correct configurations!
TeleoperatorConfig:
_target_: ufactory_lerobot.teleoperators.umi_teleop.MultipleUmiTeleopConfig
id: "multiple_teleops"
teleops:
left:
_target_: ufactory_lerobot.teleoperators.umi_teleop.UmiTeleopConfig
id: "left_teleop"
serial_number: "250801DR48FP26001318"
use_vive_tracker: False
vive_tracker_id: "LHR-555DC7BF"
use_gripper: True
tracker_to_robot_eef: [0, 0, 0, 0, 0, -90]
robot_base_pose: [400, 0, 150, -90, 0, 0]
right:
_target_: ufactory_lerobot.teleoperators.umi_teleop.UmiTeleopConfig
id: "right_teleop"
serial_number: "250801DR48FP26001295"
use_vive_tracker: False
vive_tracker_id: "LHR-2425BAD3"
use_gripper: True
tracker_to_robot_eef: [0, 0, 0, 0, 0, -90]
robot_base_pose: [400, 0, 150, 90, 0, 0]
DatasetRecordConfig:
_target_: lerobot.scripts.lerobot_record.DatasetRecordConfig
# root of local repo: /home/<user_name>/.cache/huggingface/lerobot (default)
root: "../lerobot_datas/record/ufactory/dual_xarm6_umi_datas"
repo_id: "ufactory/dual_xarm6_umi_datas"
single_task: "Pick up the purple grape and drop into the box on the left."
fps: 60
episode_time_s: 60 # max duration for one episode
reset_time_s: 20 # (invalid) time for resetting env between episodes
num_episodes: 100
push_to_hub: False

34
config/umi/umi_record_config.yaml Normal file
View File

@ -0,0 +1,34 @@
RobotConfig:
_target_: ufactory_lerobot.robots.uf_mock_robot.UFMockRobotConfig
id: "uf_mock_robot"
control_space: "cartesian"
gripper_type: 1
state_offset_action: 3
cameras:
fisheye:
_target_: ufactory_lerobot.cameras.umi_camera.configuration_umi.UmiCameraConfig
serial_number: "250801DR48FP26001295"
width: 1280
height: 1280
fps: 100
TeleoperatorConfig:
_target_: ufactory_lerobot.teleoperators.umi_teleop.UmiTeleopConfig
serial_number: "250801DR48FP26001295"
use_vive_tracker: False
vive_tracker_id: "LHR-2425BAD3"
use_gripper: True
tracker_to_robot_eef: [0, 0, 0, 0, 0, -90]
robot_base_pose: [400, 0, 400, 180, 0, 0]
DatasetRecordConfig:
_target_: lerobot.scripts.lerobot_record.DatasetRecordConfig
# root of local repo: /home/<user_name>/.cache/huggingface/lerobot (default)
root: "../lerobot_datas/record/ufactory/umi_datas"
repo_id: "ufactory/umi_datas"
single_task: "Pick up the purple grape and drop into the box on the left."
fps: 60
episode_time_s: 60 # max duration for one episode
reset_time_s: 20 # (invalid) time for resetting env between episodes
num_episodes: 100
push_to_hub: False

39
config/umi/xarm6_umi_record_config.yaml Normal file
View File

@ -0,0 +1,39 @@
RobotConfig:
_target_: ufactory_lerobot.robots.uf_robot.UFRobotConfig
id: "uf_robot"
robot_dof: 6
control_space: "cartesian"
robot_ip: "192.168.1.83"
gripper_type: 2
max_linear_velocity: 250
start_joints: [0, -14.8, -52.6, 0, 67.4, 0]
start_tcp_pose: [400, 0, 400, 180, 0, 0]
cameras:
fisheye:
_target_: lerobot.cameras.opencv.configuration_opencv.OpenCVCameraConfig
index_or_path: "/dev/v4l/by-id/usb-XVisio_Technology_XVisio_vSLAM_250801DR48FB26001396-video-index0"
width: 1280
height: 1280
fps: 100
fourcc: "YUYV"
TeleoperatorConfig:
_target_: ufactory_lerobot.teleoperators.umi_teleop.UmiTeleopConfig
serial_number: "250801DR48FP26001295"
use_vive_tracker: False
vive_tracker_id: "LHR-2425BAD3"
use_gripper: True
tracker_to_robot_eef: [0, 0, 0, 0, 0, -90]
robot_base_pose: [400, 0, 400, 180, 0, 0]
DatasetRecordConfig:
_target_: lerobot.scripts.lerobot_record.DatasetRecordConfig
# root of local repo: /home/<user_name>/.cache/huggingface/lerobot (default)
root: "../lerobot_datas/record/ufactory/xarm6_umi_datas"
repo_id: "ufactory/xarm6_umi_datas"
single_task: "Pick up the purple grape and drop into the box on the left."
fps: 60
episode_time_s: 60 # max duration for one episode
reset_time_s: 20 # (invalid) time for resetting env between episodes
num_episodes: 100
push_to_hub: False

46
pyproject.toml
View File

@ -7,7 +7,7 @@ name = "ufactory_lerobot"
version = "0.1.0"
description = "UFACTORY 机械臂与 LeRobot 框架集成"
readme = "README.md"
license = { text = "MIT" }
license = { text = "Apache-2.0" }
requires-python = ">=3.10"
authors = [
{ name = "ufactory_lerobot Contributors" },
@ -23,8 +23,31 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: Scientific/Engineering :: Artificial Intelligence",
]
dependencies = [
"numpy>=1.24",
"pyyaml",
"lerobot==0.4.3",
"xarm-python-sdk",
"opencv-python",
]
[project.scripts]
uf-robot-teleop = "ufactory_lerobot.scripts.uf_robot_teleop:main"
uf-lerobot-record = "ufactory_lerobot.scripts.uf_lerobot_record:main"
uf-lerobot-eval = "ufactory_lerobot.scripts.uf_lerobot_eval:main"
uf-vive-calibrate = "ufactory_lerobot.scripts.vive_calibrate:main"
[project.optional-dependencies]
# GELLO 遥操作
gello = [
"gello @ git+https://github.com/xArm-Developer/gello_software.git",
"dynamixel-sdk @ git+https://github.com/ROBOTIS-GIT/DynamixelSDK.git#subdirectory=python",
]
# SpaceMouse 遥操作
spacemouse = [
"spnav @ git+https://github.com/kazoo-osaro/spnav",
]
# 开发工具
dev = [
"pytest>=8.0",
"pytest-cov>=5.0",
@ -32,22 +55,15 @@ dev = [
"mypy>=1.10",
"pre-commit>=3.7",
]
camera = [
"opencv-python>=4.9",
"pyrealsense2>=2.55",
]
lerobot = [
"lerobot==0.4.3",
]
train = [
"torch>=2.2",
"torchvision>=0.17",
"huggingface-hub>=0.25",
"wandb>=0.17",
]
[tool.setuptools]
include-package-data = true
[tool.setuptools.packages.find]
where = ["."]
where = ["src"]
[tool.setuptools.package-data]
"ufactory_lerobot.devices.umi.xvlib" = ["*.so", "*.so.*"]
[tool.ruff]
line-length = 100

41
src/rules/81-vive.rules Normal file
View File

@ -0,0 +1,41 @@
# HTC Vive HID Sensor naming and permissioning
KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="0bb4", ATTRS{idProduct}=="2c87", TAG+="uaccess"
KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="0bb4", ATTRS{idProduct}=="030e", TAG+="uaccess"
KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="28de", ATTRS{idProduct}=="2101", TAG+="uaccess"
KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="28de", ATTRS{idProduct}=="2000", TAG+="uaccess"
KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="28de", ATTRS{idProduct}=="1043", TAG+="uaccess"
KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="28de", ATTRS{idProduct}=="2050", TAG+="uaccess"
KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="28de", ATTRS{idProduct}=="2011", TAG+="uaccess"
KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="28de", ATTRS{idProduct}=="2012", TAG+="uaccess"
KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="28de", ATTRS{idProduct}=="2022", TAG+="uaccess"
KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="28de", ATTRS{idProduct}=="2300", TAG+="uaccess"
SUBSYSTEM=="usb", ATTRS{idVendor}=="0bb4", ATTRS{idProduct}=="2c87", TAG+="uaccess"
SUBSYSTEM=="usb", ATTRS{idVendor}=="0bb4", ATTRS{idProduct}=="030e", TAG+="uaccess"
# HTC Camera USB Node
SUBSYSTEM=="usb", ATTRS{idVendor}=="114d", ATTRS{idProduct}=="8328", TAG+="uaccess"
# HTC Mass Storage Node
SUBSYSTEM=="usb", ATTRS{idVendor}=="114d", ATTRS{idProduct}=="8200", TAG+="uaccess"
SUBSYSTEM=="usb", ATTRS{idVendor}=="114d", ATTRS{idProduct}=="8a12", TAG+="uaccess"
#libsurvive
# HTC HMD
SUBSYSTEM=="usb", ATTRS{idVendor}=="0bb4", ATTRS{idProduct}=="2c87", MODE="0666"
# HTC HMD Pro(?)
SUBSYSTEM=="usb", ATTRS{idVendor}=="0bb4", ATTRS{idProduct}=="030e", MODE="0666"
# Light input
SUBSYSTEM=="usb", ATTRS{idVendor}=="28de", ATTRS{idProduct}=="2000", MODE="0666"
# Watchman
SUBSYSTEM=="usb", ATTRS{idVendor}=="28de", ATTRS{idProduct}=="2101", MODE="0666"
# Watchman wired
SUBSYSTEM=="usb", ATTRS{idVendor}=="28de", ATTRS{idProduct}=="2012", MODE="0666"
# Tracker wired
SUBSYSTEM=="usb", ATTRS{idVendor}=="28de", ATTRS{idProduct}=="2022", MODE="0666"
# Tracker 2018 wired
SUBSYSTEM=="usb", ATTRS{idVendor}=="28de", ATTRS{idProduct}=="2300", MODE="0666"
#KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="0bb4", ATTR{idProduct}=="2c87", MODE="0666" # HTC
#KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="0bb4", ATTR{idProduct}=="030e", MODE="0666" # HTC
#KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="28de", ATTR{idProduct}=="2000", MODE="0666" # Valve
#KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="28de", ATTR{idProduct}=="2101", MODE="0666" # Valve

81
src/rules/99-realsense-libusb.rules Normal file
View File

@ -0,0 +1,81 @@
##Version=1.1##
# Device rules for Intel RealSense devices (R200, F200, SR300 LR200, ZR300, D400, L500, T200)
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0a80", MODE:="0666", GROUP:="plugdev", RUN+="/usr/local/bin/usb-R200-in_udev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0a66", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0aa3", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0aa2", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0aa5", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0abf", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0acb", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0ad0", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="04b4", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0ad1", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0ad2", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0ad3", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0ad4", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0ad5", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0ad6", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0af2", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0af6", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0afe", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0aff", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b00", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b01", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b03", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b07", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b0c", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b0d", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b3a", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b3d", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b48", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b49", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b4b", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b4d", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b52", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b5b", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b5c", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b64", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b68", MODE:="0666", GROUP:="plugdev"
# Intel RealSense recovery devices (DFU)
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0ab3", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0adb", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0adc", MODE:="0666", GROUP:="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b55", MODE:="0666", GROUP:="plugdev"
# Intel RealSense devices (Movidius, T265)
SUBSYSTEMS=="usb", ENV{DEVTYPE}=="usb_device", ATTRS{idVendor}=="8087", ATTRS{idProduct}=="0af3", MODE="0666", GROUP="plugdev"
SUBSYSTEMS=="usb", ENV{DEVTYPE}=="usb_device", ATTRS{idVendor}=="8087", ATTRS{idProduct}=="0b37", MODE="0666", GROUP="plugdev"
SUBSYSTEMS=="usb", ENV{DEVTYPE}=="usb_device", ATTRS{idVendor}=="03e7", ATTRS{idProduct}=="2150", MODE="0666", GROUP="plugdev"
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0ad5", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
DRIVER=="hid_sensor_custom", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0ad5", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0af2", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
DRIVER=="hid_sensor*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0af2", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0afe", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
DRIVER=="hid_sensor_custom", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0afe", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0aff", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
DRIVER=="hid_sensor_custom", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0aff", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b00", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
DRIVER=="hid_sensor_custom", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b00", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b01", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
DRIVER=="hid_sensor_custom", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b01", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b3a", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
DRIVER=="hid_sensor*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b3a", RUN+="/bin/sh -c ' chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b3d", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
DRIVER=="hid_sensor*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b3d", RUN+="/bin/sh -c ' chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b4b", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
DRIVER=="hid_sensor*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b4b", RUN+="/bin/sh -c ' chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b4d", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
DRIVER=="hid_sensor*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b4d", RUN+="/bin/sh -c ' chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b5b", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
DRIVER=="hid_sensor*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b5b", RUN+="/bin/sh -c ' chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b5c", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
DRIVER=="hid_sensor*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b5c", RUN+="/bin/sh -c ' chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b64", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
DRIVER=="hid_sensor*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b64", RUN+="/bin/sh -c ' chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b68", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
DRIVER=="hid_sensor*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b68", RUN+="/bin/sh -c ' chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
# For products with motion_module, if (kernels is 4.15 and up) and (device name is "accel_3d") wait, in another process, until (enable flag is set to 1 or 200 mSec passed) and then set it to 0.
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0ad5|0afe|0aff|0b00|0b01|0b3a|0b3d|0b64|0b68", RUN+="/bin/sh -c '(major=`uname -r | cut -d \".\" -f1` && minor=`uname -r | cut -d \".\" -f2` && (([ $major -eq 4 ] && [ $minor -ge 15 ]) || [ $major -ge 5 ])) && (enamefile=/sys/%p/name && [ `cat $enamefile` = \"accel_3d\" ]) && enfile=/sys/%p/buffer/enable && echo \"COUNTER=0; while [ \$COUNTER -lt 20 ] && grep -q 0 $enfile; do sleep 0.01; COUNTER=\$((COUNTER+1)); done && echo 0 > $enfile\" | at now'"

2
src/rules/99-xvisio.rules Normal file
View File

@ -0,0 +1,2 @@
SUBSYSTEM=="usb", ATTR{idVendor}=="040e", MODE="0666", GROUP="plugdev"

1
src/rules/sensor_fisheye.rules Normal file
View File

@ -0,0 +1 @@
KERNEL=="video*", ATTRS{idVendor}=="1bcf", ATTRS{idProduct}=="2cd1", MODE:="0777", SYMLINK+="video50"

1
src/rules/sensor_serial.rules Normal file
View File

@ -0,0 +1 @@
KERNEL=="ttyUSB*", ATTRS{idVendor}=="1a86", ATTRS{idProduct}=="7522", MODE:="0777", SYMLINK+="ttyUSB50"

35
src/ufactory_lerobot/cameras/umi_camera/camera_umi.py
View File

@ -18,13 +18,8 @@ Provides the RealSenseCamera class for capturing frames from Intel RealSense cam
import logging
import time
from threading import Event, Lock, Thread
from typing import Any
import cv2 # type: ignore # TODO: add type stubs for OpenCV
import numpy as np # type: ignore # TODO: add type stubs for numpy
from numpy.typing import NDArray # type: ignore # TODO: add type stubs for numpy.typing
from lerobot.utils.errors import DeviceAlreadyConnectedError, DeviceNotConnectedError
from lerobot.cameras.camera import Camera
from ufactory_lerobot.devices.umi.xvlib import XVLib
from .configuration_umi import UmiCameraConfig
@ -60,11 +55,6 @@ class UmiCamera(Camera):
self.last_frame = None
self.xvlib = XVLib(self.serial_number)
self.xvlib.xv_color_camera_init()
self.xvlib.xv_set_color_camera_framerate(self.config.fps)
self.frame_lock = Lock()
self.new_frame_event: Event = Event()
self.thread = Thread(target=self._read_loop, daemon=True)
self.thread.start()
def __str__(self) -> str:
return f"{self.__class__.__name__}({self.serial_number})"
@ -84,18 +74,6 @@ class UmiCamera(Camera):
while time.monotonic() - start_time < self.warmup_s:
time.sleep(0.1)
def _read_loop(self):
while True:
try:
frame = self.read()
if frame is not None:
with self.frame_lock:
self.last_frame = frame
self.new_frame_event.set()
except Exception as e:
print('Read Frame Ex: {}'.format(e))
time.sleep(0.01)
def read(self, color_mode = None):
ret, img_data = self.xvlib.xv_get_color_image_rgb_data()
if ret <= 0:
@ -114,16 +92,11 @@ class UmiCamera(Camera):
return frame
def async_read(self, timeout_ms: float = 200):
if not self.new_frame_event.wait(timeout=timeout_ms / 1000.0):
thread_alive = self.thread is not None and self.thread.is_alive()
raise TimeoutError(
f"Timed out waiting for frame from camera {self} after {timeout_ms} ms. "
f"Read thread alive: {thread_alive}."
)
with self.frame_lock:
frame = self.read()
if frame is not None:
self.last_frame = frame
else:
frame = self.last_frame
self.new_frame_event.clear()
return frame
def disconnect(self) -> None:

5
src/ufactory_lerobot/robots/uf_mock_robot/uf_mock_robot.py
View File

@ -25,10 +25,9 @@ class UFMockRobot(Robot):
self.prefix = '' if not prefix else f"{prefix}."
self.config = config
self._dof = config.robot_dof
if self._dof == None or (not self._dof in (5, 6, 7)):
raise ValueError(f"Please specify the correct DOF uf_robot!, got {self._dof}")
self._control_space = self.config.control_space
if self._control_space == "joint" and (self._dof == None or (not self._dof in (5, 6, 7))):
raise ValueError(f"Please specify the correct DOF uf_robot!, got {self._dof}")
self._jnt_obs_has_vel = config.observe_joint_vel if self._control_space == "joint" else False
self._is_connected = False
self._is_calibrated =True

12
src/ufactory_lerobot/robots/uf_mock_robot/uf_mock_robot_config.py
View File

@ -1,22 +1,12 @@
from dataclasses import dataclass, field
from lerobot.cameras import CameraConfig
from lerobot.cameras.opencv import OpenCVCameraConfig
from lerobot.robots import RobotConfig
@RobotConfig.register_subclass("uf::mock_robot")
@dataclass
class UFMockRobotConfig(RobotConfig):
# cameras
cameras: dict[str, CameraConfig] = field(
default_factory=lambda: {
"fisheye": OpenCVCameraConfig(
index_or_path=6,
width=640,
height=480,
fps=30,
fourcc="MJPG"
)
}
default_factory=lambda: {}
)
robot_dof: int | None = None # Set it correctly if controlling in joint space!

19
src/ufactory_lerobot/robots/uf_robot/uf_robot_config.py
View File

@ -1,31 +1,14 @@
from dataclasses import dataclass, field
from typing import Tuple
from lerobot.cameras import CameraConfig
from lerobot.cameras.realsense import RealSenseCameraConfig
from lerobot.robots import RobotConfig
@RobotConfig.register_subclass("uf::robot")
@dataclass
class UFRobotConfig(RobotConfig):
# cameras
cameras: dict[str, CameraConfig] = field(
default_factory=lambda: {
"overhead": RealSenseCameraConfig(
serial_number_or_name="Intel RealSense D435I",
fps=30,
width=640, # 1280
height=480, # 720
# rotation=90,
),
"tool": RealSenseCameraConfig(
serial_number_or_name="Intel RealSense D435",
fps=30,
width=640, # 1280
height=480, # 720
),
}
default_factory=lambda: {}
)
robot_ip: str = "192.168.1.127"
robot_dof: int | None = None # Set it correctly if controlling in joint space!
control_space: str = "joint"

19
src/ufactory_lerobot/scripts/uf_lerobot_record.py
View File

@ -276,9 +276,9 @@ def record(cfg: RecordConfig) -> LeRobotDataset:
try:
if key == keyboard.Key.enter:
if not is_recorded:
print('[HELP] <ESC>: EXIT, <SPACE>: START, <LEFT ARROW>: RESET, <RIGH ARROW>: SAVE')
print('⌨ [ESC] Exit [Space] Start [←] Reset [→] Save')
else:
print('[HELP] <ESC>: EXIT, <LEFT ARROW>: RESET, <RIGH ARROW>: SAVE')
print('⌨ [ESC] Exit [←] Reset [→] Save')
# is_recorded = True
except Exception as e:
print(f"Error handling key release: {e}")
@ -287,9 +287,9 @@ def record(cfg: RecordConfig) -> LeRobotDataset:
listener, events = init_keyboard_listener(events=events, on_press=on_press, on_release=on_release)
print("\n********** Episode Record Loop Start **********")
print('[HELP] <ESC>: EXIT, <SPACE>: START, <LEFT ARROW>: RESET, <RIGH ARROW>: SAVE')
print('⌨ [ESC] Exit [Space] Start [←] Reset [→] Save')
else:
input('[HELP] Enter to to start record >>> ')
input('⌨ Press Enter to start record >>> ')
if is_uf_teleop:
teleop.set_teleop_enabled(True)
is_recorded = True
@ -312,7 +312,7 @@ def record(cfg: RecordConfig) -> LeRobotDataset:
if events["rerecord_episode"]:
events["rerecord_episode"] = False
events["exit_early"] = False
input('\nPress Enter to regenerate random target location >>>>> ')
input('\n⌨ Press Enter to regenerate random target location >>>>> ')
continue
if events["stop_recording"]:
continue
@ -357,9 +357,9 @@ def record(cfg: RecordConfig) -> LeRobotDataset:
dataset.clear_episode_buffer()
is_recorded = False
if is_evt:
print('[HELP] <ESC>: EXIT, <SPACE>: START, <LEFT ARROW>: RESET, <RIGH ARROW>: SAVE')
print('⌨ [ESC] Exit [Space] Start [←] Reset [→] Save')
else:
input('\nPress Enter to rerecord this episode >>>>> ')
input('\n⌨ Press Enter to rerecord this episode >>>>> ')
is_recorded = True
continue
@ -368,12 +368,13 @@ def record(cfg: RecordConfig) -> LeRobotDataset:
if is_uf_teleop:
teleop.set_teleop_enabled(False)
dataset.save_episode()
log_say(f"[Finish] Save episode {dataset.num_episodes}", cfg.play_sounds)
recorded_episodes += 1
is_recorded = False
if is_evt:
print('[HELP] <ESC>: EXIT, <SPACE>: START, <LEFT ARROW>: RESET, <RIGH ARROW>: SAVE')
print('⌨ [ESC] Exit [Space] Start [←] Reset [→] Save')
else:
input('Press Enter to record at the next episode >>>>> ')
input('⌨ Press Enter to record at the next episode >>>>> ')
is_recorded = True
print("\n********** Episode Record Loop Exit **********")

16
src/ufactory_lerobot/scripts/uf_robot_teleop.py
View File

@ -85,11 +85,11 @@ def teleop_loop(cfg: TeleopConfig):
if key == keyboard.Key.enter:
if is_paused:
if is_reset:
print('[HELP] <ESC>: EXIT, <SPACE>: RESET AND START, <LEFT ARROW>: RESET')
print('⌨ [ESC] Exit [Space] Reset / Start [←] Reset')
else:
print('[HELP] <ESC>: EXIT, <SPACE>: START, <LEFT ARROW>: RESET')
print('⌨ [ESC] Exit [Space] Start [←] Reset')
else:
print('[HELP] <ESC>: EXIT, <SPACE>: PAUSE, <LEFT ARROW>: RESET')
print('⌨ [ESC] Exit [Space] Pause [←] Pause / Reset')
except Exception as e:
print(f"Error handling key release: {e}")
if key in key_dict:
@ -97,9 +97,9 @@ def teleop_loop(cfg: TeleopConfig):
listener, events = init_keyboard_listener(events=events, on_press=on_press, on_release=on_release)
print("\n********** Teleop Control Loop Start **********")
print('[HELP] <ESC>: EXIT, <SPACE>: START, <LEFT ARROW>: RESET')
print('⌨ [ESC] Exit [Space] Start [←] Reset')
else:
input('[HELP] Enter to control robot with teleop >>> ')
input('⌨ Press Enter to start teleop >>> ')
if is_uf_teleop:
obs = robot.get_observation()
teleop.set_teleop_enabled(True, obs)
@ -121,7 +121,7 @@ def teleop_loop(cfg: TeleopConfig):
is_paused = True
if is_uf_teleop:
teleop.set_teleop_enabled(False)
print('[HELP] <ESC>: EXIT, <SPACE>: RESET AND START, <LEFT ARROW>: RESET')
print('⌨ [ESC] Exit [Space] Reset / Start [←] Reset')
elif not key_dict[keyboard.Key.left] and key_left_pressed:
key_left_pressed = False
@ -132,7 +132,7 @@ def teleop_loop(cfg: TeleopConfig):
if is_uf_teleop:
teleop.set_teleop_enabled(False)
# print('========== Teleop is paused ==========')
print('[HELP] <ESC>: EXIT, <SPACE>: START, <LEFT ARROW>: RESET')
print('⌨ [ESC] Exit [Space] Start [←] Reset')
else:
if is_reset:
is_reset = False
@ -141,7 +141,7 @@ def teleop_loop(cfg: TeleopConfig):
if is_uf_teleop:
obs = robot.get_observation()
teleop.set_teleop_enabled(True, obs)
print('[HELP] <ESC>: EXIT, <SPACE>: PAUSE, <LEFT ARROW>: RESET')
print('⌨ [ESC] Exit [Space] Pause [←] Reset')
continue
elif not key_dict[keyboard.Key.space] and key_space_pressed:
key_space_pressed = False

4
src/ufactory_lerobot/teleoperators/umi_teleop/umi_teleop.py
View File

@ -86,6 +86,10 @@ class UmiTeleop(UFBaseTeleop):
def connect(self, calibrate: bool = False) -> None:
self.tracker = ViveTracker() if self.config.use_vive_tracker else None
self.xvlib = XVLib(self.config.serial_number, not self.config.use_vive_tracker, self.config.use_gripper)
if not self.config.use_vive_tracker:
self.xvlib.xv_slam_init()
if self.config.use_gripper:
self.xvlib.xv_clamp_stream_init()
self._is_connected = True
def disconnect(self):