feat(project): 新增配置、udev 规则并完善文档结构
添加 GELLO/Pika/UMI/SpaceMouse 遥操作配置， 新增 udev 设备规则，更新 README 项目结构， 优化键盘提示格式。
This commit is contained in:
Vinman
parent
6f520b5fef
commit
e01fec8b91
3
.gitignore
vendored
3
.gitignore
vendored
@ -83,4 +83,5 @@ models/
|
||||
*.onnx
|
||||
*.engine
|
||||
*.trt
|
||||
third_party/
|
||||
*.xvcd
|
||||
ufactory_usage/
|
||||
|
||||
236
LICENSE
236
LICENSE
@ -1,27 +1,217 @@
|
||||
Copyright (c) 2026, UFACTORY Inc.
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
All rights reserved.
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
Redistribution and use in source and binary forms, with or without modification,
|
||||
are permitted provided that the following conditions are met:
|
||||
1. Definitions.
|
||||
|
||||
* Redistributions of source code must retain the above copyright notice,
|
||||
this list of conditions and the following disclaimer.
|
||||
* Redistributions in binary form must reproduce the above copyright notice,
|
||||
this list of conditions and the following disclaimer in the documentation
|
||||
and/or other materials provided with the distribution.
|
||||
* Neither the name of the copyright holder nor the names of its contributors
|
||||
may be used to endorse or promote products derived from this software
|
||||
without specific prior written permission.
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
|
||||
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
|
||||
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
|
||||
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
|
||||
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
|
||||
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
|
||||
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
|
||||
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright 2026 UFACTORY Inc.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
--------------------------------------------------------------------
|
||||
|
||||
This project includes code derived from the following third-party projects:
|
||||
|
||||
- LeRobot (https://github.com/huggingface/lerobot)
|
||||
Copyright 2024 The HuggingFace Inc. team
|
||||
Licensed under the Apache License 2.0
|
||||
|
||||
- GELLO (https://github.com/wuphilipp/gello_software)
|
||||
Copyright Philipp Wu
|
||||
Licensed under the MIT License
|
||||
|
||||
- xArm-Python-SDK (https://github.com/xArm-Developer/xArm-Python-SDK)
|
||||
Copyright UFACTORY Inc.
|
||||
Licensed under the BSD 3-Clause License
|
||||
|
||||
198
README.md
198
README.md
@ -1,54 +1,202 @@
|
||||
# uFactory LeRobot
|
||||
# UFACTORY LeRobot
|
||||
|
||||
uFactory 机械臂与 LeRobot 框架集成项目，用于机器人学习、数据采集和策略部署。
|
||||
UFACTORY 机械臂与 LeRobot 框架集成项目，支持多种遥操作方式的数据采集、策略训练和部署推理。
|
||||
|
||||
## 功能特性
|
||||
|
||||
- 🤖 uFactory 机械臂控制接口
|
||||
- 📷 多摄像头数据采集
|
||||
- 🧠 模仿学习（行为克隆、ACT、扩散策略等）
|
||||
- 🚀 策略部署与实时推理
|
||||
- 📊 训练过程可视化和日志记录
|
||||
- 🤖 UFACTORY 机械臂控制（xArm 系列）
|
||||
- 🎮 多种遥操作方式：GELLO / Pika / UMI / SpaceMouse
|
||||
- 📷 多摄像头数据采集（RealSense / UMI 相机）
|
||||
- 📊 数据集录制与管理（兼容 LeRobot 格式）
|
||||
- 🧠 模仿学习训练（ACT / Diffusion Policy 等）
|
||||
- 🚀 策略评估与实时推理
|
||||
- 🔧 Mock 机器人模拟（只用遥操作设备采集数据）
|
||||
|
||||
## 环境要求
|
||||
|
||||
- Ubuntu 22.04 / 24.04
|
||||
- Python >= 3.10
|
||||
- CUDA >= 12.0（GPU 训练推荐）
|
||||
- uFactory 机械臂（xArm 系列）
|
||||
- UFACTORY 机械臂（xArm 系列，可选）
|
||||
|
||||
## 安装
|
||||
|
||||
### 基础项目安装
|
||||
|
||||
```bash
|
||||
# 克隆仓库
|
||||
git clone <repo-url>
|
||||
git clone https://github.com/xArm-Developer/ufactory_lerobot.git
|
||||
cd ufactory_lerobot
|
||||
|
||||
# 创建虚拟环境
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate # Linux/macOS
|
||||
# .venv\Scripts\activate # Windows
|
||||
# 创建 conda 环境
|
||||
conda create -n uf_lerobot python=3.10 -y
|
||||
conda activate uf_lerobot
|
||||
|
||||
# 安装依赖
|
||||
pip install -e ".[dev]"
|
||||
# 安装项目
|
||||
pip install -e .
|
||||
```
|
||||
|
||||
## 快速开始
|
||||
包含：`lerobot==0.4.3`、`xarm-python-sdk`、`numpy`、`pyyaml`（lerobot 已自动携带 torch、opencv、wandb 等训练相关依赖）。
|
||||
|
||||
```python
|
||||
# TODO: 添加示例代码
|
||||
### 外设模块安装
|
||||
|
||||
外设依赖以可选模块形式提供，通过 `[模块名]` 安装。
|
||||
|
||||
#### GELLO 遥操作
|
||||
|
||||
适用于 GELLO 示教臂（Dynamixel 舵机方案），控制空间为关节空间。
|
||||
|
||||
```bash
|
||||
# 1. 安装 GELLO 模块
|
||||
pip install -e ".[gello]"
|
||||
|
||||
# 2. 添加串口权限（重新登录后生效）
|
||||
sudo usermod -aG dialout $USER
|
||||
```
|
||||
|
||||
#### Pika 遥操作
|
||||
|
||||
适用于 Pika Sense 手持示教器 + Vive Tracker，控制空间为笛卡尔空间。
|
||||
|
||||
```bash
|
||||
# 1. 安装外设依赖（不需要它们的间接依赖）
|
||||
pip install pysurvive agx-pypika --no-deps
|
||||
|
||||
# 2. 安装 udev 规则（重新插拔设备后生效）
|
||||
sudo cp src/rules/*.rules /etc/udev/rules.d/
|
||||
sudo udevadm control --reload-rules && sudo udevadm trigger
|
||||
```
|
||||
|
||||
> Vive Tracker 首次使用前需校准：`uf-vive-calibrate`
|
||||
|
||||
#### UMI 遥操作
|
||||
|
||||
适用于 UMI（Universal Manipulation Interface）方案，含 Vive Tracker 追踪，支持双机械臂。
|
||||
|
||||
```bash
|
||||
# 1. 安装 XVSDK（系统级依赖，仅支持 Ubuntu Focal）
|
||||
sudo dpkg -i src/xvsdk/XVSDK_focal_amd64.deb
|
||||
sudo apt install -y --fix-broken
|
||||
|
||||
# 2. 安装外设依赖
|
||||
pip install pysurvive --no-deps
|
||||
|
||||
# 3. 安装 udev 规则（重新插拔设备后生效）
|
||||
sudo cp src/rules/*.rules /etc/udev/rules.d/
|
||||
sudo udevadm control --reload-rules && sudo udevadm trigger
|
||||
```
|
||||
|
||||
> Vive Tracker 首次使用前需校准：`uf-vive-calibrate`
|
||||
|
||||
**多 UMI 设备配置**（使用两台及以上时）：
|
||||
|
||||
```bash
|
||||
# 增加 USB 缓冲区大小
|
||||
sudo sed -i '/GRUB_CMDLINE_LINUX_DEFAULT/s/quiet splash/quiet splash usbcore.usbfs_memory_mb=128/' /etc/default/grub
|
||||
sync
|
||||
sudo update-grub
|
||||
sudo reboot
|
||||
```
|
||||
|
||||
#### SpaceMouse 遥操作
|
||||
|
||||
适用于 3Dconnexion SpaceMouse / SpaceNavigator。
|
||||
|
||||
```bash
|
||||
# 1. 安装 SpaceMouse 模块
|
||||
pip install -e ".[spacemouse]"
|
||||
|
||||
# 2. 安装 udev 规则（重新插拔设备后生效）
|
||||
sudo cp src/rules/*.rules /etc/udev/rules.d/
|
||||
sudo udevadm control --reload-rules && sudo udevadm trigger
|
||||
```
|
||||
|
||||
|
||||
## 使用
|
||||
|
||||
### 1. 遥操作测试
|
||||
|
||||
测试遥操作设备与机械臂的联动，不录制数据。
|
||||
|
||||
```bash
|
||||
uf-robot-teleop -c path/to/config.yaml
|
||||
uf-robot-teleop -c path/to/config.yaml -f 60 # 指定频率
|
||||
```
|
||||
|
||||
### 2. 数据采集
|
||||
|
||||
通过遥操作录制数据集。
|
||||
|
||||
```bash
|
||||
uf-lerobot-record -c path/to/record_config.yaml
|
||||
uf-lerobot-record -c path/to/config.yaml --resume # 续录
|
||||
```
|
||||
|
||||
### 3. 策略训练
|
||||
|
||||
采集数据后，使用 LeRobot 训练管道进行模仿学习训练。
|
||||
|
||||
```bash
|
||||
lerobot-train --policy act --dataset your_dataset_name
|
||||
```
|
||||
|
||||
### 4. 策略评估
|
||||
|
||||
```bash
|
||||
uf-lerobot-eval -c path/to/eval_config.yaml
|
||||
```
|
||||
|
||||
### Mock 模式（无实体机械臂）
|
||||
|
||||
```yaml
|
||||
type: "uf::mock_robot" # 单臂模拟
|
||||
type: "uf::multiple_mock_robot" # 双臂模拟
|
||||
```
|
||||
|
||||
## 遥操作方式对比
|
||||
|
||||
| 特性 | GELLO | Pika | UMI | SpaceMouse |
|
||||
|------|-------|------|-----|------------|
|
||||
| 控制空间 | 关节空间 | 任务空间 | 任务空间 | 任务空间 |
|
||||
| 跟踪方式 | Dynamixel 舵机 | Vive Tracker | UMI SLAM / Vive | 3D 鼠标 |
|
||||
| 双臂支持 | ❌ | ❌ | ✅ | ❌ |
|
||||
| 系统依赖 | dialout 组 | — | XVSDK deb | — |
|
||||
|
||||
## 项目结构
|
||||
|
||||
```
|
||||
ufactory_lerobot/
|
||||
├── src/ # 源代码
|
||||
├── configs/ # 配置文件
|
||||
├── scripts/ # 脚本
|
||||
├── tests/ # 测试
|
||||
└── docs/ # 文档
|
||||
├── src/
|
||||
│ ├── ufactory_lerobot/
|
||||
│ │ ├── robots/ # 机器人控制
|
||||
│ │ │ ├── uf_robot/ # xArm 实体机器人
|
||||
│ │ │ ├── uf_mock_robot/ # 仿真 Mock 机器人
|
||||
│ │ ├── teleoperators/ # 遥操作器
|
||||
│ │ │ ├── gello_teleop/ # GELLO (Dynamixel 示教臂)
|
||||
│ │ │ ├── pika_teleop/ # Pika Sense (手持示教器 + Vive)
|
||||
│ │ │ ├── umi_teleop/ # UMI (含双机械臂)
|
||||
│ │ │ ├── space_mouse/ # SpaceMouse (3D 鼠标)
|
||||
│ │ ├── cameras/ # 摄像头模块
|
||||
│ │ │ └── umi_camera/ # UMI 相机
|
||||
│ │ ├── devices/ # 外部设备驱动
|
||||
│ │ │ ├── pika/ # Pika 串口驱动
|
||||
│ │ │ └── umi/ # XVLib / Vive Tracker
|
||||
│ │ ├── scripts/ # 执行脚本
|
||||
│ │ │ ├── uf_robot_teleop.py # 遥操作测试
|
||||
│ │ │ ├── uf_lerobot_record.py # 数据采集
|
||||
│ │ │ ├── uf_lerobot_eval.py # 策略评估
|
||||
│ │ │ └── vive_calibrate.py # Vive Tracker 校准
|
||||
│ │ └── utils/ # 工具函数
|
||||
│ ├── rules/ # udev 设备规则
|
||||
│ └── xvsdk/ # XVSDK 系统依赖
|
||||
├── config/ # YAML 配置文件
|
||||
│ ├── gello/
|
||||
│ ├── pika/
|
||||
│ ├── umi/
|
||||
│ └── spacemouse/
|
||||
├── pyproject.toml
|
||||
└── README.md
|
||||
```
|
||||
|
||||
## 许可证
|
||||
|
||||
本项目基于 MIT 许可证发布。详见 [LICENSE](LICENSE) 文件。
|
||||
本项目基于 Apache License 2.0 发布，详见 [LICENSE](LICENSE) 文件。
|
||||
|
||||
30
config/gello/xarm5_gello_record_config.yaml
Normal file
30
config/gello/xarm5_gello_record_config.yaml
Normal file
@ -0,0 +1,30 @@
|
||||
RobotConfig:
|
||||
_target_: ufactory_lerobot.robots.uf_robot.UFRobotConfig
|
||||
id: "uf_robot"
|
||||
robot_dof: 5
|
||||
control_space: "joint"
|
||||
robot_ip: "192.168.1.75"
|
||||
gripper_type: 1
|
||||
start_joints: [0, 0, -90, 90, 0]
|
||||
|
||||
DatasetRecordConfig:
|
||||
_target_: lerobot.scripts.lerobot_record.DatasetRecordConfig
|
||||
# root of local repo: /home/<user_name>/.cache/huggingface/lerobot (default)
|
||||
root: "../lerobot_datas/record/ufactory/xarm5_gello_datas"
|
||||
repo_id: "ufactory/xarm5_gello_datas"
|
||||
single_task: "Pick up the purple grape and drop into the box on the left."
|
||||
fps: 30
|
||||
episode_time_s: 60 # max duration for one episode
|
||||
reset_time_s: 20 # (invalid) time for resetting env between episodes
|
||||
push_to_hub: False
|
||||
|
||||
# make sure to edit with your correct configurations!
|
||||
TeleoperatorConfig:
|
||||
_target_: ufactory_lerobot.teleoperators.gello_teleop.GelloTeleopConfig
|
||||
port: "/dev/serial/by-id/usb-FTDI_USB__-__Serial_Converter_FTAJZYC7-if00-port0"
|
||||
joint_ids: [1, 2, 4, 6, 7]
|
||||
joint_signs: [1, 1, -1, 1, 1]
|
||||
start_joints: [0, 0, -90, 90, 0]
|
||||
gripper_id: 8
|
||||
torque_joint_ids: [3, 5]
|
||||
|
||||
30
config/gello/xarm6_gello_record_config.yaml
Normal file
30
config/gello/xarm6_gello_record_config.yaml
Normal file
@ -0,0 +1,30 @@
|
||||
RobotConfig:
|
||||
_target_: ufactory_lerobot.robots.uf_robot.UFRobotConfig
|
||||
id: "uf_robot"
|
||||
robot_dof: 6
|
||||
control_space: "joint"
|
||||
robot_ip: "192.168.1.68"
|
||||
gripper_type: 1
|
||||
start_joints: [0, 0, -90, 0, 90, 0]
|
||||
|
||||
DatasetRecordConfig:
|
||||
_target_: lerobot.scripts.lerobot_record.DatasetRecordConfig
|
||||
# root of local repo: /home/<user_name>/.cache/huggingface/lerobot (default)
|
||||
root: "../lerobot_datas/record/ufactory/xarm6_gello_datas"
|
||||
repo_id: "ufactory/xarm6_gello_datas"
|
||||
single_task: "Pick up the purple grape and drop into the box on the left."
|
||||
fps: 30
|
||||
episode_time_s: 60 # max duration for one episode
|
||||
reset_time_s: 20 # (invalid) time for resetting env between episodes
|
||||
push_to_hub: False
|
||||
|
||||
# make sure to edit with your correct configurations!
|
||||
TeleoperatorConfig:
|
||||
_target_: ufactory_lerobot.teleoperators.gello_teleop.GelloTeleopConfig
|
||||
port: "/dev/serial/by-id/usb-FTDI_USB__-__Serial_Converter_FTAJZYC7-if00-port0"
|
||||
joint_ids: [1, 2, 4, 5, 6, 7]
|
||||
joint_signs: [1, 1, -1, 1, 1, 1]
|
||||
start_joints: [0, 0, -90, 0, 90, 0]
|
||||
gripper_id: 8
|
||||
torque_joint_ids: [3]
|
||||
|
||||
27
config/gello/xarm7_gello_record_config.yaml
Normal file
27
config/gello/xarm7_gello_record_config.yaml
Normal file
@ -0,0 +1,27 @@
|
||||
RobotConfig:
|
||||
_target_: ufactory_lerobot.robots.uf_robot.UFRobotConfig
|
||||
id: "uf_robot"
|
||||
robot_dof: 7
|
||||
control_space: "joint"
|
||||
robot_ip: "192.168.1.85"
|
||||
gripper_type: 1
|
||||
start_joints: [0, 0, 0, 90, 0, 90, 0]
|
||||
|
||||
DatasetRecordConfig:
|
||||
_target_: lerobot.scripts.lerobot_record.DatasetRecordConfig
|
||||
# root of local repo: /home/<user_name>/.cache/huggingface/lerobot (default)
|
||||
root: "../lerobot_datas/record/ufactory/xarm6_gello_datas"
|
||||
repo_id: "ufactory/xarm6_gello_datas"
|
||||
single_task: "Pick up the purple grape and drop into the box on the left."
|
||||
fps: 30
|
||||
episode_time_s: 60 # max duration for one episode
|
||||
reset_time_s: 20 # (invalid) time for resetting env between episodes
|
||||
push_to_hub: False
|
||||
|
||||
# make sure to edit with your correct configurations!
|
||||
TeleoperatorConfig:
|
||||
_target_: ufactory_lerobot.teleoperators.gello_teleop.GelloTeleopConfig
|
||||
id: "gello_teleop"
|
||||
port: "/dev/serial/by-id/usb-FTDI_USB__-__Serial_Converter_FTAJZYC7-if00-port0"
|
||||
start_joints: [0, 0, 0, 90, 0, 90, 0]
|
||||
|
||||
34
config/pika/pika_record_config.yaml
Normal file
34
config/pika/pika_record_config.yaml
Normal file
@ -0,0 +1,34 @@
|
||||
RobotConfig:
|
||||
_target_: ufactory_lerobot.robots.uf_mock_robot.UFMockRobotConfig
|
||||
id: "uf_mock_robot"
|
||||
control_space: "cartesian"
|
||||
gripper_type: 2
|
||||
state_offset_action: 3
|
||||
|
||||
cameras:
|
||||
fisheye:
|
||||
_target_: lerobot.cameras.opencv.configuration_opencv.OpenCVCameraConfig
|
||||
index_or_path: "/dev/v4l/by-id/usb-DECXIN_CAMERA_DECXIN_CAMERA_01.00.00-video-index0"
|
||||
width: 640
|
||||
height: 480
|
||||
fps: 30
|
||||
fourcc: "MJPG"
|
||||
|
||||
DatasetRecordConfig:
|
||||
_target_: lerobot.scripts.lerobot_record.DatasetRecordConfig
|
||||
# root of local repo: /home/<user_name>/.cache/huggingface/lerobot (default)
|
||||
root: "../lerobot_datas/record/ufactory/pika_datas"
|
||||
repo_id: "ufactory/pika_datas"
|
||||
single_task: "Pick up the purple grape and drop into the box on the left."
|
||||
fps: 30
|
||||
episode_time_s: 60 # max duration for one episode
|
||||
reset_time_s: 20 # (invalid) time for resetting env between episodes
|
||||
push_to_hub: False
|
||||
|
||||
# make sure to edit with your correct configurations!
|
||||
TeleoperatorConfig:
|
||||
_target_: ufactory_lerobot.teleoperators.pika_teleop.PikaTeleopConfig
|
||||
# port: "/dev/ttyUSB0"
|
||||
scale_xyz: 1.5
|
||||
tracker_to_robot_eef: [0, 0, 0, 180, -90, 0]
|
||||
robot_base_pose: [400, 0, 400, 180, 0, 0]
|
||||
38
config/pika/xarm7_pika_record_config.yaml
Normal file
38
config/pika/xarm7_pika_record_config.yaml
Normal file
@ -0,0 +1,38 @@
|
||||
RobotConfig:
|
||||
_target_: ufactory_lerobot.robots.uf_robot.UFRobotConfig
|
||||
id: "uf_robot"
|
||||
robot_dof: 7
|
||||
control_space: "cartesian"
|
||||
robot_ip: "192.168.1.85"
|
||||
gripper_type: 10
|
||||
max_linear_velocity: 200
|
||||
start_joints: [-0.2, -13.9, -0.3, 52.6, 0.9, 66.6, 0.2]
|
||||
start_tcp_pose: [400, 0, 400, 180, 0, 0]
|
||||
|
||||
cameras:
|
||||
fisheye:
|
||||
_target_: lerobot.cameras.opencv.configuration_opencv.OpenCVCameraConfig
|
||||
index_or_path: "/dev/v4l/by-id/usb-DECXIN_CAMERA_DECXIN_CAMERA_01.00.00-video-index0"
|
||||
width: 640
|
||||
height: 480
|
||||
fps: 30
|
||||
fourcc: "MJPG"
|
||||
|
||||
DatasetRecordConfig:
|
||||
_target_: lerobot.scripts.lerobot_record.DatasetRecordConfig
|
||||
# root of local repo: /home/<user_name>/.cache/huggingface/lerobot (default)
|
||||
root: "../lerobot_datas/record/ufactory/xarm7_pika_datas"
|
||||
repo_id: "ufactory/xarm7_pika_datas"
|
||||
single_task: "Pick up the purple grape and drop into the box on the left."
|
||||
fps: 30
|
||||
episode_time_s: 60 # max duration for one episode
|
||||
reset_time_s: 20 # (invalid) time for resetting env between episodes
|
||||
push_to_hub: False
|
||||
|
||||
# make sure to edit with your correct configurations!
|
||||
TeleoperatorConfig:
|
||||
_target_: ufactory_lerobot.teleoperators.pika_teleop.PikaTeleopConfig
|
||||
# port: "/dev/ttyUSB0"
|
||||
scale_xyz: 1.5
|
||||
tracker_to_robot_eef: [0, 0, 0, 180, -90, 0]
|
||||
robot_base_pose: [400, 0, 400, 180, 0, 0]
|
||||
38
config/spacemouse/xarm7_spacemouse_record_config.yaml
Normal file
38
config/spacemouse/xarm7_spacemouse_record_config.yaml
Normal file
@ -0,0 +1,38 @@
|
||||
RobotConfig:
|
||||
_target_: ufactory_lerobot.robots.uf_robot.UFRobotConfig
|
||||
id: "uf_robot"
|
||||
robot_dof: 7
|
||||
control_space: "cartesian"
|
||||
robot_ip: "192.168.1.127"
|
||||
gripper_type: 0
|
||||
|
||||
cameras:
|
||||
overhead:
|
||||
_target_: lerobot.cameras.realsense.configuration_realsense.RealSenseCameraConfig
|
||||
serial_number_or_name: "Intel RealSense D435I"
|
||||
width: 640
|
||||
height: 480
|
||||
fps: 30
|
||||
tool:
|
||||
_target_: lerobot.cameras.realsense.configuration_realsense.RealSenseCameraConfig
|
||||
serial_number_or_name: "Intel RealSense D435"
|
||||
width: 640
|
||||
height: 480
|
||||
fps: 30
|
||||
|
||||
DatasetRecordConfig:
|
||||
_target_: lerobot.scripts.lerobot_record.DatasetRecordConfig
|
||||
# root: /home/<user_name>/.cache/huggingface/lerobot (default)
|
||||
root: "../lerobot_datas/record/ufactory/xarm7_pushT"
|
||||
repo_id: "ufactory/xarm7_pushT"
|
||||
single_task: "Push the T-block into the specified zone then go to reset position."
|
||||
fps: 10
|
||||
episode_time_s: 60 # max duration for one episode
|
||||
reset_time_s: 20 # (invalid) time for resetting env between episodes
|
||||
push_to_hub: False
|
||||
|
||||
# space mouse
|
||||
TeleoperatorConfig:
|
||||
_target_: ufactory_lerobot.teleoperators.space_mouse.SpaceMouseTeleopConfig
|
||||
max_value: 300
|
||||
use_gripper: False
|
||||
68
config/umi/dual_umi_record_config.yaml
Normal file
68
config/umi/dual_umi_record_config.yaml
Normal file
@ -0,0 +1,68 @@
|
||||
RobotConfig:
|
||||
_target_: ufactory_lerobot.robots.uf_mock_robot.MultipleUFMockRobotConfig
|
||||
id: "multiple_uf_mock_robots"
|
||||
robots:
|
||||
left:
|
||||
_target_: ufactory_lerobot.robots.uf_mock_robot.UFMockRobotConfig
|
||||
id: "left_uf_mock_robot"
|
||||
control_space: "cartesian"
|
||||
gripper_type: 1
|
||||
state_offset_action: 3
|
||||
cameras:
|
||||
fisheye:
|
||||
_target_: ufactory_lerobot.cameras.umi_camera.configuration_umi.UmiCameraConfig
|
||||
serial_number: "250801DR48FP26001318"
|
||||
width: 1280
|
||||
height: 1280
|
||||
fps: 30
|
||||
right:
|
||||
_target_: ufactory_lerobot.robots.uf_mock_robot.UFMockRobotConfig
|
||||
id: "right_uf_mock_robot"
|
||||
control_space: "cartesian"
|
||||
gripper_type: 1
|
||||
state_offset_action: 3
|
||||
cameras:
|
||||
fisheye:
|
||||
_target_: ufactory_lerobot.cameras.umi_camera.configuration_umi.UmiCameraConfig
|
||||
serial_number: "250801DR48FP26001295"
|
||||
width: 1280
|
||||
height: 1280
|
||||
fps: 30
|
||||
|
||||
# make sure to edit with your correct configurations!
|
||||
TeleoperatorConfig:
|
||||
_target_: ufactory_lerobot.teleoperators.umi_teleop.MultipleUmiTeleopConfig
|
||||
id: "multiple_teleops"
|
||||
teleops:
|
||||
left:
|
||||
_target_: ufactory_lerobot.teleoperators.umi_teleop.UmiTeleopConfig
|
||||
id: "left_teleop"
|
||||
serial_number: "250801DR48FP26001318"
|
||||
use_vive_tracker: False
|
||||
vive_tracker_id: "LHR-555DC7BF"
|
||||
use_gripper: True
|
||||
tracker_to_robot_eef: [0, 0, 0, 0, 0, -90]
|
||||
robot_base_pose: [400, 0, 150, -90, 0, 0]
|
||||
right:
|
||||
_target_: ufactory_lerobot.teleoperators.umi_teleop.UmiTeleopConfig
|
||||
id: "right_teleop"
|
||||
serial_number: "250801DR48FP26001295"
|
||||
use_vive_tracker: False
|
||||
vive_tracker_id: "LHR-2425BAD3"
|
||||
use_gripper: True
|
||||
tracker_to_robot_eef: [0, 0, 0, 0, 0, -90]
|
||||
robot_base_pose: [400, 0, 150, 90, 0, 0]
|
||||
|
||||
DatasetRecordConfig:
|
||||
_target_: lerobot.scripts.lerobot_record.DatasetRecordConfig
|
||||
# root of local repo: /home/<user_name>/.cache/huggingface/lerobot (default)
|
||||
root: "../lerobot_datas/record/ufactory/dual_xarm6_umi_datas"
|
||||
repo_id: "ufactory/dual_xarm6_umi_datas"
|
||||
single_task: "Pick up the purple grape and drop into the box on the left."
|
||||
fps: 60
|
||||
episode_time_s: 60 # max duration for one episode
|
||||
reset_time_s: 20 # (invalid) time for resetting env between episodes
|
||||
num_episodes: 100
|
||||
push_to_hub: False
|
||||
|
||||
|
||||
81
config/umi/dual_xarm6_umi_record_config.yaml
Normal file
81
config/umi/dual_xarm6_umi_record_config.yaml
Normal file
@ -0,0 +1,81 @@
|
||||
RobotConfig:
|
||||
_target_: ufactory_lerobot.robots.uf_robot.MultipleUFRobotConfig
|
||||
id: "multiple_uf_robots"
|
||||
async_connect: True
|
||||
async_configure: True
|
||||
async_action: False
|
||||
robots:
|
||||
left:
|
||||
_target_: ufactory_lerobot.robots.uf_robot.UFRobotConfig
|
||||
id: "left_uf_robot"
|
||||
robot_dof: 6
|
||||
control_space: "cartesian"
|
||||
robot_ip: "192.168.1.29"
|
||||
gripper_type: 2
|
||||
max_linear_velocity: 200
|
||||
start_joints: [-14.7, 25.6, -49.6, -282.5, 95.9, -65.9]
|
||||
start_tcp_pose: [400, 0, 150, -90, 0, 0]
|
||||
cameras:
|
||||
fisheye:
|
||||
_target_: lerobot.cameras.opencv.configuration_opencv.OpenCVCameraConfig
|
||||
index_or_path: "/dev/v4l/by-id/usb-XVisio_Technology_XVisio_vSLAM_250801DR48FB26001379-video-index0"
|
||||
width: 1280
|
||||
height: 1280
|
||||
fps: 100
|
||||
fourcc: "YUYV"
|
||||
right:
|
||||
_target_: ufactory_lerobot.robots.uf_robot.UFRobotConfig
|
||||
id: "right_uf_robot"
|
||||
robot_dof: 6
|
||||
control_space: "cartesian"
|
||||
robot_ip: "192.168.1.83"
|
||||
gripper_type: 2
|
||||
max_linear_velocity: 200
|
||||
start_joints: [14.5, 25.5, -49.6, -76.8, 95.9, 66.6]
|
||||
start_tcp_pose: [400, 0, 150, 90, 0, 0]
|
||||
cameras:
|
||||
fisheye:
|
||||
_target_: lerobot.cameras.opencv.configuration_opencv.OpenCVCameraConfig
|
||||
index_or_path: "/dev/v4l/by-id/usb-XVisio_Technology_XVisio_vSLAM_250801DR48FB26001396-video-index0"
|
||||
width: 1280
|
||||
height: 1280
|
||||
fps: 100
|
||||
fourcc: "YUYV"
|
||||
|
||||
# make sure to edit with your correct configurations!
|
||||
TeleoperatorConfig:
|
||||
_target_: ufactory_lerobot.teleoperators.umi_teleop.MultipleUmiTeleopConfig
|
||||
id: "multiple_teleops"
|
||||
teleops:
|
||||
left:
|
||||
_target_: ufactory_lerobot.teleoperators.umi_teleop.UmiTeleopConfig
|
||||
id: "left_teleop"
|
||||
serial_number: "250801DR48FP26001318"
|
||||
use_vive_tracker: False
|
||||
vive_tracker_id: "LHR-555DC7BF"
|
||||
use_gripper: True
|
||||
tracker_to_robot_eef: [0, 0, 0, 0, 0, -90]
|
||||
robot_base_pose: [400, 0, 150, -90, 0, 0]
|
||||
right:
|
||||
_target_: ufactory_lerobot.teleoperators.umi_teleop.UmiTeleopConfig
|
||||
id: "right_teleop"
|
||||
serial_number: "250801DR48FP26001295"
|
||||
use_vive_tracker: False
|
||||
vive_tracker_id: "LHR-2425BAD3"
|
||||
use_gripper: True
|
||||
tracker_to_robot_eef: [0, 0, 0, 0, 0, -90]
|
||||
robot_base_pose: [400, 0, 150, 90, 0, 0]
|
||||
|
||||
DatasetRecordConfig:
|
||||
_target_: lerobot.scripts.lerobot_record.DatasetRecordConfig
|
||||
# root of local repo: /home/<user_name>/.cache/huggingface/lerobot (default)
|
||||
root: "../lerobot_datas/record/ufactory/dual_xarm6_umi_datas"
|
||||
repo_id: "ufactory/dual_xarm6_umi_datas"
|
||||
single_task: "Pick up the purple grape and drop into the box on the left."
|
||||
fps: 60
|
||||
episode_time_s: 60 # max duration for one episode
|
||||
reset_time_s: 20 # (invalid) time for resetting env between episodes
|
||||
num_episodes: 100
|
||||
push_to_hub: False
|
||||
|
||||
|
||||
34
config/umi/umi_record_config.yaml
Normal file
34
config/umi/umi_record_config.yaml
Normal file
@ -0,0 +1,34 @@
|
||||
RobotConfig:
|
||||
_target_: ufactory_lerobot.robots.uf_mock_robot.UFMockRobotConfig
|
||||
id: "uf_mock_robot"
|
||||
control_space: "cartesian"
|
||||
gripper_type: 1
|
||||
state_offset_action: 3
|
||||
cameras:
|
||||
fisheye:
|
||||
_target_: ufactory_lerobot.cameras.umi_camera.configuration_umi.UmiCameraConfig
|
||||
serial_number: "250801DR48FP26001295"
|
||||
width: 1280
|
||||
height: 1280
|
||||
fps: 100
|
||||
|
||||
TeleoperatorConfig:
|
||||
_target_: ufactory_lerobot.teleoperators.umi_teleop.UmiTeleopConfig
|
||||
serial_number: "250801DR48FP26001295"
|
||||
use_vive_tracker: False
|
||||
vive_tracker_id: "LHR-2425BAD3"
|
||||
use_gripper: True
|
||||
tracker_to_robot_eef: [0, 0, 0, 0, 0, -90]
|
||||
robot_base_pose: [400, 0, 400, 180, 0, 0]
|
||||
|
||||
DatasetRecordConfig:
|
||||
_target_: lerobot.scripts.lerobot_record.DatasetRecordConfig
|
||||
# root of local repo: /home/<user_name>/.cache/huggingface/lerobot (default)
|
||||
root: "../lerobot_datas/record/ufactory/umi_datas"
|
||||
repo_id: "ufactory/umi_datas"
|
||||
single_task: "Pick up the purple grape and drop into the box on the left."
|
||||
fps: 60
|
||||
episode_time_s: 60 # max duration for one episode
|
||||
reset_time_s: 20 # (invalid) time for resetting env between episodes
|
||||
num_episodes: 100
|
||||
push_to_hub: False
|
||||
39
config/umi/xarm6_umi_record_config.yaml
Normal file
39
config/umi/xarm6_umi_record_config.yaml
Normal file
@ -0,0 +1,39 @@
|
||||
RobotConfig:
|
||||
_target_: ufactory_lerobot.robots.uf_robot.UFRobotConfig
|
||||
id: "uf_robot"
|
||||
robot_dof: 6
|
||||
control_space: "cartesian"
|
||||
robot_ip: "192.168.1.83"
|
||||
gripper_type: 2
|
||||
max_linear_velocity: 250
|
||||
start_joints: [0, -14.8, -52.6, 0, 67.4, 0]
|
||||
start_tcp_pose: [400, 0, 400, 180, 0, 0]
|
||||
cameras:
|
||||
fisheye:
|
||||
_target_: lerobot.cameras.opencv.configuration_opencv.OpenCVCameraConfig
|
||||
index_or_path: "/dev/v4l/by-id/usb-XVisio_Technology_XVisio_vSLAM_250801DR48FB26001396-video-index0"
|
||||
width: 1280
|
||||
height: 1280
|
||||
fps: 100
|
||||
fourcc: "YUYV"
|
||||
|
||||
TeleoperatorConfig:
|
||||
_target_: ufactory_lerobot.teleoperators.umi_teleop.UmiTeleopConfig
|
||||
serial_number: "250801DR48FP26001295"
|
||||
use_vive_tracker: False
|
||||
vive_tracker_id: "LHR-2425BAD3"
|
||||
use_gripper: True
|
||||
tracker_to_robot_eef: [0, 0, 0, 0, 0, -90]
|
||||
robot_base_pose: [400, 0, 400, 180, 0, 0]
|
||||
|
||||
DatasetRecordConfig:
|
||||
_target_: lerobot.scripts.lerobot_record.DatasetRecordConfig
|
||||
# root of local repo: /home/<user_name>/.cache/huggingface/lerobot (default)
|
||||
root: "../lerobot_datas/record/ufactory/xarm6_umi_datas"
|
||||
repo_id: "ufactory/xarm6_umi_datas"
|
||||
single_task: "Pick up the purple grape and drop into the box on the left."
|
||||
fps: 60
|
||||
episode_time_s: 60 # max duration for one episode
|
||||
reset_time_s: 20 # (invalid) time for resetting env between episodes
|
||||
num_episodes: 100
|
||||
push_to_hub: False
|
||||
@ -7,7 +7,7 @@ name = "ufactory_lerobot"
|
||||
version = "0.1.0"
|
||||
description = "UFACTORY 机械臂与 LeRobot 框架集成"
|
||||
readme = "README.md"
|
||||
license = { text = "MIT" }
|
||||
license = { text = "Apache-2.0" }
|
||||
requires-python = ">=3.10"
|
||||
authors = [
|
||||
{ name = "ufactory_lerobot Contributors" },
|
||||
@ -23,8 +23,31 @@ classifiers = [
|
||||
"Programming Language :: Python :: 3.12",
|
||||
"Topic :: Scientific/Engineering :: Artificial Intelligence",
|
||||
]
|
||||
dependencies = [
|
||||
"numpy>=1.24",
|
||||
"pyyaml",
|
||||
"lerobot==0.4.3",
|
||||
"xarm-python-sdk",
|
||||
"opencv-python",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
uf-robot-teleop = "ufactory_lerobot.scripts.uf_robot_teleop:main"
|
||||
uf-lerobot-record = "ufactory_lerobot.scripts.uf_lerobot_record:main"
|
||||
uf-lerobot-eval = "ufactory_lerobot.scripts.uf_lerobot_eval:main"
|
||||
uf-vive-calibrate = "ufactory_lerobot.scripts.vive_calibrate:main"
|
||||
|
||||
[project.optional-dependencies]
|
||||
# GELLO 遥操作
|
||||
gello = [
|
||||
"gello @ git+https://github.com/xArm-Developer/gello_software.git",
|
||||
"dynamixel-sdk @ git+https://github.com/ROBOTIS-GIT/DynamixelSDK.git#subdirectory=python",
|
||||
]
|
||||
# SpaceMouse 遥操作
|
||||
spacemouse = [
|
||||
"spnav @ git+https://github.com/kazoo-osaro/spnav",
|
||||
]
|
||||
# 开发工具
|
||||
dev = [
|
||||
"pytest>=8.0",
|
||||
"pytest-cov>=5.0",
|
||||
@ -32,22 +55,15 @@ dev = [
|
||||
"mypy>=1.10",
|
||||
"pre-commit>=3.7",
|
||||
]
|
||||
camera = [
|
||||
"opencv-python>=4.9",
|
||||
"pyrealsense2>=2.55",
|
||||
]
|
||||
lerobot = [
|
||||
"lerobot==0.4.3",
|
||||
]
|
||||
train = [
|
||||
"torch>=2.2",
|
||||
"torchvision>=0.17",
|
||||
"huggingface-hub>=0.25",
|
||||
"wandb>=0.17",
|
||||
]
|
||||
|
||||
[tool.setuptools]
|
||||
include-package-data = true
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
where = ["."]
|
||||
where = ["src"]
|
||||
|
||||
[tool.setuptools.package-data]
|
||||
"ufactory_lerobot.devices.umi.xvlib" = ["*.so", "*.so.*"]
|
||||
|
||||
[tool.ruff]
|
||||
line-length = 100
|
||||
|
||||
41
src/rules/81-vive.rules
Normal file
41
src/rules/81-vive.rules
Normal file
@ -0,0 +1,41 @@
|
||||
# HTC Vive HID Sensor naming and permissioning
|
||||
KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="0bb4", ATTRS{idProduct}=="2c87", TAG+="uaccess"
|
||||
KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="0bb4", ATTRS{idProduct}=="030e", TAG+="uaccess"
|
||||
KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="28de", ATTRS{idProduct}=="2101", TAG+="uaccess"
|
||||
KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="28de", ATTRS{idProduct}=="2000", TAG+="uaccess"
|
||||
KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="28de", ATTRS{idProduct}=="1043", TAG+="uaccess"
|
||||
KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="28de", ATTRS{idProduct}=="2050", TAG+="uaccess"
|
||||
KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="28de", ATTRS{idProduct}=="2011", TAG+="uaccess"
|
||||
KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="28de", ATTRS{idProduct}=="2012", TAG+="uaccess"
|
||||
KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="28de", ATTRS{idProduct}=="2022", TAG+="uaccess"
|
||||
KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="28de", ATTRS{idProduct}=="2300", TAG+="uaccess"
|
||||
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="0bb4", ATTRS{idProduct}=="2c87", TAG+="uaccess"
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="0bb4", ATTRS{idProduct}=="030e", TAG+="uaccess"
|
||||
# HTC Camera USB Node
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="114d", ATTRS{idProduct}=="8328", TAG+="uaccess"
|
||||
# HTC Mass Storage Node
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="114d", ATTRS{idProduct}=="8200", TAG+="uaccess"
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="114d", ATTRS{idProduct}=="8a12", TAG+="uaccess"
|
||||
|
||||
|
||||
#libsurvive
|
||||
# HTC HMD
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="0bb4", ATTRS{idProduct}=="2c87", MODE="0666"
|
||||
# HTC HMD Pro(?)
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="0bb4", ATTRS{idProduct}=="030e", MODE="0666"
|
||||
# Light input
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="28de", ATTRS{idProduct}=="2000", MODE="0666"
|
||||
# Watchman
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="28de", ATTRS{idProduct}=="2101", MODE="0666"
|
||||
# Watchman wired
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="28de", ATTRS{idProduct}=="2012", MODE="0666"
|
||||
# Tracker wired
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="28de", ATTRS{idProduct}=="2022", MODE="0666"
|
||||
# Tracker 2018 wired
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="28de", ATTRS{idProduct}=="2300", MODE="0666"
|
||||
|
||||
#KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="0bb4", ATTR{idProduct}=="2c87", MODE="0666" # HTC
|
||||
#KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="0bb4", ATTR{idProduct}=="030e", MODE="0666" # HTC
|
||||
#KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="28de", ATTR{idProduct}=="2000", MODE="0666" # Valve
|
||||
#KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="28de", ATTR{idProduct}=="2101", MODE="0666" # Valve
|
||||
81
src/rules/99-realsense-libusb.rules
Normal file
81
src/rules/99-realsense-libusb.rules
Normal file
@ -0,0 +1,81 @@
|
||||
##Version=1.1##
|
||||
# Device rules for Intel RealSense devices (R200, F200, SR300 LR200, ZR300, D400, L500, T200)
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0a80", MODE:="0666", GROUP:="plugdev", RUN+="/usr/local/bin/usb-R200-in_udev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0a66", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0aa3", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0aa2", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0aa5", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0abf", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0acb", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0ad0", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="04b4", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0ad1", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0ad2", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0ad3", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0ad4", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0ad5", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0ad6", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0af2", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0af6", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0afe", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0aff", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b00", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b01", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b03", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b07", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b0c", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b0d", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b3a", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b3d", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b48", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b49", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b4b", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b4d", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b52", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b5b", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b5c", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b64", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b68", MODE:="0666", GROUP:="plugdev"
|
||||
|
||||
# Intel RealSense recovery devices (DFU)
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0ab3", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0adb", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0adc", MODE:="0666", GROUP:="plugdev"
|
||||
SUBSYSTEMS=="usb", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b55", MODE:="0666", GROUP:="plugdev"
|
||||
|
||||
# Intel RealSense devices (Movidius, T265)
|
||||
SUBSYSTEMS=="usb", ENV{DEVTYPE}=="usb_device", ATTRS{idVendor}=="8087", ATTRS{idProduct}=="0af3", MODE="0666", GROUP="plugdev"
|
||||
SUBSYSTEMS=="usb", ENV{DEVTYPE}=="usb_device", ATTRS{idVendor}=="8087", ATTRS{idProduct}=="0b37", MODE="0666", GROUP="plugdev"
|
||||
SUBSYSTEMS=="usb", ENV{DEVTYPE}=="usb_device", ATTRS{idVendor}=="03e7", ATTRS{idProduct}=="2150", MODE="0666", GROUP="plugdev"
|
||||
|
||||
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0ad5", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
|
||||
DRIVER=="hid_sensor_custom", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0ad5", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
|
||||
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0af2", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
|
||||
DRIVER=="hid_sensor*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0af2", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
|
||||
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0afe", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
|
||||
DRIVER=="hid_sensor_custom", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0afe", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
|
||||
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0aff", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
|
||||
DRIVER=="hid_sensor_custom", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0aff", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
|
||||
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b00", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
|
||||
DRIVER=="hid_sensor_custom", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b00", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
|
||||
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b01", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
|
||||
DRIVER=="hid_sensor_custom", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b01", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
|
||||
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b3a", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
|
||||
DRIVER=="hid_sensor*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b3a", RUN+="/bin/sh -c ' chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
|
||||
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b3d", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
|
||||
DRIVER=="hid_sensor*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b3d", RUN+="/bin/sh -c ' chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
|
||||
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b4b", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
|
||||
DRIVER=="hid_sensor*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b4b", RUN+="/bin/sh -c ' chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
|
||||
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b4d", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
|
||||
DRIVER=="hid_sensor*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b4d", RUN+="/bin/sh -c ' chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
|
||||
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b5b", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
|
||||
DRIVER=="hid_sensor*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b5b", RUN+="/bin/sh -c ' chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
|
||||
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b5c", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
|
||||
DRIVER=="hid_sensor*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b5c", RUN+="/bin/sh -c ' chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
|
||||
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b64", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
|
||||
DRIVER=="hid_sensor*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b64", RUN+="/bin/sh -c ' chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
|
||||
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b68", MODE:="0777", GROUP:="plugdev", RUN+="/bin/sh -c 'chmod -R 0777 /sys/%p'"
|
||||
DRIVER=="hid_sensor*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0b68", RUN+="/bin/sh -c ' chmod -R 0777 /sys/%p && chmod 0777 /dev/%k'"
|
||||
|
||||
# For products with motion_module, if (kernels is 4.15 and up) and (device name is "accel_3d") wait, in another process, until (enable flag is set to 1 or 200 mSec passed) and then set it to 0.
|
||||
KERNEL=="iio*", ATTRS{idVendor}=="8086", ATTRS{idProduct}=="0ad5|0afe|0aff|0b00|0b01|0b3a|0b3d|0b64|0b68", RUN+="/bin/sh -c '(major=`uname -r | cut -d \".\" -f1` && minor=`uname -r | cut -d \".\" -f2` && (([ $major -eq 4 ] && [ $minor -ge 15 ]) || [ $major -ge 5 ])) && (enamefile=/sys/%p/name && [ `cat $enamefile` = \"accel_3d\" ]) && enfile=/sys/%p/buffer/enable && echo \"COUNTER=0; while [ \$COUNTER -lt 20 ] && grep -q 0 $enfile; do sleep 0.01; COUNTER=\$((COUNTER+1)); done && echo 0 > $enfile\" | at now'"
|
||||
2
src/rules/99-xvisio.rules
Normal file
2
src/rules/99-xvisio.rules
Normal file
@ -0,0 +1,2 @@
|
||||
SUBSYSTEM=="usb", ATTR{idVendor}=="040e", MODE="0666", GROUP="plugdev"
|
||||
|
||||
1
src/rules/sensor_fisheye.rules
Normal file
1
src/rules/sensor_fisheye.rules
Normal file
@ -0,0 +1 @@
|
||||
KERNEL=="video*", ATTRS{idVendor}=="1bcf", ATTRS{idProduct}=="2cd1", MODE:="0777", SYMLINK+="video50"
|
||||
1
src/rules/sensor_serial.rules
Normal file
1
src/rules/sensor_serial.rules
Normal file
@ -0,0 +1 @@
|
||||
KERNEL=="ttyUSB*", ATTRS{idVendor}=="1a86", ATTRS{idProduct}=="7522", MODE:="0777", SYMLINK+="ttyUSB50"
|
||||
@ -18,13 +18,8 @@ Provides the RealSenseCamera class for capturing frames from Intel RealSense cam
|
||||
|
||||
import logging
|
||||
import time
|
||||
from threading import Event, Lock, Thread
|
||||
from typing import Any
|
||||
|
||||
import cv2 # type: ignore # TODO: add type stubs for OpenCV
|
||||
import numpy as np # type: ignore # TODO: add type stubs for numpy
|
||||
from numpy.typing import NDArray # type: ignore # TODO: add type stubs for numpy.typing
|
||||
from lerobot.utils.errors import DeviceAlreadyConnectedError, DeviceNotConnectedError
|
||||
from lerobot.cameras.camera import Camera
|
||||
from ufactory_lerobot.devices.umi.xvlib import XVLib
|
||||
from .configuration_umi import UmiCameraConfig
|
||||
@ -60,11 +55,6 @@ class UmiCamera(Camera):
|
||||
self.last_frame = None
|
||||
self.xvlib = XVLib(self.serial_number)
|
||||
self.xvlib.xv_color_camera_init()
|
||||
self.xvlib.xv_set_color_camera_framerate(self.config.fps)
|
||||
self.frame_lock = Lock()
|
||||
self.new_frame_event: Event = Event()
|
||||
self.thread = Thread(target=self._read_loop, daemon=True)
|
||||
self.thread.start()
|
||||
|
||||
def __str__(self) -> str:
|
||||
return f"{self.__class__.__name__}({self.serial_number})"
|
||||
@ -83,18 +73,6 @@ class UmiCamera(Camera):
|
||||
start_time = time.monotonic()
|
||||
while time.monotonic() - start_time < self.warmup_s:
|
||||
time.sleep(0.1)
|
||||
|
||||
def _read_loop(self):
|
||||
while True:
|
||||
try:
|
||||
frame = self.read()
|
||||
if frame is not None:
|
||||
with self.frame_lock:
|
||||
self.last_frame = frame
|
||||
self.new_frame_event.set()
|
||||
except Exception as e:
|
||||
print('Read Frame Ex: {}'.format(e))
|
||||
time.sleep(0.01)
|
||||
|
||||
def read(self, color_mode = None):
|
||||
ret, img_data = self.xvlib.xv_get_color_image_rgb_data()
|
||||
@ -114,16 +92,11 @@ class UmiCamera(Camera):
|
||||
return frame
|
||||
|
||||
def async_read(self, timeout_ms: float = 200):
|
||||
if not self.new_frame_event.wait(timeout=timeout_ms / 1000.0):
|
||||
thread_alive = self.thread is not None and self.thread.is_alive()
|
||||
raise TimeoutError(
|
||||
f"Timed out waiting for frame from camera {self} after {timeout_ms} ms. "
|
||||
f"Read thread alive: {thread_alive}."
|
||||
)
|
||||
|
||||
with self.frame_lock:
|
||||
frame = self.read()
|
||||
if frame is not None:
|
||||
self.last_frame = frame
|
||||
else:
|
||||
frame = self.last_frame
|
||||
self.new_frame_event.clear()
|
||||
return frame
|
||||
|
||||
def disconnect(self) -> None:
|
||||
|
||||
@ -24,11 +24,10 @@ class UFMockRobot(Robot):
|
||||
super().__init__(config)
|
||||
self.prefix = '' if not prefix else f"{prefix}."
|
||||
self.config = config
|
||||
self._dof = config.robot_dof
|
||||
if self._dof == None or (not self._dof in (5, 6, 7)):
|
||||
raise ValueError(f"Please specify the correct DOF uf_robot!, got {self._dof}")
|
||||
|
||||
self._dof = config.robot_dof
|
||||
self._control_space = self.config.control_space
|
||||
if self._control_space == "joint" and (self._dof == None or (not self._dof in (5, 6, 7))):
|
||||
raise ValueError(f"Please specify the correct DOF uf_robot!, got {self._dof}")
|
||||
self._jnt_obs_has_vel = config.observe_joint_vel if self._control_space == "joint" else False
|
||||
self._is_connected = False
|
||||
self._is_calibrated =True
|
||||
|
||||
@ -1,22 +1,12 @@
|
||||
from dataclasses import dataclass, field
|
||||
from lerobot.cameras import CameraConfig
|
||||
from lerobot.cameras.opencv import OpenCVCameraConfig
|
||||
from lerobot.robots import RobotConfig
|
||||
|
||||
@RobotConfig.register_subclass("uf::mock_robot")
|
||||
@dataclass
|
||||
class UFMockRobotConfig(RobotConfig):
|
||||
# cameras
|
||||
cameras: dict[str, CameraConfig] = field(
|
||||
default_factory=lambda: {
|
||||
"fisheye": OpenCVCameraConfig(
|
||||
index_or_path=6,
|
||||
width=640,
|
||||
height=480,
|
||||
fps=30,
|
||||
fourcc="MJPG"
|
||||
)
|
||||
}
|
||||
default_factory=lambda: {}
|
||||
)
|
||||
|
||||
robot_dof: int | None = None # Set it correctly if controlling in joint space!
|
||||
|
||||
@ -1,31 +1,14 @@
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Tuple
|
||||
from lerobot.cameras import CameraConfig
|
||||
from lerobot.cameras.realsense import RealSenseCameraConfig
|
||||
from lerobot.robots import RobotConfig
|
||||
|
||||
@RobotConfig.register_subclass("uf::robot")
|
||||
@dataclass
|
||||
class UFRobotConfig(RobotConfig):
|
||||
# cameras
|
||||
cameras: dict[str, CameraConfig] = field(
|
||||
default_factory=lambda: {
|
||||
"overhead": RealSenseCameraConfig(
|
||||
serial_number_or_name="Intel RealSense D435I",
|
||||
fps=30,
|
||||
width=640, # 1280
|
||||
height=480, # 720
|
||||
# rotation=90,
|
||||
),
|
||||
"tool": RealSenseCameraConfig(
|
||||
serial_number_or_name="Intel RealSense D435",
|
||||
fps=30,
|
||||
width=640, # 1280
|
||||
height=480, # 720
|
||||
),
|
||||
}
|
||||
default_factory=lambda: {}
|
||||
)
|
||||
|
||||
robot_ip: str = "192.168.1.127"
|
||||
robot_dof: int | None = None # Set it correctly if controlling in joint space!
|
||||
control_space: str = "joint"
|
||||
|
||||
@ -276,9 +276,9 @@ def record(cfg: RecordConfig) -> LeRobotDataset:
|
||||
try:
|
||||
if key == keyboard.Key.enter:
|
||||
if not is_recorded:
|
||||
print('[HELP] <ESC>: EXIT, <SPACE>: START, <LEFT ARROW>: RESET, <RIGH ARROW>: SAVE')
|
||||
print('⌨ [ESC] Exit [Space] Start [←] Reset [→] Save')
|
||||
else:
|
||||
print('[HELP] <ESC>: EXIT, <LEFT ARROW>: RESET, <RIGH ARROW>: SAVE')
|
||||
print('⌨ [ESC] Exit [←] Reset [→] Save')
|
||||
# is_recorded = True
|
||||
except Exception as e:
|
||||
print(f"Error handling key release: {e}")
|
||||
@ -287,9 +287,9 @@ def record(cfg: RecordConfig) -> LeRobotDataset:
|
||||
|
||||
listener, events = init_keyboard_listener(events=events, on_press=on_press, on_release=on_release)
|
||||
print("\n********** Episode Record Loop Start **********")
|
||||
print('[HELP] <ESC>: EXIT, <SPACE>: START, <LEFT ARROW>: RESET, <RIGH ARROW>: SAVE')
|
||||
print('⌨ [ESC] Exit [Space] Start [←] Reset [→] Save')
|
||||
else:
|
||||
input('[HELP] Enter to to start record >>> ')
|
||||
input('⌨ Press Enter to start record >>> ')
|
||||
if is_uf_teleop:
|
||||
teleop.set_teleop_enabled(True)
|
||||
is_recorded = True
|
||||
@ -312,7 +312,7 @@ def record(cfg: RecordConfig) -> LeRobotDataset:
|
||||
if events["rerecord_episode"]:
|
||||
events["rerecord_episode"] = False
|
||||
events["exit_early"] = False
|
||||
input('\nPress Enter to regenerate random target location >>>>> ')
|
||||
input('\n⌨ Press Enter to regenerate random target location >>>>> ')
|
||||
continue
|
||||
if events["stop_recording"]:
|
||||
continue
|
||||
@ -357,9 +357,9 @@ def record(cfg: RecordConfig) -> LeRobotDataset:
|
||||
dataset.clear_episode_buffer()
|
||||
is_recorded = False
|
||||
if is_evt:
|
||||
print('[HELP] <ESC>: EXIT, <SPACE>: START, <LEFT ARROW>: RESET, <RIGH ARROW>: SAVE')
|
||||
print('⌨ [ESC] Exit [Space] Start [←] Reset [→] Save')
|
||||
else:
|
||||
input('\nPress Enter to rerecord this episode >>>>> ')
|
||||
input('\n⌨ Press Enter to rerecord this episode >>>>> ')
|
||||
is_recorded = True
|
||||
continue
|
||||
|
||||
@ -368,12 +368,13 @@ def record(cfg: RecordConfig) -> LeRobotDataset:
|
||||
if is_uf_teleop:
|
||||
teleop.set_teleop_enabled(False)
|
||||
dataset.save_episode()
|
||||
log_say(f"[Finish] Save episode {dataset.num_episodes}", cfg.play_sounds)
|
||||
recorded_episodes += 1
|
||||
is_recorded = False
|
||||
if is_evt:
|
||||
print('[HELP] <ESC>: EXIT, <SPACE>: START, <LEFT ARROW>: RESET, <RIGH ARROW>: SAVE')
|
||||
print('⌨ [ESC] Exit [Space] Start [←] Reset [→] Save')
|
||||
else:
|
||||
input('Press Enter to record at the next episode >>>>> ')
|
||||
input('⌨ Press Enter to record at the next episode >>>>> ')
|
||||
is_recorded = True
|
||||
|
||||
print("\n********** Episode Record Loop Exit **********")
|
||||
|
||||
@ -85,11 +85,11 @@ def teleop_loop(cfg: TeleopConfig):
|
||||
if key == keyboard.Key.enter:
|
||||
if is_paused:
|
||||
if is_reset:
|
||||
print('[HELP] <ESC>: EXIT, <SPACE>: RESET AND START, <LEFT ARROW>: RESET')
|
||||
print('⌨ [ESC] Exit [Space] Reset / Start [←] Reset')
|
||||
else:
|
||||
print('[HELP] <ESC>: EXIT, <SPACE>: START, <LEFT ARROW>: RESET')
|
||||
print('⌨ [ESC] Exit [Space] Start [←] Reset')
|
||||
else:
|
||||
print('[HELP] <ESC>: EXIT, <SPACE>: PAUSE, <LEFT ARROW>: RESET')
|
||||
print('⌨ [ESC] Exit [Space] Pause [←] Pause / Reset')
|
||||
except Exception as e:
|
||||
print(f"Error handling key release: {e}")
|
||||
if key in key_dict:
|
||||
@ -97,9 +97,9 @@ def teleop_loop(cfg: TeleopConfig):
|
||||
|
||||
listener, events = init_keyboard_listener(events=events, on_press=on_press, on_release=on_release)
|
||||
print("\n********** Teleop Control Loop Start **********")
|
||||
print('[HELP] <ESC>: EXIT, <SPACE>: START, <LEFT ARROW>: RESET')
|
||||
print('⌨ [ESC] Exit [Space] Start [←] Reset')
|
||||
else:
|
||||
input('[HELP] Enter to control robot with teleop >>> ')
|
||||
input('⌨ Press Enter to start teleop >>> ')
|
||||
if is_uf_teleop:
|
||||
obs = robot.get_observation()
|
||||
teleop.set_teleop_enabled(True, obs)
|
||||
@ -121,7 +121,7 @@ def teleop_loop(cfg: TeleopConfig):
|
||||
is_paused = True
|
||||
if is_uf_teleop:
|
||||
teleop.set_teleop_enabled(False)
|
||||
print('[HELP] <ESC>: EXIT, <SPACE>: RESET AND START, <LEFT ARROW>: RESET')
|
||||
print('⌨ [ESC] Exit [Space] Reset / Start [←] Reset')
|
||||
elif not key_dict[keyboard.Key.left] and key_left_pressed:
|
||||
key_left_pressed = False
|
||||
|
||||
@ -132,7 +132,7 @@ def teleop_loop(cfg: TeleopConfig):
|
||||
if is_uf_teleop:
|
||||
teleop.set_teleop_enabled(False)
|
||||
# print('========== Teleop is paused ==========')
|
||||
print('[HELP] <ESC>: EXIT, <SPACE>: START, <LEFT ARROW>: RESET')
|
||||
print('⌨ [ESC] Exit [Space] Start [←] Reset')
|
||||
else:
|
||||
if is_reset:
|
||||
is_reset = False
|
||||
@ -141,7 +141,7 @@ def teleop_loop(cfg: TeleopConfig):
|
||||
if is_uf_teleop:
|
||||
obs = robot.get_observation()
|
||||
teleop.set_teleop_enabled(True, obs)
|
||||
print('[HELP] <ESC>: EXIT, <SPACE>: PAUSE, <LEFT ARROW>: RESET')
|
||||
print('⌨ [ESC] Exit [Space] Pause [←] Reset')
|
||||
continue
|
||||
elif not key_dict[keyboard.Key.space] and key_space_pressed:
|
||||
key_space_pressed = False
|
||||
|
||||
@ -24,7 +24,7 @@ from lerobot.teleoperators import TeleoperatorConfig
|
||||
class SpaceMouseTeleopConfig(TeleoperatorConfig):
|
||||
# Port to connect to the arm
|
||||
max_value: int = 300
|
||||
deadzone: tuple = (0,0,0,0,0,0)
|
||||
deadzone: tuple = (0, 0, 0, 0, 0, 0)
|
||||
use_gripper: bool = False
|
||||
frequency: int = 10 # hz
|
||||
max_pos_speed: int = 250 # mm/s
|
||||
|
||||
@ -86,6 +86,10 @@ class UmiTeleop(UFBaseTeleop):
|
||||
def connect(self, calibrate: bool = False) -> None:
|
||||
self.tracker = ViveTracker() if self.config.use_vive_tracker else None
|
||||
self.xvlib = XVLib(self.config.serial_number, not self.config.use_vive_tracker, self.config.use_gripper)
|
||||
if not self.config.use_vive_tracker:
|
||||
self.xvlib.xv_slam_init()
|
||||
if self.config.use_gripper:
|
||||
self.xvlib.xv_clamp_stream_init()
|
||||
self._is_connected = True
|
||||
|
||||
def disconnect(self):
|
||||
|
||||
Loading…
Reference in New Issue
Block a user