Compare commits

...

9 Commits

Author SHA1 Message Date
Ivelin Ivanov dede6fe2ca
Merge 1a4f0983b7 into 0f5f7ac780 2025-04-17 15:03:03 +02:00
HUANG TZU-CHUN 0f5f7ac780
Fix broken links in `examples/4_train_policy_with_script.md` (#697) 2025-04-17 14:59:43 +02:00
ivelin 1a4f0983b7 docs: add calibrate troubleshooting sub-section for shaft out of position problem
Signed-off-by: ivelin <ivelin117@gmail.com>
2025-01-28 08:33:13 -06:00
ivelin 2920372476 docs: explain how to determine which cable is disconnected
Signed-off-by: ivelin <ivelin117@gmail.com>
2025-01-27 15:17:19 -06:00
ivelin 843c099697 fix: remove work in progress files from branch
Signed-off-by: ivelin <ivelin117@gmail.com>
2025-01-27 15:09:51 -06:00
ivelin 0825e2e14c feat: add scan_motors.py tool for troubleshooting
Signed-off-by: ivelin <ivelin117@gmail.com>
2025-01-27 14:54:44 -06:00
ivelin 8c0ad754db Merge branch 'main' of https://github.com/huggingface/lerobot into feat/scan_motors 2025-01-27 14:32:35 -06:00
ivelin 94ab721001 Merge branch 'main' of https://github.com/huggingface/lerobot into smolvlm-pusht 2025-01-22 14:07:03 -06:00
ivelin 4fb01aef68 feat: smolvlm model with pusht train notebook
Signed-off-by: ivelin <ivelin117@gmail.com>
2024-12-23 15:01:43 -06:00
4 changed files with 165 additions and 7 deletions

3
.gitignore vendored
View File

@ -171,3 +171,6 @@ dmypy.json
# Cython debug symbols # Cython debug symbols
cython_debug/ cython_debug/
# lerobot calibration cache
.cache

View File

@ -443,10 +443,54 @@ For the leader configuration, perform **Steps 123**. Make sure that you remov
*Assembly complete proceed to calibration.* *Assembly complete proceed to calibration.*
### c. Troubleshooting
Sometimes during assembly, the cables connecting the motors or the power adapter cable may be accidentally disconnected. To be sure that the motors are properly connected and functioning after assembly, use the scan_motors tool to test each arm.
The output should look similar to the example below. If the list of motor IDs is shorter than 6, there is probably a poorly connected cable or a motor is misconfigured. For example if only motor IDs [1,3] show up, that indicates that the serial cable between motor 3 and 4 is disconnected.
```bash
lerobot$ python lerobot/scripts/scan_motors.py --port /dev/ttyACM0 --brand feetech --model sts3215
Connected on port /dev/ttyACM0
Scanning all baudrates and motor indices
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 43.11it/s]
6 motor ID(s) detected for baud rate 1000000. Motor IDs: [1, 2, 3, 4, 5, 6].
Present Position 1997
Offset 0
Present Position 2098
Offset 0
Present Position 1124
Offset 0
Present Position 253
Offset 0
Present Position 3769
Offset 0
Present Position 926
Offset 0
Setting bus baud rate to 128000. Previously 1000000.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 14.42it/s]
Setting bus baud rate to 500000. Previously 128000.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 14.50it/s]
Setting bus baud rate to 115200. Previously 500000.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 14.40it/s]
Setting bus baud rate to 57600. Previously 115200.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 14.08it/s]
Setting bus baud rate to 38400. Previously 57600.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 13.88it/s]
Setting bus baud rate to 19200. Previously 38400.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 13.24it/s]
Setting bus baud rate to 250000. Previously 19200.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 14.47it/s]
Scan finished.
Disconnected from motor bus.
```
## E. Calibrate ## E. Calibrate
Next, you'll need to calibrate your SO-100 robot to ensure that the leader and follower arms have the same position values when they are in the same physical position. This calibration is essential because it allows a neural network trained on one SO-100 robot to work on another. Next, you'll need to calibrate your SO-100 robot to ensure that the leader and follower arms have the same position values when they are in the same physical position. This calibration is essential because it allows a neural network trained on one SO-100 robot to work on another.
#### a. Manual calibration of follower arm #### a. Manual calibration of follower arm
> [!IMPORTANT] > [!IMPORTANT]
@ -483,6 +527,20 @@ python lerobot/scripts/control_robot.py \
--control.arms='["main_leader"]' --control.arms='["main_leader"]'
``` ```
### c. Troubleshooting
Another known issue during calibration is related to the positioning of the motor shafts. The error message looks like this:
```
Calibration is done! Saving calibration file '.cache/calibration/moss/main_leader.json'
Activating torque on main follower arm.
Wrong motor position range detected for gripper. Expected to be in nominal range of [0, 100] % (a full linear translation), with a maximum range of [-10, 110] % to account for some imprecision during calibration, but present value is 143.7950897216797 %. This might be due to a cable connection issue creating an artificial jump in motor values. You need to recalibrate ...
```
When all joints are mounted and the robot arm is in resting position (as shown in the calibration photos) each motor shaft should be in approximately middle state ~2048 of its full [0,4096] range. You can use the `scan_motors.py` tool again as shown above to check Present Position for each motor. Motors that are too far off the mid-value in rested arm position need to be repositioned by unscrewing the attached 3D printed part and rotating it such that when screwed back on and returned to resting position, its Present Position reading is about mid-range.
## F. Teleoperate ## F. Teleoperate
**Simple teleop** **Simple teleop**

View File

@ -4,7 +4,7 @@ This tutorial will explain the training script, how to use it, and particularly
## The training script ## The training script
LeRobot offers a training script at [`lerobot/scripts/train.py`](../../lerobot/scripts/train.py). At a high level it does the following: LeRobot offers a training script at [`lerobot/scripts/train.py`](../lerobot/scripts/train.py). At a high level it does the following:
- Initialize/load a configuration for the following steps using. - Initialize/load a configuration for the following steps using.
- Instantiates a dataset. - Instantiates a dataset.
@ -21,7 +21,7 @@ In the training script, the main function `train` expects a `TrainPipelineConfig
def train(cfg: TrainPipelineConfig): def train(cfg: TrainPipelineConfig):
``` ```
You can inspect the `TrainPipelineConfig` defined in [`lerobot/configs/train.py`](../../lerobot/configs/train.py) (which is heavily commented and meant to be a reference to understand any option) You can inspect the `TrainPipelineConfig` defined in [`lerobot/configs/train.py`](../lerobot/configs/train.py) (which is heavily commented and meant to be a reference to understand any option)
When running the script, inputs for the command line are parsed thanks to the `@parser.wrap()` decorator and an instance of this class is automatically generated. Under the hood, this is done with [Draccus](https://github.com/dlwh/draccus) which is a tool dedicated for this purpose. If you're familiar with Hydra, Draccus can similarly load configurations from config files (.json, .yaml) and also override their values through command line inputs. Unlike Hydra, these configurations are pre-defined in the code through dataclasses rather than being defined entirely in config files. This allows for more rigorous serialization/deserialization, typing, and to manipulate configuration as objects directly in the code and not as dictionaries or namespaces (which enables nice features in an IDE such as autocomplete, jump-to-def, etc.) When running the script, inputs for the command line are parsed thanks to the `@parser.wrap()` decorator and an instance of this class is automatically generated. Under the hood, this is done with [Draccus](https://github.com/dlwh/draccus) which is a tool dedicated for this purpose. If you're familiar with Hydra, Draccus can similarly load configurations from config files (.json, .yaml) and also override their values through command line inputs. Unlike Hydra, these configurations are pre-defined in the code through dataclasses rather than being defined entirely in config files. This allows for more rigorous serialization/deserialization, typing, and to manipulate configuration as objects directly in the code and not as dictionaries or namespaces (which enables nice features in an IDE such as autocomplete, jump-to-def, etc.)
@ -50,7 +50,7 @@ By default, every field takes its default value specified in the dataclass. If a
## Specifying values from the CLI ## Specifying values from the CLI
Let's say that we want to train [Diffusion Policy](../../lerobot/common/policies/diffusion) on the [pusht](https://huggingface.co/datasets/lerobot/pusht) dataset, using the [gym_pusht](https://github.com/huggingface/gym-pusht) environment for evaluation. The command to do so would look like this: Let's say that we want to train [Diffusion Policy](../lerobot/common/policies/diffusion) on the [pusht](https://huggingface.co/datasets/lerobot/pusht) dataset, using the [gym_pusht](https://github.com/huggingface/gym-pusht) environment for evaluation. The command to do so would look like this:
```bash ```bash
python lerobot/scripts/train.py \ python lerobot/scripts/train.py \
--dataset.repo_id=lerobot/pusht \ --dataset.repo_id=lerobot/pusht \
@ -60,10 +60,10 @@ python lerobot/scripts/train.py \
Let's break this down: Let's break this down:
- To specify the dataset, we just need to specify its `repo_id` on the hub which is the only required argument in the `DatasetConfig`. The rest of the fields have default values and in this case we are fine with those so we can just add the option `--dataset.repo_id=lerobot/pusht`. - To specify the dataset, we just need to specify its `repo_id` on the hub which is the only required argument in the `DatasetConfig`. The rest of the fields have default values and in this case we are fine with those so we can just add the option `--dataset.repo_id=lerobot/pusht`.
- To specify the policy, we can just select diffusion policy using `--policy` appended with `.type`. Here, `.type` is a special argument which allows us to select config classes inheriting from `draccus.ChoiceRegistry` and that have been decorated with the `register_subclass()` method. To have a better explanation of this feature, have a look at this [Draccus demo](https://github.com/dlwh/draccus?tab=readme-ov-file#more-flexible-configuration-with-choice-types). In our code, we use this mechanism mainly to select policies, environments, robots, and some other components like optimizers. The policies available to select are located in [lerobot/common/policies](../../lerobot/common/policies) - To specify the policy, we can just select diffusion policy using `--policy` appended with `.type`. Here, `.type` is a special argument which allows us to select config classes inheriting from `draccus.ChoiceRegistry` and that have been decorated with the `register_subclass()` method. To have a better explanation of this feature, have a look at this [Draccus demo](https://github.com/dlwh/draccus?tab=readme-ov-file#more-flexible-configuration-with-choice-types). In our code, we use this mechanism mainly to select policies, environments, robots, and some other components like optimizers. The policies available to select are located in [lerobot/common/policies](../lerobot/common/policies)
- Similarly, we select the environment with `--env.type=pusht`. The different environment configs are available in [`lerobot/common/envs/configs.py`](../../lerobot/common/envs/configs.py) - Similarly, we select the environment with `--env.type=pusht`. The different environment configs are available in [`lerobot/common/envs/configs.py`](../lerobot/common/envs/configs.py)
Let's see another example. Let's say you've been training [ACT](../../lerobot/common/policies/act) on [lerobot/aloha_sim_insertion_human](https://huggingface.co/datasets/lerobot/aloha_sim_insertion_human) using the [gym-aloha](https://github.com/huggingface/gym-aloha) environment for evaluation with: Let's see another example. Let's say you've been training [ACT](../lerobot/common/policies/act) on [lerobot/aloha_sim_insertion_human](https://huggingface.co/datasets/lerobot/aloha_sim_insertion_human) using the [gym-aloha](https://github.com/huggingface/gym-aloha) environment for evaluation with:
```bash ```bash
python lerobot/scripts/train.py \ python lerobot/scripts/train.py \
--policy.type=act \ --policy.type=act \
@ -74,7 +74,7 @@ python lerobot/scripts/train.py \
> Notice we added `--output_dir` to explicitly tell where to write outputs from this run (checkpoints, training state, configs etc.). This is not mandatory and if you don't specify it, a default directory will be created from the current date and time, env.type and policy.type. This will typically look like `outputs/train/2025-01-24/16-10-05_aloha_act`. > Notice we added `--output_dir` to explicitly tell where to write outputs from this run (checkpoints, training state, configs etc.). This is not mandatory and if you don't specify it, a default directory will be created from the current date and time, env.type and policy.type. This will typically look like `outputs/train/2025-01-24/16-10-05_aloha_act`.
We now want to train a different policy for aloha on another task. We'll change the dataset and use [lerobot/aloha_sim_transfer_cube_human](https://huggingface.co/datasets/lerobot/aloha_sim_transfer_cube_human) instead. Of course, we also need to change the task of the environment as well to match this other task. We now want to train a different policy for aloha on another task. We'll change the dataset and use [lerobot/aloha_sim_transfer_cube_human](https://huggingface.co/datasets/lerobot/aloha_sim_transfer_cube_human) instead. Of course, we also need to change the task of the environment as well to match this other task.
Looking at the [`AlohaEnv`](../../lerobot/common/envs/configs.py) config, the task is `"AlohaInsertion-v0"` by default, which corresponds to the task we trained on in the command above. The [gym-aloha](https://github.com/huggingface/gym-aloha?tab=readme-ov-file#description) environment also has the `AlohaTransferCube-v0` task which corresponds to this other task we want to train on. Putting this together, we can train this new policy on this different task using: Looking at the [`AlohaEnv`](../lerobot/common/envs/configs.py) config, the task is `"AlohaInsertion-v0"` by default, which corresponds to the task we trained on in the command above. The [gym-aloha](https://github.com/huggingface/gym-aloha?tab=readme-ov-file#description) environment also has the `AlohaTransferCube-v0` task which corresponds to this other task we want to train on. Putting this together, we can train this new policy on this different task using:
```bash ```bash
python lerobot/scripts/train.py \ python lerobot/scripts/train.py \
--policy.type=act \ --policy.type=act \

View File

@ -0,0 +1,97 @@
"""
This script is helpful to diagnose issues with motors during calibration or later operation.
It scans all motors on a given port and displays their settings.
Example of usage:
```bash
python lerobot/scripts/scan_motors.py \
--port /dev/tty.usbmodem585A0080521 \
--brand feetech \
--model sts3215 \
```
"""
import argparse
import time
def scan_motors(port, brand, model):
if brand == "feetech":
from lerobot.common.robot_devices.motors.feetech import MODEL_BAUDRATE_TABLE
from lerobot.common.robot_devices.motors.feetech import (
SCS_SERIES_BAUDRATE_TABLE as SERIES_BAUDRATE_TABLE,
)
from lerobot.common.robot_devices.motors.feetech import FeetechMotorsBus as MotorsBusClass
elif brand == "dynamixel":
from lerobot.common.robot_devices.motors.dynamixel import MODEL_BAUDRATE_TABLE
from lerobot.common.robot_devices.motors.dynamixel import (
X_SERIES_BAUDRATE_TABLE as SERIES_BAUDRATE_TABLE,
)
from lerobot.common.robot_devices.motors.dynamixel import DynamixelMotorsBus as MotorsBusClass
else:
raise ValueError(
f"Currently we do not support this motor brand: {brand}. We currently support feetech and dynamixel motors."
)
# Check if the provided model exists in the model_baud_rate_table
if model not in MODEL_BAUDRATE_TABLE:
raise ValueError(
f"Invalid model '{model}' for brand '{brand}'. Supported models: {list(MODEL_BAUDRATE_TABLE.keys())}"
)
# Setup motor names, indices, and models
motor_name = "motor"
motor_index_arbitrary = -1 # Use an arbitrary out of range motor ID
motor_model = model # Use the motor model passed via argument
# Initialize the MotorBus with the correct port and motor configurations
motor_bus = MotorsBusClass(port=port, motors={motor_name: (motor_index_arbitrary, motor_model)})
# Try to connect to the motor bus and handle any connection-specific errors
try:
motor_bus.connect()
print(f"Connected on port {motor_bus.port}")
except OSError as e:
print(f"Error occurred when connecting to the motor bus: {e}")
return
# Motor bus is connected, proceed with the rest of the operations
try:
print("Scanning all baudrates and motor indices")
all_baudrates = set(SERIES_BAUDRATE_TABLE.values())
motors_detected = False
for baudrate in all_baudrates:
motor_bus.set_bus_baudrate(baudrate)
present_ids = motor_bus.find_motor_indices(list(range(1, 10)))
if len(present_ids) > 0:
print(f"{len(present_ids)} motor ID(s) detected for baud rate {baudrate}. Motor IDs: {present_ids}.")
motors_detected = True
for motor_idx in present_ids:
present_idx = motor_bus.read_with_motor_ids(motor_bus.motor_models, motor_idx, "ID", num_retry=2)
if present_idx != motor_idx:
raise OSError(f"Failed to access motor index {motor_idx}.")
if brand == "feetech":
print("Present Position", motor_bus.read_with_motor_ids(motor_bus.motor_models, motor_idx, "Present_Position", num_retry=2))
print("Offset", motor_bus.read_with_motor_ids(motor_bus.motor_models, motor_idx, "Offset", num_retry=2))
if not motors_detected:
print("No motors detected.")
print("Scan finished.")
finally:
motor_bus.disconnect()
print("Disconnected from motor bus.")
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("--port", type=str, required=True, help="Motors bus port (e.g. dynamixel,feetech)")
parser.add_argument("--brand", type=str, required=True, help="Motor brand (e.g. dynamixel,feetech)")
parser.add_argument("--model", type=str, required=True, help="Motor model (e.g. xl330-m077,sts3215)")
args = parser.parse_args()
scan_motors(args.port, args.brand, args.model)