Minor Updates and Cleanup (#13)

* Pin Torchvision

* Readme Cleanup

* Concurrent Resetting of Arms

* Update Bimanual to Stationary

examples/12_use_trossen_ai.md

@@ -44,36 +44,12 @@ pip uninstall -y opencv-python
 conda install -y -c conda-forge "opencv>=4.10.0"
 ```
 
-## Troubleshooting
-
-If you encounter the following error.
-
-```bash
-ImportError: /xxx/xxx/xxx/envs/lerobot/lib/python3.10/site-packages/cv2/python-3.10/../../../.././libtiff.so.6: undefined symbol: jpeg12_write_raw_data, version LIBJPEG_8.0
-```
-
-The below are the 2 known system specific solutions
-
-### System 76 Serval Workstation (serw13) & Dell Precision 7670
-
-```bash
-    conda install pytorch==2.5.1=cpu_openblas_py310ha613aac_2 -y
-    conda install torchvision==0.21.0 -y
-```
-
-### HP
-
-```bash
-    pip install torch==2.5.1+cu121 torchvision==0.20.1+cu121 torchaudio==2.5.1+cu121 --index-url https://download.pytorch.org/whl/cu121
-```
-
-
 ## Teleoperate
 
 By running the following code, you can start your first **SAFE** teleoperation:
 ```bash
 python lerobot/scripts/control_robot.py \
-  --robot.type=trossen_ai_bimanual \
+  --robot.type=trossen_ai_stationary \
   --robot.max_relative_target=5 \
   --control.type=teleoperate
 ```
@@ -82,7 +58,7 @@ By adding `--robot.max_relative_target=5`, we override the default value for `ma
 
 ```bash
 python lerobot/scripts/control_robot.py \
-  --robot.type=trossen_ai \
+  --robot.type=trossen_ai_stationary \
   --robot.max_relative_target=null \
   --control.type=teleoperate
 ```
@@ -106,52 +82,36 @@ echo $HF_USER
 
 Record 2 episodes and upload your dataset to the hub:
 
+Note: We recommend using `--control.num_image_writer_threads_per_camera=8` for best results while recording episodes.
+
 ```bash
 python lerobot/scripts/control_robot.py \
-  --robot.type=trossen_ai_bimanual \
+  --robot.type=trossen_ai_stationary \
   --robot.max_relative_target=null \
   --control.type=record \
   --control.fps=30 \
   --control.single_task="Grasp a lego block and put it in the bin." \
-  --control.repo_id=${HF_USER}/trossen_ai_bimanual_test \
+  --control.repo_id=${HF_USER}/trossen_ai_stationary_test \
-  --control.tags='["tutorial"]' \
-  --control.warmup_time_s=5 \
-  --control.episode_time_s=30 \
-  --control.reset_time_s=30 \
-  --control.num_episodes=2 \
-  --control.push_to_hub=true
-```
-
-Note: If the camera fps is unstable consider increasing the number of image writers per thread.
-
-```bash
-python lerobot/scripts/control_robot.py \
-  --robot.type=trossen_ai_bimanual \
-  --robot.max_relative_target=null \
-  --control.type=record \
-  --control.fps=30 \
-  --control.single_task="Grasp a lego block and put it in the bin." \
-  --control.repo_id=${HF_USER}/trossen_ai_bimanual_test \
   --control.tags='["tutorial"]' \
   --control.warmup_time_s=5 \
   --control.episode_time_s=30 \
   --control.reset_time_s=30 \
   --control.num_episodes=2 \
   --control.push_to_hub=true \
-  --control.num_image_writer_threads_per_camera = 8
+  --control.num_image_writer_threads_per_camera=8
 ```
 
 ## Visualize a dataset
 
 If you uploaded your dataset to the hub with `--control.push_to_hub=true`, you can [visualize your dataset online](https://huggingface.co/spaces/lerobot/visualize_dataset) by copy pasting your repo id given by:
 ```bash
-echo ${HF_USER}/trossen_ai_bimanual_test
+echo ${HF_USER}/trossen_ai_stationary_test
 ```
 
 If you didn't upload with `--control.push_to_hub=false`, you can also visualize it locally with:
 ```bash
 python lerobot/scripts/visualize_dataset_html.py \
-  --repo-id ${HF_USER}/trossen_ai_bimanual_test
+  --repo-id ${HF_USER}/trossen_ai_stationary_test
 ```
 
 ## Replay an episode
@@ -160,11 +120,11 @@ python lerobot/scripts/visualize_dataset_html.py \
 Now try to replay the first episode on your robot:
 ```bash
 python lerobot/scripts/control_robot.py \
-  --robot.type=trossen_ai_bimanual \
+  --robot.type=trossen_ai_stationary \
   --robot.max_relative_target=null \
   --control.type=replay \
   --control.fps=30 \
-  --control.repo_id=${HF_USER}/trossen_ai_bimanual_test \
+  --control.repo_id=${HF_USER}/trossen_ai_stationary_test \
   --control.episode=0
 ```
 
@@ -173,47 +133,47 @@ python lerobot/scripts/control_robot.py \
 To train a policy to control your robot, use the [`python lerobot/scripts/train.py`](../lerobot/scripts/train.py) script. A few arguments are required. Here is an example command:
 ```bash
 python lerobot/scripts/train.py \
-  --dataset.repo_id=${HF_USER}/trossen_ai_bimanual_test \
+  --dataset.repo_id=${HF_USER}/trossen_ai_stationary_test \
   --policy.type=act \
-  --output_dir=outputs/train/act_trossen_ai_bimanual_test \
+  --output_dir=outputs/train/act_trossen_ai_stationary_test \
-  --job_name=act_trossen_ai_bimanual_test \
+  --job_name=act_trossen_ai_stationary_test \
   --device=cuda \
   --wandb.enable=true
 ```
 
 Let's explain it:
-1. We provided the dataset as argument with `--dataset.repo_id=${HF_USER}/trossen_ai_bimanual_test`.
+1. We provided the dataset as argument with `--dataset.repo_id=${HF_USER}/trossen_ai_stationary_test`.
 2. We provided the policy with `policy.type=act`. This loads configurations from [`configuration_act.py`](../lerobot/common/policies/act/configuration_act.py). Importantly, this policy will automatically adapt to the number of motor sates, motor actions and cameras of your robot (e.g. `laptop` and `phone`) which have been saved in your dataset.
 4. We provided `device=cuda` since we are training on a Nvidia GPU, but you could use `device=mps` to train on Apple silicon.
 5. We provided `wandb.enable=true` to use [Weights and Biases](https://docs.wandb.ai/quickstart) for visualizing training plots. This is optional but if you use it, make sure you are logged in by running `wandb login`.
 
 For more information on the `train` script see the previous tutorial: [`examples/4_train_policy_with_script.md`](../examples/4_train_policy_with_script.md)
 
-Training should take several hours. You will find checkpoints in `outputs/train/act_trossen_ai_bimanual_test/checkpoints`.
+Training should take several hours. You will find checkpoints in `outputs/train/act_trossen_ai_stationary_test/checkpoints`.
 
 ## Evaluate your policy
 
 You can use the `record` function from [`lerobot/scripts/control_robot.py`](../lerobot/scripts/control_robot.py) but with a policy checkpoint as input. For instance, run this command to record 10 evaluation episodes:
 ```bash
 python lerobot/scripts/control_robot.py \
-  --robot.type=trossen_ai_bimanual \
+  --robot.type=trossen_ai_stationary \
   --control.type=record \
   --control.fps=30 \
   --control.single_task="Grasp a lego block and put it in the bin." \
-  --control.repo_id=${HF_USER}/eval_act_trossen_ai_bimanual_test \
+  --control.repo_id=${HF_USER}/eval_act_trossen_ai_stationary_test \
   --control.tags='["tutorial"]' \
   --control.warmup_time_s=5 \
   --control.episode_time_s=30 \
   --control.reset_time_s=30 \
   --control.num_episodes=10 \
   --control.push_to_hub=true \
-  --control.policy.path=outputs/train/act_trossen_ai_bimanual_test/checkpoints/last/pretrained_model \
+  --control.policy.path=outputs/train/act_trossen_ai_stationary_test/checkpoints/last/pretrained_model \
   --control.num_image_writer_processes=1
 ```
 
 As you can see, it's almost the same command as previously used to record your training dataset. Two things changed:
-1. There is an additional `--control.policy.path` argument which indicates the path to your policy checkpoint with  (e.g. `outputs/train/eval_act_trossen_ai_bimanual_test/checkpoints/last/pretrained_model`). You can also use the model repository if you uploaded a model checkpoint to the hub (e.g. `${HF_USER}/act_trossen_ai_bimanual_test`).
+1. There is an additional `--control.policy.path` argument which indicates the path to your policy checkpoint with  (e.g. `outputs/train/eval_act_trossen_ai_stationary_test/checkpoints/last/pretrained_model`). You can also use the model repository if you uploaded a model checkpoint to the hub (e.g. `${HF_USER}/act_trossen_ai_stationary_test`).
-2. The name of dataset begins by `eval` to reflect that you are running inference (e.g. `${HF_USER}/eval_act_trossen_ai_bimanual_test`).
+2. The name of dataset begins by `eval` to reflect that you are running inference (e.g. `${HF_USER}/eval_act_trossen_ai_stationary_test`).
 3. We use `--control.num_image_writer_processes=1` instead of the default value (`0`). On our computer, using a dedicated process to write images from the 4 cameras on disk allows to reach constent 30 fps during inference. Feel free to explore different values for `--control.num_image_writer_processes`.
 
 ## More

lerobot/common/datasets/video_utils.py

@@ -247,7 +247,7 @@ def encode_video_frames(
     imgs_dir: Path | str,
     video_path: Path | str,
     fps: int,
-    vcodec: str = "libx264",
+    vcodec: str = "libsvtav1",
     pix_fmt: str = "yuv420p",
     g: int | None = 2,
     crf: int | None = 30,

lerobot/common/robot_devices/control_utils.py

@@ -288,13 +288,16 @@ def reset_environment(robot, events, reset_time_s, fps):
     if has_method(robot, "teleop_safety_stop"):
         robot.teleop_safety_stop()
 
-    control_loop(
+    if robot.robot_type in ["trossen_ai_stationary", "trossen_ai_solo"]:
-        robot=robot,
+        time.sleep(reset_time_s)
-        control_time_s=reset_time_s,
+    else:
-        events=events,
+        control_loop(
-        fps=fps,
+            robot=robot,
-        teleoperate=True,
+            control_time_s=reset_time_s,
-    )
+            events=events,
+            fps=fps,
+            teleoperate=True,
+        )
 
 
 def stop_recording(robot, listener, display_cameras):

lerobot/common/robot_devices/motors/trossen_arm_driver.py

@@ -132,7 +132,7 @@ class TrossenArmDriver:
 
         # Move the arms to the home pose
         self.driver.set_all_modes(trossen.Mode.position)
-        self.driver.set_all_positions(self.home_pose, 2.0, True)
+        self.driver.set_all_positions(self.home_pose, 2.0, False)
 
         # Allow to read and write
         self.is_connected = True
@@ -244,7 +244,7 @@ class TrossenArmDriver:
                 self.driver.set_all_external_efforts([0.0] * 7, 0.0, True)
         elif data_name == "Reset":
             self.driver.set_all_modes(trossen.Mode.position)
-            self.driver.set_all_positions(self.home_pose, 2.0, True)
+            self.driver.set_all_positions(self.home_pose, 2.0, False)
         else:
             print(f"Data name: {data_name} value: {values} is not supported for writing.")
 
@@ -256,8 +256,8 @@ class TrossenArmDriver:
                 f"Trossen Arm Driver ({self.port}) is not connected. Try running `motors_bus.connect()` first."
             )
         self.driver.set_all_modes(trossen.Mode.position)
-        self.driver.set_all_positions(self.home_pose, 2.0, True)
+        self.driver.set_all_positions(self.home_pose, 2.0, False)
-        self.driver.set_all_positions(self.sleep_pose, 2.0, True)
+        self.driver.set_all_positions(self.sleep_pose, 2.0, False)
 
         self.is_connected = False
 

lerobot/common/robot_devices/robots/configs.py

@@ -614,9 +614,9 @@ class LeKiwiRobotConfig(RobotConfig):
     mock: bool = False
 
 
-@RobotConfig.register_subclass("trossen_ai_bimanual")
+@RobotConfig.register_subclass("trossen_ai_stationary")
 @dataclass
-class TrossenAIBimanualRobotConfig(ManipulatorRobotConfig):
+class TrossenAIStationaryRobotConfig(ManipulatorRobotConfig):
 
     # /!\ FOR SAFETY, READ THIS /!\
     # `max_relative_target` limits the magnitude of the relative positional target vector for safety purposes.
@@ -664,7 +664,13 @@ class TrossenAIBimanualRobotConfig(ManipulatorRobotConfig):
     cameras: dict[str, CameraConfig] = field(
         default_factory=lambda: {
             "cam_high": IntelRealSenseCameraConfig(
-                serial_number=130322270184,
+                serial_number=218622270304,
+                fps=30,
+                width=640,
+                height=480,
+            ),
+            "cam_low": IntelRealSenseCameraConfig(
+                serial_number=130322272628,
                 fps=30,
                 width=640,
                 height=480,

lerobot/common/robot_devices/robots/manipulator.py

@@ -160,7 +160,7 @@ class ManipulatorRobot:
     ):
         self.config = config
         self.robot_type = self.config.type
-        if not self.robot_type in ["trossen_ai_bimanual", "trossen_ai_solo"]:
+        if not self.robot_type in ["trossen_ai_stationary", "trossen_ai_solo"]:
             self.calibration_dir = Path(self.config.calibration_dir)
         self.leader_arms = make_motors_buses_from_configs(self.config.leader_arms)
         self.follower_arms = make_motors_buses_from_configs(self.config.follower_arms)
@@ -224,13 +224,17 @@ class ManipulatorRobot:
         return available_arms
 
     def teleop_safety_stop(self):
-        if self.robot_type in ["trossen_ai_bimanual", "trossen_ai_solo"]:
+        if self.robot_type in ["trossen_ai_stationary", "trossen_ai_solo"]:
-            for arms in self.follower_arms:
-                self.follower_arms[arms].write("Reset", 1)
-                self.follower_arms[arms].write("Torque_Enable", 1)
             for arms in self.leader_arms:
                 self.leader_arms[arms].write("Reset", 1)
+            for arms in self.follower_arms:
+                self.follower_arms[arms].write("Reset", 1)
+            time.sleep(2)
+            for arms in self.leader_arms:
                 self.leader_arms[arms].write("Torque_Enable", 0)
+            for arms in self.follower_arms:
+                self.follower_arms[arms].write("Torque_Enable", 1)
+            
 
     def connect(self):
         if self.is_connected:
@@ -250,8 +254,10 @@ class ManipulatorRobot:
         for name in self.leader_arms:
             print(f"Connecting {name} leader arm.")
             self.leader_arms[name].connect()
+        
+        time.sleep(2)
 
-        if self.robot_type in ["koch", "koch_bimanual", "aloha", "trossen_ai_bimanual", "trossen_ai_solo"]:
+        if self.robot_type in ["koch", "koch_bimanual", "aloha", "trossen_ai_stationary", "trossen_ai_solo"]:
             from lerobot.common.robot_devices.motors.dynamixel import TorqueMode
         elif self.robot_type in ["so100", "moss", "lekiwi"]:
             from lerobot.common.robot_devices.motors.feetech import TorqueMode
@@ -263,7 +269,7 @@ class ManipulatorRobot:
         for name in self.leader_arms:
             self.leader_arms[name].write("Torque_Enable", TorqueMode.DISABLED.value)
 
-        if not self.robot_type in ["trossen_ai_bimanual", "trossen_ai_solo"]:
+        if not self.robot_type in ["trossen_ai_stationary", "trossen_ai_solo"]:
             print("Checking if calibration is needed.")
             self.activate_calibration()
 
@@ -628,6 +634,8 @@ class ManipulatorRobot:
 
         for name in self.leader_arms:
             self.leader_arms[name].disconnect()
+        
+        time.sleep(2)
 
         for name in self.cameras:
             self.cameras[name].disconnect()

lerobot/common/robot_devices/robots/utils.py

@@ -24,7 +24,7 @@ from lerobot.common.robot_devices.robots.configs import (
     RobotConfig,
     So100RobotConfig,
     StretchRobotConfig,
-    TrossenAIBimanualRobotConfig,
+    TrossenAIStationaryRobotConfig,
     TrossenAISoloRobotConfig,
 )
 
@@ -64,8 +64,8 @@ def make_robot_config(robot_type: str, **kwargs) -> RobotConfig:
         return StretchRobotConfig(**kwargs)
     elif robot_type == "lekiwi":
         return LeKiwiRobotConfig(**kwargs)
-    elif robot_type == "trossen_ai_bimanual":
+    elif robot_type == "trossen_ai_stationary":
-        return TrossenAIBimanualRobotConfig(**kwargs)
+        return TrossenAIStationaryRobotConfig(**kwargs)
     elif robot_type == "trossen_ai_solo":
         return TrossenAISoloRobotConfig(**kwargs)
     else: