7ae368e983
Fixed bug in the action scale of the intervention actions and offline dataset actions. (scale by inverse delta)
...
Co-authored-by: Adil Zouitine <adizouitinegm@gmail.com>
2025-02-14 15:17:16 +01:00
36711d766a
Modified crop_dataset_roi interface to automatically write the cropped parameters to a json file in the meta of the dataset
...
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-14 12:32:45 +01:00
c9e50bb9b1
Optimized the replay buffer from the memory side to store data on cpu instead of a gpu device and send the batches to the gpu.
...
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-13 18:03:57 +01:00
95de8e273d
nit
...
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-13 17:12:57 +01:00
b07d95f0dd
removed uncomment in actor server
...
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-13 16:53:33 +01:00
d9a70376d8
Changed the init_final value to center the starting mean and std of the policy
...
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-13 16:42:43 +01:00
0c32008466
Changed bounds for a new so100 robot
...
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-13 15:43:30 +01:00
c462a478c7
Hardcoded some normalization parameters. TODO refactor
...
Added masking actions on the level of the intervention actions and offline dataset
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-13 14:27:14 +01:00
459f22ed30
fix log_alpha in modeling_sac: change to nn.parameter
...
added pretrained vision model in policy
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-13 11:26:24 +01:00
dc086dc21f
Added logging for interventions to monitor the rate of interventions through time
...
Added an s keyboard command to force success in the case the reward classifier fails
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-13 11:04:49 +01:00
b9217b06db
Added possiblity to record and replay delta actions during teleoperation rather than absolute actions
...
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-12 19:25:41 +01:00
6868c88ef1
[PORT-Hilserl] classifier fixes ( #695 )
...
Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co>
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-11 11:39:17 +01:00
a1d16fb400
[Port HIL-SERL] Add resnet-10 as default encoder for HIL-SERL ( #696 )
...
Co-authored-by: Khalil Meftah <kmeftah.khalil@gmail.com>
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co>
Co-authored-by: Ke Wang <superwk1017@gmail.com>
2025-02-11 11:37:00 +01:00
a7db3959f5
- Added JointMaskingActionSpace wrapper in `gym_manipulator` in order to select which joints will be controlled. For example, we can disable the gripper actions for some tasks.
...
- Added Nan detection mechanisms in the actor, learner and gym_manipulator for the case where we encounter nans in the loop.
- changed the non-blocking in the `.to(device)` functions to only work for the case of cuda because they were causing nans when running the policy on mps
- Added some joint clipping and limits in the env, robot and policy configs. TODO clean this part and make the limits in one config file only.
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-11 11:34:46 +01:00
b5f89439ff
Added sac_real config file in the policym configs dir.
...
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-10 16:08:13 +01:00
d51374ce12
Several fixes to move the actor_server and learner_server code from the maniskill environment to the real robot environment.
...
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-10 16:03:39 +01:00
b63738674c
[HIL-SERL port] Add Reward classifier benchmark tracking to chose best visual encoder ( #688 )
2025-02-06 18:39:51 +01:00
12525242ce
- Added `lerobot/scripts/server/gym_manipulator.py` that contains all the necessary wrappers to run a gym-style env around the real robot.
...
- Added `lerobot/scripts/server/find_joint_limits.py` to test the min and max angles of the motion you wish the robot to explore during RL training.
- Added logic in `manipulator.py` to limit the maximum possible joint angles to allow motion within a predefined joint position range. The limits are specified in the yaml config for each robot. Checkout the so100.yaml.
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-06 16:29:37 +01:00
7d5a9530f7
fixed bug in crop_dataset_roi.py
...
added missing buffer.pt in server dir
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-05 18:22:50 +00:00
e0527b4a6b
Added additional wrappers for the environment: Action repeat, keyboard interface, reset wrapper
...
Tested the reset mechanism and keyboard interface and the convert wrapper on the robots.
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-04 17:41:14 +00:00
efb1982eec
Added crop_dataset_roi.py that allows you to load a lerobotdataset -> crop its images -> create a new lerobot dataset with the cropped and resized images.
...
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-03 17:48:35 +00:00
2211209be5
- Added base gym env class for the real robot environment.
...
- Added several wrappers around the base gym env robot class.
- Including: time limit, reward classifier, crop images, preprocess observations.
- Added an interactive script crop_roi.py where the user can interactively select the roi in the observation images and return the correct crop values that will improve the policy and reward classifier performance.
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-03 15:07:59 +00:00
506821c7df
- Refactor observation encoder in `modeling_sac.py`
...
- added `torch.compile` to the actor and learner servers.
- organized imports in `train_sac.py`
- optimized the parameters push by not sending the frozen pre-trained encoder.
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-03 15:07:58 +00:00
f1c8bfe01e
[Port HIL-SERL] Add HF vision encoder option in SAC ( #651 )
...
Added support with custom pretrained vision encoder to the modeling sac implementation. Great job @ChorntonYoel !
2025-02-03 15:07:58 +00:00
7c89bd1018
Cleaned `learner_server.py`. Added several block function to improve readability.
...
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-03 15:07:58 +00:00
367dfe51c6
Added support for checkpointing the policy. We can save and load the policy state dict, optimizers state, optimization step and interaction step
...
Added functions for converting the replay buffer from and to LeRobotDataset. When we want to save the replay buffer, we convert it first to LeRobotDataset format and save it locally and vice-versa.
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-03 15:07:58 +00:00
e856ffc91e
Removed unnecessary time.sleep in the streaming server on the learner side
...
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-03 15:07:58 +00:00
9aabe212ea
Added missing config files `env/maniskill_example.yaml` and `policy/sac_maniskill.yaml` that are necessary to run the lerobot implementation of sac with the maniskill baselines.
...
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-03 15:07:58 +00:00
42618f4bd6
- Added additional logging information in wandb around the timings of the policy loop and optimization loop.
...
- Optimized critic design that improves the performance of the learner loop by a factor of 2
- Cleaned the code and fixed style issues
- Completed the config with actor_learner_config field that contains host-ip and port elemnts that are necessary for the actor-learner servers.
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-03 15:07:58 +00:00
36576c958f
FREEDOM, added back the optimization loop code in `learner_server.py`
...
Ran experiment with pushcube env from maniskill. The learning seem to work.
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-03 15:07:58 +00:00
322a78a378
Added server directory in `lerobot/scripts` that contains scripts and the protobuf message types to split training into two processes, acting and learning. The actor rollouts the policy and collects interaction data while the learner recieves the data, trains the policy and sends the updated parameters to the actor. The two scripts are ran simultaneously
...
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-02-03 15:07:58 +00:00
d75b44f89f
Stable version of rlpd + drq
2025-02-03 15:07:57 +00:00
1fb03d4cf2
Add type annotations and restructure SACConfig class fields
2025-02-03 15:07:57 +00:00
7d2970fdfe
Change SAC policy implementation with configuration and modeling classes
2025-02-03 15:07:50 +00:00
8105efb338
Add rlpd tricks
2025-02-03 15:06:18 +00:00
c1d4bf4b63
SAC works
2025-02-03 15:06:18 +00:00
86df8a433d
remove breakpoint
2025-02-03 15:06:18 +00:00
956c547254
[WIP] correct sac implementation
2025-02-03 15:06:18 +00:00
be965019bd
Add rlpd tricks
2025-02-03 15:06:18 +00:00
a0a50de8c9
SAC works
2025-02-03 15:06:18 +00:00
c86dace4c2
remove breakpoint
2025-02-03 15:06:18 +00:00
472a7f58ad
[WIP] correct sac implementation
2025-02-03 15:06:14 +00:00
068efce3f8
Fix for the issue https://github.com/huggingface/lerobot/issues/638 ( #639 )
2025-02-03 15:04:03 +00:00
df7310ea40
fixes to SO-100 readme ( #600 )
...
Co-authored-by: Philip Fung <no@one>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com>
2025-02-03 15:04:03 +00:00
100f54ee07
[viz] Fixes & updates to html visualizer ( #617 )
2025-02-03 15:04:03 +00:00
c2f7af3339
typo fix: batch_convert_dataset_v1_to_v2.py ( #615 )
...
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com>
2025-02-03 15:04:03 +00:00
a1b5d0faf2
fix(visualise): use correct language description for each episode id ( #604 )
...
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com>
2025-02-03 15:04:03 +00:00
d6498150bf
fix(docs): typos in benchmark readme.md ( #614 )
...
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com>
2025-02-03 15:04:03 +00:00
31c34a4a49
Fix Quality workflow ( #622 )
2025-02-03 15:04:03 +00:00
b1cfb6a710
Update README.md ( #612 )
2025-02-03 15:04:02 +00:00
Michel Aractingi
Yoel
Eugene Mironov
AdilZouitine
Pradeep Kadubandi
Philip Fung
Mishig
CharlesCNorton
Ville Kuosmanen
Simon Alibert