2516101cba
Cleaned `learner_server.py`. Added several block function to improve readability.
...
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-04-18 15:04:13 +02:00
aebea08a99
Added support for checkpointing the policy. We can save and load the policy state dict, optimizers state, optimization step and interaction step
...
Added functions for converting the replay buffer from and to LeRobotDataset. When we want to save the replay buffer, we convert it first to LeRobotDataset format and save it locally and vice-versa.
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-04-18 15:04:13 +02:00
03616db82c
Removed unnecessary time.sleep in the streaming server on the learner side
...
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-04-18 15:04:13 +02:00
93c4fc198f
Added missing config files `env/maniskill_example.yaml` and `policy/sac_maniskill.yaml` that are necessary to run the lerobot implementation of sac with the maniskill baselines.
...
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-04-18 15:04:13 +02:00
8cd44ae163
- Added additional logging information in wandb around the timings of the policy loop and optimization loop.
...
- Optimized critic design that improves the performance of the learner loop by a factor of 2
- Cleaned the code and fixed style issues
- Completed the config with actor_learner_config field that contains host-ip and port elemnts that are necessary for the actor-learner servers.
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-04-18 15:04:13 +02:00
2ae657f568
FREEDOM, added back the optimization loop code in `learner_server.py`
...
Ran experiment with pushcube env from maniskill. The learning seem to work.
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-04-18 15:04:13 +02:00
508f5d1407
Added server directory in `lerobot/scripts` that contains scripts and the protobuf message types to split training into two processes, acting and learning. The actor rollouts the policy and collects interaction data while the learner recieves the data, trains the policy and sends the updated parameters to the actor. The two scripts are ran simultaneously
...
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
2025-04-18 15:04:13 +02:00
c8b1132846
Stable version of rlpd + drq
2025-04-18 15:04:10 +02:00
ef777993cd
Add type annotations and restructure SACConfig class fields
2025-04-18 15:03:51 +02:00
760d60ad4b
Change SAC policy implementation with configuration and modeling classes
2025-04-18 15:03:51 +02:00
875c0271b7
SAC works
2025-04-18 15:03:51 +02:00
57344bfde5
[WIP] correct sac implementation
2025-04-18 15:03:51 +02:00
46827fb002
Add rlpd tricks
2025-04-18 15:03:51 +02:00
2fd78879f6
SAC works
2025-04-18 15:03:51 +02:00
e8449e9630
remove breakpoint
2025-04-18 15:03:51 +02:00
a0e2be8b92
[WIP] correct sac implementation
2025-04-18 15:03:51 +02:00
181727c0fe
Extend reward classifier for multiple camera views ( #626 )
2025-04-18 15:03:50 +02:00
d1d6ffd23c
[Port HIL_SERL] Final fixes for the Reward Classifier ( #598 )
2025-04-18 15:03:01 +02:00
e5801f467f
added temporary fix for missing task_index key in online environment
2025-04-18 15:03:01 +02:00
c6ca9523de
split encoder for critic and actor
2025-04-18 15:03:01 +02:00
642e3a3274
style fixes
2025-04-18 15:03:01 +02:00
146148c48c
Refactor SAC configuration and policy for improved action sampling and stability
...
- Updated SACConfig to replace standard deviation parameterization with log_std_min and log_std_max for better control over action distributions.
- Modified SACPolicy to streamline action selection and log probability calculations, enhancing stochastic behavior.
- Removed deprecated TanhMultivariateNormalDiag class to simplify the codebase and improve maintainability.
These changes aim to enhance the robustness and performance of the SAC implementation during training and inference.
2025-04-18 15:03:01 +02:00
8f15835daa
Refine SAC configuration and policy for enhanced performance
...
- Updated standard deviation parameterization in SACConfig to 'softplus' with defined min and max values for improved stability.
- Modified action sampling in SACPolicy to use reparameterized sampling, ensuring better gradient flow and log probability calculations.
- Cleaned up log probability calculations in TanhMultivariateNormalDiag for clarity and efficiency.
- Increased evaluation frequency in YAML configuration to 50000 for more efficient training cycles.
These changes aim to enhance the robustness and performance of the SAC implementation during training and inference.
2025-04-18 15:03:01 +02:00
022bd65125
Refactor SACPolicy for improved action sampling and standard deviation handling
...
- Updated action selection to use distribution sampling and log probabilities for better stochastic behavior.
- Enhanced standard deviation clamping to prevent extreme values, ensuring stability in policy outputs.
- Cleaned up code by removing unnecessary comments and improving readability.
These changes aim to refine the SAC implementation, enhancing its robustness and performance during training and inference.
2025-04-18 15:03:01 +02:00
63d8c96514
trying to get sac running
2025-04-18 15:03:01 +02:00
4624a836e5
Added normalization schemes and style checks
2025-04-18 15:03:01 +02:00
ad7eea132d
added optimizer and sac to factory.py
2025-04-18 15:02:59 +02:00
22a1899ff4
[HIL-SERL PORT] Fix linter issues ( #588 )
2025-04-18 15:02:44 +02:00
17a3a31b5f
[Port Hil-SERL] Add unit tests for the reward classifier & fix imports & check script ( #578 )
2025-04-18 15:02:42 +02:00
1a8b99e360
added comments from kewang
2025-04-18 15:02:13 +02:00
6db2154f28
Enhance SAC configuration and policy with new parameters and subsampling logic
...
- Added `num_subsample_critics`, `critic_target_update_weight`, and `utd_ratio` to SACConfig.
- Implemented target entropy calculation in SACPolicy if not provided.
- Introduced subsampling of critics to prevent overfitting during updates.
- Updated temperature loss calculation to use the new target entropy.
- Added comments for future UTD update implementation.
These changes improve the flexibility and performance of the SAC implementation.
2025-04-18 15:02:13 +02:00
be3adda95f
Port SAC WIP ( #581 )
...
Co-authored-by: KeWang1017 <ke.wang@helloleap.ai>
2025-04-18 15:02:13 +02:00
9d48d236c1
completed losses
2025-04-18 15:02:13 +02:00
b57d6a7776
nit in control_robot.py
2025-04-18 15:02:13 +02:00
d1f76cba8e
Update lerobot/scripts/train_hilserl_classifier.py
...
Co-authored-by: Yoel <yoel.chornton@gmail.com>
2025-04-18 15:02:13 +02:00
d78cef1fee
Fixup
2025-04-18 15:02:13 +02:00
30a808c0ae
Add human intervention mechanism and eval_robot script to evaluate policy on the robot ( #541 )
...
Co-authored-by: Yoel <yoel.chornton@gmail.com>
2025-04-18 15:02:13 +02:00
4a7f85a6ec
Reward classifier and training ( #528 )
...
Co-authored-by: Daniel Ritchie <daniel@brainwavecollective.ai>
Co-authored-by: resolver101757 <kelster101757@hotmail.com>
Co-authored-by: Jannik Grothusen <56967823+J4nn1K@users.noreply.github.com>
Co-authored-by: Remi <re.cadene@gmail.com>
Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co>
2025-04-18 15:02:13 +02:00
b43ece8934
Add pythno3-dev in Dockerfile to build and modify Readme.md , python-dev to python3-dev ( #987 )
...
Co-authored-by: makolon <smakolon385@gmail.com>
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
2025-04-17 16:17:07 +02:00
c10c5a0e64
Fix --width --height type parsing on opencv and intelrealsense scripts ( #556 )
...
Co-authored-by: Remi <remi.cadene@huggingface.co>
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
2025-04-17 15:19:23 +02:00
a8db91c40e
Fix Windows HTML visualization to make videos could be seen ( #647 )
...
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
2025-04-17 15:07:28 +02:00
0f5f7ac780
Fix broken links in `examples/4_train_policy_with_script.md` ( #697 )
2025-04-17 14:59:43 +02:00
768e36660d
[pre-commit.ci] pre-commit autoupdate ( #980 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-04-14 21:55:06 +02:00
790d6740ba
fix(installation): adding note on `ffmpeg` version during installation ( #976 )
...
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com>
2025-04-14 15:36:31 +02:00
5322417c03
fix(examples): removes extra backtick ( #948 )
2025-04-09 17:44:32 +02:00
4041f57943
feat(visualization): replace cv2 GUI with Rerun (and solves ffmpeg versioning issues) ( #903 )
2025-04-09 17:33:01 +02:00
2c86fea78a
Switch typos pre-commit to mirror ( #953 )
2025-04-08 12:44:09 +02:00
437fc29e12
[pre-commit.ci] pre-commit autoupdate ( #871 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-04-08 06:58:46 +02:00
aee86b4b18
typo fix: example_1 python script ( #631 )
...
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
2025-04-07 17:41:10 +02:00
1c873df5c0
Support for PI0+FAST ( #921 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Dana Aubakirova <118912928+danaaubakirova@users.noreply.github.com>
Co-authored-by: Remi <re.cadene@gmail.com>
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
2025-04-04 11:51:11 +02:00
Michel Aractingi
AdilZouitine
Eugene Mironov
KeWang1017
KeWang
Yoel
k1000dai
Alex Thiele
Junshan Huang
HUANG TZU-CHUN
pre-commit-ci[bot]
Caroline Pascal
Steven Palma
Simon Alibert
Junwu Zhang
mshukor