walk-these-ways-go2/README.md

# Sim-to-Real project on Unitree Go2

## Overview 

This repository is forked from [walk-these-ways](https://github.com/Improbable-AI/walk-these-ways), which is a Go1 Sim-to-Real Locomotion Starter Kit. It seems that [walk-these-ways](https://github.com/Improbable-AI/walk-these-ways) can be untilized on Unitree [A1](https://github.com/fan-ziqi/dog_rl_deploy) with simple modifications, since those robots are base on [unitree-legged-sdk](https://github.com/unitreerobotics/unitree_legged_sdk). 

However, the brand-new architecture [unitree-sdk2 ](https://github.com/unitreerobotics/unitree_sdk2)is not base on UDP anymore, so this project aims to train and deploy walk-these-ways on Unitree Go2 by modifying SDK interfaces.

## Requirements 
* pytorch 1.10 with cuda-11.3
* Isaac Gym
* Nvidia GPU with at least 8GB of VRAM

---
## Train
Clone this repository and install:

``` bash
git clone https://github.com/Teddy-Liao/walk-these-ways-go2.git
cd walk-these-ways-go2
pip install -e .
```

Start training: 
```bash
python scripts/train.py
```

`go2_gym` and `go2_gym_learn` folders are the main folders for training process.

Play the model:
```bash
cd scripts
python play.py
```
![Alt text](media/go2_training.jpg)

Go2 pretrained model is provided in [./runs](runs/gait-conditioned-agility/pretrain-go2), you can choose whether to use provide pretrained model by modifying the label line `label = "gait-conditioned-agility/pretrain-go2/train"` to your own trained model.

### Known Issues
* `flip_visual_attachments` in [go1_config](go1_gym/envs/go1/go1_config.py) should be set to `True`, otherwise errors would occur when visualizing.
* To change configuration parameters of env or the robot, you should modify parameters in [go1_config](go1_gym/envs/go1/go1_config.py), not in [legged_robot_config](go1_gym/envs/base/legged_robot_config.py)


---
## Deploy
Trained policy is only supported to be deployed through your PC or laptop now, because I am not familiar with Jetson Orin, and hope I can fix it and deploy on Jetson Orin.

### Requirements
#### Install LCM
Since [walk-these-ways](https://github.com/Improbable-AI/walk-these-ways) implement an interface based on Lightweight Communications and Marshalling ([LCM](https://github.com/lcm-proj/lcm)) to pass sensor data, motor commands, and joystick state between their code and the low-level control SDK provided by Unitree, LCM should be installed firstly in your PC or laptlop.

Clone LCM repository to any location (where you usually place installed softwares), then install LCM:
```bash
git clone https://github.com/lcm-proj/lcm.git
mkdir build
cd build
cmake ..
make
sudo make install
```

#### Unitree_SDK2
unitree_sdk2 has been inclued in `go2_gym_deploy/unitree_sdk2_bin/library/unitree_sdk2`,  you can also clone from [Unitree Robotics](https://github.com/unitreerobotics/unitree_sdk2) to make sure the sdk is updated version.

```bash
cd go2_gym_deploy
```
Delete build file
```bash
rm -r build
```
Install and build:
```bash
sudo ./install.sh
mkdir build
cd build
cmake ..
make
```

### Build lcm_position_go2
`go2_gym_deploy/unitree_sdk2_bin/lcm_position_go2.cpp` is the core file of this project, which is similar to `lcm_position.cpp` in [walk-these-ways](https://github.com/Improbable-AI/walk-these-ways), but replace unitree_legged_sdk with unitree_sdk2.

Build lcm_position_go2 and generate runfile `lcm_position_go2`
```bash
cd go2_gym_deploy
rm -r build
mkdir build
cd build
cmake ..
make -j
```

All LCM messages files in `go2_gym_deploy/lcm_types` are set as the same format shown in [walk-these-ways](https://github.com/Improbable-AI/walk-these-ways) to ensure successful connection with python files. LCM message files are provided in this project, and you can also generate customized LCM message files through the following instructions: 

`xxx_lcmt.hpp` files are generated by:
```bash
lcm-gen -x xxx.lcm
```

### Verify connection
Connect your PC/Laptop with Go2 robot with ethernet cable and check connection by:
```bash
ping 192.168.123.161
```

Check the network interface address, and copy the network interface address.
```bash
ifconfig
```
If error occurs, please check [Unitree Support](https://support.unitree.com/home/zh/developer/Quick_start) for details.

### Test communication between LCM and unitree_sdk2
```bash
cd go2_gym_deploy/build
sudo ./lcm_position_go2 enx10086
```
Aeplace `enx10086` with your own network interface address. According to the messages shown in terminal, press `Enter` for several times and the communication between LCM and unitree_sdk2 will set up.

This command will automatically shut down Unitree sport_mode Service and set the robot to LOW-LEVEL. Please make sure This will Go2 is hung up or lie on the ground.

You can verify LCM send by opening a new terminal:
```bash
cd go2_gym_deploy/build
sudo ./lcm_receive
```

### Load and run policy
Open a new terminate and run:
```bash
cd go2_gym_deploy/scripts
python deploy_policy.py
```

According to the hints shown in terminal, Press [R2] to start the controller. You can check RC mapping from [walk-these-ways](https://github.com/Improbable-AI/walk-these-ways) page.

**Caution**:
* Press [L2+B] if any unexpected situation occurs!!!
* This is research code; use at your own risk; we do not take responsibility for any damage.

Test Video on Unitree Go2: 
- Test in my bedroom: https://www.bilibili.com/video/BV1tQ4y1c7ZG/?spm_id_from=333.999.0.0&vd_source=07873ebe2a113dac57775e264a210929
- Test by other contributors: https://www.bilibili.com/video/BV1Ut421H7Fr/?spm_id_from=333.1007.top_right_bar_window_history.content.click&vd_source=07873ebe2a113dac57775e264a210929


**Please star this repository if it does help you! Many Thanks!**

## Acknowledgements
* Many thanks to [Leolar](https://github.com/NihaoyaLeolar), who provide Nvidia 3060ti and supporting.
* Many thanks to [Jony](https://github.com/jonyzhang2023) and Peter for their support and encourage me to learn basic kownledge about RL.
* Many thanks to [Simonforyou](https://github.com/Simonforyou), who provide Go2 pretrained model.

## TO DO
- [x] Do not inherit config and env from go1_gym, build customized config and env files for Go2
- [ ] Deploy on Jeston Orin Nano