105 lines
3.3 KiB
Markdown
105 lines
3.3 KiB
Markdown
# Antipodal Robotic Grasping
|
||
We present a novel generative residual convolutional neural network based model architecture which detects objects in the camera’s field of view and predicts a suitable antipodal grasp configuration for the objects in the image.
|
||
|
||
This repository contains the implementation of the Generative Residual Convolutional Neural Network (GR-ConvNet) from the paper:
|
||
|
||
#### Antipodal Robotic Grasping using Generative Residual Convolutional Neural Network
|
||
|
||
Sulabh Kumra, Shirin Joshi, Ferat Sahin
|
||
|
||
[arxiv](https://arxiv.org/abs/1909.04810) | [video](https://youtu.be/cwlEhdoxY4U)
|
||
|
||
[](https://paperswithcode.com/sota/robotic-grasping-on-cornell-grasp-dataset?p=antipodal-robotic-grasping-using-generative)
|
||
|
||
If you use this project in your research or wish to refer to the baseline results published in the paper, please use the following BibTeX entry:
|
||
|
||
```
|
||
@inproceedings{kumra2019antipodal,
|
||
title={Antipodal Robotic Grasping using Generative Residual Convolutional Neural Network},
|
||
author={Kumra, Sulabh and Joshi, Shirin and Sahin, Ferat},
|
||
booktitle={2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
|
||
year={2020},
|
||
organization={IEEE}
|
||
}
|
||
```
|
||
|
||
## Requirements
|
||
|
||
- numpy
|
||
- opencv-python
|
||
- matplotlib
|
||
- scikit-image
|
||
- imageio
|
||
- torch
|
||
- torchvision
|
||
- torchsummary
|
||
- tensorboardX
|
||
- pyrealsense2
|
||
- Pillow
|
||
|
||
## Installation
|
||
- Checkout the robotic grasping package
|
||
```bash
|
||
$ git clone https://github.com/skumra/robotic-grasping.git
|
||
```
|
||
|
||
- Create a virtual environment
|
||
```bash
|
||
$ python3.6 -m venv --system-site-packages venv
|
||
```
|
||
|
||
- Source the virtual environment
|
||
```bash
|
||
$ source venv/bin/activate
|
||
```
|
||
|
||
- Install the requirements
|
||
```bash
|
||
$ cd robotic-grasping
|
||
$ pip install -r requirements.txt
|
||
```
|
||
|
||
## Datasets
|
||
|
||
This repository supports both the [Cornell Grasping Dataset](http://pr.cs.cornell.edu/grasping/rect_data/data.php) and
|
||
[Jacquard Dataset](https://jacquard.liris.cnrs.fr/).
|
||
|
||
#### Cornell Grasping Dataset
|
||
|
||
1. Download the and extract [Cornell Grasping Dataset](http://pr.cs.cornell.edu/grasping/rect_data/data.php).
|
||
2. Convert the PCD files to depth images by running `python -m utils.dataset_processing.generate_cornell_depth <Path To Dataset>`
|
||
|
||
#### Jacquard Dataset
|
||
|
||
1. Download and extract the [Jacquard Dataset](https://jacquard.liris.cnrs.fr/).
|
||
|
||
|
||
## Model Training
|
||
|
||
A model can be trained using the `train_network.py` script. Run `train_network.py --help` to see a full list of options.
|
||
|
||
For example:
|
||
|
||
```bash
|
||
python train_network.py --dataset cornell --dataset-path <Path To Dataset> --description training_cornell
|
||
```
|
||
|
||
## Model Evaluation
|
||
|
||
The trained network can be evaluated using the `evaluate.py` script. Run `evaluate.py --help` for a full set of options.
|
||
|
||
For example:
|
||
|
||
```bash
|
||
python evaluate.py --network <Path to Trained Network> --dataset cornell --dataset-path <Path to Dataset> --iou-eval
|
||
```
|
||
|
||
## Run Tasks
|
||
A task can be executed using the relevant run script. All task scripts are named as `run_<task name>.py`. For example, to run the grasp generator run:
|
||
```bash
|
||
python run_grasp_generator.py
|
||
```
|
||
|
||
## Run on a Robot
|
||
To run the grasp generator with a robot, please use our ROS implementation for Baxter robot. It is available at: https://github.com/skumra/baxter-pnp
|