Home

Awesome

Object-Grasp-Detection-ROS

Development Environment

ROS Installation Options

ROS (Robot Operating System)


Real-time Grasp (Rotation Angle) Detection With ROS

<img src="https://github.com/yehengchen/video_demo/blob/master/video_demo/chair_pin.gif" width="75%" height="75%">

Gazebo Real-time Screw Rotation Detection - [Link]


Real-time Screw Grasp Detection With ROS

<img src="https://github.com/yehengchen/video_demo/blob/master/video_demo/grasp_detection.gif" width="75%" height="75%">

Gazebo Real-time Grasp Detection - [Link]

Parts-Arrangement-Robot - [Link]


Real-time Screw Detection With ROS

Gazebo Real-time Screw Grasp Detection - [Link]

YOLOv3_ROS object detection

Prerequisites

To download the prerequisites for this package (except for ROS itself), navigate to the package folder and run:

$ cd yolov3_pytorch_ros
$ sudo pip install -r requirements.txt

Installation

Navigate to your catkin workspace and run:

$ catkin_make yolov3_pytorch_ros

Basic Usage

  1. First, make sure to put your weights in the models folder. For the training process in order to use custom objects, please refer to the original YOLO page. As an example, to download pre-trained weights from the COCO data set, go into the models folder and run:
wget http://pjreddie.com/media/files/yolov3.weights
  1. Modify the parameters in the launch file and launch it. You will need to change the image_topic parameter to match your camera, and the weights_name, config_name and classes_name parameters depending on what you are trying to do.

Start yolov3 pytorch ros node

$ roslaunch yolov3_pytorch_ros detector.launch

Node parameters

Subscribed topics

Published topics


Ubuntu-18.04 Realsense D435


How to train (to detect your custom objects)

Training YOlOv3:

Download the dakrnet source code

git clone https://github.com/pjreddie/darknet
cd darknet

vim Makefile
...
GPU=1 # if no using GPU 0
CUDNN=1 # if no 0
OPENCV=0
OPENMP=0
DEBUG=0

make
0. Create folder for yolov3
mkdir yolov3
cd yolov3
mkdir JPEGImages labels backup cfg 

├── JPEGImages <br> │ ├── object-00001.jpg <br> │ └── object-00002.jpg <br> │ ... <br> ├── labels <br> │ ├── object-00001.txt <br> │ └── object-00002.txt <br> │ ... <br> ├── backup <br> │ ├── yolov3-object.backup <br> │ └── yolov3-object_20000.weights <br> │ ... <br> ├── cfg <br> │ ├── obj.data <br> │ ├── yolo-obj.cfg <br> │ └── obj.names <br> └── obj_test.txt...

1. Create file yolo-obj.cfg with the same content as in yolov3.cfg (or copy yolov3.cfg to yolo-obj.cfg) and:

So if classes=1 then should be filters=18. If classes=2 then write filters=21.

(Do not write in the cfg-file: filters=(classes + 5)x3)

2. Create file obj.names in the directory path_to/yolov3/cfg/, with objects names - each in new line
person
car
cat
dog
3. Create file obj.data in the directory path_to/yolov3/cfg/, containing (where classes = number of objects):
classes= 3
train  = /home/cai/workspace/yolov3/obj_train.txt
valid  = /home/cai/workspace/yolov3/obj_test.txt
names = /home/cai/workspace/yolov3/cfg/obj.names
backup = /home/cai/workspace/yolov3/backup/
4. Put image-files (.jpg) of your objects in the directory path_to/yolov3/JPEGImages
5. You should label each object on images from your dataset: [LabelImg] is a graphical image annotation tool

It will create .txt-file for each .jpg-image-file - in the same directory and with the same name, but with .txt-extension, and put to file: object number and object coordinates on this image, for each object in new line:

<object-class> <x_center> <y_center> <width> <height>

Where:

For example for img1.jpg you will be created img1.txt containing:

1 0.716797 0.395833 0.216406 0.147222
0 0.687109 0.379167 0.255469 0.158333
1 0.420312 0.395833 0.140625 0.166667
6. Create file obj_train.txt & obj_test.txt in directory path_to/yolov3/, with filenames of your images, each filename in new line,for example containing:
path_to/yolov3/JPEGImages/img1.jpg
path_to/yolov3/JPEGImages/img2.jpg
path_to/yolov3/JPEGImages/img3.jpg
7. Download pre-trained weights for the convolutional layers (154 MB): https://pjreddie.com/media/files/darknet53.conv.74 and put to the directory path_to/darknet/
wget https://pjreddie.com/media/files/darknet53.conv.74
8. Start training by using the command line:
./darknet detector train [path to .data file] [path to .cfg file] [path to pre-taining weights-darknet53.conv.74]

[visualization]
./darknet detector train path_to/yolov3/cfg/obj.data path_to/yolov3/cfg/yolov3.cfg darknet53.conv.74 2>1 | tee visualization/train_yolov3.log
9. Start testing by using the command line:
./darknet detector test path_to/yolov3/cfg/obj.data path_to/yolov3/cfg/yolov3.cfg path_to/yolov3/backup/yolov3_final.weights path_to/yolov3/test/test_img.jpg