Real-time 3D Object Detection from Point Clouds

Oct 30, 2019 2 min read

1. Introduction

2. Model Implementation

3. Initial Results

3.1 Single Class Detection

Simple Scenes

Currently the model is trained on a single class: car. For the following result, green indicates the ground truth labels, and light blue indicates the predicted results.

In a simple scene, the model seems to recognize all the car objects:

Harder

Since the model is trained using only the top view LIDAR data, it is reasonable that the model can miss the cases where thr LIDAR point cloud of the object is sparse:

Easy Mistake

However, the model still misses some obvious detection, such as in the following scene. Here, the front car in the very middle doesn’t get detected.

Why Top View?

What I think the top view (or bird’s eye view) approach can do well is that: It can detect the objects which are occluded in the front camera view. If we look at the following image, the car on the very right of the image is largely occluded:

However, viewing the point cloud from the top, these 2 cars are clearly separated in the space, and therefore the model can easily detect the targeted objects:

3.2 Multi-Class Detection

4. Final Results

5. My Thoughts

6. References

H. Su, S. Maji, E. Kalogerakis, and E. G. Learned-Miller. Multi-view convolutional neural networks for 3d shaperecognition ICCV, 2015
Bin Yang, Wenjie Luo, and Raquel Urtasun. PIXOR: Real-time 3d object detection from point clouds CVPR, 2018

Deep Learning