Deep Learning

Lidar Guided Example

Learn how to load point cloud data, preprocess datasets, define and train networks, and generate detections.

To get started:

  1. Download the code
  2. Open the code in MATLAB
  3. Follow along with the steps below.

Don’t have access?

Get a free trial
Time to Complete:
15–30 minutes
Level:
Beginner/Intermediate

Need a refresher? Try a free, interactive tutorial.

Required Products

Step 1

Load Point Cloud Data and Corresponding Labels

With a deep neural network, you can explore multiclass object detection on point cloud data. This example uses the PointPillars network, which can detect objects in the point cloud and fit oriented bounding boxes around them.
  • Create datastores to manage the data.
  • Create a fileDatastore to load the PCD files from the specified path using the pcread function.
  • Load bounding box labels using the boxLabelDatastore function for ground truth labels.

Key things to remember:

  • This demo uses a front-view point cloud rather than a full-view point cloud. There are two benefits to this approach: (1) faster training and execution, and (2) better network performance for average orientation similarity, and precision.

Step 2

Preprocess Datasets

You are ready to split datasets and explore various augmentation techniques.
  • Split the dataset into training and testing sets.
  • Perform ground truth data augmentation, which randomly adds a fixed number of car class objects to every point cloud. The image below shows 70% of the data for training and the rest for testing. This technique helps to improve the network accuracy during training by synthetically increasing the size of the training dataset.
  • Try data augmentation techniques, such as flipping, scaling, rotation, and translation of point cloud.
  • View the output.

Key things to remember:

  • This example uses a small dataset with very few frames to demonstrate the workflow. For better results, train the network on a larger dataset.

Step 3

Define Networks

Begin to define your anchor boxes, pillars for the PointPillars network, and PointPillars network.
  • Define anchor boxes.
  • Define the number of pillars for the PointPillars network.
  • Use the PointPillar object detection function to create a PointPillar object detection network.

Step 4

Train Network​​​s

Once you have defined the network, train the model on the training dataset.
  • Specify new training parameters.
  • Use the trainedPointPillarsObjectDetector function to train PointPillars.
  • Load a pretrained object detector if you do not want to train a model from scratch.

Key things to remember:

  • Training is the most time-consuming step in a deep learning workflow. Alternatively, you can use a pre-trained model to detect objects in the point cloud.

Step 5

Generate Detections

In the final step, use the trained network to detect objects in the test data and display the point cloud with bounding boxes.
  • Read the point cloud from the test data.
  • Use the detect function on the test point cloud to get the predicted bounding boxes and confidence scores.
  • Display the detected output point cloud with bounding boxes. The green bounding boxes represent cars in the output point cloud.

Follow along with the Lidar Guided Example videos:

Lidar Guided Example Videos (5 videos)

See the links below to continue with more lidar tasks or keep exploring deep learning with related applications.

Johanna Pingel

Connect with Johanna, MathWorks Deep Learning Expert

Email Johanna