More General View Adjusment of Point-cloud BEV

[phuccuongngo99]

What?

This PR allows for the option to adjust point-cloud discretization in both x, y-direction from 3D point cloud data -> the bird-eye-view (BEV) image.

In other words, this PR allows users to adjust the boundary of BEV in both width and height directions. (See screenshots further down for more details)

The existing code works well for self-driving car well as for self-driving car, we are only interested in seeing what's in front of the car.

However, other applications of 3D point clouds object detection may have different settings such as industrial factories and warehouses like what I tried to do. We may need to adjust the view freely according to our set-up.

This PR works with existing code but also allows users to change if they wish. This PR doesn't break the original code or KITTI 3D Object Detection benchmark

Why?

The current code only allows for the adjustment in the forward direction.

Users can adjust the boundary of BEV image by setting the appropriate number of minX, maxX, minY, maxY here.

Scenario 1: Default Setting

The default is given here.

Complex-YOLOv4-Pytorch/src/config/kitti_config.py

Lines 13 to 21 in a712f52

	# Front side (of vehicle) Point Cloud boundary for BEV
	boundary = {
	"minX": 0,
	"maxX": 50,
	"minY": -25,
	"maxY": 25,
	"minZ": -2.73,
	"maxZ": 1.27
	}

This is the BEV, notice how the view boundary starts from front of the car (it's because minX = 0) and we can see left and right view boundaries are symmetrical (it's because minY = -25, maxY = 25)

Scenario 2: Expected behaviour in Y-direction

If we give a different values to minY, and maxY such that minY = 0 and maxY = 50

# Front side (of vehicle) Point Cloud boundary for BEV
boundary = {
    "minX": 0,
    "maxX": 50,
    "minY": 0,
    "maxY": 50,
    "minZ": -2.73,
    "maxZ": 1.27
}

We can see that left and right boundary now starts at the left of the car

Scenario 3: Unexpected behaviour in X-direction

However, any attempt to set minX < 0 will not result in the view at the back of the car. Here we set minX = -25 and maxX = 25

# Front side (of vehicle) Point Cloud boundary for BEV
boundary = {
    "minX": -25,
    "maxX": 25,
    "minY": -25,
    "maxY": 25,
    "minZ": -2.73,
    "maxZ": 1.27
}

We see unexpected behaviour, the view doesn't include the car in the middle of the image, and the labels are not in the correct position)

How?

I added discretization options to src/config/kitti_config.py like here:

Complex-YOLOv4-Pytorch/src/config/kitti_config.py

Lines 36 to 37 in 43a046c

	DISCRETIZATION_X = (boundary["maxX"] - boundary["minX"]) / BEV_HEIGHT
	DISCRETIZATION_Y = (boundary["maxY"] - boundary["minY"]) / BEV_WIDTH

Update all related function call to take in Discretization_X and Discretization_Y instead of just Discretization

The main changes come from src/data_process/kitti_bev_utils.py where I made a more generalized conversion from 3D point cloud to BEV image.

Complex-YOLOv4-Pytorch/src/data_process/kitti_bev_utils.py

Lines 41 to 48 in 43a046c

	# Discretize Feature Map
	PointCloud = np.copy(PointCloud_)
	offset_x = np.int_(np.floor(bc['minX'] / Discretization_X))
	offset_y = np.int_(np.floor(bc['minY'] / Discretization_Y))
	PointCloud[:, 0] = np.int_(np.floor(PointCloud[:, 0] / Discretization_X)) - offset_x
	PointCloud[:, 1] = np.int_(np.floor(PointCloud[:, 1] / Discretization_Y)) - offset_y

Testing?

This PR produces expected behavior for scenario 3

Scenario 3: Unexpected behaviour in X-direction

# Front side (of vehicle) Point Cloud boundary for BEV
boundary = {
    "minX": -25,
    "maxX": 25,
    "minY": -25,
    "maxY": 25,
    "minZ": -2.73,
    "maxZ": 1.27
}

We see expected behaviour, the car is in the middle, all annotations are at the correct positions. (Note that KITTI dataset doesn't provide annotations for the cars at the back)