深度图到点云_从深度图到点云

深度图到点云

How to convert a RGBD image to points in 3D space

如何将RGBD图像转换为3D空间中的点

This tutorial introduces the intrinsic matrix and walks you through how you can use it to convert an RGBD (red, blue, green, depth) image to 3D space. RGBD images can be obtained in many ways. E.g. from a system like Kinect that uses infrared-based time-of flight detection. But also the iPhone 12 is rumored to integrate a LiDAR in its camera system. Most importantly for self-driving cars: LiDAR data from a mobile unit on a car can be combined with a standard RGB camera to obtain the RGBD data. We do not go into details about how to get the data in this article.

本教程介绍了固有矩阵，并逐步介绍了如何使用它将RGBD(红色，蓝色，绿色，深度)图像转换为3D空间。 RGBD图像可以通过多种方式获得。 例如来自Kinect之类的系统，该系统使用基于红外的飞行时间检测。 但也有传闻称iPhone 12将LiDAR集成到其相机系统中。 对于无人驾驶汽车而言，最重要的是：来自汽车移动单元的LiDAR数据可以与标准RGB摄像头组合以获得RGBD数据。 在本文中，我们不会详细介绍如何获取数据。

Fig. 1: (left) Image plane in u, v coordinates. Each pixel has a colour and a depth assigned. (right) 3D view in cartesian coordinates x, y, z.

图1 ：(左)以u，v坐标表示的图像平面。 每个像素都有指定的颜色和深度。 (右)笛卡尔坐标x，y，z中的3D视图。

It is important to know your camera’s properties if you want to understand what each pixel corresponds to in a 3D environment. The most important parameter is the focal length. It tells us how to translate a pixel coordinate into lengths. You probably have seen focal lengths like “28 mm”. This is the actual distance between the lens and the film/sensor.

如果要了解3D环境中每个像素对应的内容，了解相机的属性很重要。 最重要的参数是焦距。 它告诉我们如何将像素坐标转换为长度。 您可能已经看到像“ 28 mm”这样的焦距。 这是镜头与胶片/传感器之间的实际距离。

From a simple geometric argument (“similar triangles”) we can easily derive the position x from u and d of each pixel. The picture below is just looking at x and u but we can do exactly the same for y and v. For a pinhole camera model the focal length is the same in x and y direction. This is not always the case for a camera with a lens and we will discuss this in a future article.

通过简单的几何参数(“相似的三角形”)，我们可以轻松地从每个像素的u和d得出位置x。 下图仅显示x和u，但对于y和v可以完全相同。对于针Kong相机模型，x和y方向的焦距相同。 带镜头的相机并非总是如此，我们将在以后的文章中对此进行讨论。