Abstract

To match the virtual image and actual environment in an augmented reality (AR) system, it is necessary to complete the task of three-dimensional (3D) tracking registration. This paper proposes a new method for 3D tracking registration. Previous methods extract feature points in images to realize tracking registration. In this paper, the objects are extracted from the deep convolution neural network in the scene, and the camera pose is estimated by establishing the constraint relation of the objects. Then, 3D tracking and registration of the virtual object are realized. We design an improved single-shot multibox detector semantic segmentation network to identify and segment the scene and extract the pixel classification results of the objects in the scene. The effect of classification with this method is better. The depth of the extracted object is estimated based on the data from the left and right cameras, and the 2D image is converted into a 3D point cloud. A camera pose estimation method, combined with multiobjective information, is proposed. The camera transformation matrix is directly estimated by establishing a mathematical model. This method avoids the effect on the accuracy of the camera pose estimation when the feature points are not sufficient. Moreover, by assigning different weights to the point clouds of different objects, errors caused by the model can be reduced. The experimental results showed that the 3D registration method proposed in this paper is less than 2.5 pixels in the application scene of an augmented reality head-up display. This method had a better effect compared with that of existing methods and also improved driving safety.

© 2018 Optical Society of America

Tracking registration of optical see-through augmented reality based on the Riemannian manifold constraint

Zhe An and Yang Liu
Opt. Express 30(26) 46418-46434 (2022)

Augmented reality registration algorithm based on T-AKAZE features

Xiu Ji, Huamin Yang, Cheng Han, Jiayu Xu, and Yan Wang
Appl. Opt. 60(35) 10901-10913 (2021)

Fast and precise 6D pose estimation of textureless objects using the point cloud and gray image

Wang Pan, Feng Zhu, Yingming Hao, and Limin Zhang
Appl. Opt. 57(28) 8154-8165 (2018)

Accurate three dimensional body scanning system based on structured light

Haosong Yue, Yue Yu, Weihai Chen, and Xingming Wu
Opt. Express 26(22) 28544-28559 (2018)

Simple and efficient registration of 3D point cloud and image data for an indoor mobile mapping system

Hao Ma, Keke Liu, Jingbin Liu, Hongyu Qiu, Dong Xu, Zemin Wang, Xiaodong Gong, and Sheng Yang
J. Opt. Soc. Am. A 38(4) 579-586 (2021)