Abstract

A novel method for registering imagery with Light Detection And Ranging (LiDAR) data is proposed. It is based on the phenomenon that the back-projection of LiDAR point cloud of an object should be located within the object boundary in the image. Using this inherent geometrical constraint, the registration parameters computation of both data sets only requires LiDAR point clouds of several objects and their corresponding boundaries in the image. The proposed registration method comprises of four steps: point clouds extraction, boundary extraction, back-projection computation and registration parameters computation. There are not any limitations on the geometrical and spectral properties of the object. So it is suitable not only for structured scenes with man-made objects but also for natural scenes. Moreover, the proposed method based on the inherent geometrical constraint can register two data sets derived from different parts of an object. It can be used to co-register TLS (Terrestrial Laser Scanning) LiDAR point cloud and UAV (Unmanned aerial vehicle) image, which are obtaining more attention in the forest survey application. Using initial registration parameters comparable to POS (position and orientation system) accuracy, the performed experiments validated the feasibility of the proposed registration method.

© 2015 Optical Society of America

Full Article  |  PDF Article
OSA Recommended Articles
Hyperspectral lidar point cloud segmentation based on geometric and spectral information

Biwu Chen, Shuo Shi, Jia Sun, Wei Gong, Jian Yang, Lin Du, Kuanghui Guo, Binhui Wang, and Bowen Chen
Opt. Express 27(17) 24043-24059 (2019)

Synergistic application of geometric and radiometric features of LiDAR data for urban land cover mapping

Yuchu Qin, Shihua Li, Tuong-Thuy Vu, Zheng Niu, and Yifang Ban
Opt. Express 23(11) 13761-13775 (2015)

References

  • View by:
  • |
  • |
  • |

  1. J. Zhang, “Multi-source remote sensing data fusion: status and trends,” International Journal of Image and Data Fusion 1(1), 5–24 (2010).
    [Crossref]
  2. H. Buddenbaum, S. Seeling, and J. Hill, “Fusion of full-waveform LiDAR and imaging spectroscopy remote sensing data for the characterization of forest stands,” Int. J. Remote Sens. 34(13), 4511–4524 (2013).
    [Crossref]
  3. A. V. Kanaev, B. J. Daniel, J. G. Neumann, A. M. Kim, and K. R. Lee, “Object level HSI-LIDAR data fusion for automated detection of difficult targets,” Opt. Express 19(21), 20916–20929 (2011).
    [Crossref] [PubMed]
  4. R. Mishra and Y. Zhang, “A review of optical imagery and airborne LiDAR data registration methods,” Open Remote Sensing Journal 5(1), 54–63 (2012).
    [Crossref]
  5. P. R. O. Nnholm, M. Karjalainen, H. Kaartinen, K. Nurminen, and J. Hyypp A, “Relative orientation between a single frame image and LiDAR point cloud using linear features,” Photogrammetric Journal of Finland 23(2), 1–16 (2013).
  6. E. Mitishita, A. Habib, and A. Machado, “Photogrammetric model orientation using LiDAR dataset,” in Proceedings of the 21st ISPRS Congress (Beijing, 2008), pp. 2488–2492.
  7. R. M. Palenichka and M. B. Zaremba, “Automatic extraction of control points for the registration of optical satellite and LiDAR images,” IEEE Trans. Geosci. Rem. Sens. 48(7), 2864–2879 (2010).
    [Crossref]
  8. E. G. Parmehr, C. S. Fraser, C. Zhang, and J. Leach, “Automatic registration of optical imagery with 3D LiDAR data using statistical similarity,” ISPRS J. Photogramm. Remote Sens. 88, 28–40 (2014).
    [Crossref]
  9. K. Hormann and A. Agathos, “The point in polygon problem for arbitrary polygons,” Comput. Geom. 20(3), 131–144 (2001).
    [Crossref]
  10. F. Rottensteiner, G. Sohn, M. Gerke, and J. D. Wegner, “ISPRS test project on urban classification and 3D building reconstruction,” (2013). http://www2.isprs.org/tl_files/isprs/wg34/docs/ComplexScenes_revision_v4.pdf
  11. J. Adeyemo and F. Otieno, “Optimizing planting areas using differential evolution (DE) and linear programming (LP),” Int. J. Phys. Sci. 4(4), 212–220 (2009).

2014 (1)

E. G. Parmehr, C. S. Fraser, C. Zhang, and J. Leach, “Automatic registration of optical imagery with 3D LiDAR data using statistical similarity,” ISPRS J. Photogramm. Remote Sens. 88, 28–40 (2014).
[Crossref]

2013 (2)

P. R. O. Nnholm, M. Karjalainen, H. Kaartinen, K. Nurminen, and J. Hyypp A, “Relative orientation between a single frame image and LiDAR point cloud using linear features,” Photogrammetric Journal of Finland 23(2), 1–16 (2013).

H. Buddenbaum, S. Seeling, and J. Hill, “Fusion of full-waveform LiDAR and imaging spectroscopy remote sensing data for the characterization of forest stands,” Int. J. Remote Sens. 34(13), 4511–4524 (2013).
[Crossref]

2012 (1)

R. Mishra and Y. Zhang, “A review of optical imagery and airborne LiDAR data registration methods,” Open Remote Sensing Journal 5(1), 54–63 (2012).
[Crossref]

2011 (1)

2010 (2)

J. Zhang, “Multi-source remote sensing data fusion: status and trends,” International Journal of Image and Data Fusion 1(1), 5–24 (2010).
[Crossref]

R. M. Palenichka and M. B. Zaremba, “Automatic extraction of control points for the registration of optical satellite and LiDAR images,” IEEE Trans. Geosci. Rem. Sens. 48(7), 2864–2879 (2010).
[Crossref]

2009 (1)

J. Adeyemo and F. Otieno, “Optimizing planting areas using differential evolution (DE) and linear programming (LP),” Int. J. Phys. Sci. 4(4), 212–220 (2009).

2001 (1)

K. Hormann and A. Agathos, “The point in polygon problem for arbitrary polygons,” Comput. Geom. 20(3), 131–144 (2001).
[Crossref]

Adeyemo, J.

J. Adeyemo and F. Otieno, “Optimizing planting areas using differential evolution (DE) and linear programming (LP),” Int. J. Phys. Sci. 4(4), 212–220 (2009).

Agathos, A.

K. Hormann and A. Agathos, “The point in polygon problem for arbitrary polygons,” Comput. Geom. 20(3), 131–144 (2001).
[Crossref]

Buddenbaum, H.

H. Buddenbaum, S. Seeling, and J. Hill, “Fusion of full-waveform LiDAR and imaging spectroscopy remote sensing data for the characterization of forest stands,” Int. J. Remote Sens. 34(13), 4511–4524 (2013).
[Crossref]

Daniel, B. J.

Fraser, C. S.

E. G. Parmehr, C. S. Fraser, C. Zhang, and J. Leach, “Automatic registration of optical imagery with 3D LiDAR data using statistical similarity,” ISPRS J. Photogramm. Remote Sens. 88, 28–40 (2014).
[Crossref]

Habib, A.

E. Mitishita, A. Habib, and A. Machado, “Photogrammetric model orientation using LiDAR dataset,” in Proceedings of the 21st ISPRS Congress (Beijing, 2008), pp. 2488–2492.

Hill, J.

H. Buddenbaum, S. Seeling, and J. Hill, “Fusion of full-waveform LiDAR and imaging spectroscopy remote sensing data for the characterization of forest stands,” Int. J. Remote Sens. 34(13), 4511–4524 (2013).
[Crossref]

Hormann, K.

K. Hormann and A. Agathos, “The point in polygon problem for arbitrary polygons,” Comput. Geom. 20(3), 131–144 (2001).
[Crossref]

Hyypp, J.

P. R. O. Nnholm, M. Karjalainen, H. Kaartinen, K. Nurminen, and J. Hyypp A, “Relative orientation between a single frame image and LiDAR point cloud using linear features,” Photogrammetric Journal of Finland 23(2), 1–16 (2013).

Kaartinen, H.

P. R. O. Nnholm, M. Karjalainen, H. Kaartinen, K. Nurminen, and J. Hyypp A, “Relative orientation between a single frame image and LiDAR point cloud using linear features,” Photogrammetric Journal of Finland 23(2), 1–16 (2013).

Kanaev, A. V.

Karjalainen, M.

P. R. O. Nnholm, M. Karjalainen, H. Kaartinen, K. Nurminen, and J. Hyypp A, “Relative orientation between a single frame image and LiDAR point cloud using linear features,” Photogrammetric Journal of Finland 23(2), 1–16 (2013).

Kim, A. M.

Leach, J.

E. G. Parmehr, C. S. Fraser, C. Zhang, and J. Leach, “Automatic registration of optical imagery with 3D LiDAR data using statistical similarity,” ISPRS J. Photogramm. Remote Sens. 88, 28–40 (2014).
[Crossref]

Lee, K. R.

Machado, A.

E. Mitishita, A. Habib, and A. Machado, “Photogrammetric model orientation using LiDAR dataset,” in Proceedings of the 21st ISPRS Congress (Beijing, 2008), pp. 2488–2492.

Mishra, R.

R. Mishra and Y. Zhang, “A review of optical imagery and airborne LiDAR data registration methods,” Open Remote Sensing Journal 5(1), 54–63 (2012).
[Crossref]

Mitishita, E.

E. Mitishita, A. Habib, and A. Machado, “Photogrammetric model orientation using LiDAR dataset,” in Proceedings of the 21st ISPRS Congress (Beijing, 2008), pp. 2488–2492.

Neumann, J. G.

Nnholm, P. R. O.

P. R. O. Nnholm, M. Karjalainen, H. Kaartinen, K. Nurminen, and J. Hyypp A, “Relative orientation between a single frame image and LiDAR point cloud using linear features,” Photogrammetric Journal of Finland 23(2), 1–16 (2013).

Nurminen, K.

P. R. O. Nnholm, M. Karjalainen, H. Kaartinen, K. Nurminen, and J. Hyypp A, “Relative orientation between a single frame image and LiDAR point cloud using linear features,” Photogrammetric Journal of Finland 23(2), 1–16 (2013).

Otieno, F.

J. Adeyemo and F. Otieno, “Optimizing planting areas using differential evolution (DE) and linear programming (LP),” Int. J. Phys. Sci. 4(4), 212–220 (2009).

Palenichka, R. M.

R. M. Palenichka and M. B. Zaremba, “Automatic extraction of control points for the registration of optical satellite and LiDAR images,” IEEE Trans. Geosci. Rem. Sens. 48(7), 2864–2879 (2010).
[Crossref]

Parmehr, E. G.

E. G. Parmehr, C. S. Fraser, C. Zhang, and J. Leach, “Automatic registration of optical imagery with 3D LiDAR data using statistical similarity,” ISPRS J. Photogramm. Remote Sens. 88, 28–40 (2014).
[Crossref]

Seeling, S.

H. Buddenbaum, S. Seeling, and J. Hill, “Fusion of full-waveform LiDAR and imaging spectroscopy remote sensing data for the characterization of forest stands,” Int. J. Remote Sens. 34(13), 4511–4524 (2013).
[Crossref]

Zaremba, M. B.

R. M. Palenichka and M. B. Zaremba, “Automatic extraction of control points for the registration of optical satellite and LiDAR images,” IEEE Trans. Geosci. Rem. Sens. 48(7), 2864–2879 (2010).
[Crossref]

Zhang, C.

E. G. Parmehr, C. S. Fraser, C. Zhang, and J. Leach, “Automatic registration of optical imagery with 3D LiDAR data using statistical similarity,” ISPRS J. Photogramm. Remote Sens. 88, 28–40 (2014).
[Crossref]

Zhang, J.

J. Zhang, “Multi-source remote sensing data fusion: status and trends,” International Journal of Image and Data Fusion 1(1), 5–24 (2010).
[Crossref]

Zhang, Y.

R. Mishra and Y. Zhang, “A review of optical imagery and airborne LiDAR data registration methods,” Open Remote Sensing Journal 5(1), 54–63 (2012).
[Crossref]

Comput. Geom. (1)

K. Hormann and A. Agathos, “The point in polygon problem for arbitrary polygons,” Comput. Geom. 20(3), 131–144 (2001).
[Crossref]

IEEE Trans. Geosci. Rem. Sens. (1)

R. M. Palenichka and M. B. Zaremba, “Automatic extraction of control points for the registration of optical satellite and LiDAR images,” IEEE Trans. Geosci. Rem. Sens. 48(7), 2864–2879 (2010).
[Crossref]

Int. J. Phys. Sci. (1)

J. Adeyemo and F. Otieno, “Optimizing planting areas using differential evolution (DE) and linear programming (LP),” Int. J. Phys. Sci. 4(4), 212–220 (2009).

Int. J. Remote Sens. (1)

H. Buddenbaum, S. Seeling, and J. Hill, “Fusion of full-waveform LiDAR and imaging spectroscopy remote sensing data for the characterization of forest stands,” Int. J. Remote Sens. 34(13), 4511–4524 (2013).
[Crossref]

International Journal of Image and Data Fusion (1)

J. Zhang, “Multi-source remote sensing data fusion: status and trends,” International Journal of Image and Data Fusion 1(1), 5–24 (2010).
[Crossref]

ISPRS J. Photogramm. Remote Sens. (1)

E. G. Parmehr, C. S. Fraser, C. Zhang, and J. Leach, “Automatic registration of optical imagery with 3D LiDAR data using statistical similarity,” ISPRS J. Photogramm. Remote Sens. 88, 28–40 (2014).
[Crossref]

Open Remote Sensing Journal (1)

R. Mishra and Y. Zhang, “A review of optical imagery and airborne LiDAR data registration methods,” Open Remote Sensing Journal 5(1), 54–63 (2012).
[Crossref]

Opt. Express (1)

Photogrammetric Journal of Finland (1)

P. R. O. Nnholm, M. Karjalainen, H. Kaartinen, K. Nurminen, and J. Hyypp A, “Relative orientation between a single frame image and LiDAR point cloud using linear features,” Photogrammetric Journal of Finland 23(2), 1–16 (2013).

Other (2)

E. Mitishita, A. Habib, and A. Machado, “Photogrammetric model orientation using LiDAR dataset,” in Proceedings of the 21st ISPRS Congress (Beijing, 2008), pp. 2488–2492.

F. Rottensteiner, G. Sohn, M. Gerke, and J. D. Wegner, “ISPRS test project on urban classification and 3D building reconstruction,” (2013). http://www2.isprs.org/tl_files/isprs/wg34/docs/ComplexScenes_revision_v4.pdf

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (8)

Fig. 1
Fig. 1 Point clouds of a scene. White dots represent the point clouds of a tree crown and a building roof.
Fig. 2
Fig. 2 Optical image of the same scene. The back-projections of the white dots in LiDAR point clouds are labeled by white cross marks.
Fig. 3
Fig. 3 Three man-made control objects and their boundaries in Image A.
Fig. 4
Fig. 4 Three natural control objects and their boundaries in Image B.
Fig. 5
Fig. 5 Optical image (left) and overlay of back-projected LiDAR point cloud and optical image (right).
Fig. 6
Fig. 6 Point cloud of one TLS LiDAR scan.
Fig. 7
Fig. 7 Registration result represented on UAV image of the test area.
Fig. 8
Fig. 8 TLS LiDAR cannot acquire the upper portion of the tree crown.

Tables (2)

Tables Icon

Table 1 Bounds of the position and orientation parameters

Tables Icon

Table 2 Comparison of the registration parameters. GT stands for ground truth; RP stands for registration parameters; DF is the difference between GT and RP.

Equations (3)

Equations on this page are rendered with MathJax. Learn more.

x x 0 =f( a 1 (X X S )+ b 1 (Y Y S )+ c 1 (Z Z S ))/( a 3 (X X S )+ b 3 (Y Y S )+ c 3 (Z Z S )) y y 0 =f( a 2 (X X S )+ b 2 (Y Y S )+ c 2 (Z Z S ))/( a 3 (X X S )+ b 3 (Y Y S )+ c 3 (Z Z S ))
( a 1 a 2 a 3 b 1 b 2 b 3 c 1 c 2 c 3 )=( cosφcosκ cosφsinκ sinφ sinωsinφcosκ+cosωsinκ sinωsinφsinκ+cosωcosκ sinωcosκ cosωsinφcosκ+sinωsinκ cosωsinφsinκ+sinωcosκ cosωcosφ )
F(p)=1 1 n i=1 n R i

Metrics