The binocular vision galvanometric laser scanning (BGLS) system, which consists of a traditional galvanometric laser scanning (GLS) system and a stereo vision system, is widely applied in various advanced manufacturing fields such as 3-dimensional (3D) laser cutting and 3D laser projection positioning. The BGLS system usually needs to be recalibrated before it is put to a certain use, since the camera parameters, as well as the relative pose between the camera and the GLS, may be changed with different applications. However, a full calibration of the BGLS system requires elaborate modeling and mass calibration data, which considerably affect the efficiency of the BGLS system. A rapid on-site recalibration method is proposed, which can substantially improve the efficiency and flexibility of the BGLS system. With the method, the BGLS system needs to be carefully off-site calibrated for only one time. The on-site precise recalibration can be quickly realized by taking only two images of the laser spots projected by the GLS on a planar board. Moreover, an ingenious linear solving method is proposed to make the whole computation process more stable and timesaving. On-site recalibration and target shooting experiments are respectively conducted to verify the proposed method.
© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
Due to its superiority in scanning speed and precision, the galvanometric laser scanning (GLS) system is commonly applied in various fields such as material processing [1–5], laser projection [6,7], shape measurement [8,9], and medical imaging [10,11]. However, the GLS system itself is hard to project or process specific features on irregular three-dimensional (3D) objects. Since the relative pose between the GLS system and the 3D object is typically unknown, it is difficult to generate the specific 3D laser trajectory on the object surface. Binocular stereo vision, as a noncontact 3D sensing approach, has the ability to measure the geometry of the 3D object and align it with the GLS system to the same coordinate system. Therefore, the binocular vision has been widely employed in various 3D laser applications, such as laser cutting/engraving [12,13], laser drilling  and laser projection positioning. The system consisting of a GLS and a binocular camera is called the binocular vision galvanometric laser scanning (BGLS) system in the context of this paper.
It is necessary for a BGLS system to be calibrated in advance to implement a specific application. Qi et al  calibrate the BGLS system to cut duck feathers using the method proposed in , in which the GLS system is adapted to a pinhole camera based on the similarities between the distortion model of the GLS and the camera. However, a real GLS system does not have an optical center as that of a camera, so more optimization parameters are used to ensure the calibration accuracy. Too many parameters may lead to hard and non-convex optimization problems with a growing risk for local minima. There exist some other methods that can be used to calibrate the BGLS system such as the physical-model-based method , the Look-Up-Table (LUT) based method [18,19] and data-driven based method . All these calibration methods establish the relationship between the input control signals of the GLS
To ensure the calibration result valid, the positional relation between the GLS system and the binocular camera must keep fixed once the calibration of the BGLS system has been completed. However, the fixed camera-GLS structure may greatly lower the flexibility of the BGLS system, since the binocular camera usually needs to independently adjust its position to guarantee the 3D objects be within the field-of-view. In some situations, even the lenses of the binocular system need to be changed to adapt to the 3D objects in different sizes and/or distances. The above changes will invalid the calibration results of the BGLS system. No matter what existing method is used, a full BGLS system calibration process is quite time-consuming, since the calibration commonly requires large amount of data samplings. The requirement for frequent and time-consuming calibration severely hinder the application of the BGLS system with unfixed binocular camera. Most existing GLS calibration methods focus on eliminating the distortion and improving the accuracy of the GLS [20,21], but few literature concerns the reuse of the calibration results in the case of binocular camera changes.
In order to increase the efficiency and the flexibility of the BGLS system, we propose a rapid on-site recalibration method to support the binocular camera adjustments while working on different 3D objects. To begin with, the mapping between the input control signals and the corresponding space vectors of the laser beams is obtained through a full off-site calibration process. The calibration result is saved and then reused in the following on-site recalibration. The recalibration can be quickly and precisely realized by taking only two images of the laser spots projected by the GLS on a planar board. It is worthy of notice that the proposed recalibration method works no matter what off-site full calibration method is used.
The rest of this paper is organized as follows. The flexible structure BGLS system configuration, the off-site calibration method we used, and the proposed recalibration method are described in Section 2. Experiments on recalibration and targets shooting are demonstrated in Section 3 to validate the proposed method. Finally, the paper is concluded in Section 4.
2.1 BGLS system configuration
As shown in Fig. 1, the BGLS system to be recalibrated is composed of two parts: the GLS system and the binocular vision system. The GLS system consists of a laser transmitter, a double-mirror galvanometric scanner and a controller. The transmitter emits a laser beam, which is then deflected by the dual mirrors of the galvanometric scanner. The controller is responsible for controlling the opening/closing of the laser transmitter and the rotation angles of the dual mirrors. The binocular vision system consists of two calibrated cameras, which can realize 3D reconstruction of feature points in the field-of-view. The GLS and the two cameras are not rigidly fixed together.
2.2 The full off-site calibration
For the sake of completeness, we first briefly introduce the off-site full calibration method we used. Let represent the coordinate system of the binocular vision system in the off-site calibration. Denote the digital voltage value applied to the motor of the first mirror as , and that of the second mirror as. The symbol represents . The outgoing laser beam corresponding to a specificis represented by. In, represents the six-dimensional (6D) vector of , where represents the direction of , and represents a point on. The mapping between the 2D digital control voltages and the 6D vector is denoted as .
It’s mentioned in [19,22] that the calibration result of the data-driven method is more accurate compared to that of others in case of enough data. As a result, we use a data-driven method  to off-site calibrate the BGLS system for achieving the mapping .The whole off-site calibration process is shown in Fig. 2, which contains two main steps: 1) acquiring the training data set, and 2) fitting the regression model of via statistical learning.
Firstly, we sample the training data in . As shown in Fig. 2, control signals is successively sent to the GLS system to emit laser beams into the field-of-view of binocular system. An opaque planar board is used to intercept the laser beams to form a grid of laser spots. The laser spot grid on the board is imaged by the binocular system. By means of the binocular stereo vision algorithm, 3D coordinates of the laser spots are generated in . We denote the 3D coordinates of the laser spots on one grid as . As the planar board moves to equally spaced positions along the approximate direction of the outgoing laser beams, data sets can be obtained. Denote the 3D coordinates of the laser spot on the grid as . The 6D vector of the laser beam in is then estimated by fitting . The fitting is performed by minimizing the error measure shown in Eq. (1)
Secondly, utilizing the training data set , we solve the mapping by a statistical learning method. More specifically, a single hidden layer feedforward neural network (SLFN) , which has been applied in many applications for its good learning ability, is chosen to model the mapping . Taking the pair of control signals as input and the parameters of the outgoing laser beam as output, we establish a SLFN as shown in Fig. 2. By using the learning methods of neural networks like BP  and ELM , the SLFN can be trained with the training data set. In terms of calculation efficiency, we use the ELM to train the SLFN. The fully trained SLFN uniquely determines the mapping .
2.3 Recalibration strategy
With the mapping achieved in Section 2.2, the BGLS system can be directly put to use under the precondition that the pose between the binocular system and the GLS system keeps unchanged. However, the vision system usually needs to be adjusted on-site for the reasons mentioned in Section 1. Since the mapping is relative to the coordinate system of the stereo camera , any adjustment of the binocular camera will invalid the original calibration result of the BGLS system. In order to avoid the full calibration process in the case of camera adjustment, a rapid on-site recalibration strategy is designed in this section.
The relative position between the laser source and the galvanometric scanner is fixed, and the repetitive accuracy of the scanner is usually in the order of 1 or even smaller. As a result, we can consider that the outgoing laser beam corresponding to an arbitrary GLS control signal is unchanged in GLS’s own coordinate system. Let represent the coordinate system of the binocular vision system in the on-site recalibration. It is noteworthy that the binocular system needs to be calibrated in advance of the BGLS system recalibration. Since the stereo camera calibration is a well-studied problem, we do not detail this issue here. Compared with the calibration of the cameras, the calibration of a GLS system is more time-consuming and inconvenient to operate. As shown in Fig. 3, we manage to reuse the original calibration via a coordinate transformation between and . Here, R and T represent respectively the rotation matrix and the translation vector between and. In , the laser beams corresponding to the digitals are denoted as , whose 6D space vector is . In , the laser beams corresponding to the same digitals are denoted as , whose 6D space vector is . In the recalibration process, a planar board is put to two different positions so that two grids of laser spots on the board are formed. The images of the two laser spot grids are captured by the on-site binocular system for calculating.
The two groups of space vectors and satisfy the following two equations:Eq. (2), the two groups of space vectors are aligned by minimizing the error measure in Eq. (3)Eq. (3). A good initial estimate is the key for preventing the optimization process from being trapped into local minima and shortening the convergence time. In the following two subsections, we propose a two-step linear method to estimate the initial values of and .
2.3.1 Initial estimation of rotation matrix
As shown in Fig. 4(a), the initial rotation matrix is estimated firstly. To achieve this, the 3D coordinates on the laser beam is replaced by the corresponding 3D coordinates . In this way, the group of laser beams are transformed into a group of laser beams, denoted as . Then by further applying a proper pure rotation to , it would go through the corresponding laser spots and. So, we have Eq. (4).
For reducing the number of parameters in, we use the Roderick matrix to replace the classic rotation matrix by Eq. (5)Eq. (5) into Eq. (4), we haveEq. (5).
2.3.2 Initial estimation of translation vector
The initial translation vector is also achieved by solving a linear equation system as shown in Fig. 4(b). By applying the estimated rotation matrix, the laser beams are transformed into. The direction vector of is , and the 3D coordinate of a point on is . If is exactly accurate, a unique pure translation exists to align the two groups of laser beams and . According to the fact that the points and are on the laser beam , we have the following two linear equations:Eq. (9).
The rank of the coefficient matrix of Eq. (9) is when. We can directly derive the translation vectorby the least square solution of the linear system. Theoretically, two unparallel laser beams are sufficient to uniquely determine the translation vector. As mentioned above, the number of the laser beams is much larger than 2 in practice to overcome the random errors.
2.3.3 Transformation parameters refinement
As mentioned in Section 2.3.1, we use the pointin to replace the corresponding point on. The point is achieved from the straight line fitting with many grids of laser points in the off-site full calibration. Obviously, its position accuracy is higher than , which comes from only one grid. Due to the replacement, the relative position accuracy of is accordingly lowered relative to the one of. To guarantee the recalibration accuracy, a global optimization for the transformation parameters is necessary. Taking and as the initial values, the transformation parameters refinement is performed by minimizing the objective function in Eq. (3). This optimization task is solved by the Levenberg-Marquardt  method.
3. Experiments and analysis
Figure 5 illustrates the specific hardware of the BGLS system used in the following experiments. The GLS system makes use of an economic 520nm semiconductor laser and a TSH8050A/D galvanometer (Century Sunny, Beijing, China). Both the laser transmitter and the galvanometric scanning head are controlled by a GT-400-Scan control board (GuGao, Shenzhen, China). The binocular system consists of a tripod, two MG 419B CMOS cameras (Schneider, German) and two lenses. The software for completing the whole experimental process is installed in a personal computer with 3.1GHz CPU and 8G RAM.
3.1 Full system calibration
Two lenses with 12mm focal length are used in the original full calibration. In the sampling process, 900 pairs of control digitals are inputted to generate 900 outgoing laser beams in the field-of-view of the binocular system. The 900 pairs of digitals are uniformly spaced on the virtual digital plane as shown in Fig. 6. A total of 100 laser spot grids (as shown in Fig. 2) are used for fitting the straight lines of the laser beams. From the first grid to the last grid , the planar board moved approximately 1 meter. The 3D coordinates of the laser spots in the 100 grids are obtained by the binocular system. The space vectors of the outgoing laser beams and subsequently the mapping is computed according to the methods in Section 2.2.
3.2 Recalibration experiment
For validating the proposed recalibration method, two common situations that require recalibration are simulated. In the first case as shown in Figs. 7(a) and 7(b), the binocular system is moved to different positions from the original place (see Fig. 5). In the second case, the camera lenses of the binocular system are changed as shown in Fig. 7(c). In the recalibration experiment, we use the planar board shown in Fig. 8(a) to acquire the laser spot grids (Fig. 8(b)).
To evaluate the performance of the proposed recalibration method, we use the root mean squared error (RMSE) measure in Eq. (10) to estimate the error of the solved coordinate transformation
The experimental results are listed in Table 1. The situations (a) and (b) in Table 1 respectively correspond to the situations in Fig. 7(a) and Fig. 7(b). The situations (c) and (d) in Table 1 respectively represent the situations that the original 12mm focal length camera lens are replaced with 17mm and 35mm focal length lenses as shown in Fig. 7(c).
As shown in Table 1, the recalibration error index in each situation is less than 0.11mm, and the running time of the recalibration process is about 5s. Here, the running time includes not only the time of global optimization but also the time for estimating the initial value . The experimental results indicate that the proposed recalibration method is robust and works well in four simulated situations.
The following issues should be taken into consideration in the recalibration process. 1) The inner structure of the GLS system is forbidden to change. If the relative position between the laser emitter and the two galvanometric mirrors are changed in the experiment, the BGLS system needs a full calibration again. 2) Too strong lighting environment should be avoided for the sampling of the laser spots. 3) The field-of-view of the changed binocular system should be able to cover the effective scanning region of the GLS.
3.3 Target shooting experiments
To further verify the proposed recalibration method, we conduct target shooting experiments right after the recalibration of each simulated situation. A pattern of 49 target circles (as shown in Fig. 9(a)) is placed in the depth of field of the binocular system. The 3D coordinates of the circle centers are obtained by the binocular system. By using the originally calibrated mapping, the recalibrated coordinate transformation , and the coordinates, the digital voltages for shooting the 49 target circles are achieved byFig. 9(b). The coordinates of the laser spot centers shot on the pattern are also obtained by the binocular system. Then we use the root mean squared deviation to measure the shooting precision as shown in Fig. 9(c). The shooting performances in the four simulated situations are shown in Fig. 10.
For comparison, the target shooting experiments are also implemented right after the original full system calibration. The only difference is that the coordinate transformation is no longer needed. Theoretically, the two grids of laser spots (as shown in Fig. 4) which are used to recalibrate the system also can be utilized to directly complete the original system calibration mentioned in Section 2.2. Taking this into consideration, we recalculate the 900 space vectors with only two laser spot grids used for recalibration of situation (d) in Section 3.2. Then another mapping is obtained via the same statistical learning process. Using and calibrated in Section 3.1 respectively, we perform two additional shooting experiments, the results of which are shown in Fig. 11. The root mean squared deviation of the two additional shooting experiments are respectively listed in the first two rows of Table 2. And of the four recalibration situations mentioned above are orderly listed in the last four rows in Table 2.
As listed in the last four rows of Table 2, the shooting accuracy in the four recalibration situations is around 0.3mm. It demonstrates the good robustness and accuracy of the proposed recalibration method. By comparing the last four rows in Table 2 with the second row, we find that the accuracy loss of the recalibration relative to the original full calibration is very small.
The first row of Table2 shows that the shooting accuracy (0.794mm) based on the mapping is significantly deteriorated compared to the accuracy (0.257mm) based on the mapping obtained in Section 3.1. This result reasonably illustrates that directly calibrating the system with only two laser spot grids is undesirable. In the original full calibration for establishing , 100 grids of laser spots within the range of about 1m were collected and used to fit the laser beams. The large amount of redundant sampling data had eliminated the influence of random errors. Whereas in the situation that only two laser spot grids were used to originally calibrate the system, each laser beam is fitted by only two points. Therefore the random errors in the 3D coordinates of the laser spots inevitably resulted in the accuracy decrease.
In the recalibration, the two laser spot grids are used for solving the coordinate transformation rather than for fitting the laser beams and then calculating the mapping. The two grids provide up to 3D points to jointly determine the transformation matrix. The redundancy can also effectively eliminate the random errors. That is the reason why two laser spot grids work well for the recalibration but are undesirable for the original calibration.
3.4 Projection positioning experiment
In some industry fields, the 3D laser curve projection positioning technique is widely applied to guide workers fulfilling relative tasks, such as prepreg ply placement in composite material part fabrication, logo painting for product decoration, parts/components assembling, quality inspection and so on. By the BGLS system recalibrated in Section 3.2, we can implement the projection positioning of 3D contour curves on a freeform surface. The CAD model of a car door is shown as in Fig. 12(a), and the two red contours are the designed target contours to be laser projected. Before the projection, the transformation matrix from the model system to the binocular system was calculated by utilizing some visual feature points placed on the surface of the car door. The 3D coordinates of these feature points in the model system, together with the car door surface, are measured in advance. In real production process, the object to be laser projected is typically fixed with a positioning jig. By setting technical holes/bosses on known positions of the jig, the relative geometric relationship between the holes/ bosses and the object can be strictly guaranteed by manufacturing process control. The centers of these technical holes/bosses can be used as the visual feature points for system alignment.
The two red contours are discretized into 137 sequential points and transform into the binocular system, denoted as. Utilizing , the originally calibrated mapping, and the recalibrated coordinate transformation , the digital voltages are obtained by solvingFig. 12(b). Whereas in the point scan mode, the laser transmitter is turned off between inputting any two adjacent pairs of digital signals, and the laser marking trajectory is pointwise as shown in Fig. 12(c). With human visual observation, the laser projection trajectories perfectly overlay the designed contour on the door surface. To further quantitatively evaluate the 3D laser projection accuracy, the 3D coordinates of the 137 laser spots actually projected on the door surface, denoted as, are obtained by the binocular system. The intuitive comparison between the 3D designed contour and the actual laser point contour is shown in Fig. 13(a). The Euclidean distances between and are calculated and illustrated in Fig. 13(b). The average projection positioning error is = 0.52 mm, and the maximum of is 0.98 mm, the standard deviation = 0.44 mm.
Based on the one-to-one mapping between the input control digital and the corresponding outgoing laser beam, we propose a novel recalibration strategy for the BGLS system. With this strategy, the originally calibrated mapping can be rapidly reused in case of vision system changes. The special two-step linear method for estimating the initial rotation matrix and translation vector successively makes the whole solving process of the recalibration robust and efficient. The proposed recalibration strategy can greatly enhance the flexibility and efficiency of the BGLS system while keep high accuracy. The results of the recalibration, the shooting experiments and the projection positioning experiment comprehensively demonstrate the superiority of the proposed method.
As mentioned in Section 3.2, a prerequisite of the proposed recalibration strategy is that the inner structure of the GLS system is fixed. The recalibration may fail if the relative positions between the laser emitter and the two galvanometric mirrors are changed. Fortunately, the structure usually keep unchanged once the installation of the GLS system is completed. Besides, too strong lighting environment could probably lower the image quality of the laser spots and reduce the recalibration accuracy.
National Natural Science Foundation of China (NSFC) (51575276).
1. P. M. Yang, Y. L. Lo, and Y. H. Chang, “Laser galvanometric scanning system for improved average power uniformity and larger scanning area,” Appl. Opt. 55(19), 5001–5007 (2016). [CrossRef] [PubMed]
2. S. Naoto and T. Yukio, “Processing system with galvano scanner capable of high speed laser scanning,” U. S. patent 9,720,225 (August 1, 2017).
3. G. Cuccolini, L. Orazi, and A. Fortunato, “5 Axes computer aided laser milling,” Opt. Lasers Eng. 51(6), 749–760 (2013). [CrossRef]
4. M. Jofre, G. Anzolin, F. Steinlechner, N. Oliverio, J. P. Torres, V. Pruneri, and M. W. Mitchell, “Fast beam steering with full polarization control using a galvanometric optical scanner and polarization controller,” Opt. Express 20(11), 12247–12260 (2012). [CrossRef] [PubMed]
5. A. Kaldos, H. J. Pieper, E. Wolf, and M. Krause, “Laser machining in die making—a modern rapid tooling process,” J. Mater. Process. Technol. 155(1), 1815–1820 (2004). [CrossRef]
6. K. Tetterington, “Two-dimensional laser projection system,” U. S. Patent Application 20050052720 (March 10, 2005).
7. J. P. Callison, J. S. Pease, and R. W. Pease, “Laser projection systems,” U. S. Patent 7,142,257 (November 28 2006).
8. M. A. Stafne, L. D. Mitchell, and R. L. West, “Positional calibration of galvanometric scanners used in laser Doppler vibrometers,” Measurement 28(1), 47–59 (2000). [CrossRef]
9. U. Haruo, O. Makoto, and O. Yasuomi, “Laser measurement apparatus,” U. S. patent 7,196,795 (March 27, 2007).
10. Y. Chang, C. Wen, C. Gu, and S. Chen, “Synchronization-free light sheet microscopy based on a 2D phase mask,” Optica 4(9), 1030–1033 (2017). [CrossRef]
11. F. Ernst, R. Bruder, T. Wissel, P. Stüber, B. Wagner, and A. Schweikard, “Real time contact-free and non-invasive tracking of the human skull: First light and initial validation,” Proc. SPIE 8856, 88561G (2013). [CrossRef]
12. J. Diaci, D. Bračun, A. Gorkič, and J. Možina, “Rapid and flexible laser marking and engraving of tilted and curved surfaces,” Opt. Lasers Eng. 49(2), 195–199 (2011). [CrossRef]
13. J. M. Lee, J. H. Jang, and T. K. Yoo, “Scribing and cutting a blue LED wafer using a Qswitched Nd: YAG laser,” Appl. Phys. A 70(5), 561–564 (2000). [CrossRef]
14. M. F. Chen and Y. P. Chen, “Compensating technique of field-distorting error for the CO2 laser galvanometric scanning drilling machines,” Int. J. Mach. Tools Manuf. 47(7), 1114–1124 (2007). [CrossRef]
15. L. Qi, S. Wang, Y. X. Zhang, Z. Q. Tang, H. Yang, and X. P. Zhang, “Laser cutting of irregular shape object based on stereo vision laser galvanometric scanning system,” Opt. Lasers Eng. 68, 180–187 (2015). [CrossRef]
17. A. Manakov, H. P. Seidel, and I. Ihrke, “A mathematical model and calibration procedure for galvanometric laser scanning systems,” in Vision, Modeling, and Visualization (VMV), P. Eisert, K. Polthier, and J. Hornegger, Eds. Berlin, Germany: Eurographics Association, 2011, pp. 207–214.
18. Y. I. Abdel-Aziz and H. M. Karara, “Direct linear transformation from comparator coordinates into object space coordinates in closerange photogrammetry,” in Proc. Symp. Close-Range Photogram., Falls Church, VA, USA, 1971, pp. 1–18.
19. T. Wissel, B. Wagner, A. Schweikard, P. Stüeber, and F. Ernst, “Data-driven learning for calibrating galvanometric laser scanners,” Sensors (Basel) 15(10), 5709–5717 (2015). [CrossRef]
20. M. F. Chen, Y. P. Chen, and W. T. Hsiao, “Correction of field distortion of laser marking systems using surface compensation function,” Opt. Lasers Eng. 47(1), 84–89 (2009). [CrossRef]
21. J. Xie, S. H. Huang, Z. C. Duan, Y. Shi, and S. Wen, “Correction of the image distortion for laser galvanometric scanning system,” Opt. Laser Technol. 37(4), 305–311 (2005). [CrossRef]
23. R. Setiono, “On the solution of the parity problem by a single hidden layer feedforward neural network,” Neurocomputing 16(3), 225–235 (1997). [CrossRef]
25. G. B. Huang, Q.-Y. Zhu, and C.-K. Siew, “Extreme Learning Machine: Theory and Applications,” Neurocomputing 70(1), 489–501 (2006). [CrossRef]
26. J. J. Moré, “The Levenberg-Marquardt algorithm: Implementation and theory,” Lect. Notes Math. 630, 105–116 (1978). [CrossRef]