1. Introduction

Camera calibration is a hot research topic in the field of computer vision [1–3]. The accuracy of camera calibration directly affects the measuring accuracy of a vision system, such as industry inspection, vision-based autonomous robot navigation, and virtual reality. Thus, a simple and high-accuracy camera calibration method is of great significance. Current camera calibration methods are primarily divided into two major classes: (1) target-based calibration methods, which are applicable to the calibration of vision measuring systems for precise dimensional measurement; and (2) camera self-calibration methods that do not require targets, which are applied to vision-based autonomous robot navigation and scene reconstruction. The calibration methods of the first class are an important topic under discussion in this paper.

By adopting a 3D target of two planes [4–6] for camera calibration, this method may achieve high accuracy, but it is now relatively less used because of difficulties in machining a 3D target. Zhang [7] introduced a simple, flexible, and high-accuracy method using a freely moving planar checkerboard target, which has been widely applied to camera calibration [8]. Additionally, some researchers adopted different calibration methods using various features on planar targets [9–11]. Camera calibration methods [12–14] using 1D targets can only calibrate a single camera under certain circumstances, thereby limiting the applications of 1D targets in machine vision-based measuring systems. Considerable camera calibration methods [15–21] have been proposed based on spherical target images, besides the abovementioned target forms. The proposed method determines the intrinsic camera parameters based on the SCs. Given the particularity of a spherical target that is different from conventional 3D targets, a camera shooting a spherical target at any angle may obtain a contour image. This method is ideal for the synchronous calibration of the intrinsic parameters for multiple cameras shooting at different angles. However, it is greatly affected by image noise, so its accuracy does not satisfy requirements in high-accuracy measuring system calibration. Other methods using spinning surface [22], parallel circles [23], and coaxial circles [24] have also been described in some articles, but they are unable to achieve high-accuracy calibration.

The calibration method using 2D target is flexible and convenient, with high calibration accuracy. It has been widely applied in camera calibration, but it is unsuited to synchronously complete the calibration of multiple cameras at different positions. Use of a spherical target allows a complete outer contour of the spherical target at any angle. Such a technique is suitable for the synchronous calibration of multiple cameras at different positions, but its calibration accuracy can hardly meet high-accuracy measurement requirements. Moreover, a certain requirement must be met in the location of the spherical target: if the location is too close to the center of the camera, the calibration will be inaccurate. However, if it is too far, the contour acquisition and elliptic fitting may be subject to camera distortion, thereby affecting the calibration accuracy.

Based on the above analysis, this paper presents a new target form, namely, the grid spherical target (GST), which combines the strengths of the spherical and planar targets. The GST is different from the spherical target has been proposed in [15–21], mainly, it calibrates the camera using the latitude and longitude circles and the intersection points between latitude and longitude circles. Meanwhile, the GST-based method in this paper allows multiple cameras to shoot simultaneously and calibrate their respective intrinsic parameters, while achieving the same calibration accuracy as that using planar target. During the calibration, the linear solutions of the intrinsic and extrinsic parameters are obtained from the elliptic curves of the latitude and longitude circles in the image. Based on the coordinates of the intersecting points of the elliptic curves of the longitude and latitude lines in the image and the lens distortion, the nonlinear optimization method is adopted to obtain the optimum solutions of the intrinsic and extrinsic parameters.

About the GST method, there are some limitations. Obviously, it is difficult to achieve high accuracy manufacturing compared with checkerboards, and it can’t be applied widely because of manufacturing technology. The GST in this paper is manufactured by five-axis NC machine. In the future, with the improvement of manufacturing technology, it can be made from ceramics, and the grid on it can be manufactured by laser corrosion method and the manufacturing accuracy of it will be improved. In addition, it is an advantage of GST to achieve calibration of cameras at different positions. But if there are two cameras placed along the same horizontal axis as the sphere, then the images acquired by both cameras would be identical. So if we need to calibrate multiple cameras in the future, to avoid this ambiguity, some marker points should be pasted on the longitude intervals such as 40°, 60°, 90 °, 120°, 180°, 240° and so on. These marker points will help us to distinguish different cameras from images.

The rest of this paper is organized as follows. In section 2, a mathematical model for grid spherical imaging is introduced. The method principle of the proposed method is introduced in detail in Section 3. The proposed calibration method is verified by simulation experiments and physical experiments in Section 4. Conclusions are drawn in Section 5.

2. Mathematical model for grid spherical imaging

Figure 1 shows the model for grid spherical target imaging. $O_{\text{c}}{x}_{\text{c}}{y}_{\text{c}}{z}_{\text{c}}$ represents the camera coordinate system; $O_{\text{t}}{x}_{\text{t}}{y}_{\text{t}}{z}_{\text{t}}$ represents the target coordinate system; $C_{\text{li}(i)}$represents the equation for the i^th circle of longitude under the coordinate system $O_{\text{li(}i\text{)}}{x}_{\text{li(}i\text{)}}{y}_{\text{li(}i\text{)}}{z}_{\text{li(}i\text{)}}$_; and $e_{\text{li}(i)}$ represents the elliptic curves of $C_{\text{li}(i)}$in the image. i = 1,2…N where N is the number of circles of longitude, $C_{\text{la}(j)}$is the equation for the j^th circle of latitude under the coordinate system $O_{\text{la(}j\text{)}}{x}_{\text{la(}j\text{)}}{y}_{\text{la(}j\text{)}}{z}_{\text{la(}j\text{)}}$, and $e_{\text{la}(j)}$ represents the elliptic curves of $C_{\text{la}(j)}$ in the image. j = 1,2…M where M represents the number of circles of latitude. The origin of coordinates and Z-axis of every $O_{\text{li(}i\text{)}}{x}_{\text{li(}i\text{)}}{y}_{\text{li(}i\text{)}}{z}_{\text{li(}i\text{)}}$are the same as those of $O_{\text{t}}{x}_{\text{t}}{y}_{\text{t}}{z}_{\text{t}}$. The coordinate axis of every $O_{\text{la(}j\text{)}}{x}_{\text{la(}j\text{)}}{y}_{\text{la(}j\text{)}}{z}_{\text{la(}j\text{)}}$ is parallel to that of $O_{\text{t}}{x}_{\text{t}}{y}_{\text{t}}{z}_{\text{t}}$, but only a fixed value separates their origins of coordinates in the Z-axis direction.

 

Fig. 1 Schematic of the calibration process.

Download Full Size | PDF

$q_{}={[x_{},y_{},z_{},1]}^T$ represents the 3D coordinates of an arbitrary point on the target under the target coordinate frame $O_{\text{t}}{x}_{\text{t}}{y}_{\text{t}}{z}_{\text{t}}$. $p_{}={[u_{},v_{},1]}^T$ represents the undistorted homogeneous image coordinates of $q$ in the image. According to the pinhole camera model, they satisfy the following equation:

3. Method principle

In the calibration process, the initial solutions of intrinsic and extrinsic parameters are obtained from the elliptic curves of the latitude and longitude circles in the image. Then, the nonlinear optimization method corresponding with the coordinates of the intersection points of the longitude and latitude lines in the image is adopted for optimum solutions of the intrinsic and extrinsic parameters.

3.1 Extracting and fitting the image of the elliptic curves of the latitude and longitude circles

The Steger method [25] is used to extract the center of the image of the longitude and latitude lines on the target and then link the extracted points to line segments. Two arbitrary segments are selected, and the first point and the last point in one of segments are compared to them of the other segment. If the distance between the two nearest neighboring points of them is less than 4 pixels, and the included angle between the normal vectors (n₁, n₂ in Fig. 2(a)) of segments (s₁, s₂ in Fig. 2(a)) is smaller than the threshold value, the two segments will be categorized into a set of curves. The rest can be deduced like that, until all segments are categorized into different sets of curves. The corresponding elliptic curve equations for all sets of curves are obtained by ellipse fitting [26].

 

Fig. 2 (a) Included angle between the normal vectors n₁ and n₂. (b) Seita angle between the long axis and the y-axis in the image.

Download Full Size | PDF

For all elliptic curves, the long axis, short axis length, center of the ellipse, and included angle between the long axis and the y-axis in the image are calculated. Based on the above parameters, all elliptic curves are divided into two classes: (1) elliptic curves in which the Seita angle (as shown in Fig. 2(b)) which are included angle between the long axis and the y-axis in the image, the long axis and center of the ellipse are basically the same, and the relative deviation is smaller than the threshold value, belonging to the elliptic curves of the circles of longitude; and (2) all elliptic curves other than those of the first class, belonging to the elliptic curves of the circles of latitude. The process is shown in Fig. 3.

 

Fig. 3 Extracting and fitting the image of the elliptic curves of the circles of latitude and longitude.

Download Full Size | PDF

3.2 Obtaining the linear solutions of the intrinsic and extrinsic parameters

3.2.1 Obtaining the linear solutions of the intrinsic camera parameters

According to the geometric basics of projection, all the circles spatially intersect the absolute conic (AC) [27] at two imaginary intersection points, i.e., I = (1,I,0,0)^T, J = (1,-I,0,0)^T_, also called the imaginary circular point (CP) [27], and the projected points of the CPs in the image are called the images of circular points (ICPs). Circles that are parallel or on a same plane intersect the absolute conic at the same two CPs. According to computer vision knowledge [27], the projected curve of the absolute conic (AC) in the image is called the Image of Absolute Conic (IAC)$\omega$, and the relations between $\omega$ and the intrinsic camera parameters are as below:

As shown in Fig. 4(a), all latitude circles on GST in space are parallel, so the intersection points between any latitude circle image $e_{\text{la(}j\text{)}}$ and $\omega$ are the same two ICPs $m_I^{}$ and $m_J^{}$($m_I^{}$ and $m_J^{}$ are the conjugate imaginary numbers). The line $l_{\text{la}}$ connecting $m_I^{}$ and $m_J^{}$is called the vanishing line [27] of the plane on which a latitude circle is in the image and all latitude circles have the same vanishing line $l_{\text{la}}$ in the image. Meanwhile, $l_{\text{la}}$intersects $e_{\text{la(}j\text{)}}$ or $\omega$at $m_I^{}$and$m_J^{}$. As shown in Fig. 4(b), $l_{\text{li(}i\text{)}}$ represents the vanishing line of the plane on which every longitude circle is, and $l_{\text{li(}i\text{)}}$ intersects $e_{\text{li(}i\text{)}}$ or $\omega$at two ICPs. If we know $l_{\text{la}}$ and $l_{\text{li(}i\text{)}}$, the ICPs on $\omega$can be obtained by the intersection points between $l_{\text{li(}i\text{)}}$ and $e_{\text{li(}i\text{)}}$, or $l_{\text{la}}$and $e_{\text{la(}j\text{)}}$.

 

Fig. 4 (a) Relationship between $e_{la}$, $l_{\text{la}}$and $\omega$; (b) Relationship between $l_{\text{li(}i\text{)}}$, $\omega$ and $e_{\text{li}(i)}$.

Download Full Size | PDF

Assuming that $c_{\text{la}(j)}={[x_{\text{la}(j)},y_{\text{la}(j)},z_{\text{la}(j)},1]}^{\text{T}}$ is the center of the latitude circle $C_{\text{la}(j)}$, $o_{\text{la}(j)}={[u_{\text{la}(j)},v_{\text{la}(j)},1]}^{\text{T}}$ represents the undistorted homogeneous coordinate of $c_{\text{la}(j)}$ in the image, and the undistorted homogeneous coordinate of the center of the largest latitude circle in the image is expressed by ${\tilde{o}}_{\text{la}(j)}$. $c_{\text{li}(i)}={[x_{\text{li}(i)},y_{\text{li}(i)},z_{\text{li}(i)},1]}^{\text{T}}$ represents the center of a longitude circle $C_{\text{li}(i)}$, and $o_{\text{li}(i)}={[u_{\text{li}(i)},v_{\text{li}(i)},1]}^{\text{T}}$ represents the undistorted homogeneous coordinate of $c_{\text{li}(i)}$ in the image. The undistorted homogeneous coordinate of the center of GST in the image is expressed as o, and this point is also the point $o_{\text{li}(i)}$ corresponding to the center of any longitude circle and the point${\tilde{o}}_{\text{la}(j)}$ corresponding to the center of the largest latitude circle in the image, namely, ${\tilde{o}}_{\text{la}(j)}=o_{\text{li}(i)}=o$.

Assume that $E=\left[\begin{array}{ccc}1& & 0\\ 0& & 1\\ -a& & -b\end{array}\begin{array}{c}\\ \\ \end{array}\begin{array}{c}\\ \\ \end{array}\begin{array}{c}-a\\ -b\\ a^2+b^2-r^2\end{array}\right]$ is the matrix expression of a circle in a space plane, and ${(x-a)}^2+{(y-b)}^2=r^2$ is the analytic expression of it. The homogeneous coordinate of circle center is $O=\left[\begin{array}{c}a\\ b\\ 1\end{array}\right]$ in the space and Eq. (3) shows that the product of E and O is a homogeneous vector of the infinite line [27].

Where $L_{\infty }=\left[\begin{array}{c}0\\ 0\\ 1\end{array}\right]$ is the infinite line, $\lambda$ is a non-zero factor.

Assume that H is the homography matrix from the plane on which the circle is to the camera image plane. In Eqs. (4)-(6), e is the matrix expression of the circle in the image, o_c is the image coordinate of O and l is the image of L .

According to Eqs. (4)-(6), Eq. (7) can be obtained,

So a polar relation exists between $o_{\text{la}(j)}$ and $l_{\text{la}}$ relative to $e_{\text{la}(j)}^{}$, as Eq. (8). Thus, by solving Eq. (8), $l_{\text{la}}$ is obtained

Similarly, polar relations exist between $o_{\text{li}(i)}$ and $l_{\text{li(}i\text{)}}$ relative to $e_{\text{li}(i)}$, as shown in Eq. (10)

In addition, o in Eqs. (9) and (11) can be obtained using the method in [17]. In [17], it is mentioned that if there are three spheres in different positions, three equations will be listed as follows:

As shown in Fig. 5, C₁, C₂, C₃ are contour images of spheres, C₁*, C₂*, C₃* are the dual of them respectively, and O₁, O₂, O₃ are the images of the center of spheres respectively. l₁₂ is an eigenvector of $C_2{C}_1^{\ast }$ corresponding to the eigenvalue ${\beta }_2/{\beta }_1$, and l₁₂ can be uniquely obtained from the eigenvectors of $C_2{C}_1^{\ast }$since it is the only line having two intersection points with both conics. l₁₃, l₂₃ can also be obtained from $C_3{C}_1^{\ast }$ and $C_2{C}_3^{\ast }$ respectively. Then O₁ can be calculated from cross-product of l₁₂ and l₁₃. Similarly O₂ and O₃ can also be obtained. So we can get the o from there GSTs.

 

Fig. 5 The relationship between centers and contours of three spheres in the image.

Download Full Size | PDF

Because $l_{\text{li(}i\text{)}}$intersects $e_{\text{li(}i\text{)}}$at two ICPs, and $l_{\text{la}}$ intersects an arbitrary $e_{\text{la}(j)}^{}$at the same pair of ICPs, shooting at least three longitude circles or the largest latitude circle combined with two longitude circles can produce $\omega$, and then camera intrinsic parameters can be solved by Cholesky decomposition.

3.2.2 Linear solution of extrinsic camera parameters

For the normal vector of plane $O_{\text{t}}{x}_{\text{t}}{y}_{\text{t}}$, namely, the normal vector of the plane on which all circles of latitude of the target can be found, the coordinate of the vanishing point in the image is $v_{\text{1}}$. For the normal vector of plane $O_{\text{t}}{x}_{\text{t}}{z}_{\text{t}}$, namely, the normal vector of the plane on which a longitude circle of the target is, the coordinate of the vanishing point in the image is $v_{\text{2}}$. $v_{\text{1}}$ and $v_{\text{2}}$ can be solved by Eq. (13):

Assume that the vectors along the z-axis and y-axis under $O_{\text{t}}{x}_{\text{t}}{y}_{\text{t}}{z}_{\text{t}}$ are ${\overrightarrow{d}}_{\text{1}}={[0,0,1]}^{\text{T}}$ and ${\overrightarrow{d}}_{\text{2}}=[0,1,0]$, respectively. The relationships between $v_{\text{1}}$ and ${\overrightarrow{d}}_{\text{1}}$ and $v_{\text{2}}$ and ${\overrightarrow{d}}_{\text{2}}$ are expressed as Eq. (14):

Using Eq. (14) and combining ${\overrightarrow{d}}_{\text{1}}={[0,0,1]}^{\text{T}}$ and ${\overrightarrow{d}}_{\text{2}}=[0,1,0]$, Eq. (15) can be obtained:

$r_1{\text{,}}_{}{r}_2{\text{,}}_{}{r}_3$ represent the orthogonal unit vectors, thereby obtaining Eq. (16):

Then, the rotation matrix R from $O_{\text{t}}{x}_{\text{t}}{y}_{\text{t}}{z}_{\text{t}}$to $O_{\text{c}}{x}_{\text{c}}{y}_{\text{c}}{z}_{\text{c}}$ is obtained using Eqs. (13)–(16).

As shown in Fig. 6, the expression for the j^th circle of latitude under the coordinate system$O_{\text{la(}j\text{)}}{x}_{\text{la(}j\text{)}}{y}_{\text{la(}j\text{)}}{z}_{\text{la(}j\text{)}}$ of the j^th circle of latitude is assumed as $C_{\text{la}(j)}^{}=\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& -d_j^2\end{array}\right]$, where $d_j$ represents the radius of the j^th circle of latitude. For the j^th circle of latitude$C_{\text{la}(j)}^{}$, the coordinate system$O_{\text{la(}j\text{)}}{x}_{\text{la(}j\text{)}}{y}_{\text{la(}j\text{)}}{z}_{\text{la(}j\text{)}}$ is parallel to the target coordinate system $O_{\text{t}}{x}_{\text{t}}{y}_{\text{t}}{z}_{\text{t}}$, but only the fixed value $D$ separates them in the z direction. The value $D$ represents the inherent dimension of the target, and it is a known quantity. Therefore, the rotation matrix $\tilde{R}$ from $O_{\text{la(}j\text{)}}{x}_{\text{la(}j\text{)}}{y}_{\text{la(}j\text{)}}{z}_{\text{la(}j\text{)}}$ to $O_{\text{c}}{x}_{\text{c}}{y}_{\text{c}}{z}_{\text{c}}$ and the rotation matrix $R$ from $O_{\text{t}}{x}_{\text{t}}{y}_{\text{t}}{z}_{\text{t}}$ to $O_{\text{c}}{x}_{\text{c}}{y}_{\text{c}}{z}_{\text{c}}$ are the same, but the translation vectors $\tilde{t}$ and $t$are different, and $\tilde{t}=\text{t}+R\times {\left[\begin{array}{ccc}0& 0& D\end{array}\right]}^T$as shown in Fig. 6. The sign of D should be noted, it is positive on the positive side of the z_t-axis but negative on the negative side.

 

Fig. 6 Schematic of solving the translation vector from $O_{\text{t}}{x}_{\text{t}}{y}_{\text{t}}{z}_{\text{t}}$ to $O_{\text{c}}{x}_{\text{c}}{y}_{\text{c}}{z}_{\text{c}}$.

Download Full Size | PDF

According to multi-view geometry foundation [27], the relationship between $C_{\text{la}(j)}^{}$ and $e_{\text{la}(j)}$ is as below:

By solving Eq. (17):

By consolidating Eq. (18):

Equation (19) can be obtained from an arbitrary circle of latitude. From the projected curves of any three circles of latitude on the target, a set of no less than three equations similar to Eq. (19) is enumerated. Using the least square method, $t$ that is from $O_{\text{li(}j\text{)}}{x}_{\text{li(}j\text{)}}{y}_{\text{li(}j\text{)}}{z}_{\text{li(}j\text{)}}$ to $O_{\text{c}}{x}_{\text{c}}{y}_{\text{c}}{z}_{\text{c}}$ can be solved.

3.3 Non-linear Optimization

The 3D coordinates of the intersection points on the GST under the target coordinate system are given. The intersection points are applied to solve the camera intrinsic parameters, thereby improving the calibration accuracy.

The nonlinear optimization is a two-stage process. First, the optimal solutions of the intrinsic and extrinsic parameters are obtained from the curves of the longitude and latitude lines, and the serial numbers are identified by perspective projection positioning. The identified intersection points of the latitude and longitude lines are introduced into the objective function for optimization, and the optimum solutions of the intrinsic and extrinsic parameters are solved by nonlinear optimization.

At the first stage, nonlinear optimization establishes the objective function for optimization with the minimum difference between the projected curve of a circle of latitude and the fitted curve of the extracted image after image distortion correction, as expressed in Eq. (20):

The 3D coordinates of the intersection points on the GST under the target coordinate system are given, and solving the intrinsic camera parameters according to the intersection points may improve the calibration accuracy. Therefore, the identified intersection points of the latitude and longitude lines are introduced into the objective function for optimization, and the optimum solutions of the intrinsic and extrinsic parameters are obtained by nonlinear optimization.

The objective function for optimization is established with the minimum difference between the projected point of the intersection point on the target and the image intersection point after image distortion correction, as shown in Eq. (21):

According to the objective function for optimization in Eq. (21), the nonlinear optimum solutions of the intrinsic and extrinsic parameters are obtained by the LM method.

4. Experimental results

The effectiveness of this method is proven in simulation and physical experiments. Simulation experiments lay special stress on analyzing the root-mean-square error (RMSE) of the results relative to the true value of the GST-based method at different noise levels and those of Zhang’s camera calibration method (Zhang’s) and spherical contour (SC)-based method. Physical experiments focus on comparing GST-based method and the SC-based method relative to the results of Zhang’s.

4.1 Simulation experiments

Simulation experiments are designed to analyze the calibration accuracy of the GST-based method, Zhang’s, and SC-based method at different noise levels. GST (diameter: 120 mm; longitude/latitude interval: 20/15 degrees). Subsequently, 6 × 6 feature points are selected for nonlinear optimization. Zhang’s adopts a planar target of 6 × 6 feature points, of which the horizontal spacing of the adjacent feature points is 10 mm and the vertical spacing is 16 mm. The diameter of the SC target is 100 mm, and the clear imaging range of the camera is 180–230 mm from the lens. The target is placed within the scope, and the intrinsic parameters matrix of the camera is$K=\left[\begin{array}{ccc}1024& 0& 400\\ 0& 960& 300\\ 0& 0& 1\end{array}\right]$. The lens distortion coefficients are $\begin{array}{cc}{k}_1=-0.1,& k_2=0.08\end{array}$.

For Zhang’s and GST-based method, their targets are placed 10 times respectively. The SC-based method adopts the SDP method in the literature [28], by which the camera intrinsic parameters are solved through 10 spherical targets at different positions. The methods are tested 100 times each at different noise levels. The different noise levels are added to the simulated ideal extracted points on latitude and longitude circles in the image, so the calibration results in the simulation experiment include the effect of the curve fitting method relative to noise.

The three calibration methods are assessed according to the camera calibration results, RMSEs relative to the true value, and reprojection errors of the feature points of the target on the image plane.

In Fig. 7, the RMSEs of f_x, f_y, u₀ and v₀ obtained by the SC-based method are about two to three times greater than those by Zhang’s and GST. The RMSEs of f_x, f_y, u₀ and v₀ by Zhang’s and the proposed method are basically the same. In terms of the RMSEs of k₁ and k₂, GST is slightly poorer than Zhang’s method, and those obtained by Zhang’s are about 2/3 of those by the proposed method.

 

Fig. 7 RMSEs of the intrinsic parameters based on Zhang’s method, GST-based method and SC-based method.

Download Full Size | PDF

The reprojection error of the feature points of the target image on the image plane refers to, under the target coordinate system, the error between the target image feature points that are projected onto the image coordinate system on the basis of the calibration results and the camera model and the coordinates of the feature points extracted in the image. Figure 8 shows the reprojection error of the feature points of the target image on the image plane in the u and v directions under the image coordinate system obtained by GST and Zhang’s, where u represents the x-coordinate under the image coordinate system, and v represents the y-coordinate under the image coordinate system. In Fig. 8, similar to Zhang’s method, reprojection error based on GST increases linearly with the noise level.

 

Fig. 8 Reprojection errors of the target points. (a) Reprojection errors in the u direction; (b) Reprojection errors in the v direction.

Download Full Size | PDF

4.2. Physical experiment

In physical experiments, the effectiveness of GST is validated by comparing the calibration errors of the GST and SC-based methods relative to Zhang’s. In experiments, the focal length of the camera lens is 16 mm, with the clear imaging range of 300–400 mm and the image resolution of 1628 × 1236 pixels. The feature point of the planar checkerboard target for calibration is 10 × 10, with the horizontal/vertical spacing of 17 mm between two adjacent feature points, and the target accuracy of 0.1 mm. The diameter of GST is 100 mm, and the longitude interval is 20 degrees. The latitude lines are vertically arranged on the latitude plane with the same interval and the vertical interval is 10 mm, and the accuracy is 0.1 mm. The diameter of the spherical target in the SC-based method is 50 mm.

The physical site map is shown in Figs. 9(a)-9(c), where Fig. 9(a) represents a camera using the checkerboard target for calibration, Fig. 9(b) represents a camera using the GST for calibration, and Fig. 9(c) represents a camera using the spherical target for calibration.

 

Fig. 9 Physical map of calibration setup. (a) Zhang’s method; (b) GST-based method; (c) SC-based method.

Download Full Size | PDF

Steger method has been used in [29–31]. In [31], Steger method is used in rail wear measurement and Section 4.1 in [29] shows that in good, dim and strong light environment, the light stripe can be obtained with similar accuracy. To verify it further, Figs. 10(a) and 10(b) show two sets of experiments demonstrating the curve fitting result on different light conditions. Firstly, the points on latitude and longitude lines in the image are extracted by Steger method, and then they are fitted by curve fitting method. In uniform dark or uniform bright situation, Fig. 10(a) and Fig. 10(b) show that two sets of experiments both achieve the same curve fitting results in uniform dark and uniform bright lighting. But in non-uniform lighting condition in Figs. 10(a) and 10(b), the curve cannot be fitted well because of reduced extracted points in latitude or longitude circles. So in actual calibration scene, the lighting condition should be relatively uniform to get high-accuracy curve fitting.

 

Fig. 10 Two sets of experiments about curve fitting in dark, bright and non-uniform lighting situation. (a) Set 1; (b) Set 2.

Download Full Size | PDF

During the experiment, planar target, GST, and spherical target are respectively laid out for 10 times, and the distances from the targets to the lens are about 300–400 mm. Zhang’s method directly adopts Matlab Toolbox in [32] for calibration. Given the field of view (F.O.V.) of the camera in Zhang’s, 6 × 7 or 7 × 7 feature points of the target are extracted from every position image (as shown in Fig. 11(a)). For the GST method, the number of intersection points extracted at every position is 64 or 72 because of the extraction of light stripes, and the target image is shown as Fig. 11(b). For the spherical target, the contour is extracted by Canny algorithm [33], the SDP method in the literature [17] is adopted, and the target image in Fig. 11(c) is obtained by solving from 10 spheres.

 

Fig. 11 Images used for calibration. (a) Checkerboard target images; (b) GST images; (c) Sphere images.

Download Full Size | PDF

As a result of high machining costs of high-accuracy GST, the GST with accuracy of 0.1 mm is temporarily used to verify the effectiveness of the proposed method. Thus, we recommend the use of the checkerboard target with the same accuracy for contrast test while choosing the planar target.

The calibration results of experiments are shown in Table 1. Zhang’s is extensively applied with high accuracy. Therefore, with the results from Zhang’s as the true value, the calibration errors of the GST and SC-based methods relative to Zhang’s are analyzed.

Table 1. Comparison of the Intrinsic Parameters

View Table

From the reprojection errors, GST and Zhang’s are further analyzed. As shown in Figs. 12(a) and 12(b) their reprojection errors are 0.21 pixels. Figures 12(c) and 12(d) show the statistics forms of the reprojection errors based on the two methods. From the figures, the reprojection error in Zhang’s is basically within 0–0.5 pixels, whereas that of GST is within 0–0.8 pixels. However, the reprojection error of GST is concentrated more within the range of 0–0.2 pixels compared with that of Zhang’s.

 

Fig. 12 Reprojection errors of the target feature points. (a) Reprojection errors distribution based on Zhang’s method; (b) Reprojection errors diatribution based on GST method; (c) The statistics form of the reprojection errors based on Zhang’s method; (d) The statistics form of the reprojection errors based on GST method.

Download Full Size | PDF

To further prove the effectiveness of the proposed method, the three methods are repeatedly tested. In the specific experiments, the spherical target, planar target, and GST are placed 10 times each, with 10 target images captured at different positions. Ten sets of images are chosen for camera calibration from the 10 images, and each set contains seven images. Figures 13(a)-13(d) show the box charts and fitting curves for statistical analysis of the calibration results of 10 sets of images using planar target, GST and SC.

 

Fig. 13 Repeatability results analysis based on the three methods. (a) Repeatability of f_x based on the three methods; (b) Repeatability of f_y based on the three methods; (c) Repeatability of u₀ based on the three methods; (d) Repeatability of f_x based on Zhang’s method and GST; (e) Repeatability of f_y based on Zhang’s method and GST; (f) Repeatability of u₀ based on Zhang’s method and GST; (g) Repeatability of v₀ based on the three methods; (h) Repeatability of r based on the three methods; (i) Repeatability of v₀ based on Zhang’s method and GST; (j) Repeatability of r based on Zhang’s method and GST.

Download Full Size | PDF

Figures 13(a) and 13(b) show the statistics results of the effective focal length based on the three methods. To clarify the results, Figs. 13(d) and 13(e) show a comparison between Zhang’s and GST with their respective results enlarged. In the figures, the asterisk in black indicates the results of the 10 sets of images using each of the methods. First, along the ordinate direction, the box of the SC is the longest among the three methods, indicating that it has the highest data dispersion, and the mean values (marked with dotted horizontal lines in purple) of the results of the SC-based method differ greatly from the other two methods. As shown in Figs. 13(d) and 13(e), along the direction of ordinate, the box length of GST is basically the same as that of Zhang’s. However, according to the fitted normal distribution curves, 2.5 and 10 pixels separate their mean values respectively, but their dispersion degrees are similar. Figures 11(c) and (g) compare the results of the center coordinates of the three methods. From the figures, the dispersion of the SC-based method is high, and the mean values are greater than those of GST and Zhang. According to the normal distribution and box shown in Fig. 13(f), there is a difference of no more than five pixels between the mean value of u₀ of Zhang’s and that of GST. The normal distribution and box in Fig. 13(i) are similar in the dispersion of v₀, but a large difference of about 20 pixels is observed between their mean values. Figure 13(h) shows the statistical results of r obtained by the three methods, and the dispersion and deviation from the SC-based method are larger than those from the other two method. In Fig. 13(j), the difference between the mean value of r obtained by Zhang’s and that by GST is about 2.5, but they are similar in terms of dispersion.

Figures 14(a) and 14(b) show a comparison of the reprojection errors in the two directions of u and v. In the direction of u, there is a difference of 0.05 pixels between GST and Zhang’s in the reprojection errors, and the dispersion obtained by GST is lower than Zhang. In the direction of v, there is a difference of 0.015 pixels between GST and Zhang’s in the mean reprojection errors, and the dispersion obtained by GST is slightly lower than Zhang. Thus, the repeatability of the calibration results show that results of the GST method are similar to those of Zhang’s method.

 

Fig. 14 Reprojection errors of the target feature points.

Download Full Size | PDF

5. Conclusions

This paper presents a camera calibration method based on the GST. By combining the circles of latitude and longitude with the intersection points, this method achieves calibration of the intrinsic and extrinsic parameters. Considering target machining errors, the 3D coordinates of the intersection points, as the parameters for optimization, are introduced into nonlinear optimization, thereby further improving the calibration accuracy. The proposed method presents the strengths of both spherical target-based calibration and planar target-based calibration, while overcoming their respective shortcomings and the same calibration accuracy as Zhang’s can be obtained. In physical and simulation experiments, this method is compared with the current mainstream calibration methods, namely, Zhang’s and the SC-based method. The results prove the effectiveness of this method. Meanwhile, the GST introduced in this method has important application value in the calibration of vision measuring systems, including structured light vision sensor and multi-vision sensor without common F.O.V.