1. Introduction

Benefiting from the cooperation of wide-angle (WA) and high-resolution (HD) sub-eyes, the HeCECam exhibits great potential for practical applications, such as UAV navigation, wide-area surveillance, and fast location tracking [1]. Unlike other 3D imaging techniques, the compound eye system relies on optical imaging to obtain a number of 2D sub-images that contain spatial 3D information, from which the 3D parameters of the objects can be inverted. This purely visual 3D imaging approach is more adaptable and can be applied to outdoor 3D measurements. However, precisely extracting the 3D information of the objects from the 2D sub-images is a huge challenge for curved compound eye systems [2–4]. The key to solving this problem is to optimize the structure of the HeCECam to enhance the imaging performance, which can improve the accuracy and reliability of the image data.

The distribution unevenness of sub-eyes is a direct factor that affects the amount of and the stability of the FOV overlap, and indirectly reduces the accuracy of 3D positioning for the artificial compound eye. Within the realm of existing curved compound eye systems, the distribution modes of sub-eyes can primarily be categorized into three types: toroidal array [1,5,6], hexagonal array [7–10], and spherical subdivision based on the ortho-polyhedron [4,11,12]. Despite the toroidal array’s advantage of simplicity in design, it suffers from numerous drawbacks, for instance, a large distribution unevenness of the sub-eyes about 153% (calculated as the ratio of the maximum and minimum intervals of adjacent sub-eyes) and a lack of regional symmetry. These problems will increase the design and operating costs of the subsequent image processing algorithms. The hexagonal array mode compensates for the lack of regional symmetry, but the distribution unevenness of sub-eyes inversely increases to about 328%. It’s universally accepted that the hexagonal structure can be expanded infinitely on a plane, but not on a sphere. As the sub-eyes approach the equator line, the intervals between adjacent sub-eyes decreases until they interfere with each other. Spherical subdivision based on the ortho-polyhedron is a process that the surfaces of an ortho-polyhedron are subdivided into a number of ortho-polygons and then their vertices are mapped onto the sphere as the positions of the sub-eyes. Typified by the ortho-icosahedron, this subdivision mode has absolutely strict symmetry and reduce the distribution unevenness of sub-eyes to 138% [13]. It is a more preferable subdivision method, but still falls short of the ideal.

In 2011, Son et al. proposed an optimization to the traditional subdivision based on ortho-icosahedron [13], effectively diminishing the distribution unevenness of sub-eyes to 118.8% [12]. It obtained the optimum solution by introducing deformation coefficients and constraints. However, this method requires complex calculations. In this work, an optimization method for sub-eye distribution named TGS based on the traditional ortho-icosahedron subdivision is proposed, which reduces the distribution unevenness of sub-eyes to 117%. Moreover, the core of TGS lies in geometrical design rather than complex computation to gain great results. In brief, the basic process of the TGS can be summarized as follows: 1) The vertices on the arrises of the ortho-icosahedron are first optimized to ensure that there are strictly equal intervals between them when they are mapped onto the circumsphere of the ortho-icosahedron. 2) With these mapped points as a reference, the surface of the ortho-icosahedron is then subdivided along three different directions to obtain a number of subdivision points, and the subdivision points belonging to one of directions do not coincide with those belonging to the other two directions. 3) The adjacent subdivision points from different directions are categorized into a cluster, and the center of gravity of each cluster is the optimized point of the surface of the ortho-icosahedron. We have adopted TGS to re-design the heterogeneous compound eye array, addressing the problems of large distribution unevenness of sub-eyes and the lack of regional symmetry that exist in the previous toroidal design.

The match between the heterogeneous compound eye array and the optical relay system is another crucial factor influencing the imaging quality and data accuracy of the system. The optical relay system of the HeCECam parallels the function of the rhabdom in the insect’s compound eye [14–16], which re-images the curved focal surface formed by the heterogeneous compound eye array on a CMOS sensor. It is in essence a large FOV fisheye lens with a limited object distance, which inevitably suffers from significant pupil aberration [17]. The pupil aberration causes great deviation of the entrance pupil position at different FOVs. However, the sub-eyes are distributed on the hemisphere, with their optical axes intersecting at the hemisphere’s center. When a sub-eye match with the optical relay system, the deviation between the entrance pupil and center of the hemisphere results in the tilt of the central ray (The light with the incident angle of 0°) from the optical axis of the sub-eye, especially noticeable for sub-eyes located at the edge. A number of imaging defects are associated with such tilt, for instance, increased stray light, distortion and image quality degradation of the sub-images, crescent-shaped black edges in the sub-images, support structures of WA sub-eyes being imaged, and the FOV of HD sub-eyes being intruded. Moreover, the optical relay system, as a single-aperture optical system, inherently has the inevitable defect of gradual degradation of image quality as the FOV expands. These problems pose great challenges to the imaging performance of the system, especially in the edge FOV. In this work, the optical architecture of the HeCECam has been re-designed to focus on compensating for the tilt of each sub-eye channel.

Based on the TGS and compensation method for tilt, an optimized HeCECam is fabricated. To verify the improvements in imaging performance of it, a series of comparison imaging experiments are performed, including blank scene imaging, FOV tests, resolution verification and real-world scene imaging.

2. Optimization of distribution uniformity of compound eyes

The distribution uniformity of the sub-eyes directly affects their FOV overlap. To achieve higher uniformity of sub-eyes, we propose TGS based on the traditional ortho-icosahedron subdivision. The concept of traditional ortho-icosahedron subdivision is shown in Fig. 1(a). Each surface of an ortho-icosahedron is an equilateral triangle with identical properties. Here, one of surfaces of the polyhedron is represented by the gray triangle. This surface is uniformly subdivided into a number of sub-triangles with subdivision density s (In Fig. 1(a), $s = 8$). All the vertices are numbered as $(u,v)$, where $u,v \in [0,s]$. These numbered vertices are mapped onto the circumsphere of the ortho-icosahedron as the positions of sub-eyes, as shown by the red points in Fig. 1(b). Although the distribution unevenness of sub-eyes produced by traditional method is 138%, there is still room for improvement.

 

Fig. 1. Optimization of compound eye distribution. (a) Traditional ortho-icosahedron subdivision. (b) Spherical mapping for traditional ortho-icosahedron subdivision. (c) Illustration of TGS for the points on an arris of the ortho-icosahedron. (d) Optimization results for points on the arrises. (e) Schematic diagram of TGS for points inside the surface of the ortho-icosahedron. (f) Spatial averaging for each cluster. (g) Comparison of results of traditional ortho-icosahedron subdivision and the TGS. (h) Statistics of the angle of adjacent subdivision point.

Download Full Size | PDF

The proposed TGS optimizes the traditional method, reducing the unevenness to 117%. We summarize TGS need to meet three principles: 1) The vertices of the ortho-icosahedron remain stationary; 2) The vertices of the sub-triangles located on arrises of the polyhedron can only be moved along the arrises; 3) The positions of the subdivision points inside the surface are determined by the optimized points located on the arrises of the polyhedron, instead of the vertices of the sub-triangles. Guided by these principles, the TGS is described in detail as follows:

Figure 1(c) shows the optimization process for points on the arrises. We define the coordinates of the three vertices of the discussed surface as (Establish a cartesian coordinate system with the center O of the ortho-icosahedron as the origin):

Similarly, calculations are done in the same way for the arrises $BC$ and $AC$ to obtain the corresponding coordinates of optimized points.

Figure 1(d) shows the optimization results for the points on the arrises, where the red and blue points represent the unoptimized and optimized positions, respectively. It can be seen that there is a large unevenness near the vertices of the polyhedron when it is unoptimized, while the intervals between adjacent optimized points are strictly equal.

Figure 1(e) illustrates the subdivision process inside the surface along the directions $\alpha$, $\beta$ and $\gamma$. In the case of direction $\alpha$, $EABi$ and $EACi$ are a pair of optimized points on the arrises $AB$ and $AC$. They are connected along the circumsphere to obtain $\mathop{{EABiEACi}}\limits^\frown$ (Fig. 1(e), black solid line). A sector $EABiOEACi$ is formed by $\mathop{{EABiEACi}}\limits^\frown$ and the origin O (not drawn in Fig. 1(e)). Assuming that point G is sliding along $\overrightarrow {EABiEACi}$, the coordinates of it can be determined by:

The same calculation is performed for the other two directions $\beta$ and $\gamma$ to obtain the corresponding subdivision points inside the surface, as marked by the green and pink dots in Fig. 1(e), respectively. It is not hard to find that the positions of the points subdivided along different directions do not coincide. The degree of non-coincidence is lower near the vertices of the ortho-icosahedron (Fig. 1(e), red circle), and a larger deviation exists at the center of the surface (Fig. 1(e), brown circle). The adjacent points from different subdivision directions are defined as a cluster. To optimize the non-coincidence, each cluster is spatially averaged. The blue points in Fig. 1(f) are the final optimized positions, whose coordinates are calculated by:

Figure 1(g) shows the results of TGS optimization. It clearly reveals that the large unevenness generated by the traditional ortho-icosahedron subdivision is significantly diminished. To quantitatively measure the optimization effect of TGS, we statistics the angles ($\theta$) of adjacent subdivision points relative to the origin O before and after TGS, as shown in Fig. 1(h). The red and blue fold lines represent the unoptimized and optimized values of $\theta$, respectively. The blue one is more stable and less discrete compared to the red one, which means that the sub-eye distribution will be more uniform after TGS optimization. The parameter κ is defined to indicate the degree of unevenness of the subdivision points, which is calculated by Eq. (8). A larger value of $\kappa$ indicates a more uneven sample.

The above calculations are performed on one of the surfaces of the ortho-icosahedron. If the coordinates of all subdivision points on the circumsphere need to be obtained, they can be easily solved by using the Rodrigues’ rotation formula [18] in conjunction with the symmetry of the polyhedron.

3. Enhancement of the compatibility between heterogeneous compound eyes and the optical relay system

Pupil aberration is an unavoidable defect for the optical relay system with a large FOV. Figure 2(a) illustrates the concept of the pupil aberration. The incident rays from different FOVs intersect at different positions on the system’s optical axis, indicating that the entrance pupil position gradually changes with the expansion of the FOV. The pupil aberration has a large impact on the match of the heterogeneous compound eye array and the optical relay system. Figure 2(b) shows the relationship of them. All sub-eyes’ optical axes intersect at the center of the heterogeneous compound eye array (Fig. 2(b), black pentagram), and there is a varying degree of deviation between this center and the entrance pupil of the optical relay system at each FOV (Fig. 2(b), the enlarged green box). This results in the central ray of each sub-eye channel being tilted towards the sub-eye’s optical axis (Fig. 2(b), the different colored optical paths represent the central ray for each sub-eye channel).

 

Fig. 2. Tilt compensation between sub-eyes and the optical relay system. (a) Pupil aberration of the optical relay system. (b) Illustration of the tilt. (c) Asymmetry of the sub-eye’s FOV due to the tilt. (d) The structure of heterogeneous compound eye array. (e) A raw compound eye image taken by the HeCECam in a lab. (f) Enlarged view of the area marked by the red circle in (e). (g) Enlarged view of the WA sub-image marked by the blue circle in (f). (h) Illustration of compensation calculation for tilt. (i) Relationship between pitch angle $\varphi$ and tilt angle t.

Download Full Size | PDF

The tilt causes an asymmetry in sub-eyes’ FOV. Figure 2(c) shows the FOV range of the sub-eye channel 6 (Ch.6), where the light from −3° to +12° can enter the system. Although this asymmetry does not affect the imaging of the HeCECam, the FOV of each sub-eye channel is inconsistent, which would complicate the optical design of the system. In addition, it makes the sub-eyes face the optical relay system with a side-tilted attitude and introduces obvious imaging defects. Figure 2(d) shows the structure of the heterogeneous compound eye array. The WA and HD sub-eyes are distributed on two concentric hemispherical surfaces with different radii. The HD sub-eyes with long focal lengths are fixed on the outer surface, while the WA sub-eyes need to be embedded in tapered support structures inside the shell (Fig. 2(d), green boxes). Due to the tilt, the lateral cone of the support structure is imaged in the raw compound eye image (Fig. 2(e), green box). The bright rings appearing at the edges of several sub-images in Fig. 2(e) were caused by the scattered light from the external edges of the sub-eye lenses when they were directly illuminated by the light source, which can be effectively suppressed by wrapping a layer of black elastomer ring around the outer edge of each of the sub-eye lens. Figure 2(f) shows enlarged view of the area marked by the red circle in Fig. 2(e). As can be seen, the problems of increased stray light (Fig. 2(f), white dashed box) and the FOV of HD sub-eyes being intruded (Fig. 2(f), yellow dashed ellipse) ensue. Moreover, defects such as distortion, loss of image quality, and crescent-shaped black edges appear in the sub-images, as shown in Fig. 2(g).

To eliminate the tilt, each sub-eye channel should be compensated. Figure 2(h) shows the compensation method for the tilt. We define the angle between the central ray and the sub-eye’s optical axis as the tilt angle t, and the angle between the sub-eye’s optical axis and the system’s optical axis as the pitch angle $\varphi$. The length between the entrance pupil and the center of the heterogeneous compound eye array is defined as the Pupil-Center distance (DPC), which is a function of the pitch angle $\varphi$ and is denoted as $DPC(\varphi )$. Then, the tilt angle of any sub-eye channel can be expressed by:

We have fitted a curve of the tilt angle t about the pitch angle $\varphi$, as shown in Fig. 2(i). When $\varphi$ is less than 20°, the increase of t is basically linear with a slope of 0.224. Then the growth rate slows down and a maximum tilt angle of 9.152° is obtained at the pitch angle of 56.8°. Thus, the tilt exists to varying degrees in all but the central sub-eye channel, and are more pronounced at the edge. Furthermore, due to the fact that the optical relay system is a single aperture fisheye lens with an inherently decreased image quality in its external FOV, compensation for tilt becomes much more necessary.

4. Optimization of the optical architecture and fabrication of the prototype

According to the above compensation method for tilt, the optical architecture of the HeCECam has been optimized. Figure 3 shows the ray path of the optimized HeCECam, including four WA sub-eye channels and six HD sub-eye channels. The partial enlarged views of the WA and HD sub-eyes are respectively shown at the top and bottom of Fig. 3.

 

Fig. 3. Optical design results of the optimized HeCECam.

Download Full Size | PDF

After optimization, the FOV range of the sub-eye channel is symmetric about its optical axis and the tilt are eliminated. To improve the imaging capacity of the WA sub-eyes, the optimized design adopts a lens with a focal length and diameter of 6 mm and 4.7 mm as the WA sub-eye, instead of the previous 3.77 mm and 2.5 mm. The parameters of the HD sub-eye remain unchanged, with a focal length of 15 mm and a diameter of 5 mm. The FOV of a single WA\HD sub-eye is 32°\15° and the total FOV of the WA\HD sub-eye network is expanded from 152°\141° to 174°\154°. In this optical design, the position of each sub-eye channel corresponds to the optimized points generated by the TGS, which are located at an arris of the ortho-icosahedron with a subdivision density of 8. The pitch angle and tilt angle of each channel are shown in Table 1.

Table 1. Pitch and tilt angle of each sub-eye channel

View Table

Based on the TGS and the above optical design, the optimized heterogeneous compound eye array which consists of 195 HD sub-eyes, 31 WA sub-eyes and an integrated hemispherical shell has been fabricated. Figure 4(a₁) and 4(a₂) are the pictures of the optimized heterogeneous compound eye array. It can be divided into five identical regions (Fig. 4(a₁), blue dashed lines), and each region exhibits internal symmetry along its central axis (Fig. 4(a₂), green dot-dashed line). In the previous work (Fig. 4(a₃)), the sub-eyes were arranged in a toroidal pattern, resulting in an irregular distribution over most of the area. Figure 4(b) shows the hemispherical shell of the optimized heterogeneous compound eye array fabricated by 3D printing, whose outer diameter is 55.1 mm. The material of it is a high-performance matte black resin featuring high strength, strong resistance to deformation, and excellent stray light suppression. The HD sub-eyes are placed in the apertures on the outer surface of the shell, and the WA sub-eyes are embedded in the tapered support structures (Fig. 4(b) and (d), green boxes). In addition, the tilt of each sub-eye channel is calculated and compensated, as shown in Fig. 4(c). However, the compensation makes a difference in the height of the aperture wall that fixes the HD sub-eye (Fig. 4(d), white dashed line), which may result in poor stabilization of the HD sub-eyes or even a fall from the shell. To overcome this problem, each aperture wall is extended into a “chimney” shape, as shown in Fig. 4(e). As a result, the partial details of the integrated optimized heterogeneous compound eye array are shown in Fig. 4(f).

 

Fig. 4. The fabrication of optimized heterogeneous compound eye array. (a) Comparison of the heterogeneous compound eye array before and after optimization. (a₁) Physical picture of the optimized heterogeneous compound eye array. (a₂) A side view of the optimized heterogeneous compound eye array and illustration of the symmetry of sub-eyes. (a₃) Physical picture of the heterogeneous compound eye array in the previous work. (b) The hemispherical shell of heterogeneous compound eye array fabricated by 3D printing. (c) Illustration of the tilt compensation for sub-eye channels. (d) A section view of apertures on the hemispherical shell. (e) Enlarged view of apertures walls that are built into a chimney shape. (f) Partial enlargement of the integrated optimized heterogeneous compound eye array.

Download Full Size | PDF

5. Results and discussion

To test the imaging performance of the optimized HeCECam, a series of comparison experiments are performed, including blank scene imaging, FOV tests, resolution verification and real-world scene imaging.

The experiment on the blank scene is performed to intuitively compare the changes in sub-images’ morphology before and after optimization, and to verify that the above-mentioned imaging defects existing in the previous work have been resolved. Figure 5 shows the experimental results for blank scene imaging. The middle panels of Fig. 5(a) and (c) are the raw images taken by the optimized and unoptimized HeCECam, respectively. It can be clearly seen that the optimized sub-images feature symmetry and uniformity of distribution.

 

Fig. 5. Comparison of the results of imaging on a blank scene performed by the HeCECam before and after optimization. (a) The raw compound eye image of a blank scene taken by optimized HeCECam. (b) Enlarged view of the area marked by the red circle in (a). (c) The raw compound eye image of a blank scene taken by unoptimized HeCECam. (d) Enlarged view of the area marked by the green circle in (c).

Download Full Size | PDF

Due to the fact that the magnification of the optical relay system expands with the FOV, it is normal for sub-images in the external FOV to have differences in size compared to those in the internal FOV. However, the tilt can intensify the differences in size and change the morphology of the sub-images. The enlarged views on the left\right side of Fig. 5(a) and (c) show the WA and HD sub-images at the center\edge position after and before the optimization, respectively. When it is unoptimized, the shape of both the central WA and HD sub-images are standard circles with diameters of 61 and 121 pixels, respectively; while at the edge, the WA\HD sub-images are elliptical, with sizes of 79\165 and 66\152 pixels on the long and short axes, respectively. With the effects of the tilt eliminated, the WA\HD sub-images at both the center and the edge have a standard round shape with diameters of 102\120 and 143\145, respectively. It reveals that the morphology and size of the sub-images are well controlled.

Figures 5(b) and 5(d) show enlarged views of the areas marked by red and green circles in the raw compound eye images of Figs. 5(a) and (c), respectively. In comparison of them, it can be noticed that the problems caused by the tilt, such as the support structure being imaged (Fig. 5(d), green box), increased stray light (Fig. 5(d), gray dashed line), crescent-shaped black edge (Fig. 5(d), red dashed line), and the FOV of HD sub-eyes being intruded (Fig. 5(d), yellow dashed line), are all perfectly solved.

In the optimized design, the system's FOV is further extended. Figure 6(a) shows the experimental setup for determining the total FOV of the HeCECam. Three puppy toys are placed at different positions (0°, $+ \Phi $ and $- \Phi $) as the targets. When the target is captured by the optimized HeCECam, $\Phi $ is set to 77°; while it is set to 70.5° in the case of unoptimized HeCECam. Figure 6(b) shows the results of the FOV test. The left and right sides are the raw images taken by the optimized and unoptimized HeCECam, respectively. The eight pictures in the middle are enlarged views of several marked sub-images in the raw compound eye images, where the solid\dashed blue circles represent HD sub-images at the center\edge, and the solid\dashed green circles represent WA sub-images at the center\edge. The results reveal that the overall FOV of the HeCECam extends from 141° to 154° after the optimization.

 

Fig. 6. Comparison experiments of FOV for the optimized and unoptimized HeCECam. (a) Experimental setup for testing the FOV of the prototype. (b) The overall FOV test results of the prototype before (right) and after (left) optimization. (c) Illustration of the experimental setup to verify the FOV of a single sub-eye. (d) Experimental results of the detection range for WA sub-eye after optimization. (e) Experimental results of the detection range for HD sub-eye after optimization. (f) Experimental results of the detection range for WA sub-eye before optimization. (g) Experimental results of the detection range for HD sub-eye before optimization.

Download Full Size | PDF

Figure 6(c) shows the experimental setup for measuring the FOV of a single sub-eye. A standard ruler is placed at a distance X from the front of the prototype. The FOV of a single sub-eye ($\omega$) can be calculated by $\omega = 2\arctan ({L\textrm{/2}X} )$, where L is the length of the ruler observed by the sub-eye. Figures 6(d) and (f) show the measurement results for the optimized and unoptimized WA sub-eye, respectively. The FOV of a single WA sub-eye is calculated as being extended from 26° to 32°, which agrees with the optimized design. Similarly, Figs. 6(e) and (g) give the measurements of HD sub-eye after and before optimization, respectively. The FOV of a single HD sub-eye remains consistently 15.1° as designed.

The image quality of the HeCECam should be improved as the tilt are well compensated, especially in the external FOV. Next, we have measured and compared the resolution of the system before and after optimization. Figure 7(a) shows the experimental setup for resolution measurements. An ISO 12233 chart is respectively placed in front of the optimized and unoptimized prototypes at an object distance of X as a target. Figures 7(b) and (c) show the results for the central HD sub-eyes. The resolution values of the central HD sub-eyes are both 10lp/mm before and after optimization. At the edge, their resolution values decrease to 7lp/mm and 9lp/mm, respectively, as shown in Figs. 7(d) and (e). For the WA sub-eyes, those at the center have resolution values of 6lp/mm and 9lp/mm before and after optimization (Figs. 7(f) and (g)), and those at the edge are 4lp/mm and 8lp/mm, respectively (Figs. 7(h) and (i)). This indicates that the optimization relieves the gradual decrease in resolution of the sub-eyes as the FOV increases, resulting in a 66.6% reduction of resolution degradation for both WA and HD sub-eyes. It is important to note that this suppression on the decrease in resolution of the edge sub-eyes is a key factor reflecting whether tilt compensation is meaningful. It can be seen that the gap between the center and edge in resolution for both WA and HD sub-eyes is significantly reduced after optimization, revealing that the compensation for tilt has a remarkable effect on the improvement of the HeCECam’s image quality of the external FOV.

 

Fig. 7. Comparison experiments of resolution for the optimized and unoptimized HeCECam. (a) Illustration of experimental setup for testing the resolution of sub-eyes. (b) The resolution test results of the center HD sub-eye after optimization. (c) The resolution test results of the center HD sub-eye before optimization. (d) The resolution test results of the edge HD sub-eye after optimization. (e) The resolution test results of the edge HD sub-eye before optimization. (f) The resolution test results of the center WA sub-eye after optimization. (g) The resolution test results of the center WA sub-eye before optimization. (h) The resolution test results of the edge WA sub-eye after optimization. (i) The resolution test results of the edge WA sub-eye before optimization.

Download Full Size | PDF

In the end, to further validate the practical imaging performance of the optimized HeCECam, a real-world scene comparison imaging experiment has been performed. Figure 8(a) shows a panoramic image of a laboratory taken by a cell phone. Several objects in the lab are marked by four different colored circles, such as the operating console (red), cardboard boxes (yellow), black aviation cases (green), and cabinets (blue). They are at different distances relative to the system and are captured by sub-eyes located at different positions on the heterogeneous compound eye array. Figures 8(b₁) and (c₁) show the raw compound eye images of the lab taken by the optimized and unoptimized HeCECam, respectively, on which the sub-images marked by four circles contain the corresponding objects in the lab. These marked sub-images are categorized in four areas named center area, center-middle area, middle-edge area, and edge area, according to the location of corresponding sub-eyes.

 

Fig. 8. Comparison imaging experiments on a real-world scene for the optimized and unoptimized HeCECam. (a) A picture of a lab scene taken by a cell phone. (b) Heterogeneous compound eye images taken by optimized HeCECam. (b₁) The raw image of the lab scene. (b₂) Enlarged view of the center area of b₁. (b₃) Enlarged view of the center-middle area of b₁. (b₄) Enlarged view of the middle-edge area of b₁. (b₅) Enlarged view of the edge area of b₁. (c) Heterogeneous compound eye images taken by unoptimized HeCECam. (c₁) The raw image of the lab scene. (c₂) Enlarged view of the center area of c₁. (c₃) Enlarged view of the center-middle area of c₁. (c₄) Enlarged view of the middle-edge area of c₁. (c₅) Enlarged view of the edge area of c₁.

Download Full Size | PDF

Figures 8(b₂) and (c₂) show the enlarged views of the center area after and before optimization, respectively. The details of the cabinets at a distance of 5.4 m are well captured both before and after optimization, which reveals that the imaging quality of the HD sub-eyes in the center area is basically equal. The resolution of the optimized WA sub-image (Fig. 8(b₂), white dashed line) is significantly higher due to the replacement of the WA sub-eye with a longer focal length. Great stray light appears in the unoptimized center area (Fig. 8(c₂), yellow dashed line), but it is suppressed after optimization.

Figures 8(b₃) and (c₃) show the zoomed-in views of the center-middle area after and before optimization, respectively. It can be seen that the distribution of the optimized sub-images is obviously more uniform than that of the unoptimized one. The unoptimized WA sub-image suffers from significant distortion (Fig. 8(c₃), white dashed line) and can narrowly discriminate the black aviation cases at a distance of 2.8 m, while the optimized one maintains the standard round shape (Fig. 8(b₃), white dashed line) and has great clarity. For the HD sub-images in the center-middle area, the decrease in image quality before and after optimization is not significant and there is distinct FOV overlap in both (Fig. 8(b₃) and (c₃), blue dashed lines).

Figures 8(b₄) and (c₄) show the enlarged views of the middle-edge area after and before optimization, respectively. An operating console at a distance of 1.5 m is captured by the corresponding sub-eyes in this area. The unoptimized WA sub-image suffers from a crescent-shaped black edge and distortion (Fig. 8(c₄), white dashed line), and the outline of the operating console can be barely observed from it. Moreover, there is a problem that the FOV of HD sub-eye is intruded by the support structure (Fig. 8(c₄), yellow dashed line). After optimization, these defects are eliminated (Fig. 8(b₄), white dashed line). The optimized WA and HD sub-images have standard shapes and better sharpness than the unoptimized ones.

The enlarged views of the optimized and unoptimized edge area is respectively shown in Fig. 8(b₅) and (c₅). The decrease of image quality is most severe in this area. Although the optimized and unoptimized HD sub-eyes are both able to distinguish letters “S” and “F” on the cardboard boxes with a distance of 2.0 m, the maximum FOV of the optimized system is larger than that of the unoptimized one. This reveals that the compensated HD sub-eyes have higher resolution than the uncompensated ones under the same FOV conditions. For the WA sub-eye in edge area, it is difficult to observe the contours of the scene from the unoptimized WA sub-image, compared to the optimized one. In addition, either WA or HD sub-images, the distortion is remarkably lower after optimization.

In summary, a more uniform distribution of sub-eyes is achieved by the TGS. Tilt compensation has resulted in a significant improvement in the image quality at external FOV of the system, and has solved the problems of distortion, crescent-shaped black edges, support structure being imaged, increased stray light, and the FOV of HD sub-eyes being intruded.

6. Conclusion

In this work, we optimize the structure of the HeCECam by two proposed optimization methods: an optimization method for sub-eye distribution named the TGS and a compensation method for tilt, which significantly improves the detection performance of the system. The TGS is an optimization for the traditional ortho-icosahedron subdivision, which reduces the distribution unevenness of sub-eyes down to 117% from the previous 152%, and provides regional symmetry to the heterogeneous compound eye array. Tilt compensation can eliminate imaging defects that existed in previous work, such as a decrease in image quality, crescent-shaped black edges and distortion in the sub-images, increased stray light, support structures being imaged, and the FOV of HD sub-eyes being intruded. Based on two methods, we re-design the heterogeneous compound eye array of the HeCECam and fabricate an optimized prototype with stronger imaging performance. The overall FOV of the HeCECam has been extended from 141° to 154°, and imaging capabilities of it have been improved, especially in the external FOV. To verify the imaging performance of the optimized system, a series of comparison experiments have been performed, including blank scene imaging, FOV tests, resolution verification and real-world scene imaging. The results show that the previous imaging defects have been perfectly eliminated, and the optimized HeCECam has stronger resolving power and wider FOV. These structure optimizations enable HeCECam not only to have the potential to continue expanding the FOV, but also to perform better in subsequent practical applications, like high-definition panoramic image reconstruction and precise target location. In fact, two proposed optimization methods are not limited to the HeCECam and can be applied to most compound eye cameras, especially for the compound eyes with complex optical relay systems. However, tilt compensation may not be applicable for some special compound eyes, such as multi-camera arrays and fiber-optic compound eyes. Our next works focus on fast target location and identification, and panoramic and local image reconstruction, based on the optimized HeCECam.