The planar compound eye has the advantages of simple structure and no requirement for complex relay optical elements, but the field of view (FOV) is very difficult to expand. Overcoming the limitation of FOV, especially with simple structures, is a great challenge for the development of planar compound eyes. Different from the existing designs that only considering refraction, this article proposes a catadioptric planar compound eye based on the reflection and refraction to expand the FOV. In the proposed design, the incident light from a large angle is reflected into the lenslet array by two rotationally symmetric mirrors whose surface equations are optimized by mathematical and optical softwares. The FOV of the proposed catadioptric planar compound eye theoretically can reach 96.6°, which is much wider than the opening record of 70°. Moreover, no distortion of the imaging system can be obtained theoretically in this design. Simulation results show a linearity of better than 99% for the most of the incident angles. The verification experiments show that the FOV of the proposed device can reach 90.7° while the FOV of the corresponding planar compound eye without mirrors is 41.6°. The proposed catadioptric planar compound eye has the great potential in monitoring, detection and virtual reality since the FOV has been widen significantly.
© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
Bionic compound eye, an optical system composed of multiple sub eyes, are learned from the biological compound eye in nature [1–6]. Compound eyes are particularly notable for their compact size, high sensitivity to motion and exceptionally wide FOV [7–10]. There are two types of artificial compound eye, namely curved compound eye and planar compound eye [11, 12]. The planar compound eye has the advantage of simple structure, but the FOV is very difficult to increase.
The FOV of the planar compound eye mainly depends on the lenslet array which is made up of a plurality of lenslets. Lenslet array is often used to image and measure 3D objects [13, 14]. Each lenslet of the array can image a picture on the image sensor individually. The lenslet array is arranged as a curved surface in the curved compound eye [15–18] and a planar surface in the planar compound eye [19–21]. The curved compound eye can obtain large FOV while its structure is complex and requires optical relay elements which brings difficulties to structural analysis, optical design and processing. Because the commercialized image sensors are basically plane, the curved compound eye imaging on plane image sensors cause a series of problems due to mismatch. The planar compound eye can be combined with the plane image sensors directly, which is beneficial to the commercialization of the device. However, the FOV of the planar compound eye is difficult to improve due to the influence of planar lenslet array. As such, breaking the limitation of the FOV is of great importance in the development of the planar compound eye for the potential commercialization.
The existing methods for expanding the FOV of the planar compound eye are mostly based on the structural parameters optimization for lenslet array and pinhole array, such as the pitch between lenslet array and pinhole array, the pitch between adjacent lenslet array. A kind of artificial compound eye called APCO (Apposition Compound eye Objective) design a pitch difference between lenslet array and pinhole array to wider FOV to 21° . Through setting up the pitch difference between the optics and each channel, the eCley (Electronic Cluster Eye) obtain the FOV of 58° × 46° . The use of cluster eye structure which is composed of three microlens arrays with different pitches raised the FOV of the planar compound eye to 70° . Comparing with APCO, the image resolution and distortion in marginal channels of Cley have been greatly improved. Wu, Jiang etc. al. have reviewed the state of the art of the development for artificial compound eyes and pointed out the opening record for the manufactured planar compound eye is 70° . Bruckner et al. proposed a conceptual design of artificial neural superposition eye, just like the neural archetype based on the reflective and refractive superposition, which theoretically widen the FOV significantly to 115° × 86° . But the artificial neural superposition compound eyes are too complicated to be manufactured.
In order to expand the FOV of the planar compound eye, this paper proposes another design, a catadioptric planar compound eye, through reflection and refraction to collect more light. This design utilizes two mirrors type structure that, without changing the pitch between lenslet array and pinhole array and the pitch between lenslet array, and obtain the light incident on the edge to conquer the limitation even under a simple structure. The proposed catadioptric structure is designed not only to expand the FOV, but also designed to be distortion free. Theoretically, the image obtained by the catadioptric planar compound eye is clear and distortionless. The paper also manufactured and verified the proposed design by the experiments. 3D printing is applied to the fabrication of mirrors, making the processing and verification of the proposed design more convenient.
The paper is organized as follows. Section 2 describes the conceptual design of the catadioptric planar compound eye. Section 3 describes the principle analysis of the catadioptric planar compound eye. Section 4 and section 5 show the surface optimization of the two mirrors and the prototype design of the device. Experiments and results are then shown in Section 6 and Section 7 concludes the paper.
2. Conceptual design
The conceptual design of the catadioptric planar compound eye are presented in Fig. 1, which structured by two mirrors with different surface type to collect lights on the edge part. The key of enlarging the FOV of the planar compound eye is to collect light that is incident at a large angle relative to the axis of symmetry. The catadioptric planar compound eye is divided into a optical reflection part and a optical refraction part. The reflection part is composed of two mirrors, namely the primary mirror and the secondary mirror. The purpose of the reflection part is to reflect as much light as possible to the refraction part. The optical refraction part is the lenslet array which makes the light to image on the image sensors. The refraction part and the image sensors together can be viewed as a simple planar compound eye. The primary mirror is in the interior of the design and the secondary mirror is in the periphery part of the design. The surface types of the primary mirror and the secondary mirror are parabolic surface and free-form surface, respectively. Both the primary mirror and secondary mirror are designed as rotating bodies for simple design, low cost and easy to be fabricated.
The light path of the structure is as followings:
- The light from the object arrives on the surface of the secondary mirror. Then the light is reflected from the secondary mirror to the primary mirror.
- The light is reflected from the primary mirror to the layer of lenslet array.
- The light is refracted in the lenslet array and passes through the layer of lenslet array.
- At last, the light is imaged on the surface of the image sensor.
After the conceptual design, the principle and the optimization of the design is introduced in the following.
3. Principle analysis
The principle analysis aims to obtain the characteristics and laws of light propagation in this design from the point of view of optics and mathematics. Figure 2 is the analysis model. In order to make the analysis easier, the direction of light which used to be analysed in this paper is opposite to the direction of the light propagation. In addition, the light which is perpendicular to X axis before refraction is selected to be analyzed in this paper. Since the primary mirror and secondary mirror are rotationally symmetric, the right half of a cross section is selected for analysis in this paper. As shown in Fig. 2, the blue curve represents the primary mirror; the red curve shows the secondary mirror and the green line represents the light path.
Supposing the vertex coordinate of the paraboloid of primary mirror is (h, k), and the focal coordinate is (h, k + p). Then the parabola equation can be obtained as follows:Fig. 2, it can be acquired that h < 0, k < 0, p > 0. Eq.(2) can be solved as follows: Fig. 2. P0 is the intersection between light and X axis. The reflection points of light on the surfaces of the primary mirror and the secondary mirror are P1 and P2, respectively. And the point on the object plane emitting the light is set as P3. It can take the optical properties of a parabola into account. After reflected by the parabola, the light which passing through the focus of the parabola will be changed to parallel to the parabolic axis of symmetry. So F, P1 and P2 are on the same line. Extending the line in which P1 and P2 are located, the line will pass through F point. Draw a straight line through the P2, which is perpendicular to the tangent of the P2 point on the secondary mirror. The vector of this line is set as . Equally, the straight line determined by P2 and P1 is set as . So as the point of P2 and P3, it is set as . The direction of the vector and the vector respectively represent the direction of incident light before and after reflection. The vector indicates the direction of normal of P2 point. Suppose that The coordinate of P2 is (x, y). The coordinate of can be written as : 26]. The scaling ratio a is set as a constant, thus the mapping from the incident light to the emergent light is linear and the design is free from distortion. The details of derivation of can be got in . The coordinate of the vector is: Eq.(11) in mathematical software such as MAPLE and so on.
4. Surface optimization
The prototype is mainly composed of the primary mirror, the secondary mirror, lenslet array and image sensor. The surface types of the primary mirror and the secondary mirror determine the FOV of the design directly. Lenslet array and image sensor are combined into a planar compound eye.
In order to optimize the surfaces of the primary mirror and the secondary mirror, the factors that affect the surface of mirrors need to be analyzed. The main factors which affect the surface equation of the secondary mirror are the scaling ratio a, the aperture diameter of the secondary mirror and the value of k which determines the surface equation of the primary mirror. The FOV and imaging quality are the main evaluation criteria which determine the optimization of parameters combinations. To ensure the quality of the imaging, the aperture value of the secondary mirror is set between 1mm and 6mm with the step of 1mm. The FOV of different combinations can be obtained from the simulation of optical software. According to the triangle formula, the formula of FOV is:
Figure 3(a) shows the light overlap phenomenon that occurs when the parameters are not selected properly. Light overlap causes the image quality degradation, the combination of parameters should be ruled out. The design that avoids the light overlap phenomenon is shown in Fig. 3(b) in which the primary and secondary mirror are fully utilized. Through the analysis of the parameter combinations, many combinations of parameters that meet the requirements are selected. The equations of the secondary mirror are obtained by MAPLE software.
After a series of calculation, the optimized parameters are obtained for wider FOV and high quality of imaging. The final set is a = 0.2, k = −7/2 and aperture diameter is from 6 mm to 56 mm. Then the equation of the primary mirror is :
According to the simulation result and Eq.(12), the theoretical FOV of the optimized structure is obtained as 96.6°, which is much wider than the existing report of 70°. In addition, as shown in Fig. 4(a), the pitch of the horizontal light is 0.1mm on the right side of the main reflector, and the uniform distribution of the reflected light can be observed on the detector. Figure 4(b) and Table 1 present the linearity and FOV of the incident light at different angles and demonstrate simulation results with low distortion. The linearity of the majority of the incident light rays is better than 99% all cross different incident angles from 0° to 45°.
The surface of the secondary mirror is free-form and 3D printing is employed to manufacture a variety of complex surfaces. The material used for 3D printed mirrors is Epoxy Resin Composites (ERC). Figure 5(a) exhibits the prototype of the primary mirror and the secondary mirror processed by 3D printing. While traditional methods such as ultra-precision diamond turning machining process have the advantage of high precision, the procedure is very complex, time consuming and expensive. On the other hand, the 3D printing technology has the advantages of less material loss, short time consumption, straightforward fabrication process and cost effective. In addition, complicated freeform optical elements can be fabricated directly from their modelling. Thus, 3D printing has the potential for efficient optical elements fabrication and rapid prototyping at the product design stage. Nevertheless, much more accurate 3D printed optical elements have been reported, where nearly diffraction limited optical lenses have been successfully fabricated [27–29].
Lenslet is the important component in planar compound eye system. A kind of square plano convex lenslet is designed and fabricated by PMMA. The lenslet has a 4 mm × 4 mm square shape with a spherical surface, whose equivalent optical focal distance is 8.95 mm. The lenslet array consists of 9 lenslets which are arranged in a 3 × 3 matrix. The picture of lenslet array is shown in Fig. 5(b). In order to verify the imaging property, the light convergence and the incoherent irradiance of the single lenslet and lenslet array are simulated to check the image of parallel lights at image sensors. The results are presented in Fig. 6(a). Figure 6(b) shows the spot diagram of single lenslet. The RMS radius is 5.870 µm, and the GEO radius (geometrical spot radius) is 10.242 µm. There are only 9 spots at the image sensor in Figs. 6(c) and 6(d), which shows good light convergence of the lenslet array. These results indicate that the image quality of the fabricated lenslet array is high.
The results of the combination of the lenslet array and the designed reflecting mirrors are shown in Fig. 7. The light passing through two different reflectors can reach the surface of microlens well and converge on the imaging surface.
5. Prototype design
The mechanical design of the prototype should consider the assembling of the reflectors, the pitch between lenslet array and the image sensors. Figure 8 is the schematic diagram of the proposed catadioptric planar compound eye. The primary mirror is fixed by transparent acrylic board 2 and the secondary mirror are mounted with a common symmetry axis and the apex of the primary mirror share the same plane with the lower end of the secondary mirror. Two mirrors and board 2 are fixed by acrylic board 1 and studs as a whole which can adjust the distance relative to the camera in a certain range. The 3 × 3 lenslet array is fixed 9 mm behind the board 4. The camera is fixed on the bottom board which is made of aluminium alloy. The displacement devices and spring are used to adjust the distance between board 4 and the camera. In order to prevent the noisy light from the environment and the effect of dust on the device, four acrylic boards 3 are designed to seal the space between the board 1 and the board 4. To illustrate the internal structure, two pieces of acrylic boards 3 are removed from the side in Fig. 8.
The photograph of fabricated prototype is demonstrated in Fig. 9(b), while the corresponding schematic diagram is shown in Fig. 9(a). The dimensions of the protype are 230 mm × 107 mm × 145 mm. The distance between the lenslet array and the bottom of the secondary mirror is 5.47 mm. The initial distance between the lenslet array and the CMOS sensor is 14 mm and can be adjusted in a certain range. Note that the system volume is extremely increased by introducing the reflecting mirrors. It has the advantage of no distortion compared to the conventional wide FOV imaging systems such as fish-eye lens cameras. If smaller microlenses are utilized, the volume can be reduced dramatically. The CMOS used in the prototype is PYTHON 25K. The size of the CMOS is 23 mm × 23 mm. The resolution is 5120 px × 5120 px and the pixel size is 4.5 µ m × 4.5 µ m. In order to verify the proposed design, the corresponding simple planar compound eye, without reflectors of the prototype, is shown in Figs. 9(c) and 9(d).
6. Experiments and results
In order to compare the FOV of the catadioptric planar compound eye and the simple planar compound eye, the verification experiments are carried out at the same conditions. The FOV in the experiments is calculated based on the content of the captured target image and the distance from the experimental device to the target. The image target is shown in Fig. 10, of which the size is 220 cm × 145 cm. The 50 letters and numbers printed on the target are divided in 5 rows and 10 columns. The distance between adjacent letters or numbers in the photographed target is 21 cm. The distance between the photographed target and the camera is 83 cm. The catadioptric mirror is adjusted to be 77cm in front of the tested target. The value of the FOV can be obtained according to the triangle relation.
Figure 11 shows the picture captured by the simple planar compound eye. It can be seen from the image, there are three characters ‘8, 9, a’ in a horizontal line. The FOV is thus calculated as 41.6° according to the triangle relation. There are several ghost images in Fig. 11 which may lead to image quality degeneration. The ghost images are mainly induced by the cross-talk of different channels. However, the cross-talk is greatly suppressed with the two reflecting mirrors, as shown in Fig. 12 because only light rays with particular angles can form an image via the microlens. Figure 12 displays the raw image captured by the catadioptric planar compound eye. The images captured by the system is arranged 3 lines and 3 columns because the lenslet array used in the system is a 3 × 3 array. There is a black area in the middle of each picture because of the obstacle of the primary mirror. The black area from different channels differ from each other slightly. From Fig. 12, the corresponding letters and numbers can be identified clearly from different channels. The third line of the target in Fig. 10 is selected to be analyzed, where ‘U,W,X,Y,Z,0,1,2’ can be identified easily in Fig. 12. The distance between ‘U’ and ‘2’ in the target is 168 cm. According to the triangle relation, the FOV of the catadioptric planar compound eye imaging experiment is calculated as 90.7°. After the process of the haze removal using dark channel prior algorithm , the numbers and letters within the field of view of the device are stitched together and presented in Fig. 13.
Comparably, the FOV of the compound eye has been widen from 41.6° to 90.7° by the proposed design. The FOV has been dramatically increased for 118%. In addition, the FOV of the catadioptric planar compound eye in simulation is 96.6°. The difference between the simulation and the experiments is caused by the errors from the surface fabrication and assembling. The fabrication errors of the microlenses and the primary and secondary mirrors cause disparity from the designed freeform surfaces. The assembling errors such as the perpendicularity between the microlenses and the reflecting mirrors, the concentricity between the primary and secondary mirrors also cause the FOV loss.
In order to further display the imaging quality, this paper uses the catadioptric compound eye to imaging the Chinese characters which has bigger size than the letters and numbers in the above. Imaging enhancement algorithm such as multi-scale retinex [31, 32] is adopted in this paper. Figure 14 shows the images of Chinese characters after imaging enhancement. The image collected can be divided into 9 regions, and each regions varies from each other. Figure 15 shows the images obtained by simple image stitching. The quality of each region is affected by distortion which is mainly induced by the fabrication of two mirrors. The less distorted pictures can be obtained by more precision fabrication of the mirrors and using distortion correction method. If image fusion method is adopted, the full image with better quality can be obtained easily, which is not the focus of this paper.
In this work, a design of catadioptric planar compound eye is proposed to expand the FOV of the traditional planar compound eye system. The surface of the double mirrors are optimized by simulation and fabricated by 3D printing, which is an attempt for optical components processing by additive manufacturing. The simulation results confirm that the design has a large field angle(96.6°) and low distortion(The linearity of the incident light rays is better than 99%). The experimental results show that the FOV of the catadioptric planar compound eye reaches 90.7° while the FOV of the simple planar compound eye is 41.6°. To the best of authors’ knowledge, the proposed compound eye achieves the largest FOV in the opening record of fabricated planar compound eyes. The proposed catadioptric planar compound eye, with large FOV and low distortion, has great potential for commercial applications such as monitoring, detection and virtual reality.
National Natural Science Foundation of China (No. 51775164, 51575156, 51675156 and 51705122) and the Fundamental Research Funds for the Central Universities (No. JZ2017HGPA0165, PA2017GDQT0024).
References and links
1. J. Duparre, P. Schreiber, A. Matthes, E. Pshenay-Severin, A. Brauer, A. Tunnermann, R. Volkel, M. Eisner, and T. Scharf, “Microoptical telescope compound eye,” Opt. Express 13(3), 889–903 (2005). [CrossRef] [PubMed]
3. A. Brueckner, J. Duparre, R. Leitel, P. Dannberg, A. Braeuer, and A. Tuennermann, “Thin wafer-level camera lenses inspired by insect compound eyes,” Opt. Express 18(24), 24379–24394 (2010). [CrossRef]
4. H. R. Fallah and A. Karimzadeh, “Design and simulation of a high-resolution superposition compound eye,” J. Mod. Opt. 54(1), 67–76 (2007). [CrossRef]
5. M. Shankar, R. Willett, N. Pitsianis, T. Schulz, R. Gibbons, R. T. Kolste, J. Carriere, C. Chen, D. Prather, and D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. 47(10), B1–B10 (2008). [CrossRef] [PubMed]
6. G. Druart, N. Guerineau, R. Haidar, S. Thetas, J. Taboury, S. Rommeluere, J. Primot, and M. Fendler, “Demonstration of an infrared microcamera inspired by Xenos peckii vision,” Appl. Opt. 48(18), 3368–3374 (2009). [CrossRef] [PubMed]
7. T. Wang, W. Yu, C. Li, H. Zhang, Z. Xu, Z. Lu, and Q. Sun, “Biomimetic compound eye with a high numerical aperture and anti-reflective nanostructures on curved surfaces,” Opt. Lett. 37(12), 2397–2399 (2012). [CrossRef] [PubMed]
9. J. W. Duparre and F. C. Wippermann, “Micro-optical artificial compound eyes,” Bioinspir. Biomim. 1(1), R1–R16 (2006). [CrossRef]
10. Y. M. Song, Y. Xie, V. Malyarchuk, J. Xiao, I. Jung, K.-J. Choi, Z. Liu, H. Park, C. Lu, R.-H. Kim, R. Li, K. B. Crozier, Y. Huang, and J. A. Rogers, “Digital cameras with designs inspired by the arthropod eye,” Nature 497(7447), 95–99 (2013). [CrossRef] [PubMed]
13. H. Yoo, “Depth extraction for 3D objects via windowing technique in computational integral imaging with a lenslet array,” Opt. Lasers Eng. 51(7), 912–915 (2013). [CrossRef]
14. Y. Piao, M. Zhang, J.-J. Lee, D. Shin, and B.-G. Lee, “Orthoscopic integral imaging display by use of the computational method based on lenslet model,” Opt. Lasers Eng. 52(1), 184–188 (2014). [CrossRef]
18. J. Duparre, P. Dannberg, P. Schreiber, A. Brauer, and A. Tunnermann, “Artificial apposition compound eye fabricated by micro-optics technology,” Appl. Opt. 43(22), 4303–4310 (2004). [CrossRef] [PubMed]
19. J. Tanida, T. Kumagai, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, “Thin observation module by bound optics (TOMBO): concept and experimentalverification,” Appl. Opt. 40(11), 1806–1813 (2001). [CrossRef]
20. F. Fan, Q. Hao, and X. Cheng, “Retina-like sensor based on a lens array with a large field of view,” Appl. Opt. 54(36), 10692–10697 (2015). [CrossRef]
22. A. Brueckner, J. Duparre, P. Dannberg, R. Leitel, and A. Braeuer, “Driving microoptical imaging systems towards miniature camera applications,” Proc. SPIE 7716, 77160J (2010). [CrossRef]
23. J. Duparre, P. Schreiber, and R. Volkel, “Theoretical analysis of an artificial superposition compound eye for application in ultra flat digital image acquisition devices,” Proc. SPIE 5249, 408–418 (2004). [CrossRef]
24. S. Wu, T. Jiang, G. Zhang, B. Schoenemann, F. Neri, M. Zhu, C. Bu, J. Han, and K.-D. Kuhnert, “Artificial compound eye: a survey of the state-of-the-art,” Artif. Intell. Rev. 48(4), 573–603 (2017). [CrossRef]
25. A. Brueckner, J. Duparre, P. Dannberg, A. Braeuer, and A. Tuennermann, “Artificial neural superposition eye,” Opt. Express 15(19), 11922–11933 (2007). [CrossRef]
26. E. Koese and R. K. Perline, “Double-mirror catadioptric sensors with ultrawide field of view and no distortion,” Appl. Opt. 53(4), 528–536 (2014). [CrossRef]
27. S. Thiele, K. Arzenbacher, T. Gissibl, H. Giessen, and A. M. Herkommer, “3D-printed eagle eye: Compound microlens system for foveated imaging,” Sci. Adv. 3(2), e1602655 (2017). [CrossRef] [PubMed]
28. T. Gissibl, S. Thiele, A. Herkommer, and H. Giessen, “Two-photon direct laser writing of ultracompact multilens objectives,” Nat. Photonics 10, 554–560 (2016). [CrossRef]
29. S. Juodkazis, “Manufacturing: 3D printed micro-optics,” Nat. Photonics 10, 449–501 (2016). [CrossRef]
30. K. M. He, J. Sun, and X. O. Tang, “Single image haze removal using dark channel prior,” IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2011). [CrossRef]
31. Z. Rahman, D. J. Jobson, and G. A. Woodell, “Multi-scale retinex for color image enhancement,” in Proceedings of IEEE Conference on Image Processing (IEEE, 1996), pp. 1003–1006. [CrossRef]
32. D. J. Jobson, Z. Rahman, and G. A. Woodell, “A multiscale retinex for bridging the gap between color images and the human observation of scenes,” IEEE Trans. Image Process. 6(7), 965–976 (1997). [CrossRef] [PubMed]