We propose an optical pseudoscopic to orthoscopic conversion method for integral imaging using a lens-array holographic optical element (LAHOE), which solves the pseudoscopic problem. The LAHOE reconstructs an array of diverging spherical waves when a probe wave with the phase-conjugated condition is imposed on it, while an array of converging spherical waves is reconstructed in ordinary reconstruction. For given pseudoscopic elemental images, the array of the diverging spherical waves integrates the orthoscopic three-dimensional images without a distortion. The principle of the proposed method is verified by the experiments of displaying the integral imaging on the LAHOE using computer generated and optically acquired elemental images.
© 2014 Optical Society of America
Integral imaging (InIm) is a promising three-dimensional (3D) display technology, since it provides quasi-continuous viewpoint images with a full parallax, and satisfies various physiological depth cues without special glasses [1–4]. The typical InIm system is composed of a lens-array, a charge coupled device (CCD), and a flat-panel display. In the recording process of the InIm, which is named pickup process, the information of 3D objects is recorded through the lens-array. The recorded images on the CCD in the pickup process is called elemental images. The elemental images contain various perspective view images, and 3D images are reconstructed when the elemental images are displayed with the same lens-array as that of the pickup process.
Pseudoscopic (PS) problem arises in the display process of the InIm, and is a bottleneck for implementing a real-time pickup and display system [5–13]. Since directions of the pickup and display are opposite in the InIm, an observer watches depth-reversed 3D images if the elemental images are not handled properly with a post-processing. To solve the PS problem, various approaches have been proposed by help of data processing such as a centrosymmetrical rotating of each elemental image , smart pixel mapping algorithm [8, 9], and algorithm based on the interweaving process [10, 11]. However, the computational cost in these numerical conversion methods prevents the practical applications of the InIm. Hence specially designed optical components for solving the PS problem without the data processing also have been studied intensively. Jang and Javidi proposed a simple optical conversion from PS to orthoscopic (OS) images by using a convex mirror-array in the display process , and Arai et al. proposed a gradient index lens-array and multiple layers of convex lens-arrays . Though these optical components for converting the PS images to the OS images are helpful to directly display the elemental images captured in the pickup process, complicated fabrication processes or requirements for the inconvenient pickup process are still problematic. In particular, the difference in a focal length between the lens-array in the pickup process and the optical components in the display process leads to distortions in the displayed 3D images.
In this paper, we propose a solution for the PS problem in the InIm by a phase-conjugated reconstruction on a lens-array holographic optical element (LAHOE). The LAHOE reconstructs an array of diverging spherical waves, when a probe wave is a phase-conjugation of reference wave in the recording process. For a given PS elemental images, the array of diverging spherical waves integrates the OS 3D images. Meanwhile, if we use the OS elemental images, an array of converging spherical waves is necessary to integrate the OS 3D images. Since the LAHOE can reconstruct both of the diverging and converging spherical waves according to the reconstruction condition, it is a promising solution for displaying the 3D images without the PS problem. First, the PS problem and the principles of the conversion methods from the PS to the OS images are introduced in Section 2. Then, the detail principles of the proposed method are explained based on wave vector analysis. Lastly, the experimental setup and results for the pickup and the display are presented in Section 3.
2.1. PS problem and methods for converting from PS to OS images in InIm
In the typical configuration of the InIm pickup system, a front side of 3D object is captured as a form of the elemental images through the lens-array. When the elemental images are displayed with the same lens-array in the display process, the observer views the depth reversed 3D images as demonstrated in Fig. 1. For example, while a green pixel is located closer to the lens-array than a red pixel in the pickup process of Fig. 1(a), the observer perceives that the red pixel is placed closer than the green pixel to himself or herself in the display process as shown in Fig. 1(b).
As addressed in the previous section, the solution for the PS problem is to rotate each elemental image by 180º centrosymmetrically by using the numerical or the optical methods. These methods convert the reconstructed 3D images from the PS real images to the OS virtual images, or from the PS virtual images to the OS real images. When the numerically converted elemental images from PS to OS images are displayed, the OS 3D images are reconstructed as shown in Fig. 1(c). The optical conversion methods for the PS problem change the propagating direction of each pixel on the elemental images without any additional data processing by using optical elements such as the convex mirror-array. Figure 1(d) shows the schematic diagram for the PS problem solved integral imaging when the optical conversion method is used in the display process. Though we demonstrate the optical conversion method using the convex mirror-array in Fig. 1(d), the principles of other optical elements for converting the PS images to the OS images can also be understood as a similar mechanism. In contrast to the conventional lens-array shown in Fig. 1(b), these optical elements make pixels on the upper and lower parts in each elemental image propagate to upward and downward directions, respectively. In the optical conversion methods, an effective focal length of total optical components should be considered to be equal to a focal length of the conventional lens-array in the pickup process. Otherwise, the optical conversion methods cause distortions of the reconstructed 3D images .
2.2. Phase-conjugated reconstruction on LAHOE for solving PS problem
In the LAHOE, the array of converging spherical waves is recorded in a holographic material by interfering a signal wave which is passed through a reference lens-array and a plane reference wave as shown in Fig. 2(a) . In an ordinary reconstruction shown in Fig. 2(b), the recorded signal wave can be read out by illuminating the probe wave which is identical to the reference wave in the recording process. The LAHOE performs the optical function of a concave mirror-array for Bragg matched light. A converging angle of the reconstructed wave on the LAHOE is identical to that of the reference lens-array. The configuration for displaying 3D images using the LAHOE is same as that of a reflection-type InIm [12, 15] except that the projected elemental images should satisfy the Bragg matching condition. The LAHOE can be implemented with inexpensive manufacturing process and it has a thin film structure. These characteristics are strong benefits compared to the conventional optical elements.
In holographic lenses, the wavefront is switchable between a convergence and a divergence by changing the reconstruction condition of the ordinary reconstruction and phase-conjugated reconstruction [16, 17]. As described previously, when the LAHOE is reconstructed using the probe wave which is identical to the reference wave in the recording process, the reconstructed wavefront is the array of converging spherical wave, and causes the PS problem. However, when we impose the probe wave with the phase-conjugation of the reference wave in the recording process of the LAHOE, the reconstructed wave is also a phase-conjugation of the original signal wave: the array of diverging spherical wave with traveling backwards as shown in Fig. 2(c). By using the LAHOE for displaying the InIm with the phase-conjugated probe wave, the PS problem can be simply solved by the reconstructed diverging spherical waves. Also the diverging angle of each spherical wave (θHOE) is identical to the converging angle of the reference lens-array and can be calculated as
If the elemental images are captured with the lens-array which has same specifications as the reference lens-array, the reconstructed 3D images on the LAHOE by the phase-conjugated probe wave are free from the PS problem, and the distortion of the 3D images due to the focal length difference is also prevented. Figure 3 shows the schematic diagram of displaying the PS problem solved 3D images by the phase-conjugated reconstruction on the LAHOE, when the PS elemental images of Fig. 1(b) are projected on it. The phase-conjugated reconstruction on the LAHOE converts the propagation direction, from the converging to diverging, of displayed pixels in each elemental image, and thus the observer can see the PS problem solved OS 3D images.
The LAHOE can display the OS 3D images not only from the PS elemental images, but also from the OS elemental images, by controlling the reconstructed wavefront of the LAHOE between the divergence and convergence respectively. If the OS elemental images are given, the ordinary reconstruction of LAHOE integrates the OS 3D images, while the phase-conjugated reconstruction integrates the OS 3D images with the PS elemental images. Since the phase-conjugation is easily achievable by changing an incident direction of the probe wave, the LAHOE is a promising solution for the displaying the InIm without the PS problem.
In the LAHOE, a grating vector varies according to the position, because the signal wave is an array of spherical waves. By using the approach which is presented in , the region in a volume hologram, where a spherical signal wave and a plane reference wave are interferred, can be divided into n number of sub-regions. In each sub-region a wave vector of signal wave can be approximated by a single plane wave vector. In Figs. 4(a)-4(c), we note topmost and bottommost regions of the volume hologram as the 1st and the nth sub-regions, and represent the reference wave and signal wave in the recording process as R and S. Also, the probe waves and reconstructed waves are represented as R and S for the ordinary reconstruction, and R* and S* for the phase-conjugated reconstruction, respectively. The angle between the wave vector of signal wave and z axis varies from -θHOE/2 to θHOE/2 as the position changes, while an incidence angle of the reference wave is fixed to θr.
Figures 4(d) and 4(e) demonstrate wave vector space diagrams for the ordinary reconstruction and phase-conjugated reconstruction on the LAHOE at the 1st and the nth sub-regions. Considering the interference between two plane waves, a relationship among the grating vector (K), a wave vector of reference wave (kr), and signal wave (ks) is given by
When the volume hologram is reconstructed with the probe wave which is identical to the reference wave in the recording process, a wave vector of reconstructed wave (kd) is represented as the sum of the grating vector and wave vector of the probe wave:Fig. 4(d). On the other hand, if the probe wave is the phase-conjugation of the reference wave in the reocrding process as shown in Fig. 4(e), the wave vector of reconstructed wave is represented as19].
As shown in Eq. (4), the reconstructed wave has the wavevector ks* = -ks, when the probe wave with the wavevector kr* is imposed. As a result, the whole reconstructed wave, which is superposition of n number of plane waves in every sub-region, becomes a diverging spherical wave propagating along the -z direction as shown in Fig. 4(c). From the wave vector analaysis, it is verified that the volume hologram recorded with the converging spherical wave as the signal wave reconstructs the diverging spherical wave traveling backwards when we impose the phase-conjugatged probe wave.
3. Experimental results
We compare wavefront profiles between the ordinary and phase-conjugated reconstruction of the LAHOE. Figure 5(a) shows the experimental setup for capturing intensity profiles of the LAHOE, where a CCD and a telecentric lens are used. The telecentric lens relays an optical field of the reconstructed wavefront from near front planes of the LAHOE to the CCD plane. We move the telecentric lens and CCD along the z direction to capture intensity images on imaging planes at different distances from the LAHOE. The LAHOE shown in Fig. 5(a) is optically recorded on a photopolymer as the holographic material using 532 nm laser. The reference lens-array used in the recording process has focal length of 41.9 mm, and lens pitch of 5.4 mm and 7 mm in the horizontal and vertical directions, respectively. A collimated plane wave is used for the reference wave in the recording process with an incidence angle (θr) of 45º. The intensity profiles at the plane of 20 mm and 40 mm in front of the LAHOE with the ordinary and phase-conjugated reconstruction are presented in Figs. 5(b) and 5(c), respectively. In the ordinary reconstruction of Fig. 5(b), the captured image around 40 mm in front of the LAHOE shows the array of focused points as expected, because the focal length of the reference lens-array is 41.9 mm. On the other hand, when we flip the LAHOE and impose the same probe wave for the phase-conjugated illumination, the reconstructed wavefront changes into the diverging spherical waves as shown in Fig. 5(c).
Though the typically used lens-array in the InIm is a square lens-array and has a short focal length, it is hard to capture the variation in the reconstructed wavefront of LAHOE when we use the reference lens-array with the short focal length. Instead, we use the reference lens-array whose focal length is longer than 40 mm, and the reconstructed wavefront slowly converges or diverges, which is enough to compare the reconstruction characteristics of LAHOE according to the reconstruction conditions. Also, though the reference lens-array which we use in the experiments of Fig. 5 is not the square lens-array but the rectangular lens-array, the characteristics of LAHOE for switching the reconstructed wave from the convergence to the divergence with the phase-conjugated reconstruction do not depend on the shape (rectangle or square) of the reference lens-array.
In the display experiments, we develop a full-color LAHOE which is recorded in a reflective hologram scheme, which has the advantage on a wavelength multiplexing . In the recording process for the full-color LAHOE, a square lens-array with 1 mm lens pitch and 3.3 mm focal length, which is the widely used specification of the lens-array in the InIm [11, 14], is used as the reference lens-array, and the wavelength multiplexing is performed on the photopolymer by using 473 nm, 532 nm, and 671 nm lasers. The detail specifications for the recording process of the full-color LAHOE and experiments for the display process are listed in Table 1.
The display experiments are performed for two different elemental images: computer generated elemental images and optically acquired elemental images. In both display experiments using two different elemental images, we demonstrate that the integrated 3D images using the LAHOE are switchable between the PS and OS images by changing the incidence direction of projected images on the LAHOE. Figure 6(a) shows the experimental setup for the display, which is composed of the telecentric lens and an image projector. The incidence angle of projected elemental images is set to 45º for Bragg matching condition. For the phase-conjugated reconstruction, on the other hand, we flip the LAHOE with maintaining the path of probe wave. Figures 6(b) and 6(c) are the computer generated and optically acquired elemental images which have information of the PS real images. The photograph of experimental setup for optical pickup process is shown in Fig. 6(d).
For the computer generated elemental images, the letters ‘1’, ‘2’, and ‘3′ are used as 3D objects, which have depth information of 40 mm, 20 mm, and 0 mm. In the computer pickup process, the letter ‘2’ occludes the letter ‘1’, and the letter ‘3′ occludes the letter ‘2’ according to their depth information. For the optically acquired elemental images, the letters ‘I’ and ‘P’ are used as 3D objects, which are distant from the lens-array by about 35 mm and 15 mm, respectively. The letter ‘P’ occludes the letter ‘I’ as shown in Fig. 6(c). Though we use the optically acquired elemental images for the PS real objects, the elemental images for the PS virtual images  can also be used for the display process of the LAHOE.
Figures 7 and 8 show the experimental results of the display experiments. Figures 7(a) and 7(b) are the experimental results when the computer generated elemental images are projected, and Figs. 8(a) and 8(b) are the experimental results with the optically acquired elemental images. Figures 7(a) and 8(a) are the camera captured images at different positions for the ordinary reconstruction on the LAHOE, while Figs. 7(b) and 8(b) are the camera captured images for the phase-conjugated reconstruction on the LAHOE. In the ordinary reconstruction, the letter ‘1’ and the letter ‘I’ are located closest to the observer, while the letter ‘2’ and the letter ‘P’ occlude them, respectively. However, when the LAHOE is reconstructed with the phase-conjugation, the resultant images show the correct disparity and occlusion effects, since the PS real images in Figs. 7(a) and 8(a) are converted into the OS virtual images in Figs. 7(b) and 8(b). The disparities shown in the ordinary reconstruction and phase-conjugated reconstruction on the LAHOE are totally inverted. In Media 1 and Media 2 of Fig. 8, the continuous viewpoint images as the camera moves are presented with the full parallax, when the optically acquired elemental images are used.
In this paper, we have proposed the use of the phase-conjugated reconstruction for displaying the InIm on the LAHOE to solve the PS problem. The phase-conjugated reconstruction on the LAHOE makes the array of diverging spherical waves, which has the identical effect to the rotation of the pixels in each elemental image by 180 º centrosymmetrically, and provides the PS problem solved 3D images. The reconstruction profiles of the LAHOE in the ordinary and phase-conjugated reconstruction are described using the wave vector analysis. The experimental results for measuring of intensity profiles verify the wavefront of the LAHOE is switchable between the arrays of the converging and diverging spherical waves according to the reconstruction condition. The display experiments, which use the computer generated and optically acquired elemental images, confirm that the phase-conjugated reconstruction on the full-color LAHOE provides correct 3D images without the PS problem.
This work was supported by the IT R&D program of MSIP/KEIT (fundamental technology development for digital holographic contents). The authors acknowledge the support by Bayer Material Science AG for providing the photopolymer Bayfol HX film used for recording the LAHOE.
References and links
1. B. Lee, “Three-dimensional displays, past and present,” Phys. Today 66(4), 36–41 (2013). [CrossRef]
2. B. Javidi and F. Okano, eds., Three Dimensional Television, Video, and Display Technology (Springer, 2002).
3. S.- Park, J. Yeom, Y. Jeong, N. Chen, J.-Y. Hong, and B. Lee, “Recent issues on integral imaging and its applications,” J. Inf. Disp. 15(1), 37–46 (2014). [CrossRef]
4. G. Lippmann, “La photograhie integrale,” Comptes Rendus Acad. Sci., Paris, CR (East Lansing, Mich.) 146, 446–451 (1908).
6. H. E. Ives, “Optical properties of a Lippmann lenticulated sheet,” J. Opt. Soc. Am. 21(3), 171–179 (1931). [CrossRef]
8. M. Martínez-Corral, B. Javidi, R. Martínez-Cuenca, and G. Saavedra, “Formation of real, orthoscopic integral images by smart pixel mapping,” Opt. Express 13(23), 9175–9180 (2005). [CrossRef] [PubMed]
9. D. H. Shin, B. G. Lee, and E.-S. Kim, “Modified smart pixel mapping method for displaying orthoscopic 3D images in integral imaging,” Opt. Lasers Eng. 47(11), 1189–1194 (2009). [CrossRef]
14. K. Hong, J. Yeom, C. Jang, J. Hong, and B. Lee, “Full-color lens-array holographic optical element for three-dimensional optical see-through augmented reality,” Opt. Lett. 39(1), 127–130 (2014). [CrossRef] [PubMed]
16. R. R. A. Syms and L. Solymar, “Analysis of volume holographic cylindrical lenses,” J. Opt. Soc. Am. 72(2), 179–186 (1982). [CrossRef]
18. Y. Luo, J. Castro, J. K. Barton, R. K. Kostuk, and G. Barbastathis, “Simulations and experiments of aperiodic and multiplexed gratings in volume holographic imaging systems,” Opt. Express 18(18), 19273–19285 (2010). [CrossRef] [PubMed]
20. R. R. A. Syms, Practical Volume Holography (Clarendon, 1990).
21. J.-S. Jang and B. Javidi, “Formation of orthoscopic three-dimensional real images in direct pickup one-step integral imaging,” Opt. Eng. 42(7), 1869–1870 (2003). [CrossRef]