Open Access
Add to BibSonomyAbstract
In this paper we describe an aerial 3D image that occludes far background scenery based on coarse integral volumetric imaging (CIVI) technology. There have been many volumetric display devices that present floating 3D images, most of which have not reproduced the visual occlusion. CIVI is a kind of multilayered integral imaging and realizes an aerial volumetric image with visual occlusion by combining multiview and volumetric display technologies. The conventional CIVI, however, cannot show a deep space, for the number of layered panels is limited because of the low transmittance of each panel. To overcome this problem, we propose a novel optical design to attain an aerial 3D image that occludes far background scenery. In the proposed system, a translucent display panel with 120 Hz refresh rate is located between the CIVI system and the aerial 3D image. The system modulates between the aerial image mode and the background image mode. In the aerial image mode, the elemental images are shown on the CIVI display and the inserted translucent display is uniformly translucent. In the background image mode, the black shadows of the elemental images in a white background are shown on the CIVI display and the background scenery is displayed on the inserted translucent panel. By alternation of these two modes at 120 Hz, an aerial 3D image that visually occludes the far background scenery is perceived by the viewer.
© 2014 Optical Society of America
1. Introduction
Since an aerial display that shows a floating 3D image in the air can give a great impact to the viewers, many kinds of aerial 3D displays have been developed so far. Basically they generate a real image in the air to show a floating image. A toy that generates a floating real image by setting two concave mirrors face-to-face is well-known, though it is not an electronic display that can show an arbitrary graphical image.
Electronic aerial 3D displays are roughly divided into two categories: multiview aerial displays and volumetric aerial displays. Multiview images with 360 degree light field are realized by using multiple projectors [1], by using time-division multiplexing technology with a rotating screen with directionality of light diffusion [2,3], or by using both of them together [4]. Floating volumetric images controlled electronically have been realized by use of varifocal optical devices, such as liquid crystal lenses [5] and mirror scanners [6,7].
Volumetric displays are prominent in the sense that it solves the problem of vergence-accommodation conflict. The oldest volumetric display is based on the varifocal optics using a vibrating mirror [8]. Besides the varifocal optics, volumetric displays can also be realized by rotating a screen [9] or by layering translucent screens [10].
The most serious disadvantage of most volumetric displays is its inability to express visual occlusion and specular light. There have been some trials to overcome this problem. For example Cossairt et al. used a projector and a rotating screen with limited diffusion of light [11]. Yendo et al. combined a rotating LED array and a rotating parallax barrier to attain a 360 degree high density light field, which can induce focal accommodation of the viewer [12]. These solutions, however, generate the 3D image not in the air but inside the hardware.
One solution to realize an aerial volumetric image that can express visual occlusion and specular light is coarse integral volumetric imaging (CIVI) [13–15]. This method combines multiview and volumetric display solutions and achieves natural depth representation. An aerial image is achieved by generating a real image in the air by the lenses. In this paper we propose an aerial 3D display that occludes the background scenery located as far as the display height or width based on the CIVI technology.
This paper is organized as follows. In Section 2 the principle of CIVI is reviewed. In Section 3 a new CIVI display system that enables occlusion of far background scenery is proposed. In Section 4 the experimental result of the prototype system is given. In Section 5 we summarize this paper.
2. Coarse integral volumetric imaging
Integral imaging [16], which combines a convex lens array with a high resolution display panel, is a kind of multiview display solution that can show not only horizontal parallax but also vertical parallax. In the conventional integral imaging, the number of pixels each elemental lens covers is usually the same as the number of views, which means that the viewer perceives each elemental lens as a single pixel. Therefore the focus of viewer’s eyes is fixed close to the lens sheet, which makes it hard to show realistic images far beyond the screen or popping up from the screen.
In the coarse integral imaging (CII), each elemental lens is large enough to cover pixels dozens of times more than the number of views. Since a real image or a virtual image can be observed through the coarse elemental lenses, CII can express 3D space far beyond the screen or popping up from the screen. We can express 3D objects popping up in the air by generating a real image, which emerges when the distance between the display panel and the lens array is farther than the focal length of the elemental lenses.
In the actual real-image CII, we usually use a large aperture Fresnel lens in addition to the convex lens array to converge light, for the optical aberration becomes smaller. The distance between the display panel and the lens array is kept almost the same as the focal length of the elemental lenses of the lens array so that the light may be collimated by each elemental lens. Then the large convex Fresnel lens converges the collimated light and generates a real image at the focal length of the large Fresnel lens away from its surface.
The distance between the large convex lens and the convex lens array can vary. When the distance is short, the optics is similar to that of the traditional integral imaging and the total system size can be kept compact. It can also be a multiview system where the whole image observed in each eye switches alternately when we keep the distance between the lens array and the large aperture Fresnel lens long enough to generate a real image of the lens array, which corresponds to the viewing zone where the view for each eye changes alternately.
Though CII can show images off the screen, the problem of vergence-accommodation conflict still exists, for it can generate only one image plane. One way to solve the problem of vergence-accommodation conflict is to introduce volumetric approach using multilayered panels in addition to the multiview approach [17–23]. In the scheme of coarse integral imaging, Kakeya et al. proposed coarse integral volumetric imaging (CIVI) as shown in Fig. 1. With this configuration vergence-accommodation conflict is reduced since each 3D pixel is displayed at the real image layer near the right depth.
In the CIVI system the layered real images generated by the lenses are curved and distorted. Without taking into account this distortion, smooth connection of the elemental images cannot be realized. We can know how the images are distorted by calculating the paths of light rays from the viewpoint to the display panel. Once the distortion data are obtained, texture mapping can be used to correct the distortion [24]. As for the distortion in the direction of depth, we calculate refraction of the light rays from the points on the display panel to find the points where the light rays converge, which can be used to decide on which panel each pixel should be drawn. Thus the image with correct geometry can be presented to the viewer.
3. Realization of background occlusion
Though CIVI can realize presentation of volumetric image with occlusion, the conventional CIVI cannot show a deep space, for the number of layered panels is limited because of the low transmittance of each panel. In this section a novel optical design to attain an aerial 3D image that occludes far background scenery is proposed.
One way to show far background scenery is to insert a translucent display panel immediately in front of the large aperture lens of the CIVI system. In this case the distance between the popping-up image and the background scenery can be as far as the focal length of the large aperture lens of the CIVI system. Also the resolution of the background image is as fine as the resolution of the inserted display panel. In this optical set-up, however, the background image can be seen through the aerial image floating in front as shown in Fig. 2.
One way to realize visual occlusion of the background behind the aerial image is to draw a white shadow in the background image so that the aerial image may not be disturbed by the texture of the background image. The position of the shadow, however, moves as the viewer moves. Also multiple shadows are needed when the image is observed by multiple viewers, who inevitably see the shadows for other viewers as shown in Fig. 3.
To realize proper visual occlusion, we propose use of time-division multiplexing for CIVI. In the proposed system, a translucent display panel with 120 Hz refresh rate is located between the CIVI system and the aerial 3D image. Also the display panels of the CIVI system are replaced with TFT panels that have a short response time to attain 120 Hz refresh rate. When t = n/60 + 1/120 second (n: integer), the elemental images in a black background are shown on the CIVI display panels and a white image (uniformly translucent) is shown on the inserted translucent panel, which gives the same image given by the conventional CIVI, though the brightness is weakened due to the limited transmittance of the inserted panel. When t = n/60 second, black shadows of the elemental images in a white background are shown on the CIVI display panels and the background scenery image is shown on the translucent panel, which gives the background scenery partly occluded by the floating black shadow. By quick alternation between these two modes, the viewer can perceive an aerial 3D image occluding the far background scenery by the afterimage effect as shown in Fig. 4.
In the proposed system, not only the aerial image but also the shadow of it changes in accordance with the viewer’s position, for both of them are presented with the multiview configuration of CIVI. Therefore the position of the shadow can be kept at the proper position for any viewpoint. The areal image is not affected by what kind of background image is used, for the aerial image and the background image are presented independently by use of time-division multiplexing. Thus we can show an aerial image that occludes a high-resolution background scenery far behind it. Here note that the background is presented in 2D, not in 3D.
4. Experiment
We made a prototype system based on the principle explained in the previous section. Figure 5 shows the picture of the prototype system. As for the CIVI system, we used a half mirror to merge the images of two different depths instead of layering two panels to avoid the emergence of moiré pattern in a simple way [15]. As for the display panels, we used three sets of BenQ XL2420T, which can show 24-inch Full HD images with 120 Hz refresh rate. Two of the panels are used as the panels of the CIVI system, while the backlight of the other panel is removed and placed immediately in front of the large aperture lens.
As for the optical design of CIVI, we used the same modules as those of the previous work [25], where the decentered elemental lenses had been used to attain a high resolution image. The optical parameters were as follows:
- ・ Focal length of elemental lenses: 90 mm;
- ・ Size and number of elemental lenses: 38 mm × 38 mm, 10 × 6 = 60;
- ・ Size and number of elemental images: 48 mm × 48 mm, 10 × 6 = 60;
- ・ Distance between elemental lenses and display panels (2 layers): 90 mm, 93 mm;
- ・ Distance between elemental lenses and large aperture lens: 380 mm;
- ・ Focal length of large aperture lens: 325 mm;
- ・ Size of large aperture lens: 400 mm × 300 mm.
Since BenQ XL2420T uses a TN panel and the polarization is inclined by 45 degrees, polarization after reflection by a half mirror is perpendicular to the polarization without reflection. To keep the polarization in the same orientation, a half wave plate was inserted between the half mirror and one of the display panels. Also the polarization of the light from the CIVI system was kept the same as that of the backside polarization filter of the LC panel placed in front of the large aperture lens. To widen the narrow viewing angle in the vertical direction, which is one of the major weak points of TN panels, vertical diffusers were placed in front of the display panels of the CIVI system. Here note that all the lenses and the diffusers should be made of material that preserves polarization.
The images observed by the viewer from different viewing angles are shown in Fig. 6. As the figure shows, a popping-up image with the proper occlusion of background is attained, though the occluded background is not perfectly erased. This leakage of light is caused partially by the crosstalk of time-division multiplexing system, where the pair of alternating frames is not perfectly separated. In addition to that, crosstalk can be stronger due to the reflection of light among the lenses, which increases the background light level. These problems are expected to be overcome by using display panels with shorter response time and the lenses with less reflection.
5. Conclusion
This paper proposes an aerial 3D display that occludes the far background scenery. In the proposed system, a translucent display panel with 120 Hz refresh rate is located between the CIVI system and the aerial 3D image. The CIVI system composed of display panels with 120 Hz refresh rate shows the elemental images in a black background and the shadows of the elemental images in a white background alternately, while the inserted translucent display panel shows a white image (uniformly translucent) and the background scenery synchronously. By quick alternation between these two modes, an aerial 3D image that visually occludes the far background scenery is realized. The validity of the proposed method is confirmed by the prototype system.
Acknowledgment
This research is partially supported by the Grant-in-Aid for Scientific Research, MEXT, Japan, Grant numbers: 22680008 and 25280070.
References and links
1. S. Yoshida, S. Yano, and H. Ando, “Prototyping of glasses-free table-style 3D display for tabletop tasks,” SID 2010 Digest. 41(1), 211–214 (2010). [CrossRef]
2. A. Jones, I. McDowall, H. Yamada, M. Bolas, and P. Debevec, “Rendering for an interactive 360° light field display,” ACM Trans. Graph. 26(3), 40 (2007). [CrossRef]
3. X. Xia, H. Wang, H. Li, Y. Peng, Z. Zheng, X. Liu, and W. Shen, “360-degree floating light field three-dimensional display based on a high-frame-rate color projector,” SID 2013 Digest.404–407 (2013).
4. Y. Takaki and S. Uchida, “Table screen 360-degree three-dimensional display using a small array of high-speed projectors,” Opt. Express 20(8), 8848–8861 (2012). [CrossRef] [PubMed]
5. S. Suyama, M. Date, and H. Takada, “Three-dimensional display system with dual-frequency liquid-crystal varifocal lens,” Jpn. J. Appl. Phys. 39(Part 1, No. 2A), 480–484 (2000). [CrossRef]
6. D. Miyazaki and K. Matsushita, “Volume-scanning three-dimensional display that uses an inclined image plane,” Appl. Opt. 40(20), 3354–3358 (2001). [CrossRef] [PubMed]
7. D. Miyazaki, K. Shiba, K. Sotsuka, and K. Matsushita, “Volumetric display system based on three-dimensional scanning of inclined optical image,” Opt. Express 14(26), 12760–12769 (2006). [CrossRef] [PubMed]
8. A. C. Traub, “Stereoscopic display using rapid varifocal mirror oscillations,” Appl. Opt. 6(6), 1085–1087 (1967). [CrossRef] [PubMed]
9. G. E. Favalora, J. Napoli, D. M. Hall, R. K. Dorval, M. G. Giovinco, M. J. Richmond, and W. S. Chun, “100 million voxel volumetric display,” Proc. SPIE 4712, 300–312 (2002). [CrossRef]
10. A. Sullivan, “DepthCube solid state 3D volumetric display,” Proc. SPIE 5291, 279–284 (2004). [CrossRef]
11. O. S. Cossairt, J. Napoli, S. L. Hill, R. K. Dorval, and G. E. Favalora, “Occlusion-capable multiview volumetric three-dimensional display,” Appl. Opt. 46(8), 1244–1250 (2007). [CrossRef] [PubMed]
12. T. Yendo, T. Fujii, M. Tanimoto, and M. P. Tehrani, “The Seelinder: cylindrical 3D display viewable from 360 degrees,” J. Vis. Commun. Image R. 21(5–6), 586–594 (2010). [CrossRef]
13. H. Kakeya, “Coarse integral imaging and its applications,” Proc. SPIE 6803, 680317 (2008). [CrossRef]
14. H. Kakeya, “Improving image quality of coarse integral volumetric display,” Proc. SPIE 7237, 723726 (2009). [CrossRef]
15. H. Kakeya, “Realization of undistorted volumetric multiview image with multilayered integral imaging,” Opt. Express 19(21), 20395–20404 (2011). [CrossRef] [PubMed]
16. G. Lippmann, “La photograhie integrale,” Comptes Rendus Acad. Sci. 146, 446–451 (1908).
17. B. Lee, S. Jung, S.-W. Min, and J.-H. Park, “Three-dimensional display by use of integral photography with dynamically variable image planes,” Opt. Lett. 26(19), 1481–1482 (2001). [CrossRef] [PubMed]
18. J.-H. Park, S. Jung, H. Choi, and B. Lee, “Integral imaging with multiple image planes using a uniaxial crystal plate,” Opt. Express 11(16), 1862–1875 (2003). [CrossRef] [PubMed]
19. S.-W. Min, B. Javidi, and B. Lee, “Enhanced three-dimensional integral imaging system by use of double display devices,” Appl. Opt. 42(20), 4186–4195 (2003). [CrossRef] [PubMed]
20. Y. Kim, J.-H. Park, H. Choi, J. Kim, S.-W. Cho, and B. Lee, “Depth-enhanced three-dimensional integral imaging by use of multilayered display devices,” Appl. Opt. 45(18), 4334–4343 (2006). [CrossRef] [PubMed]
21. Y. Kim, H. Choi, J. Kim, S.-W. Cho, Y. Kim, G. Park, and B. Lee, “Depth-enhanced integral imaging display system with electrically variable image planes using polymer-dispersed liquid-crystal layers,” Appl. Opt. 46(18), 3766–3773 (2007). [CrossRef] [PubMed]
22. R. Yasui, I. Matsuda, and H. Kakeya, “Combining volumetric edge display and multiview display for expression of natural 3D images,” Proc. SPIE 6055, 60550Y (2006). [CrossRef]
23. H. Ebisu, T. Kimura, and H. Kakeya, “Realization of electronic 3D display combining multiview and volumentric solutions,” Proc. SPIE 6490, 64900Y (2007). [CrossRef]
24. S. Sawada and H. Kakeya, “Coarse integral volumetric imaging with flat screen and wide viewing angle,” J. Electron. Imaging 21(1), 011004 (2012). [CrossRef]
25. S. Sawada and H. Kakeya, “Integral volumetric imaging using decentered elemental lenses,” Opt. Express 20(23), 25902–25913 (2012). [CrossRef] [PubMed]
