Open Access
Add to BibSonomyAbstract
A lenticular-type super multi-view (SMV) display normally requires an ultra high-resolution flat-panel display. To reduce this resolution requirement, two or more views are generated around each eye with an interval smaller than the pupil diameter. Cylindrical lenses constituting a lenticular lens project a group of pixels of the flat-panel display to generate a group of viewing zones. Pixel groups generating left and right viewing zones through the same cylindrical lens are partitioned to separate the two zones. The left and right pixel groups for different cylindrical lenses are interlaced horizontally. A prototype SMV display is demonstrated.
©2011 Optical Society of America
1. Introduction
Substantial research has been conducted to develop glasses- and glassesless-type three-dimensional (3D) displays [1–4]. A super multi-view (SMV) display [5–9] has been developed as a glassesless-type 3D display that is free from the visual fatigue caused by the accommodation-vergence conflict and provides smooth motion parallax. Because the SMV display requires generation of a large number of views, numerous studies so far have focused on increasing the number of views. In the present study, we develop a technique that enables reduction of the number of views required for the SMV display.
Humans perceive depth using four physiological factors: vergence, binocular disparity, motion parallax, and accommodation [1]. Conventional 3D schemes, including glasses-type two-view display and glassesless-type two-view and multi-view displays, have two physiological problems: the accommodation-vergence conflict [10] and imperfect motion parallax. When two parallax images are displayed to the left and right eyes, the depth of 3D images can be perceived correctly by vergence, which allows for depth perception by using the angle between the lines of sight of the left and right eyes when the both eyes look at the same point. Accommodation, which creates depth perception when the focal length of the eye lenses changes, does not work correctly because the eyes focus on the display screen instead of the 3D images because the two images are displayed on the screen. This conflict causes visual fatigue because of the close interaction between vergence and accommodation; the human visual system makes the eyes focus on the position at which vergence perceives the depth. A two-view display does not compensate for motion parallax, which is the change in retinal images caused by a shift in eye position. A multi-view display provides discontinuous motion parallax because the pitch of multiple viewing zones is usually the average interocular distance or half of it; thus, the retinal image does not change until the eye moves onto the adjacent viewing zone. The absence of motion parallax or discontinuous motion parallax reduces the presence and realism effects of 3D images because humans unconsciously predict retinal image change caused by eye movement.
An SMV display generates dense viewing zones to make their pitch smaller than the pupil diameter [5, 6], as shown in Fig. 1 . Because two or more viewing zones exist in the pupil, two or more rays passing through one point of a 3D image enter the pupil simultaneously through the viewing zones; thus, the eyes can focus on that point according to the depth information perceived by vergence. The accommodation responses are evoked by the SMV display technique to prevent the accommodation-vergence conflict. Because the pitch of the viewing zones is smaller than the pupil diameter, the retinal image changes smoothly with eye movement. Therefore, an SMV display provides smooth and continuous motion parallax.
Because the average pupil diameter of humans is 5 mm, the viewing zone pitch of SMV displays should be less than 5 mm. The total width of the multiple viewing zones should be at least twice as large as the interocular distance to provide enough viewing area and effective use of the entire viewing zones. Therefore, display systems that can generate dense viewing zones should be developed to realize SMV displays. An SMV display with 45 views [5–7] was first demonstrated using a focused light array. A SMV display system with 30 views [8] using fan-like array projection optics was also reported. High-density directional (HDD) displays [11–16] were also developed to realize the SMV display condition. While SMV displays project a numerous parallax images with rays converging to viewing zones, HDD displays project a large number of directional images with nearly parallel rays. Parallax images are perspective projections of a 3D scene, and directional images are orthographic projections. Reducing the projection angle pitch of HDD displays satisfies the SMV display condition. HDD displays with 64 and 128 ray directions [11–13] were constructed with a multi-projection scheme consisting of an array of projection imaging systems, and those with 30 and 72 ray directions [14–16] were constructed with a flat-panel system consisting of a flat-panel display and a lenticular lens.
To construct SMV displays, numerous projectors are required for a multi-projection system, and an ultra-high resolution flat-panel display is required for a flat-panel system. The multi-projection and flat-panel systems were recently combined to increase the number of views [9]. This technique enabled the construction of an SMV display with 256 views.
A head mount display (HMD)-type SMV display has also been developed [17, 18]. Because the eye position is fixed for HMD displays, the number of views can be reduced. A high-speed projector employing a digital micromirror device was used to generate multiple views for one eye.
As another method to resolve the accommodation-vergence conflict, a multi-focal display [19, 20] has been developed in which several two-view images are aligned in the depth direction, generating different two-view images at different depth positions. The positions of the two viewing zones for the left and right eyes are identical for all the two-view images. Although fewer display images are required for this technique, motion parallax is not provided.
In the present study, we propose a flat-panel display system that generates fewer views to satisfy the SMV condition. We offer a flat-panel system that generates two or more views for each eye with an interval smaller than the pupil diameter. This technique reduces the resolution of the flat-panel display required to construct an SMV display.
2. Conventional flat-panel SMV display
Figure 2 shows the formation of viewing zones for a conventional flat-panel SMV display. Massive viewing zones are generated without discontinuity. A lenticular lens is attached to a flat-panel display, and a group of pixels corresponds to each cylindrical lens constituting the lenticular lens. The cylindrical lenses magnify pixels in each group to generate viewing zones. All magnified images of the pixel groups by all lenses are superimposed at a pre-defined distance by making the lens pitch slightly smaller than the pitch of the pixel groups. The number of views is equal to the number of pixels in each pixel group; when the pixel group consists of n pixels, n viewing zones are generated. For color image generation, the number of subpixels (R, G, and B subpixels) in the pixel group is three times the number of views.
When the total width of the viewing zones is twice the average interocular distance (assumed to be 63 mm here) and the pitch of the viewing zones is the average pupil diameter (5 mm), the required number of views is 26. Therefore, a flat-panel display with a resolution 26 times larger than the 3D resolution is required. This requirement of an ultra-high resolution flat-panel display is the main difficulty of a flat-panel SMV display.
3. SMV display with a lower resolution flat-panel display
In the present study, to allow a lower resolution flat-panel display to be used in an SMV display, we propose an SMV display technique that produces two or more viewing zones only around each eye, with an interval smaller than the pupil diameter. Therefore, the total number of views can be reduced, thus reducing the resolution required for the flat-panel display.
Figure 3 illustrates the viewing zones formed in the proposed technique. The left pixel groups that generate the left viewing zones for the left eye are indicated by white boxes, and those for the right eye are represented by black boxes. The left and right pixel groups corresponding to the same cylindrical lens are partitioned to separate the left and right viewing zones; the left and right pixel groups corresponding to different cylindrical lenses are interlaced horizontally. Between the left and right pixel groups corresponding to the same lens, 2n pixel groups corresponding to the other lenses are arranged. Figures 3(a) and 3(b) show the cases when n = 1 and n = 2, respectively.
Fig. 3 Formation of viewing zones for the left and right eyes in the proposed technique: (a) n = 1; (b) n = 2.
Here, w represents the width of the left as well as the right viewing zones. The width of the region between the left and right viewing zones is given by 2nw. The distance between the centers of the left and right viewing zones is given by 2nw + w, and this distance is made identical to the interocular distance denoted by P. Thus,
When the number of viewing zones in the left and right viewing zones is denoted by V, the pitch of viewing zones, represented by d, is given by
Therefore, increasing n can reduce the pitch of the viewing zones. The SMV display condition can be achieved by properly choosing n and V. The formation of viewing zones by a conventional SMV display corresponds to the case when n = 0; the left and right viewing zones are connected. In the region between the left and right viewing zones, viewing zones that provide the same parallax images as those of the left and right viewing zones are generated. These viewing zones are produced when the pixel groups are projected by lenses other than a lens that projects the pixel groups to generate the left and right viewing zones.
Because the width w of the viewing zones for each eye decreases as n increases, the allowable range of eye movement decreases. The introduction of an eye tracking system can solve this problem. The use of such a system with a multi-view display [21] and an integral imaging display [22] has been reported. The positions of the left and right viewing zones can be moved by altering the grouping of pixels as shown in Fig. 4 . The use of the eye tracking system in effect allows a single viewer.
The requirements for the lenticular lens are considered next. The distance between the viewing zones and the lenses is denoted by l, and that between the lenses and the pixels of the flat-panel display is denoted by l′. The pixel pitch of the flat-panel display is denoted by p. From the similarity of triangles,
When the focal length of the lenses is represented by f, the lens maker’s formula gives 1/f = 1/l + 1/l′. The pixel groups are imaged to generate the left and right viewing zones. Therefore, the focal length can be given by the following equation.
When n is increased to reduce the pitch of the viewing zones, the total viewing zone width w decreases according to Eq. (1); thus, the focal length f increases. Therefore, the proposed technique requires a longer focal length for the lenses and thus a thicker lenticular lens. When the width of the pixel group, Vp, is much smaller than the width w of the viewing zones for each eye, the focal length can be approximated by Vpl/w, which is equal to l′.
4. Prototype system
An SMV display was constructed to demonstrate the proposed technique of partitioning viewing zones for the left eye and those for the right eye.
A flat-panel display with a slanted subpixel arrangement [23], which was previously developed to construct a 16-view display, was used. A photograph of the slanted subpixel arrangement is shown in Fig. 5 . Subpixels of the same color have different horizontal positions in each 3D pixel, which consists of 12 × 4 subpixels (4 × 4 subpixels in each R, G, and B color). The lenses of the lenticular lens deflect rays emitted from subpixels with different horizontal positions into different horizontal directions to generate multiple viewing zones. In the present study, each 3D pixel was divided into left and right pixel groups. The resolution of the flat-panel display was 1,024 × 768 pixels, and the screen size was 2.57 inches. The horizontal pitch of the subpixels was 4.25 μm, and the horizontal pitch of subpixels of the same color was p = 12.75 μm. The vertical pitch of the subpixels was 12.75 μm. The 3D resolution was 256 × 192 pixels.
The prototype SMV display was designed for n = 1. Each of the left and right viewing zones consisted of eight viewing zones, i.e., V = 8. The width of the each of the left and right viewing zones was w = 21.0 mm from Eq. (1). The width of the region between the two viewing zones was 42.0 mm. The pitch of the viewing zones was v = 2.6 mm from Eq. (2), which is sufficiently smaller than the average pupil diameter (5 mm).
The lenticular lens was designed to generate viewing zones 350 mm in front of it (l= 350 mm). The focal length of the lenses was f= 1.69 mm from Eq. (4). The pitch of the lenses given by 2Vp(2n − 1)/(1 + Vp/w) was calculated to be 0.203 mm. The lenticular lens was made of PMMA. The aspherical lens surface was optimized to minimize the radii of the spot diagrams in the viewing area whose width was twice as large as the interocular distance (126 mm) using the lens design software. Although an eye tracking system is not used in the present study, the lens was designed to cover the viewing area described above so that we could enhance the system in the future by introducing an eye tracking system. Spot diagrams for different positions in the viewing area are shown in Fig. 6 .
Fig. 6 Spot diagrams for our designed lenticular lens: Positions from the center of the viewing area are (a) 0.0 mm, (b) 10.5 mm, (c) 21.1 mm, (d) 31.6 mm, (e) 43.9 mm, (f) 56.2 mm, and (g) 68.6 mm. Radii of the spot diagrams are (a) 0.45 mm, (b) 0.45 mm, (c) 0.42 mm, (d) 0.36 mm, (e) 0.52 mm, (f) 0.82 mm, and (g) 1.19 mm.
Figure 7 shows a photograph of the proposed SMV display. Figure 8 shows photographs of the 3D images generated by this SMV display that were captured by a camera with pupil diameter set to ~2 mm. Figures 8(a)−8(c) and 8(g)−8(i) show images captured at three different horizontal positions (−38 mm, −33 mm, and −25 mm from the center) in the left viewing zones, and Figs. 8(d)−8(f) and 8(j) −8(l) show those captured at different three horizontal positions ( +25 mm, +33 mm, and +38 mm from the center) in the right viewing zones. Sixteen parallax images were generated by computer graphics software, which were then combined to generate a single image that was displayed on the flat-panel display.
Fig. 8 3D images generated by our prototype SMV display: (a)–(c) and (g) –(i) are images captured at different positions in the left viewing zones, and (d)– (f) and (j)– (l) are those in the right viewing zones.
The formation of the viewing zones was evaluated by measuring the intensity distributions in the viewing zones. The intensity distribution was measured by using a cooled CCD camera with a sensor plane placed at a distance of 350 mm where the viewing zones were generated. A white image was displayed in one viewing zone at which the intensity distribution was measured, and black images were displayed in the other viewing zones. The intensity distributions were measured for a width of 126 mm (twice the interocular distance), in consideration of the future enlargement of the viewing area by introducing the eye tracking system. Because the pitch of the viewing zones was 2.6 mm, the intensity distributions of 48 viewing zones were measured. Figure 9 shows the results; blue lines illustrate the results of the eight viewing zones for the left eye and red lines illustrate those for the right eye. The centers of the left and right view zones are −31.5 mm and +31.5 mm, respectively.
We examined the possibility of focusing on 3D images produced by the prototype SMV display. Five sets of three vertical lines were displayed at –40 mm, –20 mm, 0 mm, +20 mm, and +40 mm from the display screen, where a minus sign indicates that the lines were displayed behind the screen and a plus sign indicates that the lines were displayed in front of the screen. A camera, placed at the left viewing zones, was focused on each of the three lines. The aperture diameter of the camera lens was set at 5 mm, which is the average pupil diameter in humans; Fig. 10 shows the captured images. The minimum line width was obtained for the lines on which the camera was focused. The results reveal that the prototype display has the possibility of producing 3D images on which the human eye can focus. Indeed, a human eye could observe the change in the blurring of the line patterns.
Fig. 10 Focusing on three lines at different depth positions produced by our prototype SMV display. We focused the camera on the three lines at the following distances from the screen: (a) −40 mm, (b) −20 mm, (c) 0 mm, (d) +20 mm, and (e) +40 mm.
5. Discussion
From the results of measurement of the intensity distributions in the viewing zones shown in Fig. 9, the average interval between the viewing zones was 2.6 mm, which coincides well with the designed value. The crosstalk between viewing zones was higher in the periphery than in the center. This result is explained by the radii of the spot diagrams of the lenticular lens shown in Fig. 6. The increase of the viewing area by using the eye tracking mechanism might be limited by the crosstalk between viewing zones.
The proposed technique can reduce the pitch of the views by increasing the parameter n. Decreasing the pitch of the viewing zones leads to the requirement of a longer focal length lenticular lens (as described in Sec. 3), thus requiring lenses with greater radius of curvature and thickness. This means that higher accuracy is required in the fabrication of the lenticular lens to reduce the pitch of the viewing zones. The depth of the grooves of the lenticular lens depends on the radius of curvature and the lens pitch. The depth of the grooves of the lenticular lens fabricated for the prototype display was 6.1 μm. The accuracy of the fabrication process might be the main limiting factor determining the maximum value of n.
The width of the eight viewing zones for each eye of the prototype system was 21.0 mm, and the distance between the two viewing zones was 63.0 mm; thus, the system supported interocular distances between 47.0 mm and 79.0 mm, assuming a pupil diameter of 5.0 mm. Almost all the people could easily observe 3D images. However, the observation freedom in the depth direction depended on the interocular distance of each person.
The prototype system was constructed using a flat-panel display with a slanted subpixel arrangement. Therefore, the lenticular lens was not slanted. If a normal flat-panel display with the RGB stripe subpixel arrangement is used, the lenticular lens should be slanted [24]. When the lenticular lens is slanted, the viewing zones are also slanted at the same angle. In this case, the viewing zones that observers see depend on not only the horizontal but also the vertical positions of eyes. In contrast, the viewing zones generated by our prototype system are not slanted. Therefore, the prototype system has a large tolerance to the vertical eye positions.
The viewing area can be increased when an eye tracking system is introduced. The eye tracking system is required to detect not only the horizontal positions of the eyes but also their depth positions. Detection of the vertical eye position is not necessarily required because the lenticular lens of the prototype system is not slanted. However, detecting it would allow the system to provide quasi-vertical parallax.
6. Conclusion
We proposed an SMV display system that concentrates the viewing zones only around the left and right eyes to reduce the resolution required for the flat-panel display constituting the SMV display. We developed a prototype of such an SMV display system that generated eight viewing zones for each eye. The pitch of the viewing zones was 2.6 mm, and an LCD panel with a slanted subpixel arrangement was used. The screen size was 2.57 inches and the 3D resolution was 256 × 192 pixels.
The proposed technique reduces the pitch of the viewing zones, which decreases the horizontal width of the observation area. To overcome this problem, we will introduce an eye tracking system to our prototype SMV display system in the future.
Acknowledgments
The authors would like to thank Seiko Epson Corporation for providing the LCD panels. The present study was partly supported by a Grant-in-Aid for Scientific Research, No. (B) 20360153, from the Japan Society for the Promotion of Science (JSPS).
References and links
1. T. Okoshi, Three-Dimensional Imaging Techniques (Academic Press, New York, 1976).
2. T. Okoshi, “Three-dimensional displays,” Proc. IEEE 68(5), 548–564 (1980).
3. J.-H. Park, K. Hong, and B. Lee, “Recent progress in three-dimensional information processing based on integral imaging,” Appl. Opt. 48(34), H77–H94 (2009). [PubMed]
4. J.-Y. Son and B. Javidi, “Three-dimensional imaging methods based on multiview images,” J. Display Technol. 1(1), 125–140 (2005).
5. Y. Kajiki, H. Yoshikawa, and T. Honda, “Hologram-like video images by 45-view stereoscopic display,” Proc. SPIE 3012, 154–166 (1997).
6. T. Honda, Y. Kajiki, K. Susami, T. Hamaguchi, T. Endo, T. Hatada, and T. Fujii, “Three-dimensional display technologies satisfying ‘super multiview condition’,” SPIE Critical Reviews CR76, p.218–249.
7. T. Honda, Y. Kajiki, S. Susami, T. Hamaguchi, T. Endo, T. Hatada, and T. Fujii, “A display system for natural viewing of 3-D images,” in Three-dimensional television, video and display technologies, B. Javidi, F. Okano ed. (Springer-Verlag, Berlin Heidelberg, Germany, 2002) p.461–487.
8. T. Honda, D. Nagai, and M. Shimomatsu, “Development of 3-D display system by a fan-like array of projection optics,” Proc. SPIE 4660, 191–199 (2002).
9. Y. Takaki and N. Nago, “Multi-projection of lenticular displays to construct a 256-view super multi-view display,” Opt. Express 18(9), 8824–8835 (2010). [PubMed]
10. W. A. Ijsselsteijn, H. de Ridder, and J. Vliegen, “Effects of stereoscopic filming parameters and display duration on the subjective assessment of eye strain,” Proc. SPIE 3957, 12–22 (2000).
11. Y. Takaki, “High-Density Directional Display for Generating Natural Three-Dimensional Images,” Proc. IEEE 94(3), 654–663 (2006).
12. H. Nakanuma, H. Kamei, and Y. Takaki, “Natural 3D display with 128 directional images used for human-engineering evaluation,” Proc. SPIE 5664, 28–35 (2005).
13. K. Kikuta and Y. Takaki, “Development of SVGA resolution 128-directional display,” Proc. SPIE 6490, 64900U, 64900U-8 (2007).
14. Y. Takaki, “Thin-type natural three-dimensional display with 72 directional images,” Proc. SPIE 5664, 56–63 (2005).
15. Y. Takaki and T. Dairiki, “72-directional display having VGA resolution for high-appearance image generation,” Proc. SPIE 6055, 60550X, 60550X-8 (2006).
16. M. Tsuboi, M. Fujioka, Y. Takaki, and T. Horikoshi, “Real Time Rendering for a Full Parallax 3D Display Using High-Density Directional Images,” in Proceedings of the 13th International Display Workshops (IDW’06), pp. 1379–1380 (2006).
17. S.-K. Kim, D.-W. Kim, M.-C. Park, Y. M. Kwon, and J.-Y. Son, “Development of a HMD-type multifocus 3D display system using LEDs,” Proc. SPIE 6392, 63920B, 63920B-9 (2006).
18. S.-K. Kim, D.-W. Kim, Y. M. Kwon, and J.-Y. Son, “Evaluation of the monocular depth cue in 3D displays,” Opt. Express 16(26), 21415–21422 (2008). [PubMed]
19. S. Liu and H. Hua, “A systematic method for designing depth-fused multi-focal plane three-dimensional displays,” Opt. Express 18(11), 11562–11573 (2010). [PubMed]
20. K. Akeley, S. J. Watt, A. R. Girshick, and M. S. Banks, “A stereo display prototype with multiple focal distances,” ACM Trans. Graph. 23(3), 804–813 (2004).
21. R. S. Brar, P. Surman, I. Sexton, and K. Hopf, “MUTED: Multi-user 3-D display,” J. Soc. Inf. Disp. 18(9), 654–661 (2010).
22. G. Park, J.-H. Jung, K. Hong, Y. Kim, Y.-H. Kim, S.-W. Min, and B. Lee, “Multi-viewer tracking integral imaging system and its viewing zone analysis,” Opt. Express 17(20), 17895–17908 (2009). [PubMed]
23. Y. Takaki, “Multi-view 3-D display employing a flat-panel display with slanted pixel arrangement,” J. Soc. Inf. Disp. 18(7), 476–482 (2010).
24. C. van Berkel and J. A. Clarke, “Characterization and optimization of 3D-LCD module design,” Proc. SPIE 3012, 179–186 (1997).
