Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Binocular full-color holographic three-dimensional near eye display using a single SLM

Open Access Open Access

Abstract

A binocular full-color holographic three-dimensional near eye display system using a single spatial light modulator (SLM) is proposed. In the display system, the frequency spectrum shifting operation and color spectrum shifting operation are adopted to realize the frequency division multiplexing (FDM) and frequency superposition multiplexing (FSM) by manipulating the frequency spectrums of each color- and view-channel sub-holograms. The FDM combined with polarization multiplexing will be used to implement binocular display using a single SLM, and the FSM working with a bandpass filter for each view-channel will be used to achieve full-color display from single frame hologram. The optical analysis and experiments with 3D color objects confirm the feasibility of the proposed system in the practical application.

© 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

Since the inception from the fifties of the last century, as a novel near-eye display (NED) concept, VR/AR has been highly respected for its unique visual experience of on-the-spot and revolutionized the wearable way. Especially with the rise of the metaverse concept, NED technologies such as VR/AR are blooming again. Due to the human stereo visual feature, the 3D display is an indispensable technology to enhance the realism and immersion sense for VR/AR. Based on the structural characteristics of VR/AR, the binocular parallax stereogram is the straightforward and most direct solution for 3D display. However, the vergence-accommodation conflict (VAC) is an unavoidable defect of such technology [1].

To mitigate the VAC problem, some multi-plane display methods have been proposed by tunable lens focusing [2,3], multi-plane sweeping [4,5], polarization multiplexing [6], and so on, aiming to change the focal distance of the eye lens instead of just providing parallax images on a fixed display panel. Those methods can only partially alleviate the VAC problem because of the limited number of depth planes. Furthermore, the light field displays can provide a continuous depth change by recreating a physical wavefront, which has also been introduced in VR/AR display to decrease the VAC problem [79]. But the resolution of these methods is not satisfactory due to the limited bandwidth of the display devices. In addition, the retinal projection display technology has also been traded as a good solution for alleviating the VAC problem in VR/AR systems [10,11], however, the lack of defocusing blur effect will influence the 3D visual experience.

Fortunately, holography can reproduce a real 3D scene in high resolution matching with true depth information simultaneously, which is considered to be one of the most potential 3D solutions. Thus, holography-based VR/AR display system express excellent performance [1215]. However, current holographic display relies on computers to generate hologram and spatial light modulator (SLM) to load digital hologram. Due to the huge information and complex-value encoding of computer-generated holograms (CGHs), the CGHs generation algorithm becomes one of the barriers to the high-performance real-time application of dynamic holographic display. Thus, many kinds of algorithms, such as phase-only hologram encoding methods [1621] and speckle noise elimination methods [22,23] have been proposed to improve the reconstruction quality. Besides, the look-up table methods [2426], wavefront recording plane methods [27,28] polygon methods [29,30], layer-based methods [31,32], fractional Fourier transform-based methods [33,34], and convolutional neural network-based methods [12,13,3537] are proposed to accelerate the CGHs calculation speed. These algorithms have greatly improved the performance of CGHs, but the hologram generation algorithms still need to be promoted in terms of computing speed, imaging quality, and system universality, comprehensively. On the other hand, the performance of the holographic display system is also limited by the hardware device used to load the CGHs. Currently, the pixel size of commercial SLM, such as LCOS and DMD, is tens of wavelength, which causes the diffraction angle to be just 3∼5 degrees. To solve this problem, multiple devices spatially [38,39] and sequentially [40,41] splicing methods, and curved hologram methods [42,43] are proposed, which involuntary increase the volume of the holographic display system. When considering the full-color display, the system volume or refreshing rate burden will be greater [4447]. Therefore, multiplexing with more functions based on reducing the demand for hardware equipments will be more conducive to the development of holographic display.

Actually, the NED and holography just can complement each other in different aspects. The holography can provide true focal depth information on binocular parallax images, and the NED can reduce the requirement of wide viewing angle. Therefore, a binocular full-color holographic three-dimensional near eye display system using single SLM has been proposed here. The frequency division multiplexing (FDM) technology is used in this display system to separate the spectrum of sub-holograms corresponding to the binocular images. Further, a waveplate is adopted on the Fourier plane to modulate the polarization state of binocular information into orthogonality. Thus, two orthogonal polarizers or two circular polarizers with opposite polarization are set in front of each eyepiece for selecting the correct 3D information. Finally, a binocular holographic true 3D display system can be achieved using single SLM. Meanwhile, the frequency superposition multiplexing (FSM) method working with a bandpass filter for each view-channel have been introduced to achieve full-color display from single frame hologram. To confirm the feasibility of the proposed method, optical experiments and analysis using 3D color objects are carried out and the results are comparatively discussed.

2. Principle of the proposed system

Figure 1 shows the structure diagram of the proposed binocular full-color holographic three-dimensional near eye display system, which consists of only one SLM. The laser light sources are expanded and collimated as a multi-wavelength plane wave for illumination. The phase-only type binocular full-color hologram, which encodes 6 sub-holograms corresponding to the left and right view RGB information, is loaded on the SLM. Even though the left and right view information are encoded into a single hologram, the information is separated by the FDM method which can divide the Fourier plane into two parts corresponding to the frequency spectrum of the left and right view image. One waveplate is used to modulate the polarization of the light source before illuminating, the other is set on the Fourier plane of only one channel to modulate the polarization of the binocular information into orthogonality. The left view and right view reconstruction images are separated by a beam splitter and two polarizers corresponding to the polarization of each channel. Then, the binocular holographic display can be realized on single SLM. For the full-color implementation on this single SLM-based binocular display system, an angular compensation element (ACE) is introduced before the light source illuminates onto the SLM, which can increase the separation angle of color dispersion for full-color display by encoding RGB sub-holograms simultaneously. Thus, a FSM method working with a bandpass filter for each view-channel is adopted for avoiding the affiliated diffraction in the full-color display by manipulating the main diffraction spectrum of each color-channel to the corresponding spatial position. Meanwhile, the two waveplates should be an achromatic waveplate to eliminate the polarization modulation difference of multi-wavelength in the full-color display.

 figure: Fig. 1.

Fig. 1. Structure diagram of the proposed binocular full-color holographic three-dimensional near eye display system, ACE: angular compensation element, B.S.: beam splitter, SLM: spatial light modulator.

Download Full Size | PDF

2.1 Frequency spectrum shifting scheme of FDM for binocular display

In the FDM method, although the frequency domain space is separated into sub-regions for each sub-hologram, the spectrum of each sub-hologram should be shifted to their corresponding region. Because different spatial position represents different frequency value on the Fourier plane of the hologram. Thus, in the spectrum division process for FDM, the reconstruction quality by filtering the different spatial positions varies greatly. However, when the spectral region with the same frequency value of each sub-hologram is moved to the filter window through the frequency spectrum shifting operation, the reconstructed image quality will be the same even though the filters are located at different spatial positions. Theoretically, it is possible to arbitrarily shift the spatial location of the frequency spectrum on the Fourier plane by multiplying a corresponding phase factor, which can be expressed as follows:

$${H_{fs}}({f_x},{f_y}) = F\{ {h_{ori}}(x,y)\exp [ - j2\pi ({f_a}x + {f_b}y)]\} = {H_{ori}}({f_x}\textrm{ + }{f_a},{f_y} + {f_b}),$$
where Hfs(fx, fy) and Hori(fx + fa, fy + fb) mean the frequency spectrums of the frequency-shifted hologram and original hologram, respectively. After multiplying the phase factor exp[-j2π(fax + fby)], the frequency spectrum will shift values of fa and fb in horizontal (fx) and vertical (fy) directions, respectively.

Nevertheless, the phase factor will act as a grating of the off-axis reference beam in the diffraction reconstruction process. Thus, the propagation direction of reconstruction will be changed by multiplying the phase factor. The spatial position of the reconstructed image will also be changed. An image & frequency shift multiplexing (IFSM) method was proposed in our previous work to overcome such problems [48]. When the hologram is pixelated shifted in the spatial domain, its frequency spectrum is expressed as follows:

$${H_{ss}}({f_{x\_a}},{f_{y\_b}}) = F\{ {h_{ori}}(x + a,y + b)\} = {H_{ori}}({f_x},{f_y})\exp [ - j2\pi ({f_{s\_a}}x + {f_{s\_b}}y)],$$
where Hss(fx_a, fy_b) and Hori(fx, fy) represent the frequency spectrum of spatial shifted hologram and original hologram, respectively. We can find that when the hologram has spatial shifting values of a and b in the x and y directions, respectively, the spatial position of the frequency spectrum keeps the same as that of the original hologram. While there will be a phase factor on the spectrum, which means the direction of propagation through the Fourier plane changes but the spatial position remains the same. Thanks to the movement invariance of the holograms, the reconstructed image will shift the same displacement along with the pixelated shift value of the hologram. The hologram pixelated shifting process is called image shifting in the IFSM method. Therefore, the spatial position arbitrarily shifting of the frequency spectrum can be realized by using image shifting to compensate for the displacement of the reproduced image caused by the frequency shifting process. The same frequency shifting scheme can also be used to implement the FDM for binocular display.

The hologram has a strong linear superposition property. Two sub-holograms of binocular images can be encoded into a single hologram by the FDM method, in which the frequency spectrums of left and right view images will occupy different spatial areas on the Fourier plane. Then, the polarization state of each channel can be modulated by using a waveplate in the optical light path separately. Therefore, the binocular holographic display using single SLM can be easily realized by this polarization multiplexing method working with the FDM method. Figure 2 shows the digital generation process of the binocular hologram. First, the sub-holograms of the left view and right view 3D images are generated computationally. Here in this work, the novel look-up table method [24] is used for hologram generation. The image-shifting process can be encoded into the pre-calculating process of the principal fringe patterns, which can simplify the image-shifting process. Second, the calculated sub-holograms are Fourier transformed into their frequency spectrum domain for band limiting, which can eliminate the spectral aliasing effect after FDM. Third, the band-limited sub-holograms are inverse transformed into their spatial domain for the frequency-shifting process. By multiplying different phase factors, the frequency spectrums of each channel are shifted to different spatial positions, while the spatial position of reconstructed images keep the same. Finally, the two sub-holograms of different view are linearly superposed together for a single-frame binocular hologram. The frequency spectrums of each view-channel will occupy the different spatial positions on the Fourier plane when this hologram is loaded on the SLM.

 figure: Fig. 2.

Fig. 2. Digital generation process of the binocular hologram, FT: Fourier Transform, IFT: Inverse Fourier Transform, IS: Image Shift-, FS: Frequency Shift, IFS: Image and Frequency Shift.

Download Full Size | PDF

2.2 Color spectrum shifting scheme of FSM for full-color display

As we all know, a hologram is a kind of grating that can cause color dispersion when illuminated with a multi-wavelength light source. That means any hologram can reproduce red, green, and blue color reconstructed images containing the same information when illuminated with RGB color light sources simultaneously. Here we call the different color frequency spectrums from a single monochromatic hologram caused by color dispersion as the color spectrums. Which, only the color of the reproduced image or color spectrum matching the calculating wavelength of the hologram is the main spectrum, while the others are the affiliated reconstruction that should be removed or filtered out. Thus, the previous mentioned ISFM method can realize a full-color holographic display on a single SLM by superimposing and passing through the main frequency spectrums of RGB reconstructed images together using a bandpass filter. However, the ISFM method can just shift the frequency spectrum of each sub-hologram. Because all the color spectrums are dispersed from one hologram and contain the same information, the frequency shifting operation mentioned in the previous subsection can only move the color spectrums as a whole rather than shifting any one of the color spectrums separately. Because the diffraction angle of color dispersion is proportional to the wavelength and inversely proportional to the period of the grating. Limited by the pixel pitch of the current commercial SLM, the diffraction angle is small. Thus, the reconstructed images from the dispersion will be partially superimposed and the color spectrums will be close to each other, as shown in Fig. 3(a). The separation angle between different color spectrum is just the diffraction angle difference, as shown in Fig. 3(b). Then, it makes the bandwidth of the bandpass filter should be small to avoid the crosstalk of the close color spectrums. However, the small size bandpass filter will dramatically decrease the reconstruction quality.

 figure: Fig. 3.

Fig. 3. Color spectrums and reconstructed images distribution caused by color dispersion when the RGB light source illuminated (a) coaxially and (c) with different incident angles by the angular compensation element (ACE); the diffraction angles distribution corresponding to RGB when the light source illuminated (b) coaxially and (d) by ACE.

Download Full Size | PDF

On the other hand, the hologram loaded on the SLM is not a volume hologram, the diffracted propagation direction will be changed along with the direction of incident light. Therefore, the RGB color spectrums and RGB color reconstructed images will be separated large when the RGB light source illuminating with different incident angles, as shown in Fig. 3(c). In this situation, the separation angle between RGB color reconstruction equals the summation of the incident angle and diffraction angle. Detailly, as shown in Fig. 3(d), the angles θR, θG and θB are the diffraction angle corresponding to each color from the hologram, which are the same with shown in Fig. 3(b). While, when the red color light source incident with an angle αR instead of perpendicular illumination, but the diffraction angle kept the same with θR. Therefore, the angle between the diffracted light with z-axis will be θR+αR. On the other hand, if the incident angle for green and blue color light source are 0 and R, respectively, the separation angle between red, green and blue color light will be increased by αR. Even though the diffraction angle cannot be enlarged, the separation angle can be simply enlarged by increasing the difference of incident angles. Further, the color spectrums can be shifted separately by introducing different off-axis incident angles. The detail color spectrum shifting amount can be given by the following equation:

$${d_{css}} = f \cdot \sin \alpha ,$$
where f is the focal length of the lens in 4f system, and α represents the incident angle changing amount. The enlarged separation angle between each component of color spectrums will help for passing through a large bandwidth spectrum area when spatially filtering the affiliated spectrums on the Fourier plane. Therefore, the bandwidth of the bandpass filter in the ISFM method for full-color display can be enlarged. The different incident angles of the light source can be achieved by the special designed angular compensation element (ACE) which contains two primes and a plane plate, as shown in Fig. 3(b). Finally, the full-color holographic display on a single-frame hologram can be realized by using a single-hole bandpass filter for each view-channel. By combining with the FDM method, the full-color function can be introduced into the binocular holographic display on a single SLM using double-hole bandpass filter.

3. Experimental setup and results

3.1 Optical setup of the proposed system

Figure 4 shows the optical experimental setup of the proposed binocular full-color holographic 3D display system. In which a 1920 × 1080 pixels reflective type LCOS with a pixel pitch of 8 µm is used as the SLM. The hologram loaded on the SLM is encoded by 6 sub-holograms corresponding to the left and right view RGB component information. Three lasers with wavelengths of 660 nm, 532 nm, and 473 nm are used as the light sources in the experimental setup. The laser light sources are expanded as three plane waves by a 40× objective lens, a pinhole spatial filter, and a field lens with a focal length of 300 mm, separately.

 figure: Fig. 4.

Fig. 4. Optical setup of the binocular full-color display system: (a) photograph of the overall setup, (b) double-hole bandpass filter in the 4-f system, and (c) binocular viewing eyepiece system.

Download Full Size | PDF

A special designed ACE is set before the plane waves illuminating onto the SLM, which bends the plane waves into different incident angles. The ACE consists of a plate glass and two prisms with angles of 2° and -2°, in which the two prisms are symmetrically distributed at both ends of the plate glass. The expanded RGB plane waves correspondingly pass through the positive prism, plate glass, and negative prism by aperture, respectively. Such element will bend the incident angles of RGB plane waves on the SLM into 1°, 0°, and -1°, respectively. A special designed double-hole bandpass filter is set on the Fourier plane of the 4-f lens system to remove the disturbing spectrums of affiliated diffraction. To separate the binocular information, a commercial achromatic half waveplate with a nominal working frequency range of 450∼650 nm is set behind only one hole of the bandpass filter to adjust the polarization of the binocular frequency spectrums as orthogonally. Thus, the fast axis of the waveplate is set as 45° with vertical or horizontal axis, as shown in Fig. 4(b). To match the waveplate, a polarized beam splitter is employed to separate the left and right view information from the orthogonally polarized information in different directions. Finally, each view reconstructed image can be propagated into the corresponding eyepiece by two depolarized beam splitters, as shown in Fig. 4(c). Such a binocular eyepiece system is placed on the light path after the mirror reflected the reconstruction light field, which is not shown in Fig. 4(a). The reconstructed experiment results are captured by a Nikon camera with a lens, which is consistent with the viewing effect of direct viewing by the human eyes.

3.2 Optical verification and analysis of the proposed display system

To verify the effectiveness of the FDM in the proposed binocular display system, the analysis of frequency spectrums for different cases is carried out. Figure 5 shows the frequency spectrum results of a comparative experiment with and without the frequency shifting operation for FDM. The first and third row are the spectrums of the left and right view original hologram without frequency shifting operation, the second row is the spectrums of the left view hologram with negative frequency shifting, and the fourth row is the spectrums of the right view hologram with positive frequency shifting. Since the RGB light sources are illuminated with different incident angles, the focused zero orders of RGB light source are three bright spots arranged in sequence with the corresponding color. As shown in the spectrum results, the frequency spectrum of the original hologram is concentrated in the low-frequency region and overlaps with the zero order. Because of the simultaneous incidence of RGB light source and no Bragg diffraction effect, each hologram can be reconstructed in every color. The spectrums of each color always overlap onto the zero order of the corresponding color. Therefore, the spectrums of the color hologram encoding RGB sub-holograms simultaneously superimpose three component information at each spectrum area. That is the frequency spectrum labeled by fCr contains both the spectrums labeled by fRr, fGr, and fBr, which are the frequency spectrums of RGB sub-holograms diffracted by the ‘r’ color light source, respectively. However, the spectrums labeled by fGr and fBr are the affiliated diffraction parts that should be removed. The other two spectrums labeled by fCg and fCb have the same problem. In this situation, the affiliated diffraction spectrum is impossible to be removed away from the main spectrum for full-color display. On the other hand, when the left and right view hologram encoding together, their main spectrum will also superimpose at the same spatial position, which is also impossible to modify the polarization separately.

 figure: Fig. 5.

Fig. 5. Frequency spectrum comparative results of left and right view holograms with and without frequency shifting process. (Captured by CCD)

Download Full Size | PDF

When introducing the frequency shifting operation, not only the spectrum of the left and right view hologram can be separated, but the spectrums of affiliated diffraction can also be shifted away from the main spectrum for full-color display. To avoid the spectrums crosstalk of left and right view and their conjugate diffraction, in this experiment, the frequency shifting amount of the R sub-hologram is (-0.0224, 0.0385)/µm whose corresponding spatial displacement is (-4.44, 7.62) mm. As shown in the second row of Fig. 5, both the spectrums labeled by fRr, fRg, and fRb have been shifted to the upper left from their corresponding zero order by frequency shifting on the left view R sub-hologram, for which they are both the spectrum of R sub-hologram. In addition, the frequency spectrum of G and B sub-holograms have been shifted to a small upper left and lower left corresponding to their zero order. The detailed frequency shifting amount of R, G, and B sub-holograms and their corresponding spatial displacement are listed in Table 1 based on the diffraction equation and Abbe imaging equation. Furthermore, three main spectrums for RGB component labeled by fRr, fGg, and fBb overlap at the same spatial position, and the affiliated diffraction have been separated away from the main spectrums. Therefore, a bandpass filter can be used on the Fourier plane to pass through the main spectrums and filter out the affiliated diffraction spectrums for a full-color holographic display. In order to prevent spectrum aliasing of left and right view sub-holograms and their conjugate spectrums, the frequency spectrum shifting of right view sub-holograms are carried toward the right side of the zero-order as shown in the fourth row of Fig. 5. And also, the detail frequency shifting amount of right view R, G and B sub-holograms and their corresponding spatial displacement are also listed in Table 1. Finally, the double-hole bandpass filter is set on the Fourier plane to easily remove the affiliated diffraction and spatially separate the binocular information, which verifies the effectiveness of the frequency-shifting operation for FDM. Further, the waveplate is introduced in the system to modulate the reconstruction of the light fields of left and right view scenes into orthogonal polarization. Then, the binocular full-color holographic display using single SLM can be realized.

Tables Icon

Table 1. Parameters of the frequency shifting amount for left and right view RGB sub-holograms

To illustrate the availability of the FSM for full-color display, the frequency spectrums comparison results of the color display using color spectrum shifting operation or not are carried out in Fig. 6. Figures 6(a), (b) and (c) are the frequency spectrums distribution of left view color-hologram, right view color-hologram, and binocular color-hologram, respectively. By introducing the FDM mentioned before, the affiliated diffraction spectrums can also be shifted away from the main color spectrum. The full-color display can also be realized by introducing the double-hole bandpass filter on the Fourier plane. However, limited by the pixel size of the SLM, the maximum color dispersion angle of first-order diffraction are just 0.45° between R and G color, and 0.216° between G and B color, which results in the spectrums caused by color dispersion, such as fRr, fGr, and fBr, cannot separate large enough, as shown in the expanded area of Figs. 6(a) and (b). In order to avoid the spectrums crosstalk of left and right view, in this work, the frequency shifting amount for left view and right view hologram are both quarter of the maximum frequency in horizontal and vertical directions, respectively. Thus, the theoretical spatial distance of the neighboring spectrums labeled by fGb and fBg away from main color spectrums fRr, fGg, and fBb are 0.7821 mm and 0.6954 mm, respectively. The actual distance in the experiment results as shown in Fig. 6(c) are roughly estimated as 0.9 mm and 0.8 mm, which is consistent with the theory. The length bar is based on the actual ruler as shown on the left edge of Fig. 6(c). Such a proximity distance results in the aperture size of the bandpass filter cannot be large. Further, the frequency band is limited by a small range in the optical filtering process, which decreases the quality of reproduced images.

 figure: Fig. 6.

Fig. 6. Frequency spectrum comparative results of left and right view holograms using color spectrum shifting (CSS) operation or not: the left column is the frequency spectrum results without CSS operation, and the right column is the frequency spectrum results with CSS operation; the first row is the left view results (Captured by CCD), the second row is the right view results (Captured by CCD), and the third row is the binocular view frequency spectrum results (Captured on a white screen by digital camera with lens).

Download Full Size | PDF

As shown in the right column of Fig. 6, when introducing the color spectrum shifting operation, the color dispersion separation angle becomes larger, and the affiliated diffraction spectrums have been shifted far away from the center overlapping color spectrums. Instead, the neighbor spectrums are the conjugate spectrums, as shown in Fig. 6(f). Based on the theoretical equations, the distance between the conjugate B color spectrum of the left view R sub-hologram labeled by fBr_L_con with center color spectrum is 1.89 mm, which is far enough away from the color spectrums. The real distance in this experiment can be calculated by roughly scaling with the left ruler as 2 mm, which is consistent with the theory. Therefore, the color spectrum shifting operation can increase the dispersion separation angle, so that the affiliated diffraction spectrums can be shifted away from the center-overlapping color spectrums. Then, a large bandwidth optical filtering can be achieved, as shown in Figs. 6(d) and (e).

3.3 Experimental results of the proposed display system

Figure 7 shows the reconstruction results of the binocular hologram based on a single SLM. The recorded image of the left view hologram is a ‘Cube’, and the right view hologram recorded image is a ‘Soccer’. After the corresponding frequency shifting process, the RGB sub-holograms of the left and right view are encoded into a single frame binocular color hologram and loaded on the SLM. Figures 7(a), (b) and (c) show the reconstructed results of a single frame binocular color hologram by passing through only the left, right and both view spectrum information, respectively. Due to the ISFM process, the left and right view full-color spectrums are set only on two separated positions. To demonstrate the effect of the spectrum manipulation process, the on-off of the left and right view spectrums are just controlled by bandpass filter. It’s worth mentioned that the waveplate is not set on the Fourier plane yet. From the experimental results, it can be proved that the proposed system can well separate the information of the single frame binocular hologram in the frequency domain by the FDM and polarization multiplexing method.

 figure: Fig. 7.

Fig. 7. Reconstruction results of the proposed binocular full-color holographic display system from single SLM: (a), (b), (c) are the results captured at the same position by passing through the color spectrums of only left view, only right view and both, respectively; (d)-(h) show the specific frames of the reconstructed Visualization 1 result from the binocular hologram with orthogonal polarization under different polarization state detection. (Captured by digital camera with lens)

Download Full Size | PDF

When the waveplate is inserted behind one of the spectrums to modulate the polarization state, the result is shown in Visualization 1 and some specific frames are shown in the second row of Fig. 7. Since the color display in this experiment requires both RGB light sources, an achromatic waveplate is used as the waveplate to prevent the polarization directions of different colors from being different. To check the polarization of the reconstructed light field of the binocular hologram, a polarizer is temporarily placed in front of the capturing camera. Because the proposed system simultaneously encodes 6 sub-holograms into a single frame hologram, there should be no strobing problem in both the left and right view display results, which can also be found in Visualization 1. With the polarization changed from vertical to horizontal, the left view reconstructed image ‘Cube’ gradually disappears, while the right view reconstructed image ‘Soccer’ gradually becomes clearer. Therefore, it is proved that the reconstructed light field of the left and right view hologram by this system from a single SLM has perfect orthogonal polarizations.

Figure 8 shows the acquisition 3D results of the proposed binocular holographic display system on the binocular AR viewing eyepiece which is shown in Fig. 4(c). The test scenario is designed by both a ‘Cube’ and a ‘Soccer’ located at different depths to illuminate the 3D display performance. Due to the different reflection light paths of the eyepiece for the left and right view, the reconstructed images from the right view have a mirror effect. Such effect can be simply compensated by mirroring the right view 3D input image. While, due to the small 3D perspective, the left and right view input image have just slightly different. To well distinguish the left and right view reconstruction, the mirror effect is not eliminated in Fig. 8. The reconstructed light field is firstly separated into left and right view images by a polarization beam splitter. Then, each view reconstruction is reflected into the left and right eyepiece by mirrors, and each eyepiece is fused with the real scene, the printed label marked with ‘Front Image’ and ‘Rear Image’, with the reconstrued light field by a depolarization beam splitter. The left and right column of Fig. 8 show the reconstruction results captured from the left and right eyepiece by a digital camera with a lens, focusing on the front and rear images, respectively. It can be seen from the experimental results that the proposed system can reconstruct binocular full-color 3D images with no crosstalk from the left and right view on a single SLM using a static single-frame hologram. The dynamic refreshing rate can be easily used to scanning expand the viewing angle by the time-multiplexing method.

 figure: Fig. 8.

Fig. 8. Optical 3D binocular reconstruction on the binocular eyepiece: (a) left view reconstruction focused on the front image ‘Cube’ and (b) rear image ‘Soccer’; (c) right view reconstruction focused on the front image ‘Cube’ and (d) rear image ‘Soccer’. (Captured by digital camera with lens)

Download Full Size | PDF

3.4 Discussion and limitations

Honestly, this paper just focuses on achieving more functions, binocular information display and full-color, on single SLM for holographic display. Thus, there are still more works should be done for this system to be used in actual VR/AR display. First, the hologram calculation algorithm used in this work is the novel look-up table method. Such method can accelerate the calculation speed than traditional ray tracking method, but the image quality and calculation speed are still far away from actual application, which can be improved by the recent convolution neural network-based algorithm [12,13]. Second, the binocular viewing eyepiece is not well designed, which just consists of two beam splitters for simply exhibiting the reconstruction results of each view. The two optical axes of each beam splitter are set with 65 mm width for pupillary distance and 12° tilted angle with each other. The center positions of the two reconstructed images are set at 300 mm and 350 mm in front of each eyepiece. Because there is no field lens in each eyepiece, the field of view is limited in several degree, which is far away from using in commercial near eye display. Such limitations can be settled down by the advanced meta-surface or multi-functional volume holographic optical elements [1315]. Third, since the proposed method does not use time-division multiplexing, the saved refreshing resources will be used to increase the field of view in the future. Therefore, there still need more works in the future to make the proposed system for actual application.

4. Conclusions

In this paper, a binocular full-color holographic three-dimensional near eye display system using single SLM is proposed, in which the FDM combined with polarization multiplexing is introduced in this system for separately modulating the polarization of binocular information, and the FSM working with color spectrum shifting operation is adopted to realize full-color display on single frame hologram by a large bandwidth bandpass filter for each view-channel. The frequency shifting process is used to operate the spatial position of the frequency spectrum of each sub-hologram for FDM and FSM. Meanwhile, the image shifting operation is also needed to compensate for the displacement of reconstruction caused by the frequency shifting process. The optical spectrums distribution and full-color binocular reconstruction results of different cases are carried out to prove the effectiveness of the FDM and FSM in the proposed binocular display system. The 3D scenario with different depths is also shown to illuminate the 3D display performance of the proposed system. The proposed display system realizes binocular and full-color holographic true 3D display simultaneously in a single SLM without increasing the burden of hardware equipment and refreshing rate, which provides a good technical solution for the combination of holographic 3D display and NED. In addition, the refresh rate resources can be combined with time division multiplexing technology to expand the viewing angle and field of view, thereby realizing richer functions.

Funding

National Natural Science Foundation of China (NSFC) (61905008, 62275006); Natural Science Foundation of Beijing Municipality (4222063).

Acknowledgments

We want to thank Rubik’s Brand Ltd. for permitting the usage of a Rubik’s Cube in our paper.

Disclosures

The authors declare no conflicts of interest.

Data availability

The data underlying the results presented in this paper are not publicly available but can be obtained from the authors upon reasonable request.

References

1. K. Yin, Z. He, J. Xiong, J. Zou, K. Li, and S.-T. Wu, “Virtual reality and augmented reality displays: advances and future perspectives,” JPhys Photonics 3(2), 022010 (2021). [CrossRef]  

2. N. Matsuda, A. Fix, and D. Lanman, “Focal surface displays,” ACM Trans. Graph. 36(4), 1–14 (2017). [CrossRef]  

3. S. Liu, D. Cheng, and H. Hua, “An optical see-through head mounted display with addressable focal planes,” in Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality (IEEE2008), pp. 33–42.

4. S. Liu, Y. Li, P. Zhou, Q. Chen, and Y. Su, “Reverse-mode PSLC multi-plane optical see-through display for AR applications,” Opt. Express 26(3), 3394–3403 (2018). [CrossRef]  

5. V. Joshi, M. Groom, and L. C. Chien, “Glasses-free 3-D visualization with multi-layered transparent cholesteric films,” Opt. Express 27(12), 16847–16854 (2019). [CrossRef]  

6. G. Tan, T. Zhan, Y. H. Lee, J. Xiong, and S. T. Wu, “Polarization-multiplexed multiplane display,” Opt. Lett. 43(22), 5651–5654 (2018). [CrossRef]  

7. L. Shi, F. C. Huang, W. Lopes, W. Matusik, and D. Luebke, “Near-eye light field holographic rendering with spherical waves for wide field of view interactive 3d computer graphics,” ACM Trans. Graph. 36(6), 1–17 (2017). [CrossRef]  

8. K. Hong, J. Yeom, C. Jang, J. Hong, and B. Lee, “Full-color lens-array holographic optical element for three-dimensional optical see-through augmented reality,” Opt. Lett. 39(1), 127–130 (2014). [CrossRef]  

9. H. Deng, C. Chen, M. Y. He, J. J. Li, H. L. Zhang, and Q. H. Wang, “High-resolution augmented reality 3D display with use of a lenticular lens array holographic optical element,” J. Opt. Soc. Am. A 36(4), 588–593 (2019). [CrossRef]  

10. Z. Wang, K. Tu, Y. Pang, G. Q. Lv, Q. B. Feng, A. T. Wang, and H. Ming, “Enlarging the FOV of lensless holographic retinal projection display with two-step Fresnel diffraction,” Appl. Phys. Lett. 121(8), 081103 (2022). [CrossRef]  

11. W. Song, X. Li, Y. Zheng, Y. Liu, and Y. Wang, “Full-color retinal-projection near-eye display using a multiplexing-encoding holographic method,” Opt. Express 29(6), 8098–8107 (2021). [CrossRef]  

12. Y. Wang, P. Chakravarthula, Q. Sun, and B. Chen, “Joint neural phase retrieval and compression for energy- and computation-efficient holography on the edge,” ACM Trans. Graph. 41(4), 1–16 (2022). [CrossRef]  

13. J. Kim, M. Gopakumar, S. Choi, Y. Peng, W. Lopes, and G. Wetzstein, “Holographic Glasses for Virtual Reality,” in ACM SIGGRAPH 2022 Conference Proceedings (SIGGRAPH ‘22), pp. 33:1–9.

14. A. Maimone, A. Georgiou, and J. S. Kollin, “Holographic near-eye displays for virtual and augmented reality,” ACM Trans. Graph. 36(4), 1–16 (2017). [CrossRef]  

15. W. Song, X. Liang, S. Li, D. Li, R. Paniagua-Domínguez, K. H. Lai, Q. Lin, Y. Zheng, and A. I. Kuznetsov, “Large-Scale Huygens’ Metasurfaces for Holographic 3D Near-Eye Displays,” Laser Photonics Rev. 15(9), 2000538 (2021). [CrossRef]  

16. X. M. Sui, Z. H. He, G. F. Jin, and L. C. Cao, “Spectral-envelope modulated double-phase method for computer-generated holography,” Opt. Express 30(17), 30552–30563 (2022). [CrossRef]  

17. J. A. Davis, E. D. Wolfe, I. Moreno, and D. M. Cottrell, “Encoding complex amplitude information onto phase-only diffractive optical elements using binary phase Nyquist gratings,” OSA Continuum 4(3), 896 (2021). [CrossRef]  

18. P. Chakravarthula, Y. Peng, J. Kollin, H. Fuchs, and F. Heide, “Wirtinger holography for near-eye displays,” ACM Trans. Graph. 38(6), 1–13 (2019). [CrossRef]  

19. S. Jiao, D. Zhang, C. Zhang, Y. Gao, T. Lei, and X. Yuan, “Complex-Amplitude Holographic Projection with a Digital Micromirror Device (DMD) and Error Diffusion Algorithm,” IEEE J. Select. Topics Quantum Electron. 26(5), 1–8 (2020). [CrossRef]  

20. Y. Wu, J. Wang, C. Chen, C. J. Liu, F. M. Jin, and N. Chen, “Adaptive weighted Gerchberg-Saxton algorithm for generation of phase-only hologram with artifacts suppression,” Opt. Express 29(2), 1412–1427 (2021). [CrossRef]  

21. L. Chen, H. Zhang, Z. He, X. Wang, L. Cao, and G. Jin, “Weighted constraint iterative algorithm for phase hologram generation,” Appl. Sci. 10(10), 3652 (2020). [CrossRef]  

22. D. Wang, N. N. Li, Y. L. Li, Y. W. Zheng, and Q. H. Wang, “Curved hologram generation method for speckle noise suppression based on the stochastic gradient descent algorithm,” Opt. Express 29(26), 42650–42662 (2021). [CrossRef]  

23. I. Ducina, T. Shimobabab, M. Makowski, K. Kakarenkoa, A. Kowalczyka, J. Suszeka, M. Biedaa, A. Kolodziejczyka, and M. Sypek, “Holographic projection of images with step-less zoom and noise suppression by pixel separation,” Opt. Commun. 340, 131–135 (2015). [CrossRef]  

24. S. C. Kim and E. S. Kim, “Effective generation of digital holograms of three-dimensional objects using a novel look-up table method,” Appl. Opt. 47(19), D55–D62 (2008). [CrossRef]  

25. Y. Pan, X. Xu, S. Solanki, X. Liang, R. B. Tanjung, C. Tan, and T. C. Chong, “Fast CGH computation using S-LUT on GPU,” Opt. Express 17(21), 18543–18555 (2009). [CrossRef]  

26. J. Jia, Y. Wang, J. Liu, X. Li, Y. Pan, Z. Sun, B. Zhang, Q. Zhao, and W. Jiang, “Reducing the memory usage for effective computer-generated hologram calculation using compressed look-up table in full-color holographic display,” Appl. Opt. 52(7), 1404–1412 (2013). [CrossRef]  

27. T. Shimobaba, N. Masuda, and T. Ito, “Simple and fast calculation algorithm for computer-generated hologram with wavefront recording plane,” Opt. Lett. 34(20), 3133–3135 (2009). [CrossRef]  

28. D. Pi, J. Liu, Y. Han, S. Yu, and N. Xiang, “Acceleration of computer-generated hologram using wavefront-recording plane and look-up table in three-dimensional holographic display,” Opt. Express 28(7), 9833–9841 (2020). [CrossRef]  

29. Y. Pan, Y. Wang, J. Liu, X. Li, and J. Jia, “Fast polygon-based method for calculating computer-generated holograms in three-dimensional display,” Appl. Opt. 52(1), A290–A299 (2013). [CrossRef]  

30. D. Im, J. Cho, J. Hahn, B. Lee, and H. Kim, “Accelerated synthesis algorithm of polygon computer-generated holograms,” Opt. Express 23(3), 2863–2871 (2015). [CrossRef]  

31. Y. Zhao, L. Cao, H. Zhang, D. Kong, and G. Jin, “Accurate calculation of computer-generated holograms using angular-spectrum layer-oriented method,” Opt. Express 23(20), 25440–25449 (2015). [CrossRef]  

32. H. Zhang, L. Cao, and G. Jin, “Computer-generated hologram with occlusion effect using layer-based processing,” Appl. Opt. 56(13), F138–F143 (2017). [CrossRef]  

33. C. L. Chang, J. Xia, and W. Lei, “Holographic image projection using fractional Fourier transformation,” Opt. Commun. 285(1), 24–28 (2012). [CrossRef]  

34. Z. Zhang, S. Chen, H. Zheng, Z. Zeng, H. Gao, Y. Yu, and A. K. Asundi, “Full-color holographic 3D display using slice-based fractional Fourier transform combined with free-space Fresnel diffraction,” Appl. Opt. 56(20), 5668–5675 (2017). [CrossRef]  

35. L. Shi, B. Li, C. Kim, P. Kellnhofer, and W. Matusik, “Towards real-time photorealistic 3D holography with deep neural networks,” Nature 591(7849), 234–239 (2021). [CrossRef]  

36. Y. Peng, S. Choi, N. Padmanaban, and G. Wetzstein, “Neural holography with camera-in-the-loop training,” ACM Trans. Graph. 39(6), 1–14 (2020). [CrossRef]  

37. S.-C. Liu and D. Chu, “Deep learning for hologram generation,” Opt. Express 29(17), 27373–27395 (2021). [CrossRef]  

38. H. Sasaki, K. Yamamoto, K. Wakunami, Y. Ichihashi, R. Oi, and T. Senoh, “Large size three-dimensional video by electronic holography using multiple spatial light modulators,” Sci. Rep. 4(1), 6177 (2015). [CrossRef]  

39. T. Kozacki, G. Finke, P. Garbat, W. Zaperty, and M. Kujawińska, “Wide angle holographic display system with spatiotemporal multiplexing,” Opt. Express 20(25), 27473–27481 (2012). [CrossRef]  

40. B. Lee, D. Yoo, J. Jeong, S. Lee, D. Lee, and B. Lee, “Wide-angle speckleless DMD holographic display using structured illumination with temporal multiplexing,” Opt. Lett. 45(8), 2148–2151 (2020). [CrossRef]  

41. Y. Z. Liu, X.-N. Pang, S. Jiang, and J. W. Dong, “Viewing-angle enlargement in holographic augmented reality using time division and spatial tiling,” Opt. Express 21(10), 12068–12076 (2013). [CrossRef]  

42. R. Kang, J. Liu, G. Xue, X. Li, D. Pi, and Y. Wang, “Curved multiplexing computer-generated hologram for 3D holographic display,” Opt. Express 27(10), 14369–14380 (2019). [CrossRef]  

43. Y. F. Ma, J. Wang, Y. Wu, F. M. Jin, Z. K. Zhang, Z. X. Zhou, and N. Chen, “Large field-of-view holographic display by gapless splicing of multisegment cylindrical holograms,” Appl. Opt. 60(24), 7381–7390 (2021). [CrossRef]  

44. J. Roh, K. Kim, E. Moon, S. Kim, B. Yang, J. Hahn, and H. Kim, “Full-color holographic projection display system featuring an achromatic Fourier filter,” Opt. Express 25(13), 14774–14782 (2017). [CrossRef]  

45. Y. Zhao, K. C. Kwon, M. U. Erdenebat, S. H. Jeon, M. L. Piao, and N. Kim, “Implementation of full-color holographic system using non-uniformly sampled 2D images and compressed point cloud gridding,” Opt. Express 27(21), 29746–29758 (2019). [CrossRef]  

46. Y. Zhao, K. C. Kwon, Y. L. Piao, S. H. Jeon, and N. Kim, “Depth-layer weighted prediction method for a full-color polygon-based holographic system with real objects,” Opt. Lett. 42(13), 2599–2602 (2017). [CrossRef]  

47. Y. Lim, K. Hong, H. Kim, H.-E. Kim, E.-Y. Chang, S. Lee, T. Kim, J. Nam, H.-G. Choo, J. Kim, and J. Hahn, “360-degree tabletop electronic holographic display,” Opt. Express 24(22), 24999–25009 (2016). [CrossRef]  

48. S.-F. Lin, H.-K. Cao, and E.-S. Kim, “Single SLM full-color holographic three-dimensional video display based on image and frequency-shift multiplexing,” Opt. Express 27(11), 15926–15942 (2019). [CrossRef]  

Supplementary Material (1)

NameDescription
Visualization 1       Visualization results in different polarization state from a binocular holographic 3D display system

Data availability

The data underlying the results presented in this paper are not publicly available but can be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (8)

Fig. 1.
Fig. 1. Structure diagram of the proposed binocular full-color holographic three-dimensional near eye display system, ACE: angular compensation element, B.S.: beam splitter, SLM: spatial light modulator.
Fig. 2.
Fig. 2. Digital generation process of the binocular hologram, FT: Fourier Transform, IFT: Inverse Fourier Transform, IS: Image Shift-, FS: Frequency Shift, IFS: Image and Frequency Shift.
Fig. 3.
Fig. 3. Color spectrums and reconstructed images distribution caused by color dispersion when the RGB light source illuminated (a) coaxially and (c) with different incident angles by the angular compensation element (ACE); the diffraction angles distribution corresponding to RGB when the light source illuminated (b) coaxially and (d) by ACE.
Fig. 4.
Fig. 4. Optical setup of the binocular full-color display system: (a) photograph of the overall setup, (b) double-hole bandpass filter in the 4-f system, and (c) binocular viewing eyepiece system.
Fig. 5.
Fig. 5. Frequency spectrum comparative results of left and right view holograms with and without frequency shifting process. (Captured by CCD)
Fig. 6.
Fig. 6. Frequency spectrum comparative results of left and right view holograms using color spectrum shifting (CSS) operation or not: the left column is the frequency spectrum results without CSS operation, and the right column is the frequency spectrum results with CSS operation; the first row is the left view results (Captured by CCD), the second row is the right view results (Captured by CCD), and the third row is the binocular view frequency spectrum results (Captured on a white screen by digital camera with lens).
Fig. 7.
Fig. 7. Reconstruction results of the proposed binocular full-color holographic display system from single SLM: (a), (b), (c) are the results captured at the same position by passing through the color spectrums of only left view, only right view and both, respectively; (d)-(h) show the specific frames of the reconstructed Visualization 1 result from the binocular hologram with orthogonal polarization under different polarization state detection. (Captured by digital camera with lens)
Fig. 8.
Fig. 8. Optical 3D binocular reconstruction on the binocular eyepiece: (a) left view reconstruction focused on the front image ‘Cube’ and (b) rear image ‘Soccer’; (c) right view reconstruction focused on the front image ‘Cube’ and (d) rear image ‘Soccer’. (Captured by digital camera with lens)

Tables (1)

Tables Icon

Table 1. Parameters of the frequency shifting amount for left and right view RGB sub-holograms

Equations (3)

Equations on this page are rendered with MathJax. Learn more.

H f s ( f x , f y ) = F { h o r i ( x , y ) exp [ j 2 π ( f a x + f b y ) ] } = H o r i ( f x  +  f a , f y + f b ) ,
H s s ( f x _ a , f y _ b ) = F { h o r i ( x + a , y + b ) } = H o r i ( f x , f y ) exp [ j 2 π ( f s _ a x + f s _ b y ) ] ,
d c s s = f sin α ,
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.