## Abstract

We propose a full-color complex holographic display system design comprised of three R/G/B amplitude-only spatial light modulators and an achromatic Fourier filter. A key feature of the design is a single achromatic Fourier bandpass filter for robust axial R/G/B color matching, whereby the R/G/B holographic image light fields can be three-dimensionally aligned. The synthesis algorithm producing the full-color computer-generated holograms for this system is described and a full-color optical reconstruction of the designed holographic three-dimensional images is experimentally demonstrated.

© 2017 Optical Society of America

## 1. Introduction

Nowadays, full-color holographic three-dimensional (3D) display systems are being actively researched [1–3]. The main goal of the research is to achieve full-color high-resolution holographic 3D displays featuring wide-viewing angles [4] and extended viewing windows [5] from which observers can enjoy comfortable holographic 3D images. However, this is beyond the ability of current display technology and requires technological breakthroughs to solve many challenging issues. A particular challenge is creating a display device architecture supporting extremely small pixels of around 1μm pitch [6, 7] and complex light modulation [8]. The real time generation of large scale computer-generated holograms (CGH) requires acceleration and parallelism [9–13], but the understanding of the related information theory and algorithms is still in its infancy. In practice, the research on the system level configurations of holographic 3D display systems has preceded the research on display device architectures. Various smart system configurations have been proposed to resolve the limitations of holographic 3D display systems, but performance factors, such as viewing-angles, are still limited [14–16]. Recently, with the increasing public interest in virtual reality (VR) and augmented reality (AR) wearable displays, VR and AR display applications based on holographic technology are increasingly desired. Personal wearable VR and AR smart displays are less affected by limited viewing angles and viewing-window size [16, 17].

When constructing a full-color holographic 3D display system, full-color image generation is a fundamental issue. Normally, we need to combine three separate R/G/B image light fields without three-dimensional alignment mismatches. In terms of achromatic alignment, although the use of a single spatial light modulator (SLM) with R/G/B color pixels is a perfect solution, it inevitably reduces the bandwidth of the holographic image by one third. As an alternative, the optical combination of separate R/G/B beam lines modulated by three separate mono SLMs, an approach common in commercial projection display systems, is preferred since the use of three SLMs preserves the bandwidth and resolution of the full-color holographic images. However, for this approach, volumetric matching of the R/G/B complex holographic light fields is a significant problem. The R/G/B components of the holographic image light field should perfectly overlap not only in the transverse plane but also in the axial direction. However, the axial-direction alignment of R/G/B components is a particularly difficult problem to solve.

This paper proposes a holographic 3D display system configuration comprised of three amplitude-only SLMs for R/G/B components and an achromatic Fourier band-pass filter providing robust axial color alignment capabilities, and develops an optimal full-color CGH synthesis algorithm for this system. In the proposed system, the key is the axial alignment of the R/G/B carrier waves, then, for that, the use of a single achromatic Fourier filter configuration is essential. Section 2 presents the wave optic modeling to support a full-color holographic 3D projection display and full-color CGH synthesis algorithm. Section 3 presents the design of the full-color holographic 3D display projector, its experimental implementation, and the experimental results, which verify the proposed design. Finally, section 4 gives the concluding remarks.

## 2. Wave optic model of the full-color holographic projection display system and the design of computer-generated holograms

The wave optic model of the full-color holographic projection display system is described and, based on that model, the full-color CGH synthesis method is explained. The core of the system is the single-sideband 4-ƒ transformation of amplitude SLM to complex SLM [16, 18]. Since the 4-ƒ transformation sets up the virtual complex plane that can display complex CGH patterns, in CGH design, the complex CGH should be synthesized for the virtual complex SLM plane of the system. In Fig. 1, the complex CGH plane and the retina plane of the observer’s eye are denoted by the planes ${x}_{1}{y}_{1}$ and ${x}_{2}{y}_{2}$, respectively. The eye lens plane ($uv$ plane) is placed in the intermediate position. The mathematical relationship between the optical field at the virtual complex SLM, $P\left({x}_{1},{y}_{1}\right)$, and that at the retinal space of the observer, $Q\left({x}_{2},{y}_{2}\right)$ is described by the forward and inverse cascaded Fresnel transforms, CdFr and ICdFr, as given by [19]

In this projection type holographic display, the observation distance is designed to be equal to the focal length of the projection lens as illustrated in Fig. 1. The cascaded Fresnel transform $Q\left({x}_{2},{y}_{2}\right)=CdFr\left\{P\left({x}_{1},{y}_{1}\right)\right\}$ is composed of two elementary transforms $Fr{T}_{1}$ and $Fr{T}_{2}$ defined, respectively, as

and,The first integral transform, $W\left(u,v\right)=Fr{T}_{1}\left\{P\left({x}_{1},{y}_{1}\right)\right\}$, is represented by*uv*plane) and $k$ is the wave number $2\pi /\lambda $. The second integral transform, $Q\left({x}_{2},{y}_{2}\right)=Fr{T}_{2}\left\{t\left(u,v\right)W\left(u,v\right)\right\}$ is represented by

*x*- and

*y*-axis. The $\Delta {x}_{1}$($\Delta {y}_{1}$) and $\Delta {x}_{2}$ ($\Delta {y}_{2}$), pixel sizes of the input and output planes are proportional to each other and independent of the wavelength. That means that in this scheme the computational grid of the retinal plane is independent of color, i.e., wavelength, and it is this independence that allows, the proposed scheme, to realize the color matching of the R/G/B CGH components. Therefore, the full-color CGH image can be observed when the R/G/B holographic image light components,$Q\left({x}_{2},{y}_{2};{\lambda}_{red}\right)$,$Q\left({x}_{2},{y}_{2};{\lambda}_{green}\right)$, and $Q\left({x}_{2},{y}_{2};{\lambda}_{blue}\right)$ are imaged in the retina plane. In the CGH projection display system, the full color CGH is synthesized by ICdFr. The target image light field of the 3D object in the retina space is projected to the complex light field on the retina plane, $Q\left({x}_{2},{y}_{2};\lambda \right)$. $Q\left({x}_{2},{y}_{2};\lambda \right)$ can be calculated using the point cloud CGH synthesis algorithm [20], depth map CGH synthesis algorithm [12, 21] or the polygon CGH synthesis algorithm [19, 22, 23]. It is important that the 3D object should be geometrically transformed into the retina space through the geometric imaging transform [19].

The key idea of the CGH synthesis at the virtual SLM plane ${x}_{1}{y}_{1}$ is that the color matched holographic field, $Q\left({x}_{2},{y}_{2};\lambda \right)$, is initially set up on the retina plane ${x}_{2}{y}_{2}$ and the optical field at the retina space is transferred to the CGH plane in the object space through ICdFr as

## 3. Color-CGH alignment in the holographic projection display system with an achromatic Fourier filter

The holographic 3D projection display system needs to combine the R/G/B beam lines and achieves this using an X-cube or a trichroic prism as illustrated in Fig. 2(a). The previous section has already emphasized the importance of robustly implementing the color-matched system such that the R/G/B beam lines are in near perfect alignment. In order to obtain a color-matched 3D holographic image, the R/G/B beam lines must be precisely aligned not only in the collinear axial-direction but also in the transverse direction. Axial color matching is essential for creating full-color 3D images featuring realistic natural accommodation effects.

Figures 2(a) and 2(b) compare the designs of the conventional projection system and the proposed system, respectively, in terms of R/G/B alignment and achromaticity. In the conventional configuration shown in Fig. 2(a), the R/G/B beam lines are separated and independently aligned. Each R/G/B beam line has a single-sideband filter for representing effective complex holograms on the virtual SLM plane as shown in Fig. 2(c). In the conventional setup, the DC signal is positioned at the center of the Fourier plane and the signal and conjugate terms are separated in the upper and lower parts of the DC term. The single side-band filter aperture allows each signal terms that contains complex holographic information to bypass the aperture and be transformed into the complex hologram on the virtual SLM plane. The R/G/B beams lines are combined by the X-cube, but it should be noted that, from the observer’s point of view, the absolute positions of the R/G/B signal terms in the Fourier filtering plane are separate due to color dispersion. Assuming that the 4-*f* Fourier filter is the same for all R/G/B beam lines, we can represent the R/G/B filtering operations as in the combined schematic shown in Fig. 2. (c). Figure 2(e) shows the optical field distributions in the Fourier domain. The positions of the R/G/B hologram signals and their conjugate terms are dispersed according to their relative wavelengths. The DC signal is white and focused into a point at the center of the filter domain, which is on the optical axis. The R signal is furthest from the center (DC), and the B signal is nearest. In the conventional setup, the single sideband filtering of each color components is conducted through the apertures at the different positions in the Fourier filtering domain, and this approach inevitably leads to axial misalignment in the propagation direction of the R/G/B signal beams after the optical Fourier transform. Here, the filter through which the hologram signal passes is represented by the circular aperture for each color.

In order to resolve the difficulties associated with axial color matching, a novel projection structure is proposed, as shown in Fig. 2(b) that allows the R/G/B carrier waves to be independently adjusted. A new system configuration is proposed that facilitates beam line adjustment. As shown in Fig. 2(b), the R/G/B SLMs are filtered by a single achromatic Fourier filter. A single achromatic adjusted 4-ƒ system is employed to filter the signal terms of the R/G/B beam lines. A key feature of the proposed system is the manipulation of the R/G/B carrier waves to align the R/G/B component signals into the center of the signal band-pass filter on the optical axis. As illustrated in Fig. 2(d), the effect of this filter is straightforward axial color matching alignment. The R/G/B panels virtually overlap at the entrance plane of each 4-ƒ system. Figure 2(f) shows the signal and conjugate distributions in the Fourier filter domain. Unlike the conventional system shown in Fig. 2(e), the aperture center of the Fourier filter is tuned to the optic axis. This is a design feature which makes the system less sensitive to geometric and lens aberration. Regarding the aberration, the alignment of the signal beamlines along the optical axis is superior to that of the conventional setup in terms of aberration tolerance. If the signal beams are distant from the center of lens, the holographic image is relatively more vulnerable to the lens aberration than the beams are nearly aligned on the optical axis as shown in Fig. 2(d).

The numerically simulated observation results of the conventional and proposed full-color CGH projection systems are presented in Fig. 3. The CGH observation simulation images shown in Figs. 3(a)-3(c), and created by the conventional system clearly show a color-matched image at ${d}_{m}=0$, but color-dispersion-induced image quality deterioration at all the out of focus positions, ${d}_{m}=80\text{mm}$ and ${d}_{m}=160\text{mm}$. Figures 3(d)-3(f) show the numerically simulated observation results obtained by the proposed system at the focal planes, ${d}_{m}=0$, ${d}_{m}=80\text{mm}$, and ${d}_{m}=160\text{mm}$. The simulation results show a natural accommodation effect such that the observation color images at the different focal planes are all matched and without color dispersion.

In order to verify the proposed idea, we implemented the proposed design shown in Fig. 2(b) and conducted a test experiment. Figure 4(a) shows the prototype implementation of the proposed design, which is composed of three amplitude-type SLMs (Epson L3C07U-85G10) of 8.5μm pixel pitch and 1920 × 1080 resolution, and 2-inch lenses and mirrors. Two chromatic lenses of focal lengths 7.5cm and 40cm were used as a Fourier transfer lens and a projection lens, respectively. After collimation through the projection lens, the complex SLM is observed by a CMOS camera. Since the proposed system needs less optical components than the conventional color holographic system containing three separate 4-ƒ filters, it is more efficient, both spatially and economically. Figures 4(b)-4(e) compare the experimental results of the full-color CGH reconstruction with the results of the numerical simulation. Figures 4(d) and 4(e), the optical reconstruction results of the full-color polygon CGH demonstrate the accommodation effect and confirm not only the axial but also the transverse color matching. The experimental results also agree quite well with the simulation results in terms of the accommodation effect and axial/transverse color matching. In the experiment, the calibration of R/G/B color intensity was technically important. Accordingly, the optical powers of the R/G/B laser beams were carefully calibrated so as to be at comparable levels. The proposed method allows the easy color alignment and display of high-resolution photorealistic full-color CGHs. The vertical dark line seen in the center of the experimental results is caused by the imperfect joining of two prisms in the X-cube, and the artifact can be suppressed by the use of partial coherent light emitting diode (LED) light sources.

## 4. Conclusion

We have proposed a novel full color holographic projection system featuring an achromatic Fourier filter and analyze the wave optic modeling for displaying the color CGHs. The color CGH is obtained by incoherently combining the monochromatic CGHs of the respective R/G/B wavelengths. The system configuration was explained and analyzed with regard to the critical hologram image alignment issue. Recognizing that color hologram matching is difficult for system configurations in which the DC beam line goes straight along the optical axis, we overcome the issue by proposing a system configuration in which the image signal is overlaid on the optical axis. This system configuration has the additional benefit of increasing spatial efficiency. We believe that the proposed system can be an essential element in the development of commercial projective full-color holographic displays and full-color holographic head mounted displays for VR and AR applications.

## Funding

The Cross-Ministry Giga KOREA Project’ grant from the Ministry of Science, ICT and Future Planning, Korea; Samsung Display Co. Ltd.

## Acknowledgments

Jinyoung Roh^{1} and Kwangsoo Kim^{2} are co-first authors, contributed equally to this paper. Hwi Kim* and Joonku Hahn** are the corresponding author and the co-corresponding author, respectively.

## References and links

**1. **L. Onural, F. Yaras, and H. Kang, “Digital holographic three-dimensional video displays,” Proc. IEEE **99**(4), 576–589 (2011). [CrossRef]

**2. **B. Lee, “Three-dimensional displays, past and present,” Phys. Today **66**(4), 36–41 (2013). [CrossRef]

**3. **J. Hong, Y. Kim, H. J. Choi, J. Hahn, J.-H. Park, H. Kim, S.-W. Min, N. Chen, and B. Lee, “Three-dimensional display technologies of recent interest: principles, status, and issues [Invited],” Appl. Opt. **50**(34), H87–H115 (2011). [CrossRef] [PubMed]

**4. **J. Hahn, H. Kim, Y. Lim, G. Park, and B. Lee, “Wide viewing angle dynamic holographic stereogram with a curved array of spatial light modulators,” Opt. Express **16**(16), 12372–12386 (2008). [CrossRef] [PubMed]

**5. **S. Kim, J. Choi, J. Hahn, and H. Kim, “Viewing-window extension of holographic display using high-order diffraction,” in Proceedings of Digital Holography and Three-dimensional Imaging Conference 2017 (Optical Society of America, 2017), Th3A.6.

**6. **R. Stahl, V. Rochus, X. Rottenberg, S. Cosemans, L. Haspeslagh, S. Severi, G. V. Plas, G. Lafruit, and S. Donnay, “Modulator sub-wavelength diffractive light modulator for high definition holographic displays,” J. Phys. Conf. Ser. **415**, 012057 (2013), doi:. [CrossRef]

**7. **K.-I. Aoshima, N. Funabashi, K. Machida, Y. Miyamoto, K. Kuga, T. Ishibashi, N. Shimidzu, and F. Sato, “Submicron magneto-optical spatial light modulation device for holographic displays driven by spin-polarized electrons,” J. Displ. Technol. **6**(9), 374–380 (2010). [CrossRef]

**8. **S. Reichelt, R. Häussler, G. Fütterer, N. Leister, H. Kato, N. Usukura, and Y. Kanbayashi, “Full-range, complex spatial light modulator for real-time holography,” Opt. Lett. **37**(11), 1955–1957 (2012). [CrossRef] [PubMed]

**9. **H. Niwase, N. Takada, H. Araki, Y. Maeda, M. Fujiwara, H. Nakayama, T. Kakue, T. Shimobaba, and T. Ito, “Real-time electroholography using a multiple-graphics processing unit cluster system with a single spatial light modulator and the InfiniBand network,” Opt. Eng. **55**(9), 093108 (2016). [CrossRef]

**10. **H. Araki, N. Takada, H. Niwase, S. Ikawa, M. Fujiwara, H. Nakayama, T. Kakue, T. Shimobaba, and T. Ito, “Real-time time-division color electroholography using a single GPU and a USB module for synchronizing reference light,” Appl. Opt. **54**(34), 10029–10034 (2015). [CrossRef] [PubMed]

**11. **H. Niwase, N. Takada, H. Araki, H. Nakayama, A. Sugiyama, T. Kakue, T. Shimobaba, and T. Ito, “Real-time spatiotemporal division multiplexing electroholography with a single graphics processing unit utilizing movie features,” Opt. Express **22**(23), 28052–28057 (2014). [CrossRef] [PubMed]

**12. **N. Takada, T. Shimobaba, H. Nakayama, A. Shiraki, N. Okada, M. Oikawa, N. Masuda, and T. Ito, “Fast high-resolution computer-generated hologram computation using multiple graphics processing unit cluster system,” Appl. Opt. **51**(30), 7303–7307 (2012). [CrossRef] [PubMed]

**13. **D. Im, J. Cho, J. Hahn, B. Lee, and H. Kim, “Accelerated synthesis algorithm of polygon computer-generated holograms,” Opt. Express **23**(3), 2863–2871 (2015). [CrossRef] [PubMed]

**14. **H. Sasaki, K. Yamamoto, K. Wakunami, Y. Ichihashi, R. Oi, and T. Senoh, “Large size three-dimensional video by electronic holography using multiple spatial light modulators,” Sci. Rep. **4**(1), 6177 (2015), doi:. [CrossRef] [PubMed]

**15. **T. Ichikawa, K. Yamaguchi, and Y. Sakamoto, “Realistic expression for full-parallax computer-generated holograms with the ray-tracing method,” Appl. Opt. **52**(1), A201–A209 (2013). [CrossRef] [PubMed]

**16. **E. Moon, M. Kim, J. Roh, H. Kim, and J. Hahn, “Holographic head-mounted display with RGB light emitting diode light source,” Opt. Express **22**(6), 6526–6534 (2014). [CrossRef] [PubMed]

**17. **A. Maimone, A. Georgiou, and J. S. Kollin, “Holographic near-eye displays for virtual and augmented reality,” ACM Trans. Graph. **36**(85), 1–16 (2017).

**18. **H. Kim, C.-Y. Hwang, K.-S. Kim, J. Roh, W. Moon, S. Kim, B.-R. Lee, S. Oh, and J. Hahn, “Anamorphic optical transformation of an amplitude spatial light modulator to a complex spatial light modulator with square pixels [invited],” Appl. Opt. **53**(27), G139–G146 (2014). [CrossRef] [PubMed]

**19. **D. Im, E. Moon, Y. Park, D. Lee, J. Hahn, and H. Kim, “Phase-regularized polygon computer-generated holograms,” Opt. Lett. **39**(12), 3642–3645 (2014). [CrossRef] [PubMed]

**20. **S.-C. Kim and E.-S. Kim, “Effective generation of digital holograms of three-dimensional objects using a novel look-up table method,” Appl. Opt. **47**(19), D55–D62 (2008). [CrossRef] [PubMed]

**21. **J.-S. Chen and D. P. Chu, “Improved layer-based method for rapid hologram generation and real-time interactive holographic display applications,” Opt. Express **23**(14), 18143–18155 (2015). [CrossRef] [PubMed]

**22. **K. Matsushima, “Exact hidden-surface removal in digitally synthetic full-parallax holograms,” SPIE proc. Practical Holography XIX#5742, 25–32 (2005). [CrossRef]

**23. **K. Matsushima, M. Nakamura, and S. Nakahara, “Silhouette method for hidden surface removal in computer holography and its acceleration using the switch-back technique,” Opt. Express **22**(20), 24450–24465 (2014). [CrossRef] [PubMed]