## Abstract

In the calculation of large-scale computer-generated holograms, an approach called “tiling,” which divides the hologram plane into small rectangles, is often employed due to limitations on computational memory. However, the total amount of computational complexity severely increases with the number of divisions. In this paper, we propose an efficient method for calculating tiled large-scale holograms using ray-wavefront conversion. In experiments, the effectiveness of the proposed method was verified by comparing its calculation cost with that using the previous method. Additionally, a hologram of 128K × 128K pixels was calculated and fabricated by a laser-lithography system, and a high-quality 105 mm × 105 mm 3D image including complicated reflection and translucency was optically reconstructed.

© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

## 1. Introduction

Holographic 3D displays satisfy all depth cues in human perception by reconstructing wavefronts from objects. Displaying 3D images at a wide range of depths and high resolution is a significant point of the holographic display technique. There have been some attempts to synthesize holograms computationally (known as computer-generated holography, CGH) for electronic display. However, computing high-quality holograms with a large number of pixels remains a major challenge because simulating the diffraction of the light-waves of such holograms requires a heavy computational load.

Point-source [1–8] and polygon-based methods [9–12], sometimes called physically based methods, are the most widely used frameworks in CGH. According to these methods, objects are decomposed into a set of emissive point-sources or polygons. The wavefront is generated based on the physical model of diffraction, and thus depth cues can be accurately retrieved. Various means of accelerating the calculation have also been proposed, including look-up Tables [1–3], GPU acceleration [4], recurrence relations [5,6], intermediate planes [7], and similarity between polygons [10,11]. Additionally, material appearance has been enhanced in recent studies [8,12]. However, calculating point-source or polygon-based holograms with highly realistic traits—such as complicated reflections, refraction, translucency, and subsurface scattering—remains a major challenge because these methods require an enormous number of points or polygons.

As an alternative approach, the holographic-stereogram (HS)-based method has also been developed. HS-based CGH [13] reproduces 2D images from multiple viewpoints and autostereoscopic 3D display is achieved. Moreover, the light-field (LF) can be reconstructed when the number of viewpoints increases and full-parallax is implemented [14]. In this case, the technique can be called ray-based CGH. Use of light rays enables expression of realistic appearance and occlusion, because advanced techniques of 3D computer graphics (3DCG) or image-based rendering can be directly applied; however, the resolution of reconstructed images is degraded in deep-3D scenes due to the sampling and diffraction of light-rays.

Phase-added stereograms (PAS) [15] are a mixture of ray- and wavefront-based reconstructions. Degradation of image resolution can be prevented even when a 3D image is distant from the hologram plane by adding phase information to each ray. This approach has lately been improved into fast PAS (FPAS) [16] and fully computed HS [17].

Another way to avoid resolution loss in ray-based CGH is to convert the ray information into the wavefront near the object [18–21]. In this approach, a ray-sampling (RS) plane [18,19] is defined near the object and the LF of an object passing through the RS plane is densely sampled. The captured LF is converted into the wavefront via Fourier transformation and the obtained wavefront is numerically propagated to the hologram plane using diffraction theory. Therefore, the method is suitable for displaying deep-3D scenes that represent complicated physical phenomena such as refraction and reflection, because information concerning these properties is already included in captured rays. Additionally, a wide range of depths can be adapted with multiple RS planes and mutual-occlusion culling [19]. The feasibility of reconstructing realistic 3D video has also been demonstrated [20]. Recently, we have proposed an accelerated method, introducing a set of parallel light rays called an orthographic image [21]. Ineffective calculation of propagation can be omitted using this method; thus, the acceleration of the calculation can be achieved without degradation of the image quality.

In the calculation of large-scale (over a giga-pixel) holograms, a technique called “rectangular tiling” [22] is often employed because of limitations on computational memory. In this framework, the hologram plane is divided into small rectangular tiles with the same number of pixels. The wavefront at each hologram tile is computed by superposition of the propagation result from all wave-primitives, which are point-sources, polygons, or object planes. Some researchers have reported efficient methods for calculating divided holograms using the GPU-cluster system [23,24] and decomposition of both the object and hologram planes [25].

Basically, methods using intermediate planes [7] are faster than a simple superposition of point-sources because of the use of fast Fourier transforms (FFTs). However, in terms of tiling calculation, the total computational cost sharply increases with the number of division; thus, the size of a tile is set to be as large as possible. The framework of the calculation using ray-wavefront conversion [18–21] is similar to that of methods using intermediate planes rather than HS-based methods. Therefore the problem of computation with the number of tiles mentioned above emerges in the synthesis of large-scale holograms.

In this paper, we propose a method to effectively calculate large-scale holograms by separation of the input data. At the same time, this method also achieves realistic material reproduction and resolution at great depth, which are advantages of the previous RS-plane method. Our proposed method enables an efficient parallel calculation of large-scale and realistic CGHs. In experiments, we confirmed that tiled holograms can be computed faster using our proposed method than using the previous method. We also calculated and fabricated a high-resolution hologram of 128K × 128K pixels and showed that a realistic 3D image of size 105 mm × 105 mm can be observed over a wide viewing angle of $\pm 23.3$.

## 2. Hologram calculation using orthographic ray-sampling plane

In this section, the CGH calculation using an orthographic ray-sampling (ORS) plane [21] is briefly described in the one-dimensional case. The scheme of this calculation is shown in Fig. 1.

According to the method, an intermediate “ORS” plane is defined. This plane needs to be placed near objects so that rays from the objects are adequately sampled. The LF passing through this plane is captured as orthographic images at different angles. An orthographic image is a set of spatially distributed parallel light rays. The propagation from the ORS plane to the hologram plane is simulated based on diffraction theory, which can be written as a convolution as follows:

*z*, such as Fresnel diffraction or angular-spectrum propagation [26]. We showed that the angular spectrum of the RS plane, ${W}_{\text{RS}}\left({f}_{x}\right)$, can be written as follows [21]:

## 3. Calculation of tiled holograms using an orthographic ray-sampling plane

In the calculation of large-scale holograms, it is difficult to process all wavefront data at once, because a large working memory is required. Thus, it is necessary to partition the object and/or hologram data. A reasonable means of partitioning is to divide the hologram plane into small rectangular tiles [22]. This division enables parallel computation of the hologram plane.

Figure 2 shows the concept of (a) the original tiling and (b) the proposed tiling calculation. In the conventional methods for partitioning object and hologram data, each hologram tile must be calculated as a sum of the propagation results from all wave-primitives, such as point-sources, polygons, or object planes. If the point sources or polygons are divided into subsets, all combinations among an object subset and a hologram tile need to be calculated. In the case of propagation of the wavefront from the object plane to the hologram plane, e.g., the shifted-Fresnel diffraction [22], the object plane is divided into tiles of the same size as those of hologram plane. The wave propagation must be calculated for all combinations of tiles at the object and hologram planes, as shown in Fig. 3.

Let the object and hologram planes be divided into *K* and *N* tiles, respectively. Here, it is assumed that the object plane is *m* times larger than the hologram plane (thus $K=mN$). In this scheme, the wavefront at the *n*-th tile of the hologram can be calculated as follows:

*k*-th tile of the object plane and the transfer function of propagation between the

*k*-th object tile and the

*n*-th hologram tile, respectively. ${w}_{\text{CGH}}\left(x;n\right)$ is calculated as the summation of the propagation result from all tiles of the object plane. As we can see from Eq. (4), propagation must be computed

*K*times to obtain one tile of the hologram, meaning that

*NK*propagations are required for whole hologram. Therefore, the computational cost becomes larger as the number of tiles increases.

In this paper, we propose an efficient method to calculate large-scale tiled holograms by applying the orthographic ray-sampling method. This scheme makes it possible to independently compute each small tile of the hologram plane from a small amount of input data that affects the small tile.

Figure 4 shows the concept of the proposed method. The key idea is the fact that each hologram tile can be computed from only the light rays that are incident upon it from the object plane. A small tile does not require all light ray data, but only a small number of rays. Therefore, a small amount of data concerning a tile of the ORS plane is required for computing the propagation. Note that whole-wavefront data of an object tile is required in the conventional scheme.

The proposed scheme can be described as below using the framework of the orthographic RS method. The contribution from the orthographic image of the angle ${\alpha}_{k}$ to the *n*-th hologram tile can be obtained by the following simple geometric calculation

*n*-th hologram tile considering the diffraction effect. A light-ray expands traveling through the distance

*z*by diffraction; thus ${\text{W}}_{\text{ray}}$ is larger than ${\text{W}}_{\text{tile}}$. When the sampling pitch of the orthographic rays is $\text{\Delta}o$, a single light-ray can be treated as a plane wave of width $\text{\Delta}o$. The width of the plane wave propagating a distance

*z*, ${\delta}_{\text{ray}}$, can be approximated as follows:Here, $\lambda $ is the wavelength. Let the angular resolution of the human eye be ${\text{\theta}}_{\text{eye}}$. Because of the main concept of this paper, we assume that the hologram reconstructs virtual 3D images and the observer looks at the 3D image from an arbitrary point. In the framework of orthographic-ray sampling, $\text{\Delta}o$ is proportional to the propagation distance

*z*as follows:Thus

In the calculation of ${O}_{{\alpha}_{k}}$, the width of the orthographic image is limited by the rectangular function; thus, only light-rays is required in this limitation. Considering that all wavefront data are required in the original scheme explained above, the input data can be reduced to the rate *r* by dividing the hologram plane into *N* tiles:

*r*can be derived as the ratio of ${\text{W}}_{\text{ray}}$ and the area of the object plane, $mN{\text{W}}_{\text{tile}}$, assuming both planes have the same sampling pitch. In the typical case, ${\text{\theta}}_{\text{eye}}=2.9\times {10}^{-4}\left[\text{rad}\right]$, and if the wavelength $\text{\lambda}=633\text{}\left[\text{nm}\right]$,Thus, if the width of the tile is larger than $2.2\times {10}^{-3}\left[\text{m}\right]$, the input data required to compute one tile of the hologram is approximately reduced to 1/

*mN*~2/

*mN*. If the hologram plane has the same area as the object plane,

*m*= 1, the rate is 1/

*N*~2/

*N.*

According to this scheme, the huge wavefront data set can be separated into a small input data set for light rays attached to each tile of the hologram plane. Therefore, the efficient calculation of tiled large-scale holograms can be achieved. Additionally, unlike other ray-based methods, the diffraction of light-rays can be considered; thus, the resolution of the reconstructed image does not degrade, even over a significant distance from the hologram plane.

## 4. Estimation of computational complexity

In this section, let us estimate and compare the computational complexities of the conventional and proposed methods to calculate the tiled hologram. We assume that the object plane is employed to efficiently calculate the propagation of the object wave. The propagation distance is selected so as to satisfy that every point on the object plane affects all the area of the hologram plane.

Let the hologram plane have *X* × *X* pixels. Firstly, when calculating a hologram without using the rectangular-tiling technique shown in Fig. 5(a), the computational complexity can be estimated as follows:

In the conventional framework of rectangular tiling [22], the computational complexity of a tiled hologram can be approximately estimated as follows. The schematic diagram is shown in Fig. 5(b). The hologram plane is divided into *N* × *N* tiles of *X/N* × *X/N* pixels. The object plane has *mX* × *mX* pixels and is divided into *mN* × *mN* tiles of *X/N* × *X/N* pixels. Note that it is assumed that each tile is computed independently, meaning that the result of the calculation is not shared between each tile. The entire computational complexity ${C}^{\text{conv}}$ measured by complex multiplication can be written as:

*H*. ${C}_{\text{HOLO}}$ is used to compute the inverse Fourier transform to obtain the wavefront at one hologram tile ${w}_{\text{h}}$. When the object and hologram planes are tiled, as shown in Fig. 5(b), each term of Eq. (15) is given by:

*C*, is derived as:

We also estimate the computational complexity of the proposed method. We define the entire computational cost, ${C}^{\text{ORS}}$, in the same manner as Eq. (15) as follows

Secondly, the cost required for multiplying by the transfer function, ${C}_{H}^{\text{ORS}}$, is also written as:

Finally, the cost for an inverse-Fourier transform to obtain the wavefront at the hologram plane, ${C}_{\text{HOLO}}^{\text{ORS}}$, can be written as:

Substituting the above equations into Eq. (20), ${C}^{\text{ORS}}$ can be written as:

*N*.

In practice, the entire computational cost slightly increases with the number of tiles, *N*, because of the overhead cost that proportionally increases with the number of FFTs. However, the overhead cost is sufficiently small compared with the overall complexity. The diffraction effect of rays could also be a problem if tiles are too small. The width of each tile, ${\text{W}}_{\text{tile}}$, decreases with *N*. On the other hand, the width of a ray considering the diffraction effect, ${\delta}_{\text{ray}}$, does not change with *N*. When ${\text{W}}_{\text{tile}}$ is much more smaller than ${\delta}_{\text{ray}}$, the overall cost, ${C}^{\text{ORS}}$, becomes heavier than the estimation. However, ${\text{W}}_{\text{tile}}$ is larger than ${\delta}_{\text{ray}}$ in most cases; thus, the proposed method works effectively.

Figure 6 shows the estimated computational complexity calculated from the equations above. The resolution of the hologram is set to 16K × 16K, and we assumed that ${C}_{\text{W}}=0$ and $m=1$. In the next section, we examined this estimation experimentally.

## 5. Experimental results

5. 1. Measurement of the computational time

In the first experiment, we confirmed the efficiency of the proposed method by measuring the computational time for synthesizing CGHs. We calculated holograms of 16K × 16K pixels as shown in Fig. 7; the number of tiles with the conventional RS-plane method [18] was changed to that with the proposed method. The wavelength, $\text{\lambda}$, was set to 633 nm, and the sampling pitch, $\text{\Delta}x$, was 1.0µm. Each orthographic image has 512 × 512 pixels, and they were rendered from 16 × 16 different angles using 3DCG software Blender [28]. The propagation distance was 60 mm.

Figure 8 shows the experimentally measured calculation time including time for rendering when the number of divisions in one direction, N, changes. 3D images of the simulated reconstruction are shown in Fig. 9. Note that we used an Intel Core i7 4790K CPU (4.0GHz) and a 32GB RAM for rendering orthographic images and calculating holograms. The overall behavior of the measured calculation time agreed with the theoretical estimation of the computational cost, as shown in Fig. 6. The calculation time remains almost constant at 12 minutes, regardless of the number of tiles using the proposed method, as opposed to the radical increase that occurs with the conventional method. From this result, we conclude that the proposed method is effective for calculation of large-tiled holograms. Moreover, the result shows that the hologram plane can be easily divided and computed independently without additional computing cost; thus, parallel computing of holograms will greatly improve the calculation time.

#### 5. 2. Calculation of over-10-Giga-pixel CGH

In the second experiment, we calculated a CGH with 128K × 128K pixels and printed the synthesized CGH as a binary mask. For the CGH calculation, we created a 3DCG scene of “Whiskey and Glasses” containing realistic material features such as reflection, refraction, absorption, and anisotropic reflection, as rendered by the physically based “Cycles” rendering engine in Blender. Each orthographic image has 1,024 × 1,024 pixels rendered in 64 × 64 different angles. The wavelength, $\text{\lambda}$, was set to 633 nm, and the sampling pitch, $\text{\Delta}x$, was 0.8µm. The maximum diffraction angle, ${\theta}_{\text{max}}$, is given by:

*x*and

*y*axes, respectively. Some examples of rendered orthographic images are shown in Fig. 10. The required rendering time was 23 hours using the computing environment mentioned above and an additional graphical processing unit (GeForce GTX 980).

The geometrical setup of the computed CGH is shown in Fig. 11. The ORS plane was placed 180 mm behind the hologram one. The size of the hologram and ORS planes was 105 mm × 105 mm. The hologram plane was divided into 8 × 8 tiles, each having 16K × 16K pixels. Without division of the CGH plane, we need at least 260 GB of RAM. After tiling, the required memory became 4 GB. The computed wavefront interfered with the reference wave, a spherical wave of $x=0\text{mm},\text{}y=140\text{mm},\text{z}=-800\text{mm}$, and the fringe pattern was obtained. The computing time needed to synthesize this hologram was 13 hours using the same PC in the first experiment without a GPU.

The calculated fringe pattern was binarized and printed as an amplitude mask by the laser-lithography system (Heidelberg Instruments DWL 66 + ) of “Kan-Dai Digital Holo Studio” [29]. At this time, the interference-fringe data is converted from a binary bit map image to CAD data and fabricated on a glass substrate.

By illuminating the CGH with a spherical wave of He-Ne laser light, the 3D image can be observed with the naked eye as a virtual image at a distance of 180 mm from the hologram plane. The optical reconstructions are shown in Fig. 12 (see Visualization 1). It can be seen that the label of the whiskey bottle looked distorted through the wine glass by the translucency and refraction effects in Fig. 12(d). Even for similar effects such as refraction, a different translucency can be observed from the absorption and scattering in ice cubes. Additionally, the ice bucket can be recognized as an anisotropic-reflecting-metal object.

## 6. Conclusion

In this paper, we propose an efficient method to calculate large-scale and realistic CGHs using separation of the input data by the ORS plane method. We theoretically and experimentally verified the effectiveness of the proposed framework in terms computational cost and also verified that the synthesized CGH can reproduce 105mm × 105mm 3D images at a viewing angle $\pm 23.3$ degrees from the CGH fabricated by the laser-lithography system. The optical reconstruction provides high resolution, even in the deep scene and highly realistic material appearance including translucency, refraction, and anisotropic reflection. For future improvements, parallel computing using the GPU will greatly accelerate the calculation time. Additionally, optical reconstruction in full color must be realized.

## Funding

Japan Society for the Promotion of Science (JSPS) (KAKWNHI 15K04691); MEXT strategic research foundation at private universities (2013-2017).

## Acknowledgment

A part of this work is supported by Kan-Dai Digital Holo-Studio in Kansai University.

## References and links

**1. **M. Lucente, “Interactive computation of holograms using a look-up table,” J. Electron. Imaging **2**(1), 28–34 (1993). [CrossRef]

**2. **S.-C. Kim and E.-S. Kim, “Effective generation of digital holograms of three-dimensional objects using a novel look-up table method,” Appl. Opt. **47**(19), D55–D62 (2008). [CrossRef] [PubMed]

**3. **Y. Pan, X. Xu, S. Solanki, X. Liang, R. B. A. Tanjung, C. Tan, and T.-C. Chong, “Fast CGH computation using S-LUT on GPU,” Opt. Express **17**(21), 18543–18555 (2009). [CrossRef] [PubMed]

**4. **T. Shimobaba, T. Ito, N. Masuda, Y. Ichihashi, and N. Takada, “Fast calculation of computer-generated-hologram on AMD HD5000 series GPU and OpenCL,” Opt. Express **18**(10), 9955–9960 (2010). [CrossRef] [PubMed]

**5. **H. Yoshikawa, “Fast computation of Fresnel holograms employing difference,” Opt. Rev. **8**(5), 331–335 (2001). [CrossRef]

**6. **T. Nishitsuji, T. Shimobaba, T. Kakue, and T. Ito, “Fast calculation of computer-generated hologram using run-length encoding based recurrence relation,” Opt. Express **23**(8), 9852–9857 (2015). [CrossRef] [PubMed]

**7. **T. Shimobaba, N. Masuda, and T. Ito, “Simple and fast calculation algorithm for computer-generated hologram with wavefront recording plane,” Opt. Lett. **34**(20), 3133–3135 (2009). [CrossRef] [PubMed]

**8. **T. Ichikawa, T. Yoneyama, and Y. Sakamoto, “CGH calculation with the ray tracing method for the Fourier transform optical system,” Opt. Express **21**(26), 32019–32031 (2013). [CrossRef] [PubMed]

**9. **D. Leseberg and C. Frère, “Computer-generated holograms of 3-D objects composed of tilted planar segments,” Appl. Opt. **27**(14), 3020–3024 (1988). [CrossRef] [PubMed]

**10. **H. Kim, J. Hahn, and B. Lee, “Mathematical modeling of triangle-mesh-modeled three-dimensional surface objects for digital holography,” Appl. Opt. **47**(19), D117–D127 (2008). [CrossRef] [PubMed]

**11. **Y.-M. Ji, H. Yeom, and J.-H. Park, “Efficient texture mapping by adaptive mesh division in mesh-based computer generated hologram,” Opt. Express **24**(24), 28154–28169 (2016). [CrossRef] [PubMed]

**12. **H. Nishi and K. Matsushima, “Rendering of specular curved objects in polygon-based computer holography,” Appl. Opt. **56**(13), F37–F44 (2017). [CrossRef] [PubMed]

**13. **T. Yatagai, “Stereoscopic approach to 3-D display using computer-generated holograms,” Appl. Opt. **15**(11), 2722–2729 (1976). [CrossRef] [PubMed]

**14. **M. Yamaguchi, “Light-field and holographic three-dimensional displays,” J. Opt. Soc. Am. A **33**(12), 2348–2364 (2016). [CrossRef] [PubMed]

**15. **M. Yamaguchi, H. Hoshino, T. Honda, and N. Ohyama, “Phase-added stereogram: calculation of hologram using computer graphics technique,” Proc. SPIE **1914**, 25–31 (1993). [CrossRef]

**16. **H. Kang, E. Stoykova, and H. Yoshikawa, “Fast phase-added stereogram algorithm for generation of photorealistic 3D content,” Appl. Opt. **55**(3), A135–A143 (2016). [CrossRef] [PubMed]

**17. **H. Zhang, Y. Zhao, L. Cao, and G. Jin, “Fully computed holographic stereogram based algorithm for computer-generated holograms with accurate depth cues,” Opt. Express **23**(4), 3901–3913 (2015). [CrossRef] [PubMed]

**18. **K. Wakunami and M. Yamaguchi, “Calculation for computer generated hologram using ray-sampling plane,” Opt. Express **19**(10), 9086–9101 (2011). [CrossRef] [PubMed]

**19. **K. Wakunami, H. Yamashita, and M. Yamaguchi, “Occlusion culling for computer generated hologram based on ray-wavefront conversion,” Opt. Express **21**(19), 21811–21822 (2013). [CrossRef] [PubMed]

**20. **H. Sasaki, K. Yamamoto, K. Wakunami, Y. Ichihashi, R. Oi, and T. Senoh, “Large size three-dimensional video by electronic holography using multiple spatial light modulators,” Sci. Rep. **4**(1), 6177 (2015). [CrossRef] [PubMed]

**21. **S. Igarashi, T. Nakamura, and M. Yamaguchi, “Fast method of calculating a photorealistic hologram based on orthographic ray-wavefront conversion,” Opt. Lett. **41**(7), 1396–1399 (2016). [CrossRef] [PubMed]

**22. **R. P. Muffoletto, J. M. Tyler, and J. E. Tohline, “Shifted Fresnel diffraction for computational holography,” Opt. Express **15**(9), 5631–5640 (2007). [CrossRef] [PubMed]

**23. **N. Takada, T. Shimobaba, H. Nakayama, A. Shiraki, N. Okada, M. Oikawa, N. Masuda, and T. Ito, “Fast high-resolution computer-generated hologram computation using multiple graphics processing unit cluster system,” Appl. Opt. **51**(30), 7303–7307 (2012). [CrossRef] [PubMed]

**24. **Y. Pan, X. Xu, and X. Liang, “Fast distributed large-pixel-count hologram computation using a GPU cluster,” Appl. Opt. **52**(26), 6562–6571 (2013). [CrossRef] [PubMed]

**25. **P. Lobaz, “Reference calculation of light propagation between parallel planes of different sizes and sampling rates,” Opt. Express **19**(1), 32–39 (2011). [CrossRef] [PubMed]

**26. **J. W. Goodman, *Introduction to Fourier Optics* (McGraw-Hill, 1996).

**27. **K. Matsushima, “Shifted angular spectrum method for off-axis numerical propagation,” Opt. Express **18**(17), 18453–18463 (2010). [CrossRef] [PubMed]

**28. ** Blender Foundation, “Blender,” https://www.blender.org.

**29. ** Kansai University, “Kan-Dai Digital Holo Studio,” http://holography.ordist.kansai-u.ac.jp/digitalholostudio.