In the calculation of large-scale computer-generated holograms, an approach called “tiling,” which divides the hologram plane into small rectangles, is often employed due to limitations on computational memory. However, the total amount of computational complexity severely increases with the number of divisions. In this paper, we propose an efficient method for calculating tiled large-scale holograms using ray-wavefront conversion. In experiments, the effectiveness of the proposed method was verified by comparing its calculation cost with that using the previous method. Additionally, a hologram of 128K × 128K pixels was calculated and fabricated by a laser-lithography system, and a high-quality 105 mm × 105 mm 3D image including complicated reflection and translucency was optically reconstructed.
© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
Holographic 3D displays satisfy all depth cues in human perception by reconstructing wavefronts from objects. Displaying 3D images at a wide range of depths and high resolution is a significant point of the holographic display technique. There have been some attempts to synthesize holograms computationally (known as computer-generated holography, CGH) for electronic display. However, computing high-quality holograms with a large number of pixels remains a major challenge because simulating the diffraction of the light-waves of such holograms requires a heavy computational load.
Point-source [1–8] and polygon-based methods [9–12], sometimes called physically based methods, are the most widely used frameworks in CGH. According to these methods, objects are decomposed into a set of emissive point-sources or polygons. The wavefront is generated based on the physical model of diffraction, and thus depth cues can be accurately retrieved. Various means of accelerating the calculation have also been proposed, including look-up Tables [1–3], GPU acceleration , recurrence relations [5,6], intermediate planes , and similarity between polygons [10,11]. Additionally, material appearance has been enhanced in recent studies [8,12]. However, calculating point-source or polygon-based holograms with highly realistic traits—such as complicated reflections, refraction, translucency, and subsurface scattering—remains a major challenge because these methods require an enormous number of points or polygons.
As an alternative approach, the holographic-stereogram (HS)-based method has also been developed. HS-based CGH  reproduces 2D images from multiple viewpoints and autostereoscopic 3D display is achieved. Moreover, the light-field (LF) can be reconstructed when the number of viewpoints increases and full-parallax is implemented . In this case, the technique can be called ray-based CGH. Use of light rays enables expression of realistic appearance and occlusion, because advanced techniques of 3D computer graphics (3DCG) or image-based rendering can be directly applied; however, the resolution of reconstructed images is degraded in deep-3D scenes due to the sampling and diffraction of light-rays.
Phase-added stereograms (PAS)  are a mixture of ray- and wavefront-based reconstructions. Degradation of image resolution can be prevented even when a 3D image is distant from the hologram plane by adding phase information to each ray. This approach has lately been improved into fast PAS (FPAS)  and fully computed HS .
Another way to avoid resolution loss in ray-based CGH is to convert the ray information into the wavefront near the object [18–21]. In this approach, a ray-sampling (RS) plane [18,19] is defined near the object and the LF of an object passing through the RS plane is densely sampled. The captured LF is converted into the wavefront via Fourier transformation and the obtained wavefront is numerically propagated to the hologram plane using diffraction theory. Therefore, the method is suitable for displaying deep-3D scenes that represent complicated physical phenomena such as refraction and reflection, because information concerning these properties is already included in captured rays. Additionally, a wide range of depths can be adapted with multiple RS planes and mutual-occlusion culling . The feasibility of reconstructing realistic 3D video has also been demonstrated . Recently, we have proposed an accelerated method, introducing a set of parallel light rays called an orthographic image . Ineffective calculation of propagation can be omitted using this method; thus, the acceleration of the calculation can be achieved without degradation of the image quality.
In the calculation of large-scale (over a giga-pixel) holograms, a technique called “rectangular tiling”  is often employed because of limitations on computational memory. In this framework, the hologram plane is divided into small rectangular tiles with the same number of pixels. The wavefront at each hologram tile is computed by superposition of the propagation result from all wave-primitives, which are point-sources, polygons, or object planes. Some researchers have reported efficient methods for calculating divided holograms using the GPU-cluster system [23,24] and decomposition of both the object and hologram planes .
Basically, methods using intermediate planes  are faster than a simple superposition of point-sources because of the use of fast Fourier transforms (FFTs). However, in terms of tiling calculation, the total computational cost sharply increases with the number of division; thus, the size of a tile is set to be as large as possible. The framework of the calculation using ray-wavefront conversion [18–21] is similar to that of methods using intermediate planes rather than HS-based methods. Therefore the problem of computation with the number of tiles mentioned above emerges in the synthesis of large-scale holograms.
In this paper, we propose a method to effectively calculate large-scale holograms by separation of the input data. At the same time, this method also achieves realistic material reproduction and resolution at great depth, which are advantages of the previous RS-plane method. Our proposed method enables an efficient parallel calculation of large-scale and realistic CGHs. In experiments, we confirmed that tiled holograms can be computed faster using our proposed method than using the previous method. We also calculated and fabricated a high-resolution hologram of 128K × 128K pixels and showed that a realistic 3D image of size 105 mm × 105 mm can be observed over a wide viewing angle of .
2. Hologram calculation using orthographic ray-sampling plane
According to the method, an intermediate “ORS” plane is defined. This plane needs to be placed near objects so that rays from the objects are adequately sampled. The LF passing through this plane is captured as orthographic images at different angles. An orthographic image is a set of spatially distributed parallel light rays. The propagation from the ORS plane to the hologram plane is simulated based on diffraction theory, which can be written as a convolution as follows:26]. We showed that the angular spectrum of the RS plane, , can be written as follows :
3. Calculation of tiled holograms using an orthographic ray-sampling plane
In the calculation of large-scale holograms, it is difficult to process all wavefront data at once, because a large working memory is required. Thus, it is necessary to partition the object and/or hologram data. A reasonable means of partitioning is to divide the hologram plane into small rectangular tiles . This division enables parallel computation of the hologram plane.
Figure 2 shows the concept of (a) the original tiling and (b) the proposed tiling calculation. In the conventional methods for partitioning object and hologram data, each hologram tile must be calculated as a sum of the propagation results from all wave-primitives, such as point-sources, polygons, or object planes. If the point sources or polygons are divided into subsets, all combinations among an object subset and a hologram tile need to be calculated. In the case of propagation of the wavefront from the object plane to the hologram plane, e.g., the shifted-Fresnel diffraction , the object plane is divided into tiles of the same size as those of hologram plane. The wave propagation must be calculated for all combinations of tiles at the object and hologram planes, as shown in Fig. 3.
Let the object and hologram planes be divided into K and N tiles, respectively. Here, it is assumed that the object plane is m times larger than the hologram plane (thus ). In this scheme, the wavefront at the n-th tile of the hologram can be calculated as follows:Eq. (4), propagation must be computed K times to obtain one tile of the hologram, meaning that NK propagations are required for whole hologram. Therefore, the computational cost becomes larger as the number of tiles increases.
In this paper, we propose an efficient method to calculate large-scale tiled holograms by applying the orthographic ray-sampling method. This scheme makes it possible to independently compute each small tile of the hologram plane from a small amount of input data that affects the small tile.
Figure 4 shows the concept of the proposed method. The key idea is the fact that each hologram tile can be computed from only the light rays that are incident upon it from the object plane. A small tile does not require all light ray data, but only a small number of rays. Therefore, a small amount of data concerning a tile of the ORS plane is required for computing the propagation. Note that whole-wavefront data of an object tile is required in the conventional scheme.
The proposed scheme can be described as below using the framework of the orthographic RS method. The contribution from the orthographic image of the angle to the n-th hologram tile can be obtained by the following simple geometric calculationEq. (9) into Eq. (5), the Fourier transform of the contributing orthographic rays, , can be written as follows:
In the calculation of , the width of the orthographic image is limited by the rectangular function; thus, only light-rays is required in this limitation. Considering that all wavefront data are required in the original scheme explained above, the input data can be reduced to the rate r by dividing the hologram plane into N tiles:
According to this scheme, the huge wavefront data set can be separated into a small input data set for light rays attached to each tile of the hologram plane. Therefore, the efficient calculation of tiled large-scale holograms can be achieved. Additionally, unlike other ray-based methods, the diffraction of light-rays can be considered; thus, the resolution of the reconstructed image does not degrade, even over a significant distance from the hologram plane.
4. Estimation of computational complexity
In this section, let us estimate and compare the computational complexities of the conventional and proposed methods to calculate the tiled hologram. We assume that the object plane is employed to efficiently calculate the propagation of the object wave. The propagation distance is selected so as to satisfy that every point on the object plane affects all the area of the hologram plane.
Let the hologram plane have X × X pixels. Firstly, when calculating a hologram without using the rectangular-tiling technique shown in Fig. 5(a), the computational complexity can be estimated as follows:22] or shifted-angular-spectrum propagation . The second term is the computational cost for FFT and inverse FFT. The third term corresponds to the multiplication with the transfer function.
In the conventional framework of rectangular tiling , the computational complexity of a tiled hologram can be approximately estimated as follows. The schematic diagram is shown in Fig. 5(b). The hologram plane is divided into N × N tiles of X/N × X/N pixels. The object plane has mX × mX pixels and is divided into mN × mN tiles of X/N × X/N pixels. Note that it is assumed that each tile is computed independently, meaning that the result of the calculation is not shared between each tile. The entire computational complexity measured by complex multiplication can be written as:Fig. 5(b), each term of Eq. (15) is given by:Eqs. (16)-(18) into Eq. (15), the entire computational complexity, C, is derived as:
We also estimate the computational complexity of the proposed method. We define the entire computational cost, , in the same manner as Eq. (15) as follows
Secondly, the cost required for multiplying by the transfer function, , is also written as:
Finally, the cost for an inverse-Fourier transform to obtain the wavefront at the hologram plane, , can be written as:
Substituting the above equations into Eq. (20), can be written as:Eq. (19) disappears because of separation of input data using conversion from orthographic images to angular spectra. Therefore, the computational complexity does not increases with the number of divisions, N.
In practice, the entire computational cost slightly increases with the number of tiles, N, because of the overhead cost that proportionally increases with the number of FFTs. However, the overhead cost is sufficiently small compared with the overall complexity. The diffraction effect of rays could also be a problem if tiles are too small. The width of each tile, , decreases with N. On the other hand, the width of a ray considering the diffraction effect, , does not change with N. When is much more smaller than , the overall cost, , becomes heavier than the estimation. However, is larger than in most cases; thus, the proposed method works effectively.
Figure 6 shows the estimated computational complexity calculated from the equations above. The resolution of the hologram is set to 16K × 16K, and we assumed that and . In the next section, we examined this estimation experimentally.
5. Experimental results
5. 1. Measurement of the computational time
In the first experiment, we confirmed the efficiency of the proposed method by measuring the computational time for synthesizing CGHs. We calculated holograms of 16K × 16K pixels as shown in Fig. 7; the number of tiles with the conventional RS-plane method  was changed to that with the proposed method. The wavelength, , was set to 633 nm, and the sampling pitch, , was 1.0µm. Each orthographic image has 512 × 512 pixels, and they were rendered from 16 × 16 different angles using 3DCG software Blender . The propagation distance was 60 mm.
Figure 8 shows the experimentally measured calculation time including time for rendering when the number of divisions in one direction, N, changes. 3D images of the simulated reconstruction are shown in Fig. 9. Note that we used an Intel Core i7 4790K CPU (4.0GHz) and a 32GB RAM for rendering orthographic images and calculating holograms. The overall behavior of the measured calculation time agreed with the theoretical estimation of the computational cost, as shown in Fig. 6. The calculation time remains almost constant at 12 minutes, regardless of the number of tiles using the proposed method, as opposed to the radical increase that occurs with the conventional method. From this result, we conclude that the proposed method is effective for calculation of large-tiled holograms. Moreover, the result shows that the hologram plane can be easily divided and computed independently without additional computing cost; thus, parallel computing of holograms will greatly improve the calculation time.
5. 2. Calculation of over-10-Giga-pixel CGH
In the second experiment, we calculated a CGH with 128K × 128K pixels and printed the synthesized CGH as a binary mask. For the CGH calculation, we created a 3DCG scene of “Whiskey and Glasses” containing realistic material features such as reflection, refraction, absorption, and anisotropic reflection, as rendered by the physically based “Cycles” rendering engine in Blender. Each orthographic image has 1,024 × 1,024 pixels rendered in 64 × 64 different angles. The wavelength, , was set to 633 nm, and the sampling pitch, , was 0.8µm. The maximum diffraction angle, , is given by:Fig. 10. The required rendering time was 23 hours using the computing environment mentioned above and an additional graphical processing unit (GeForce GTX 980).
The geometrical setup of the computed CGH is shown in Fig. 11. The ORS plane was placed 180 mm behind the hologram one. The size of the hologram and ORS planes was 105 mm × 105 mm. The hologram plane was divided into 8 × 8 tiles, each having 16K × 16K pixels. Without division of the CGH plane, we need at least 260 GB of RAM. After tiling, the required memory became 4 GB. The computed wavefront interfered with the reference wave, a spherical wave of , and the fringe pattern was obtained. The computing time needed to synthesize this hologram was 13 hours using the same PC in the first experiment without a GPU.
The calculated fringe pattern was binarized and printed as an amplitude mask by the laser-lithography system (Heidelberg Instruments DWL 66 + ) of “Kan-Dai Digital Holo Studio” . At this time, the interference-fringe data is converted from a binary bit map image to CAD data and fabricated on a glass substrate.
By illuminating the CGH with a spherical wave of He-Ne laser light, the 3D image can be observed with the naked eye as a virtual image at a distance of 180 mm from the hologram plane. The optical reconstructions are shown in Fig. 12 (see Visualization 1). It can be seen that the label of the whiskey bottle looked distorted through the wine glass by the translucency and refraction effects in Fig. 12(d). Even for similar effects such as refraction, a different translucency can be observed from the absorption and scattering in ice cubes. Additionally, the ice bucket can be recognized as an anisotropic-reflecting-metal object.
In this paper, we propose an efficient method to calculate large-scale and realistic CGHs using separation of the input data by the ORS plane method. We theoretically and experimentally verified the effectiveness of the proposed framework in terms computational cost and also verified that the synthesized CGH can reproduce 105mm × 105mm 3D images at a viewing angle degrees from the CGH fabricated by the laser-lithography system. The optical reconstruction provides high resolution, even in the deep scene and highly realistic material appearance including translucency, refraction, and anisotropic reflection. For future improvements, parallel computing using the GPU will greatly accelerate the calculation time. Additionally, optical reconstruction in full color must be realized.
Japan Society for the Promotion of Science (JSPS) (KAKWNHI 15K04691); MEXT strategic research foundation at private universities (2013-2017).
A part of this work is supported by Kan-Dai Digital Holo-Studio in Kansai University.
References and links
1. M. Lucente, “Interactive computation of holograms using a look-up table,” J. Electron. Imaging 2(1), 28–34 (1993). [CrossRef]
4. T. Shimobaba, T. Ito, N. Masuda, Y. Ichihashi, and N. Takada, “Fast calculation of computer-generated-hologram on AMD HD5000 series GPU and OpenCL,” Opt. Express 18(10), 9955–9960 (2010). [CrossRef] [PubMed]
5. H. Yoshikawa, “Fast computation of Fresnel holograms employing difference,” Opt. Rev. 8(5), 331–335 (2001). [CrossRef]
6. T. Nishitsuji, T. Shimobaba, T. Kakue, and T. Ito, “Fast calculation of computer-generated hologram using run-length encoding based recurrence relation,” Opt. Express 23(8), 9852–9857 (2015). [CrossRef] [PubMed]
7. T. Shimobaba, N. Masuda, and T. Ito, “Simple and fast calculation algorithm for computer-generated hologram with wavefront recording plane,” Opt. Lett. 34(20), 3133–3135 (2009). [CrossRef] [PubMed]
15. M. Yamaguchi, H. Hoshino, T. Honda, and N. Ohyama, “Phase-added stereogram: calculation of hologram using computer graphics technique,” Proc. SPIE 1914, 25–31 (1993). [CrossRef]
17. H. Zhang, Y. Zhao, L. Cao, and G. Jin, “Fully computed holographic stereogram based algorithm for computer-generated holograms with accurate depth cues,” Opt. Express 23(4), 3901–3913 (2015). [CrossRef] [PubMed]
20. H. Sasaki, K. Yamamoto, K. Wakunami, Y. Ichihashi, R. Oi, and T. Senoh, “Large size three-dimensional video by electronic holography using multiple spatial light modulators,” Sci. Rep. 4(1), 6177 (2015). [CrossRef] [PubMed]
21. S. Igarashi, T. Nakamura, and M. Yamaguchi, “Fast method of calculating a photorealistic hologram based on orthographic ray-wavefront conversion,” Opt. Lett. 41(7), 1396–1399 (2016). [CrossRef] [PubMed]
23. N. Takada, T. Shimobaba, H. Nakayama, A. Shiraki, N. Okada, M. Oikawa, N. Masuda, and T. Ito, “Fast high-resolution computer-generated hologram computation using multiple graphics processing unit cluster system,” Appl. Opt. 51(30), 7303–7307 (2012). [CrossRef] [PubMed]
26. J. W. Goodman, Introduction to Fourier Optics (McGraw-Hill, 1996).
28. Blender Foundation, “Blender,” https://www.blender.org.
29. Kansai University, “Kan-Dai Digital Holo Studio,” http://holography.ordist.kansai-u.ac.jp/digitalholostudio.