We present a method to increase the resolution of holographically projected images by the use of a collective matrix of two phase-only spatial light modulators. As a result of fine alignment and common coherent illumination of the modulators, a synthetic aperture is formed with a doubled number of active pixels. In this paper we present theoretical analysis and numerical simulations which stand in good agreement for different distances between the apertures of modulators. We have achieved experimentally controlled field interference from both modulators on the projection screen, exhibiting increased resolution in one spatial direction and residual effects of the dead space between modulators. Then we have proved numerically possibility to extend our approach on bigger synthetic apertures consisted of more than two modulators.
© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
The ongoing development of liquid crystal on silicon (LCoS) spatial light modulators (SLMs) induces growth in many branches of science and technology, including the holographic projection . The advantages of diffractive image formation are significant and include high efficiency, lack of mechanical parts and compactness of the optical setup , as opposed to classical imaging with lossy modulation of light by selective absorption. On the other hand, the main limitation of the holographic concept lies in the low pixel count of the spatial light modulators, which leads to the insufficient resolution of the projected images. Usually, the effective loss of image points compared to the pixel count of the spatial light modulator is close to 50% . The commonly used SLMs have a resolution of 1920 by 1080 pixels, which allows one to obtain the holographically projected images with small angular size and merely SVGA resolution (800 by 600 points), typical for displays from two decades ago.
In order to increase the informative capacity and the angular size of projected images, the reduction of pixel pitch and increase of number of pixels is required. Currently the high-end, off-the-shelf modulators with resolutions of 4k or even 8k  and the pixel size below 4µm are commercially available. However, their production technology needs to be further refined in terms of flicker, linearity  and refresh rates. Additionally, it is noticeable that denser pixel structure leads to lower fill factor and as a consequence to poor SLM performance. Thirdly, the transmission of 4k signal with high frame rates is problematic due to bottlenecks in digital media transmission standards, e.g. DisplayPort or HDMI. In this paper we prove experimentally that it is possible to obtain holographic image projection with higher and fully scalable definition by a coherent superposition of multiple lower-resolution spatial light modulators working as a single synthetic-aperture SLM (SA-SLM).
The idea of using multiple phase SLMs in computer holography was previously used to decrease the point spread function , to expand the field of view of virtual images [6–9] or to achieve a full-color operation . Nevertheless, the combinations of SLMs was incoherent, i.e. the intensity fields were superimposed without the phase shift control  or without overlapping of light fields from separate SLMs . Ochoa et al. recently demonstrated the reduction of on-axis Point Spread Function (PSF) spots [13, 14] by interference of two spatially apodized beams reflected from a single SLM. Nevertheless as opposed to the presented method, significant energy losses were present and no real image formation was shown except for numerical convolution with obtained PSFs.
In this approach we present the theoretical calculations, numerical simulations and experimental results of holographic image projection from a projector with active resolution of 3840 by 1080 pixels, constructed by locating side-by-side and carefully positioning two Full HD (1920 by 1080) SLMs with pixel pitch of 8µm. The proposed method could potentially be used with higher number of component SLMs forming a larger collective arrays. Such case showed here with 4 SLMs is based on numerical simulations only due to lack of hardware to perform the experiment.
2. Projection by means of synthetic aperture consisting of two modulators
The desired growth of the effective aperture of the spatial light modulator can be obtained by arranging several SLMs in a coherent, collectively functioning matrix. The key task is to synchronize the modulators so that they constitute a uniform surface with coherent phase modulation on a large area, referred to as the synthetic aperture. Obviously, such area will not be homogeneous due to non-zero spacing between respective modulators, referred here as dead space or gap. Such formed synthetic-aperture SLM is loaded with holograms of a diffusive type, i.e. iterated with random initial phase. As a result it is possible to reconstruct the entire encoded image from any part of the hologram . Therefore, the impact of dead area between SLMs on the quality of the reconstructed image is low, but it is still analyzed in this paper in order to find the optimal value of the gap width yielding the satisfactory image resolution.
For theoretical derivation it is assumed that the pair of light modulators is loaded with far-field Fraunhofer holograms iterated with Iterative Fourier Transform Algorithm (IFTA) algorithm [16, 17] and additionally multiplied with a quadratic phase factor of a positive lens in order to include the optical power of the missing projection lens focusing correctly on the fixed acquisition plane. Such holograms are here referred to as Fourier holograms. The choice of such holograms was dictated by the possible applications in far field image projection where the resolution is proportional to the extent of the hologram, or the number of its active pixels.
The synthetic aperture is obtained as a result of superposition of two perfectly flat and aligned spatial light modulators loaded with respective fragments of computer-generated hologram with an added phase factor of a converging lens. The process is schematically depicted in Fig. 1.
Two identical spatial light modulators with dimension A x B and distant by Δ form the following aperture transmittance (Fig. 2):
The Point Spread Function in the focal plane of the virtual lens shown in Fig. 1 is proportional to the Fourier transform of the transmittance T(x, y) given as follows:
With an accuracy to a non-important constant, Eq. (2) defines the following output intensity distribution:
The modulation 4A2sinc2(fx A), depends on the aperture size along the OX axis and the factor cos2 [πfx (A + Δ)] is the result of interference generated by two identical apertures. This factor corresponds to intensity distribution formed by two infinitesimally narrow slits being perpendicular to the OX axis and passing through the centers of the both rectangular apertures with coordinates (−(A + Δ)/2, 0) and ((A + Δ)/2, 0) (Fig. 2).
In the particular case of Δ = 0, the intensity distribution (5) transforms into I(fx) = 4A2B2sinc2(fx2A). The main peak has the width Δfx = 1/A, defined by a distance between the first minima. The growing gap Δ between two SLMs causes the narrower main peak at the expense of sidelobes which may have a negative influence on the resolution obtained in the holographic projection. Intensities and number of the sidelobes generally increase with the width of the gap Δ (Fig. 3).
In order to verify the theoretical conclusions, numerical simulation were performed on the matrix of 65, 5362 complex points with sampling of 1µm × 1µm and wavelength of λ = 632.8nm. The phase patterns displayed on virtual SLMs were assumed in accordance with the scheme depicted in Fig. 1. The numerical propagation was performed at a distance of z = 1000mm by the Modified Convolution Method . As a result, the images of the PSFs for two spatial light modulators separately and their superposition were obtained. The theoretically calculated cross sections of PSFs are shown in Fig. 3(a) and the numerically verified 2-D shapes of PSFs are presented in Fig 3(b).
Two effects can be observed in Fig. 3 for growing values of Δ: narrowing of the central peak in PSFs and inevitable raise of spurious sidelobes. These opposite efects require more accurate study of image resolution by means of the modulated transfer function (MTF) as a function of the dead space.
MTFs for different values of Δ were obtained as Fourier transforms of the PSF spots obtained from numerical simulations and are shown in Fig. 4. We intentionally show only selected curves for the following gaps: Δ = 0, Δ = 0.2A where the MTF is still monotonic, Δ = 0.7A corresponding to the gap width validated in the experiment, and Δ = 1.3A, where the MTF drops to zero before the apparent final cut-off frequency.
According to the detailed analysis of the calculated MTFs the parameter Δ = 0.7A seems to be the most convenient in the experimental set-up. This gap corresponds to the physical size of plastic borders of the used Holoeye Pluto SLMs. Moreover its width is sufficiently small then the MTF has substantial values in a wide range of spacial frequencies. On the other hand in this case, the MTF function is not monotonic, hence it was questionable to estimate the image resolution based on any of the commonly used MTF criteria. Therefore, in this paper we decided to use the USAF 1951 test pattern and to observe the densest resolved group. Such observation is subjective by nature, hence the averaging of observations from four independent people was used to make it more objective. The given USAF 1951 group was noted as resolved when it was independently recognized by at least three of four observers.
In order to avoid the speckle patterns resulting from G-S algorithm, the numerical reconstruction of USAF 1951 test pattern was performed as an intensity integration of 25 reconstructions of Fourier holograms with different initial random phase patterns . The results for single SLM and both SLMs with the distance Δ = 0.7A are presented in Fig. 5. For fair comparison, the simulations of a single modulator were performed with a single SLM aperture centered on the calculation matrix and the apertures of SLMs were enforced in each iteration of the G-S loop.
One can notice a minor decrease of contrast in group 4 of the USAF pattern. The spatial frequencies corresponding to this group fall in the local minimum of the MTF for Δ = 0.7, see Fig. 4. For this reason we conclude that further increasing of the gap between the SLMs above Δ = 0.7 would lead to unacceptable deficiencies in the image contrast.
We also performed the same simulations for different values of Δ. The image resolutions assesed by four observers are presented in Table 1. The apparent resolution in the horizontal direction reach a plateau between Δ = 0.2 and Δ = 0.7. Then the resolution decreases because the interference from the gap between SLM apertures becomes the dominant effect. The resolution in the vertical direction is constant, because the effect of synthetic aperture works only in the horizontal (x) direction, see Fig. 2.
The surfaces of spatial light modulators available on the market are not perfectly flat. In order to successfully create the synthetic aperture from two modulators, firstly it was necessary to establish and improve the optical flatness of each SLM by superimposing appropriate phase corrective functions [3, 20].
Two Holoeye Pluto VIS-014 SLMs with a resolution of 1920 by 1080 pixels and the pixel pitch 8µm (Fig. 6(a)) were examined in the Mach-Zehnder interferometer in order to gain the correction functions (Fig. 6(b, c)), which allows to obtain the quasi-plane wave in the reflection. The functions have been found as the composition of Zernike polynomials up to 5th order  and applied in the holographic computation process. Obviously, the corrected fields are observed in the first diffractive orders of the SLMs. The corrective factors vary with time due to thermal expansion of the SLM then they need to be periodically re-checked.
The panels were illuminated with a linearly-polarized light from a He-Ne laser. The angle of polarization was the same for both interfering beams and optimized for the maximal diffraction efficiency.
Measurement of Point Spread Functions
The optical setup without the optical imaging elements was built to verify the numerical simulations. The scheme of the optical setup is presented in Fig. 7. One SLM panel was fixed, while the other one was mounted on a set of motorized actuators enabling precise motion along X, Y, Z axes, rotation and leaning controlled from the in-house software developed in C++. All six degrees of alignment of one SLM against the other were iteratively controlled so as to obtain the pixel-in-pixel precision, being the crucial for the synthetic aperture concept.
He-Ne laser beam expanded by a pinhole and a lens illuminated the SLM surface at a normal angle. Reflected beams modulated by SLMs travelled through a non-polarizing beam splitter and could interfere anywhere within the cone shown in red in Fig. 7. The resulting field was registered directly on a monochromatic CMOS matrix of the Basler acA3800-14um camera with a pixel size of 1.67 × 1.67µm. After displaying the lens phase on the SLMs, the point spread function spots were recorded in non-compressed RAW files for one and two SLMs at the propagation distance z = 1000mm. The apertures of modulators were separated by a distance of Δ = 10.8mm = 0.7A. The minimal and maximal values of Δ were limited by the physical size of SLM frame and the size of the beam-splitter reducing the effective aperture of the optical setup. As was noted before the value of distance Δ was established as 0.7A to avoid the strong side-lobes in the PSF spots.
Narrowing of the width of the main peak of the PSF can be observed as a result of the coherent interference of beams incoming from both SLMs to the camera. Due to the non-zero gap between the SLMs, the low-intensity sidelobes appear in the point spread function spots, as predicted by simulations and theory.
The above results were obtained in the optimal position at the image plane, i.e. in its center. Isoplanarity could provide the uniform resolution in the entire image plane. In order to confirm the isoplanarity of the constructed optical setup, five PSF spots were registered in different locations of the image plane at a distance z = 1000mm. They were located in the center and in corners of the rectangular image area with dimensions 25 × 19mm, as shown in Fig. 9. The spots were generated by especially designed hologram comprising a set of phase patterns of saw-tooth diffractive gratings. The cross-sections of the analyzed PSFs are shown in Fig. 10. The similarity of all spots confirms the isoplanarity of the optical system used in this work. It also proves that no significant distortions of polarization  were observed, possibly due to the use of identical panels and illumination angles in both interfering arms.
Holographic projection of images
After the confirmation of the isoplanar nature of the optical setup it was possible to use it for image projection at the CMOS matrix of the camera. The synthetic aperture Spatial Light Modulator was loaded with computer generated phase only Fourier holograms calculated with the Gerchberg-Saxton (GS) algorithm . An additional quadratic phase of a focusing lens with focal length equal to distance to the acquisition plane was added to holograms. The images reconstructed in this way suffered from high-contrast speckle noise. In order to properly study the achieved imaging resolutions we used the time dependent random phase (TDRP) method  so as to time-average the noise. A set of 25 holograms with different random initial phase states was summed in each photograph. The acquisition time of 500 ms allowed the intensity integration of all sub-holograms. The USAF-1951 resolution test was chosen as the input image.
The obtained results are depicted in Fig. 11. In the case of a single SLM, the hologram was calculated on a smaller matrix so as to make a fair comparison. In the images reconstructed with the superposition of SLMs, the vertical resolution has increased, while the horizontal resolution was preserved, in accordance with the theory and simulations. Generally, experimentally obtained resolutions given in Tab. 2 are in agreement with numerical simulations for Δ = 0.7A, but with somewhat declined vertical resolution resulting from experimental distortions of the output image. These distortions were caused by instable interference of fields propagating from two SLMs due to thermal drift, non-uniform spatial response , liquid crystal flicker effects [23, 24], and errors of stepper motors in the SLM positioning hardware. The negative effect of the SLM flicker  was minimized by temporal synchronization of both used panels, driven by a single graphic board in the PC computer. For this reason the mitigation of flicker e.g. by cooling down the panel  was not necessary in this conceptual work. The isoplanatic nature of the optical setup enables to predict the uniform reconstruction quality in the entire visible image plane.
The visibility of the whole encoded test pattern proves that the number of image points is sustained in the process of image formation with SA-SLM and could even be increased by adding more modulators to the collective matrix. Alternatively, any spatial division of the SLM surface  could lead to the reduction of the informative capacity of the displayed images.
4. Collective matrix of four modulators
The possibility of increasing the resolution of projected images with the use of synthetic aperture is not limited to only two SLM modulators. The concept of a collective matrix allows one to create the synthetic aperture from a larger number of SLMs. The method allows particularly easy rescaling to e.g. 4, 8, 16 or even more modulators. In order to validate this claim, we provide here additional simulation of holographic image reconstruction with a SA-SLM consisting of 4 perfectly synchronized and aligned spatial light modulators (Fig. 12(a)). The projection distance was z = 1000mm. The result of simulations performed on a matrix of 65, 5362 points are shown in Fig. 12(b). The shown bitmap is an integration of 25 sub-holograms with different speckle distributions. The image resolution was assessed again based on observations of the USAF-1951 pattern from four independent people and gathered in Tab. 3. The experimental validation of this case is a matter of our future work.
The image resolution estimated according to our methodology is in this case approximately four times higher than in the case of projection performed by the single SLM.
Adding more modulators has increased the projection resolution in both x and y directions. This is also noticeable in the direct comparison of numerically projected Mandrill test image from one, two and four SLMs, shown in Fig. 13. For instance, the faint ring in the mandrill’s iris is only resolved with two or four SLMs. The 1-pixel wide vertical and horizontal black lines were intentionally added to the test image in order to underline the one-dimensional and two-dimensional growth of image resolution for 2 and 4 SLMs, respectively.
The main challenge of future experimental validation of this case seems to be the uniform illumination and precise alignment of such large matrices, as real-time calculations of such big data amounts are feasible with GPU (Graphics Processing Unit) computing. The mentioned problem of fine positioning of multiple SLMs can potentially be overcome by their joint fabrication and packaging on a common silicon wafer.
This paper proves that the concept of the synthetic-aperture spatial light modulator in computer holography can be realized experimentally in a good agreement with simulations and theory. The method allows scalable enlargement of the surface and pixel count of phase-modulating devices for far-field holographic projection of images with higher resolution. The resolution-limiting factor is the presence of sidelobes in PSFs, which constraints the maximal allowed spacing between SLMs. The gap width can be decreased by modified process of manufacturing of an array of SLMs on a single silicon wafer. Alternatively one can tailor the lightpath to effectively remove the dead space between SLMs , obviously for the price of higher complexity of the optical setup. Nevertheless for an optimal value of the gap width Δ = 0.7A, the experiment with a group of independent viewers has confirmed the twofold one-dimensional growth of image resolution in the isoplanatic holographic projection setup at a distance of 100cm.
The proposed method has a potential of numerous applications in the Fourier optics domain and computer holography. The demonstrated enlargement of the effective aperture of available liquid-crystal phase modulators considerably increases the possibilities of shaping light fields with the electronically controlled diffraction. The coherent superposition of modulators considerably enlarges the effective aperture and increases the informative capacity, which could be advantageous mainly in the field of holographic projection of intensity fields.
The method opens other opportunities, currently limited by parameters of spatial light modulators, e.g. high-resolution holographic beam manipulation in optical tweezers , where the critical parameter is the size and positioning of the optical spot used to manipulate the microscopic objects. The pixel count of the SLM has a significant influence on the fine positioning of the spot. In this case the problem of side-lobes must be addressed in order not to create spurious ghost traps. On the other hand high contrast between the central peak of the PSF and the adjacent dark fringes (see Fig. 8) and proper phase relations could possibly enhance the optical trapping force. The use of SA-SLM also allows higher frame rates, which are problematic for high-end modulators due to bottlenecks in data transfer and pixel addressing.
The SA-SLM concept can be easily extended to a different ranges of the electromagnetic spectrum. For instance, in the UV region it could be used for dense stereolighography and high-resolution exposures of photo-resist layers [27, 28]. In the infrared region it could potentially be used for ultrafast photo-magnetic data recording in transparent media . The combination of highly localized PSFs and a thresholding medium should effectively eliminate the weaker sidelobes falling bellow the treshold, thus significantly increasing the recording density. Shifting of the required level of thresholding can be carried out by altering the recording medium and thus the magnetic anisotropy barrier, i.e. the energy required to switch the magnetic state.
The spectral range of applications is only limited by the proper operation of light modulators, e.g. in far infrared and THz regions SA-SLMs could be the optimal way of obtaining high pixel count for real-time beam modulation [30, 31].
The unique property or the proposed solution is the scalability to higher numbers of combined modulators. The main limitation is the ability to illuminate coherently the growing surface of the synthetic aperture, fine alignment of SLMs and the data calculation. Nevertheless, having overcome these technical challenges, one could possibly use a collective matrix of any number of SLMs thus surpassing the pixel count of available high-end modulators. This could lead to unprecedented image resolutions unaccessible for conventional imaging systems.
Polish National Science Center under grant no. 2015/17/B/ST7/03754.
References and links
1. I. Ducin, T. Shimobaba, M. Makowski, K. Kakarenko, A. Kowalczyk, J. Suszek, M. Bieda, A. Kolodziejczyk, and M. Sypek, “Holographic projection of images with step-less zoom and noise suppression by pixel separation,” Opt. Commun. 340, 131–135 (2015). [CrossRef]
2. M. Makowski, “Minimized speckle noise in lens-less holographic projection by pixel separation,” Opt. Express 21, 29205–29216 (2013). [CrossRef]
3. M. Makowski, I. Ducin, K. Kakarenko, J. Suszek, and A. Kowalczyk, “Performance of the 4k phase-only spatial light modulator in image projection by computer-generated holography,” Photonics Lett. Pol. 8, 26–28 (2016).
4. K. Wakunami, P.-Y. Hsieh, R. Oi, T. Senoh, H. Sasaki, Y. Ichihashi, M. Okui, Y.-P. Huang, and K. Yamamoto, “Projection-type see-through holographic three-dimensional display,” Nat. Commun. 7, 12954 (2016). [CrossRef] [PubMed]
7. M. Kujawinska, T. Kozacki, C. Falldorf, T. Meeser, B. M. Hennelly, P. Garbat, W. Zaperty, M. Niemelä, G. Finke, M. Kowiel, and T. Naughton, “Multiwavefront digital holographic television,” Opt. Express 22, 2324–2336 (2014). [CrossRef] [PubMed]
8. J. Hahn, H. Kim, Y. Lim, G. Park, and B. Lee, “Wide viewing angle dynamic holographic stereogram with a curved array of spatial light modulators,” Opt. Express 16, 12372–12386 (2008). [CrossRef] [PubMed]
9. H. Sasaki, K. Yamamoto, K. Wakunami, Y. Ichihashi, R. Oi, and T. Senoh, “Large size three-dimensional video by electronic holography using multiple spatial light modulators,” Sci. Reports 4, 6177 (2014). [CrossRef]
10. H. Nakayama, N. Takada, Y. Ichihashi, S. Awazu, T. Shimobaba, N. Masuda, and T. Ito, “Real-time color electroholography using multiple graphics processing units and multiple high-definition liquid-crystal display panels,” Appl. Opt. 49, 5993–5996 (2010). [CrossRef]
11. F. Yaras and L. Onural, “Color holographic reconstruction using multiple slms and led illumination,” Proc. SPIE 7237, 72370O (2009). [CrossRef]
12. S. B. Hasan and T. Kozacki, “Method for enhancing the resolution of holographic displays,” Photonics Lett. Pol. 1, 193–195 (2009).
13. N. A. Ochoa and C. Pérez-Santos, “Super-resolution with complex masks using a phase-only lcd,” Opt. letters 38, 5389–5392 (2013). [CrossRef]
14. N. A. Ochoa and Y. P. de León, “Super-resolution with an slm and two intensity images,” Opt. Commun. 416, 36–40 (2018). [CrossRef]
15. J. Goodman, An Introduction to Fourier Optics (McGraw-Hill, 1968).
16. F. Wyrowski and O. Bryngdahl, “Iterative fourier-transform algorithm applied to computer holography,” JOSA A 5, 1058–1065 (1988). [CrossRef]
17. R. Gerchberg and W. Saxton, “A practical algorithm for the determination of the phase from image and diffraction plane pictures,” Optik 35, 237–246 (1972).
18. M. Sypek, “Light propagation in the fresnel region. new numerical approach,” Opt. Commun. 116, 43–48 (1995). [CrossRef]
19. M. Makowski, I. Ducin, K. Kakarenko, A. Kowalczyk, M. Bieda, and J. Suszek, “Optimized computation method for real-time holographic formation of color images,” Photonics Lett. Pol. 6, 81–83 (2014).
20. S. Osten, S. Krüger, and A. Hermerschmidt, “New hdtv (1920x1080) phase-only slm,” Proc. SPIE 6487, 64870X (2007). [CrossRef]
21. V. Lakshminarayanan and A. Fleck, “Zernike polynomials: a guide,” J. Mod. Opt. 58, 545–561 (2011). [CrossRef]
22. V. Lopez, A. González-Vega, A. Aguilar, J. Landgrave, and J. García-Márquez, “Non-uniform spatial response of the lcos spatial light modulator,” Opt. Commun. 366, 419–424 (2016). [CrossRef]
24. K. Kakarenko, M. Zaremba, I. Ducin, M. Makowski, A. Siemion, A. Siemion, J. Suszek, M. Sypek, D. Wojnowski, Z. Jaroszewicz, and A. Kolodziejczyk, Utilization of the phase flicker of a lcos spatial light modulator for improved diffractive efficiency,” Photonics Lett. Pol. 2, 128–130(2010).
25. Y. Calderón-Hermosillo, J. García-Márquez, R. Espinosa-Luna, N. A. Ochoa, V. López, A. Aguilar, E. Noé-Arias, and Y. Alayli, “Flicker in a twisted nematic spatial light modulator,” Opt. Lasers Eng. 51, 741–748 (2013). [CrossRef]
27. T. Yoon, C.-S. Kim, K. Kim, and J.-R. Choi, “Emerging applications of digital micromirror devices in biophotonic fields,” Opt. & Laser Technol. 104, 17 – 25 (2018). [CrossRef]
30. A. M. Tareki, W. Kim, J. Guo, and R. G. Lindquist, “Longitudinal stratified liquid crystal structures to enable practical spatial light modulators in the terahertz regime,” Proc. SPIE 9940, 994013 (2016). [CrossRef]
31. M. N. F. Hoque, G. Karaoglan-Bebek, M. Holtz, A. A. Bernussi, and Z. Fan, “High performance spatial light modulators for terahertz applications,” Opt. Commun. 350, 309 – 314 (2015). [CrossRef]