Open Access
Add to BibSonomy
Abstract
Time-multiplexed light-field displays (TMLFDs) can provide natural and realistic three-dimensional (3D) performance with a wide 120° viewing angle, which provides broad potential applications in 3D electronic sand table (EST) technology. However, current TMLFDs suffer from severe crosstalk, which can lead to image aliasing and the distortion of the depth information. In this paper, the mechanisms underlying the emergence of crosstalk in TMLFD systems are identified and analyzed. The results indicate that the specific structure of the slanted lenticular lens array (LLA) and the non-uniformity of the emergent light distribution in the lens elements are the two main factors responsible for the crosstalk. In order to produce clear depth perception and improve the image quality, a novel ladder-type LCD sub-pixel arrangement and a compound lens with three aspheric surfaces are proposed and introduced into a TMLFD to respectively reduce the two types of crosstalk. Crosstalk simulation experiments demonstrate the validity of the proposed methods. Structural similarity (SSIM) simulation experiments and light-field reconstruction experiments also indicate that aliasing is effectively reduced and the depth quality is significantly improved over the entire viewing range. In addition, a tabletop 3D EST based on the proposed TMLFD is presented. The proposed approaches to crosstalk reduction are also compatible with other lenticular lens-based 3D displays.
© 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
As one of the most promising three-dimensional (3D) display technologies, light-field displays have attracted significant interests in recent years [1–9]. Unlike binocular disparity-based 3D displays, light-field displays can reconstruct real 3D light-field distributions, which provide viewers with natural and realistic 3D perception. Light-field displays are of particular interest for their potential use in 3D electronic sand tables (ESTs). Recent developments in advanced real-time terrain rending make it possible to acquire high-definition topographic maps containing massive data. The use of light-field displays to visualize terrain in a realistic and intuitive manner can effectively improve the efficiency of military command, navigation, and air-traffic control systems [10].
An ideal 3D EST system should produce clear 3D images with a large viewing angle so that it can be shared simultaneously with multiple viewers in different directions. Multi-projection-based light-field displays [11–13] can produce realistic, high-definition 3D scenes with a wide viewing angle. However, these systems are generally bulky and a considerable degree of calibration is required. To simplify the structure, several scanning light-field display approaches [14,15] based on one or several high-refresh-rate projectors have been proposed. However, high-speed components only offer a limited number of colors or gray scales, so these systems generally have serious shortcomings when displaying full-color animation and interactive content. For this reason, flat-panel-based light-field displays have recently received greater research attention due to their ability to render full-color, high contrast, and dynamic 3D images with a simple structure [16–18]. As one of the most common types of flat-panel-based light-field display, integral imaging (II) can provide the human eye with full and continuous motion parallax and true color [19]. There has recently been a large volume of research reported on II [20–23], and II display systems have been effectively improved in terms of their structural design and display effects. However, due to the limited capacity of 2D display panels to display spatial information, the inherent constrained relationship between the light-field reconstruction angle and the spatial display resolution has restricted the practical applications of II displays.
Recently, our group demonstrated a novel LCD flat-panel light-field display based on a time-multiplexing technique [24]. It was verified to be an effective approach for achieving a wide 120° viewing angle while ensuring image clarity. In addition, up to 192 dense viewpoints could be constructed within the viewing range, which ensures the correct geometric occlusion and smooth motion parallax in complicated 3D scenes. Therefore, the proposed time-multiplexed light-field display (TMLFD) is well-suited for the virtual display of terrain information in 3D EST systems.
However, current TMLFDs suffer from severe crosstalk issues. Crosstalk is the perception of one or more unintended adjacent views in addition to the desired one, which leads to image aliasing for viewers [25–28]. As the disparity between adjacent views increases, the more serious aliasing appears [29]. According to the 3D perspective relationship, the viewing disparity is proportional to the depth of field (DOF) in the 3D display. As a result, 3D scenes with a larger DOF usually suffer from more serious image aliasing, which degrades the depth quality and lowers the visual experience. Figure 1 presents a light-field reconstruction image of large-depth urban buildings based on a TMLFD. We can see that, in a depth range of 0–15 cm, the viewing disparity is very small, thus the aliasing between the images is slight and acceptable. However, in a depth range of 15–22 cm, the image quality gradually deteriorates with an increase in the depth, which damages the depth information and causes visual fatigue. This illustrates that crosstalk has a detrimental effect on the depth perception for 3D scenes, especially for larger depth ranges, thus restricting the application of light-field displays in 3D EST technology.
In order to improve the depth perception and image quality, the present study focuses on solving the issue of crosstalk in TMLFDs. In Section 2, a brief review of TMLFDs is presented. In Sections 3 and 4, the mechanisms responsible for the two main types of crosstalk in current TMLFD systems are described in detail. A novel ladder-type LCD sub-pixel arrangement and a compound lens with three aspheric surfaces are then proposed and introduced into a TMLFD to suppress the two types of crosstalk. Crosstalk simulation experiments demonstrate that the crosstalk is effectively suppressed using the proposed methods. In Section 5, structural similarity (SSIM) computational simulations and light-field reconstructions show that aliasing is eliminated and the depth quality is significantly improved over the entire viewing range.
2. Brief review of TMLFDs
Figure 2(a) presents the basic structure of a TMLFD system. The light source consists of several multi-directional backlight units (MDBUs) with the same structure arranged side by side; these are responsible for the illumination of the LCD panel. Each MDBU consists of three linear LED light bars and a linear Fresnel lens (LFL), as illustrated in Fig. 2(b). The three light bars are symmetrically distributed on a circular arc, the center of which is the midpoint of the LFL. $\theta $ represents the angle between the midlines of the side and middle LED light bars. The midlines of all light bars are directed toward the center of the LFL, and the radius is set to the focal length of the LFL, which ensures that the light ray bundles emitting from each LED are collimated into parallel beams at the minor field curvature when passing through the LFL. When the three LED light bars work alternately in a sequential manner, three types of time-sequential collimated backlight with directions of $- \theta $, 0°, and $+ \theta $ can be achieved. A lenticular lens array (LLA) is placed in front of the LCD. As shown in Fig. 3, the focal length of the lens elements ($f$), the pitch of the lens elements ($p$), and the incident angle ($\theta $) are designed to satisfy the imaging relationship $f = {p / {\tan \theta }}$, which ensures that collimated light beams from different directions can converge at the same point to generate a voxel. Voxels can be regarded as the basic display cell for a light-field display. With the modulation of a holographic functional screen (HFS), a voxel can emit multiple continuous distributed light beams of various intensities and colors in different directions, which can be used to realistically simulate a point within a 3D scene. (Details of the modulation mechanisms of an HFS are provided in our previous work [24]). The re-modulated light beams emitted from all voxels intersect in space, thus dense viewpoints are constructed. Moreover, by switching the three groups of collimated backlights at a high speed and synchronously refreshing the corresponding light field coded images on the LCD, the three viewing zones (left, central, and right) constructed by a voxel at different time intervals can be spliced together as a continuous whole due to the visual persistence effect. Because the field of view (FOV) of the lens in our prototype is designed to be 40°, a wide 120° viewing angle can be achieved in a time-multiplexed manner. In addition, because all of the pixels on the panel contribute to one viewing zone rather than the whole 120° range at each moment, the resolution of the perceived 3D image will not be greatly reduced. Therefore, the inherent constrained relationship between the viewing angle and resolution is effectively solved. The parameters of the TMLFD system are detailed in Table 1.
Table 1. Configuration of the TMLFD system
However, the current TMLFD configuration can also suffer from crosstalk, which affects the reconstruction quality, particularly by damaging the depth perception. Because this issue has not been addressed in previous researches, the goal of this paper is to resolve crosstalk in TMLFDs. In the next two sections, two types of crosstalk – structural and optical – are described, and two corresponding approaches to resolving this crosstalk are proposed to improve the image quality of TMLFDs.
3. Structural crosstalk
3.1 Analysis
In the light-field reconstruction process, the intensity and color information of the viewpoints are loaded onto the sub-pixels of the LCD. Figure 4(a) shows a conventional LCD sub-pixel arrangement. In the horizontal direction, the adjacent red, green, and blue (RGB) sub-pixels that make up a pixel are arranged periodically. In the vertical direction, the sub-pixels of each column usually have the same basic color. In this sub-pixel arrangement, when the LLA is installed horizontally in front of the LCD, the sub-pixels in each column with the same color will be magnified by the linear lenticular lens [30,31]. As a result, a strong periodic RGB rainbow pattern appears, which affects the image quality and visual comfort. A fundamental solution for this is to position the LLA so that it is slanted at a particular angle [32,33], which can break the periodic positional relationship between the LLA and the sub-pixel columns. In our previous prototype, the slant angle of the LLA was set to 14.04°, which effectively suppressed the rainbow pattern.
Fig. 4. (a) Conventional LCD sub-pixel arrangement. (b) Formation of structural crosstalk in the light-field display. (c) Positional relationship between the slanted lens elements and the sub-pixels. (d) Composition of the mth viewpoint.
However, the use of a slanted LLA introduces considerable crosstalk to the voxel generation process. Figure 4(b) demonstrates the production of crosstalk when the LLA is placed on the LCD with slant angle $\varphi $, with the construction of the central viewing zone used as an example. The region marked by the red dashed line represents the sub-pixels in the middle row that are covered by the lens elements in the horizontal direction. Theoretically, these sub-pixels will participate in the generation of a voxel on the HFS, and only the light beams cast from the corresponding sub-pixels are expected to be perceived at each viewpoint position. However, because the lens is slanted, the actual region that completely covers these sub-pixels will expand to include the adjacent rows, as shown by the region marked by the black dashed line in Fig. 4(c). This indicates that, in addition to the expected sub-pixels in the middle row, some sub-pixels from the upper and lower rows will participate in the generation of the voxel. As a result, the light beams perceived at each viewpoint (e.g. the ${m^{th}}$ viewpoint shown in Fig. 4(d)) consist of not only the current view but also, unintendedly, the adjacent views, which results in severe crosstalk. We refer to this type of crosstalk as structural crosstalk because it arises only from the slanted positioning of the LLA and is not related to the optical properties of the system. Structural crosstalk exists across the entire viewing range, with its degree the same for each viewpoint. It can be calculated as the proportion of the area of the unintended views to the total area. Its derivation is shown in Eq. (1).
It should be noted that similar structural crosstalk also exists in other lenticular lens-based 3D displays, and many crosstalk reduction methods have been proposed. Image processing [34,35], where the intensity values of the sub-pixels are corrected in synthetic images, has been reported to be one of the most effective methods for reducing structural crosstalk. However, this method requires the measurement and calibration of 3D display devices, thus the accuracy of intensity value correction cannot be guaranteed for a light-field display like ours with high-density viewpoints. The combination of eye-tracking techniques and 3D displays [36] have also been proposed to eliminate structural crosstalk. After capturing the observer’s eye position in real-time, only the sub-pixels of the corresponding viewpoint at the current position are turned on while the others are turned off, thus crosstalk between adjacent viewpoints can be avoided. However, this method cannot transmit corresponding viewpoints to multiple observers at the same time, restricting the potential application of light-field displays. Therefore, no current method for the reduction of structural crosstalk can effectively solve this issue in our light-field display.
3.2 Reduction method
In the present study, we propose a novel method that can simultaneously eliminate rainbow patterns and structural crosstalk. The key to this method is introducing a novel ladder-type LCD sub-pixel arrangement to the light-field display. Figure 5(a) presents the ladder-type sub-pixel arrangement in detail. Unlike the conventional arrangement, the sub-pixels are periodically arranged in the order of R, G, and B in both the horizontal and vertical directions in the ladder-type LCD sub-pixel arrangement. With the sub-pixels of the same color in adjacent rows completely staggered in the vertical direction, this ladder-type sub-pixel arrangement can effectively prevent the appearance of rainbow patterns when the LLA is horizontally positioned on the LCD. In addition, the black matrix between adjacent horizontal sub-pixels is designed to be sufficiently narrow, which prevents the appearance of a black/white moiré pattern in the horizontal direction. Figure 5(b) illustrates the structure of an LCD utilizing the ladder-type sub-pixel arrangement with a non-slanted LLA. The region marked by the red dashed line represents the sub-pixels in the middle row that are expected to generate a voxel. Because the lenticular direction is parallel to the vertical edge of the sub-pixel, the edge of the lens element can exactly cover the vertical edge of this group of sub-pixels, as shown in Fig. 5(c). Thus, the light beams that generate the voxel all originate from the expected sub-pixels. It also ensures that only the light beams cast from the current view can be perceived at each viewpoint position (e.g., the ${m^{th}}$ viewpoint shown in Fig. 5(d)). As a result, structural crosstalk can be fundamentally avoided in our light-field display, while rainbow patterns are also effectively eliminated.
Fig. 5. (a) The ladder-type sub-pixel arrangement for an LCD. (b) Generation of a voxel based on the improved LCD panel. (c) Positional relationship between the non-slanted lens elements and the sub-pixels. (d) Composition of the mth viewpoint.
To verify the proposed crosstalk reduction method, two experiments were conducted. The first verified the ability of the proposed method to eliminate the rainbow patterns. In this experiment, a 32-inch TFT-LCD prototype with a resolution of 3840×2160 (4K) and the ladder-type sub-pixel arrangement was fabricated and employed in our light-field display. We compared the visual results for a white image displayed on a light-field display using a traditional 4K LCD panel and the new 4K LCD panel. We used the same non-slanted LLA in this comparison. The pitch of the lenticular lens elements was 0.31 mm. Figure 6(a) presents the results for the conventional panel; a visible striped rainbow pattern was observed on the display, which significantly degraded the image quality. Figure 6(b) shows the results for the new LCD panel. The rainbow pattern disappears and the white field is pure and uniform. As this comparison illustrates, by adopting the proposed method, the rainbow pattern can be effectively eliminated without slanting the LLA.
Fig. 6. Visual effects for the entire white image displayed on a light-field display for (a) a conventional LCD panel and (b) the new LCD panel with ladder-type sub-pixel arrangement. The inserts in (a) and (b) are magnified images.
A crosstalk simulation experiment was also conducted to verify the effectiveness of the proposed method in reducing structural crosstalk. In this experiment, the crosstalk was simulated by measuring the luminance distribution of the viewpoints along the horizontal direction on a virtual receiving screen. To avoid the influence of optical aberrations on the results, an ideal lens was employed in the simulation. The pitch of the lens elements was set at 0.31 mm. By tracing the corresponding LCD pixels in sequence, the luminance distributions of the odd and even viewpoints were obtained separately. We compared the luminance distribution based on the original structure using the conventional sub-pixel arrangement and a lens array with a slanted angle of 14.04° and the optimized structure using the ladder-type sub-pixel arrangement and a non-slanted lens array. Figure 7 and Fig. 8 present the corresponding results for the two systems, respectively, taking nine viewpoints as an example. Crosstalk can be quantified using Eq. (2).
where ${I_{current}}$ refers to the luminance distribution of the current viewpoint, and ${I_{noise}}$ refers to the total luminance distribution leaking from the adjacent viewpoints within the range of the current viewpoint (e.g., the areas marked by the red rectangle in Fig. 7(a) and Fig. 7(b) represent the luminance distribution of the 5th viewpoint and the luminance distribution leaking from the adjacent 4th and 6th viewpoints, respectively). It can be seen in Fig. 7 that, before structural optimization, both the illuminance distribution of the current viewpoint and the illuminance distribution leaking from adjacent viewpoints are observable from a particular viewpoint range, indicating that the crosstalk is relatively high. Using Eq. (2), the average crosstalk over the entire viewing angle is calculated to be 20.3%. However, by adopting the optimized structure, the illuminance distribution becomes more concentrated and only the illuminance distribution of the current viewpoint is detected within the respective viewpoint range. Thus, the average crosstalk over the entire viewing angle is successfully reduced to 0%. The experiment result of crosstalk simulation confirms that structural crosstalk is completely reduced with the use of the optimized structure. Taken together, these two experiments demonstrate the validity of the proposed method for structural crosstalk reduction.4. Optical crosstalk
4.1 Analysis
One of the basic optical properties of a lens element is the uniformity of the distribution of the emergent light, and it is also an important factor influencing the crosstalk of a light-field system. In an ideal situation, the light rays emitted from the LCD panel converge at a beam spot (voxel) on the HFS after passing through the lens elements and are then uniformly distributed in different spatial directions, as shown in Fig. 9(a). Because the intervals of the sub-pixels (Δh) are equal, the light distribution zones (LDZs) for these sub-pixels have the same width across the overall viewing range (Δw represents the width of the LDZ). However, in practical applications, due to aberrations in the lens elements, the emergent light rays may deviate from the expected path, resulting in a non-uniform light distribution (Fig. 9(b)). According to Snell’s law, for light rays with a larger incident height, these aberrations are worse. As a result, the widths of the LDZs corresponding to the sub-pixels are no longer consistent but rather increase with the viewing angle within each viewing zone.
Fig. 9. (a) Emergent light distribution in an ideal situation. (b) Emergent light distribution in a practical situation.
Figure 10 illustrates the relationship between the uniformity of the emergent light distribution and crosstalk in the process of viewpoint construction by taking the construction of the central viewing zone as an example. Under a uniform light distribution, the viewpoints can be constructed with high accuracy, as shown in Fig. 10(a). The light beams (marked in red, green, and blue) emitted from different lens elements intersect in the viewing plane. Because the widths of their LDZs are equal, they can accurately coincide within the expected viewpoint range. In this case, the perceived light beams at the constructed viewpoint all originate from the expected sub-pixels, which ensures crosstalk-free 3D perception. Figure 10(b) illustrates the process of viewpoint construction with a non-uniform light distribution. Because the LDZs corresponding to the light rays from different lenses have different widths, they cannot overlap accurately in the expected viewpoint range. By reverse-tracing the rays converging to the target viewpoint position (the tracing paths are represented by the black dotted lines), the practically perceived view at the viewpoint position can be determined, as shown by the areas marked by the black slanted lines. Figure 10(c) shows that both the current view and a portion of the adjacent views can be perceived at the target viewpoint position, which results in crosstalk.
Fig. 10. (a) Construction process for the viewpoints with a uniform light distribution. (b) Generation of crosstalk with a non-uniform light distribution. (c) Tracing results for light rays from the viewpoint position.
This type of crosstalk is referred to as optical crosstalk because it arises from the optical properties of the lens elements only. To the best of our knowledge, this is the first time that optical crosstalk caused by the non-uniformity of the light distribution has been revealed and analyzed. It is worth mentioning that similar forms of optical crosstalk also exist in other lenticular lens-based 3D displays. However, conventional 3D displays usually have a narrow viewing angle and a limited number of viewpoints, so their optical crosstalk is slight and has little influence on the 3D effect. In contrast, for 3D displays with a wide viewing angle and dense viewpoints, reducing optical crosstalk should be considered at the design stage.
4.2 Reduction method
In the present study, in order to suppress optical crosstalk and improve the 3D imaging quality of a TMLFD, a compound cylindrical lens structure with three aspheric surfaces was designed. Compared with a traditional cylindrical lens, the optimized compound aspherical lens can effectively optimize the aberrations, which ensures uniform light distribution in the viewpoint construction process. The aspheric model uses the base radius of the curvature and the conic constant. The aspheric surface formula is given in Eq. (3):
Fig. 11. (a) Layout of the optimized compound lens. (b) Corrected light distribution for the compound lens.
Table 2. Structural parameters for the optimized compound lens
Table 3. Parameters of the aspheric surface
Figure 11(b) illustrates the simulation results for the corrected light distribution based on the optimized compound lens elements. (Given the symmetry of the system, we only present the right side of the viewing range for clarity.) We can see that, after optimization, the emergent light rays are uniformly distributed and the widths of the LDZs in the central and marginal positions are mostly consistent. Figure 12 illustrates the change in the width of the LDZs before and after lens optimization. Compared with the standard cylindrical lens, the distribution of the emergent light after lens optimization was in good agreement with the ideal distribution across the entire viewing range.
Fig. 12. Change in the width of the LDZs with the viewing angle in the (a) central and (b) right-sided viewing zones.
In order to verify the effectiveness of the proposed method for the reduction of optical crosstalk, a crosstalk simulation experiment was conducted. In this experiment, we simulated the luminance distribution of the viewpoints using a standard lens and a compound aspheric lens. To avoid the influence of structural crosstalk, the slant angle of the lens was set at 0° and a ladder-type sub-pixel arrangement was adopted. The crosstalk across the overall viewing range is obtained using Eq. (2) and illustrated in Fig. 13. It can be seen that the crosstalk of the light-field display before optimization gradually increases with an increase in the viewing angle in each viewing zone. The maximum crosstalk is 17.3% at 60 degrees. Figure 14 shows the corresponding luminance distribution at the 60-degree position before optimization. By employing the optimized compound lens, the crosstalk is almost eliminated from the overall viewing range. The crosstalk at 60 degrees is reduced to 0.26%. The corresponding luminance distribution at the 60-degree position after optimization is presented in Fig. 15. Based on these results, this experiment demonstrated the validity of the proposed method for the reduction of optical crosstalk.
Fig. 13. Change in crosstalk with the viewing angle in the (a) central and (b) right-sided viewing zones.
Fig. 14. Luminance distribution of the (a) odd and (b) even viewpoints with a standard lens at the position of the maximum viewing angle.
Fig. 15. Luminance distribution of the (a) odd and (b) even viewpoints with a compound aspheric lens at the position of the maximum viewing angle.
5. Experiments and discussion
In order to verify the improvement in the image quality based on the proposed crosstalk reduction methods, practical experiments and subsequent analysis were conducted. In Section 5.1, SSIM computational simulation is adopted to quantitatively examine the improvement in the 3D image quality based on the proposed crosstalk reduction methods. In Section 5.2, a series of optical experiments are conducted to evaluate the improvement in viewing quality of the reconstructed light-field images as perceived by the human eye. In Section 5.3, a tabletop 3D EST system based on the improved TMLFD is developed.
5.1 SSIM computational simulation experiments
To quantitatively examine the improvement in the 3D image quality, we first simulated the 3D images as perceived by the human eye from different perspectives before and after the implementation of the crosstalk reduction methods. Because the 3D image perceived by the human eye is composed of a series of generated voxels, simulated images can be obtained by calculating all of the voxels for each perspective. The SSIM evaluation method was adopted to quantify the similarity between the simulated images and their corresponding original images (i.e., the ideal images). The SSIM values of the viewpoints with different perspectives were calculated. A higher SSIM value indicated that the image perceived by the human eye in the simulation had a stronger similarity with the original image.
The results of the simulation are illustrated in Fig. 16. Figure 16(a) presents the original light-field images of a 3D urban terrain scene from three different perspectives, and Fig. 16(b) displays the corresponding depth maps. Figure 16(c), (e), and (g) present three groups of simulated light-field images, and Fig. 16(d), (f), and (h) show the corresponding SSIM values. In the first group of simulations, a conventional LCD sub-pixel arrangement and a standard cylindrical lens array were employed. The slant angle between the lens array and the LCD panel was set to 14.04°. As shown in Fig. 16(d), the simulated light-field images had a low similarity due to the effect of crosstalk (average SSIM value = 0.8027), especially for buildings with a large depth. In the second group of simulations, the ladder-type sub-pixel arrangement was employed, and the slant angle of the lens array was 0°. A comparison of Fig. 16(d) and (f) reveals that the SSIM values were significantly higher (average = 0.8969), especially for the center perspective. These results verify the effectiveness of the proposed method for the reduction of structural crosstalk. In the third group of simulations, in order to further improve the image quality of the marginal perspectives (i.e., the left 60° and the right 60°), a compound aspheric lens array was employed to reduce the optical crosstalk. It can be seen in Fig. 16(h) that the SSIM values of the two marginal perspectives increased to 0.9417 and 0.9445, respectively. The average SSIM value increased to 0.9443, which confirms that the optical crosstalk was successfully suppressed and that the simulation images were extremely close to the ideal images. The simulation results suggest that the two proposed crosstalk reduction methods significantly improve the 3D image quality.
Fig. 16. Computational simulation results for a 3D urban terrain scene from different perspectives. (a) Original light-field images from different perspectives. (b) Depth maps from different perspectives. (c) Simulated images of the light-field display with a conventional LCD sub-pixel arrangement and a standard cylindrical lens array. (d) SSIM values corresponding to Fig. 16(c). (e) Simulated images of the light-field display with the ladder-type LCD sub-pixel arrangement and a standard cylindrical lens array. (f) SSIM values corresponding to Fig. 16(e). (g) Simulated images of the light-field display with both the ladder-type LCD sub-pixel arrangement and a compound aspheric lens array. (h) SSIM values corresponding to Fig. 16(g).
5.2 Light-field reconstruction experiments
A series of light-field reconstruction experiments of a 3D urban terrain scene based on the proposed TMLFD system was conducted. The maximum depth of the 3D scene was 22 cm (from the top of the buildings to the ground). Figure 17(a) presents the practical reconstructed 3D images from different perspectives produced by employing a conventional LCD panel and a standard cylindrical lens array. The diagonal of the LCD panel was 32 inches and the resolution was 3840×2160 (4K). The pitch of the lens elements was 0.31 mm, and the focal length was 0.44 mm. The lens array was installed with a slant angle of 14.04° with respect to the LCD panel. We can see from the results that parts of the buildings with a large depth (17–22 cm) suffered from severe aliasing, which degrades the depth quality and lowers the visual experience.
Fig. 17. Light-field reconstruction effects for different perspectives of a 3D urban terrain scene. The inserts are the magnified images. (a) Reconstructed images of the light-field display with a conventional LCD sub-pixel arrangement and a standard cylindrical lens array. (b) Reconstructed images of the light-field display with the ladder-type LCD sub-pixel arrangement and a standard cylindrical lens array. (c) Reconstructed images of the light-field display with both the ladder-type LCD sub-pixel arrangement and a compound aspheric lens array.
To verify the effectiveness of the proposed structural crosstalk reduction method, we replaced the conventional LCD panel with a 32-inch 4K LCD panel based on the ladder-type sub-pixel arrangement, with the lens array installed horizontally on the panel. The reconstructed 3D images are presented in Fig. 17(b). A comparison of Fig. 17(a) and (b) shows that the redesigned system produced clearer reconstructed images and offered more detailed depth information after the structural crosstalk had been reduced, especially for the center perspective. However, some image aliasing remained in the marginal perspectives (i.e., the left 60° and the right 60°). According to the previous analysis, this remaining image aliasing can be attributed to optical crosstalk. In order to reduce this crosstalk, a compound aspheric lens array was employed in the TMLFD prototype. The reconstructed 3D images are shown in Fig. 17(c). The results illustrate that the remaining aliasing was almost completely eliminated for all perspectives, and the overall depth information for the buildings was accurately reproduced. The complete light-field reconstruction effect across the entire viewing range is shown in Visualization 1. These optical experimental results are consistent with the SSIM computational simulations, proving that the crosstalk in the light-field display was successfully suppressed and that the viewing quality of the 3D images perceived by the human eye was significantly improved.
5.3 Application of the improved TMLFD in an EST system
A tabletop 3D EST system that could be employed in air-traffic control and geographic information systems was fabricated based on the proposed TMLFD. Figure 18(a) shows the exterior of the EST. The diagonal of the viewing area was 32 inches. The display screen was slanted at a 20° angle to the ground. Observers could perceive clear 3D images and depth information with a wide 120° viewing angle. By rendering 3D images with backward ray tracing [37], a real-time dynamic and interactive air-traffic control simulation system based on the EST was realized at a frame rate of 25 fps, as shown in Fig. 18(b) and Visualization 2. The movement and zoom operations of the terrain were controlled using the movement of a mouse and its wheel, respectively, while the flight path of the aircraft was controlled via mouse click. Combined with satellite mapping technology, the 3D visualization of geographic information based on the EST was also realized. Figure 19 and Visualization 3 present the 3D reconstructed images of complicated mountain terrain from different perspectives. The outline of the mountains and the changes in the terrain topography were accurately reproduced.
Fig. 18. (a) Exterior of the EST. Observers can perceive clear 3D images and depth information with a wide 120° viewing angle. (b) Real-time dynamic and interactive 3D light-field display of an air-traffic control simulation system (see Visualization 2).
Fig. 19. Reconstructed 3D images of complicated mountain terrain from different perspectives (see Visualization 3).
6. Conclusion
In this paper, the mechanisms responsible for the generation of two types of crosstalk in current TMLFD systems were identified and analyzed. Structural crosstalk was attributed to the slanted lenticular lens array using in the TMLFD, while optical crosstalk occurred due to the non-uniformity of the distribution of the emergent light from the lens elements. Crosstalk can cause image aliasing and disturb the depth information. In order to improve the depth perception and image quality, a novel ladder-type LCD sub-pixel arrangement and a compound lens with three aspheric surfaces were introduced to ameliorate the two types of crosstalk. Crosstalk simulation experiments demonstrated the validity of the proposed methods, while SSIM computational simulations and light-field reconstruction experiments confirmed that aliasing was successfully eliminated and the depth quality was significantly improved over the entire viewing range. A tabletop EST based on the proposed TMLFD was also fabricated and tested.
Funding
National Key Research and Development Program of China (2017YFB1002900); National Natural Science Foundation of China (61705014, 61905019); Fundamental Research Funds for the Central Universities (2019PTB-018, 2019RC13).
Disclosures
The authors declare no conflicts of interest. This work is original and has not been published elsewhere.
References
1. Q. Ma, L. Cao, Z. He, and S. Zhang, “Progress of three-dimensional light-field display [Invited],” Chin. Opt. Lett. 17(11), 111001 (2019). [CrossRef]
2. W. Song, Q. Cheng, P. Surman, Y. Liu, Y. Zheng, Z. Lin, and Y. Wang, “Design of a light-field near-eye display using random pinholes,” Opt. Express 27(17), 23763–23774 (2019). [CrossRef]
3. Z. Qin, J. Wu, P. Chou, Y. Chen, C. Huang, N. Balram, and Y. Huang, “Revelation and addressing of accommodation shifts in microlens array-based 3D near-eye light field displays,” Opt. Lett. 45(1), 228–231 (2020). [CrossRef]
4. Z. Zhao, J. Liu, Z. Zhang, and L. Xu, “Bionic-compound-eye structure for realizing a compact integral imaging 3D display in a cell phone with enhanced performance,” Opt. Lett. 45(6), 1491–1494 (2020). [CrossRef]
5. J. Wen, X. Yan, X. Jiang, Z. Yan, F. Fan, P. Li, Z. Chen, and S. Chen, “Integral imaging based light field display with holographic diffusor: principles, potentials and restrictions,” Opt. Express 27(20), 27441–27458 (2019). [CrossRef]
6. H. Zhang, H. Deng, J. Li, M. He, D. Li, and Q. Wang, “Integral imaging-based 2D/3D convertible display system by using holographic optical element and polymer dispersed liquid crystal,” Opt. Lett. 44(2), 387–390 (2019). [CrossRef]
7. B. Chen, L. Ruan, and M. Lam, “Light field display with ellipsoidal mirror arrray and single projector,” Opt. Lett. 27(15), 21999–22016 (2019). [CrossRef]
8. T. Nakamura and M. Yamaguchi, “Rapid calibration of a projection-type holographic light-field display using hierarchically upconverted binary sinusoidal patterns,” Appl. Opt. 56(34), 9520–9525 (2017). [CrossRef]
9. X. Gao, X. Sang, W. Zhang, and B. Yan, “Viewing resolution and viewing angle enhanced tabletop 3D light field display based on voxel superimposition and collimated backlight,” Opt. Commun. 474, 126157 (2020). [CrossRef]
10. X. Yu, X. Sang, X. Gao, B. Yan, D. Chen, B. Liu, L. Liu, C. Gao, and P. Wang, “360-degree tabletop 3D light-field display with ring-shaped viewing range based on aspheric conical lens array,” Opt. Express 27(19), 26738–26748 (2019). [CrossRef]
11. L. Ni, Z. Li, H. Li, and X. Liu, “360-degree large-scale multiprojection light-field 3D display system,” Appl. Opt. 57(8), 1817–1823 (2018). [CrossRef]
12. X. Xia, X. Zhang, L. Zhang, P. Surman, and Y. Zheng, “Time-multiplexed multi-view three-dimensional display with projector array and steering screen,” Opt. Express 26(12), 15528–15538 (2018). [CrossRef]
13. S. Yoshida, “fVisiOn: 360-degree viewable glasses-free tabletop 3D display composed of conical screen and modular projector arrays,” Opt. Express 24(12), 13194–13203 (2016). [CrossRef]
14. Y. Takaki and S. Uchida, “Table screen 360-degree three-dimensional display using a small array of high-speed projectors,” Opt. Express 20(8), 8848–8861 (2012). [CrossRef]
15. Y. Takaki and J. Nakamura, “Generation of 360-degree color three-dimensional images using a small array of high-speed projectors to provide multiple vertical viewpoints,” Opt. Express 22(7), 8779–8789 (2014). [CrossRef]
16. D. Teng, Y. Xiong, L. Liu, and B. Wang, “Multiview three-dimensional display with continuous motion parallax through planar aligned OLED microdisplays,” Opt. Express 23(5), 6007–6019 (2015). [CrossRef]
17. E. Chen, J. Cai, X. Zeng, S. Xu, Y. Ye, Q. F. Yan, and T. Guo, “Ultra-large moiré-less autostereoscopic threedimensional light-emitting-diode displays,” Opt. Express 27(7), 10355–10369 (2019). [CrossRef]
18. H. Sasaki, N. Okaichi, H. Watanabe, M. Kano, M. Miura, M. Kawakita, and T. Mishina, “Color moiré reduction and resolution enhancement of flat-panel integral three-dimensional display,” Opt. Express 27(6), 8488–8503 (2019). [CrossRef]
19. B. Javidi, A. Carnicer, J. Arai, T. Fujii, H. Hua, H. Liao, M. Martínez-Corral, F. Pla, A. Stern, L. Waller, Q. H. Wang, G. Wetzstein, M. Yamaguchi, and H. Yamamoto, “Roadmap on 3D integral imaging: sensing, processing, and display,” Opt. Express 28(22), 32266–32293 (2020). [CrossRef]
20. Z. Yan, X. Yan, Y. Huang, X. Jiang, Z. Yan, Y. Liu, Y. Mao, Q. Qu, and P. Li, “Characteristics of the holographic diffuser in integral imaging display systems: A quantitative beam analysis approach,” Opt. Lasers Eng. 139, 106484 (2021). [CrossRef]
21. Z. Wang, G. Lv, Q. Feng, A. Wang, and H. Ming, “Resolution priority holographic stereogram based on integral imaging with enhanced depth range,” Opt. Express 27(3), 2689–2702 (2019). [CrossRef]
22. Z. Yan, X. Yan, X. Jiang, H. Gao, and J. Wen, “Integral imaging based light field display with enhanced viewing resolution using holographic diffuser,” Opt. Commun. 402, 437–441 (2017). [CrossRef]
23. F. Shateri, S. Behzadfar, and Z. Kavehvash, “Improved depth resolution and depth-of-field in temporal integral imaging systems through non-uniform and curved time-lens array,” Opt. Express 28(5), 6261–6276 (2020). [CrossRef]
24. B. Liu, X. Sang, X. Yu, X. Gao, L. Liu, C. Gao, P. Wang, Y. Le, and J. Du, “Time-multiplexed light field display with 120-degree wide viewing angle,” Opt. Express 27(24), 35728–35739 (2019). [CrossRef]
25. C. H. Lee, G. W. Seo, J. H. Lee, T. H. Han, and J. G. Park, “Auto-stereoscopic 3D displays with reduced crosstalk,” Opt. Express 19(24), 24762–24774 (2011). [CrossRef]
26. X. Li, Q. Wang, Y. Tao, D. Li, and A. Wang, “Crosstalk reduction in multi-view autostereoscopic three-dimensional display based on lenticular sheet,” Chin. Opt. Lett. 9(2), 021001–21003 (2011). [CrossRef]
27. X. Gao, X. Sang, X. Yu, P. Wang, X. Cao, L. Sun, B. Yan, J. Yuan, K. Wang, C. Yu, and W. Dou, “Aberration analyses for improving the frontal projection three-dimensional display,” Opt. Express 22(19), 23496–23511 (2014). [CrossRef]
28. J. L. Feng, Y. J. Wang, S. Y. Liu, D. C. Hu, and J. G. Lu, “Three-dimensional display with directional beam splitter array,” Opt. Express 25(2), 1564–1572 (2017). [CrossRef]
29. I. Tsirlin, L. M. Wilcox, and R. S. Allison, “The effect of crosstalk on the perceived depth from disparity and monocular occlusions,” IEEE Trans. Broadcast. 57(2), 445–453 (2011). [CrossRef]
30. B. R. Lee, J. Y. Son, O. O. Chernyshov, H. Lee, and I. K. Jeong, “Color moiré simulations in contact-type 3-D displays,” Opt. Express 23(11), 14114–14125 (2015). [CrossRef]
31. V. Saveljev and S.-K. Kim, “Experimental observation of moiré angles in parallax barrier 3D displays,” Opt. Express 22(14), 17147–17157 (2014). [CrossRef]
32. Z. Zhuang, P. Surman, L. Zhang, R. Rawat, S. Wang, Y. Zheng, B. Yan, and X. Sun, “Moiré-reduction method for slanted-lenticular-based quasi-three-dimensional displays,” Opt. Commun. 381, 314–322 (2016). [CrossRef]
33. Y. Kim, G. Park, J.-H. Jung, J. Kim, and B. Lee, “Color moiré pattern simulation and analysis in three-dimensional integral imaging for finding the moiré-reduced tilted angle of a lens array,” Appl. Opt. 48(11), 2178–2187 (2009). [CrossRef]
34. X. F. Li, Q. H. Wang, and D. H. Li, “Image processing to eliminate crosstalk between neighboring view images in three-dimensional lenticular display,” J. Disp. Technol. 7(8), 443–447 (2011). [CrossRef]
35. M. Zhou, H. Wang, W. Li, S. Jiao, T. Hong, S. Wang, X. Sun, X. Wang, J. Kim, and D. Nam, “A Unified Method for Crosstalk Reduction in Multiview Displays,” J. Disp. Technol. 10(6), 500–507 (2014). [CrossRef]
36. S. K. Kim, K. H. Yoon, S. K. Yoon, and H. Ju, “Parallax barrier engineering for image quality improvement in an autostereoscopic 3D display,” Opt. Express 23(10), 13230–13244 (2015). [CrossRef]
37. S. Xing, X. Sang, X. Yu, C. Duo, B. Pang, X. Gao, S. Yang, Y. Guan, B. Yan, J. Yuan, and K. Wang, “High efficient computer-generated integral imaging based on the backward ray-tracing technique and optical reconstruction,” Opt. Express 25(1), 330–338 (2017). [CrossRef]
