Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Lissajous MEMS laser beam scanner with uniform and high fill-factor projection for augmented reality display

Open Access Open Access

Abstract

MEMS Laser beam scanning (LBS) has been identified as a key advancement for augmented reality (AR) displays due to its ability to create compact optical systems that generate bright, high-contrast images with minimal heat dissipation. This innovation can be attributed to the focus-free, efficient light-on-demand pixel projection mechanisms integral to LBS. The LBS, specifically in Lissajous-mode, outperforms the raster-mode in terms of larger scan angles and stability to external vibrations, by leveraging a MEMS mirror operating at bi-axial resonance. However, it tends to be hampered by small mirror aperture, low fill-factor, and inconsistent uniformity of image projection. In this research, a unique gimbal-less Lissajous MEMS scanner was proposed. It employs a bi-axial high frequency of 12,255 Hz and 7,182 Hz to achieve a resolution of 640 × 360 pixels and a video refresh rate of 57 Hz, all while maintaining a high image fill factor of 85.11%. The robust structure of the mirror is proven to sustain stable scanning under broad spectrum of external vibration disturbance up to 2,000 Hz. Furthermore, the large mirror diameter of 2 mm improves refined pixel projection and increased optical etendue for exit pupil. Mathematic model of Lissajous pixel-cells and image reconstruction simulation were established to validate the LBS's ability to generate a uniform and densely pixelated visual effect that fits for typical AR head-up display (AR-HUD). In a pioneering move, performance metric of figure-of-merit was defined to evaluate AR light-engines using varied picture-generation techniques, laying a foundation for guiding future AR system development.

© 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

Augmented reality (AR), which seamlessly overlays virtual data onto real-world environments, is ushering in a profound shift in human perception and interaction. Consequently, the creation of high-performance display devices that mix virtual digital contents with real-world experiences has become a focal point in the field of AR. The key requirement for these systems is the development of a compact, bright, high-contrast light engine for displays that generates minimal heat or less power consumption, while stable enough to sustain external mechanical shock or disturbance [1,2].

On one hand, AR displays can be generally classified into panel-based and scanning display systems. Panel-based displays [3], such as digital light processing (DLP) and liquid crystal displays (LCD), modulate light intensity through a global pixel lighting and then pixel-by-pixel shadowing. As illustrated in Fig. 1, for a use case of AR display, although the image display occupation usually needs only 30% pixels on, the whole matrix of panel illuminated at all times and results in limited luminous efficiency, low image contrast and significant heat generation. Moreover, these technologies require multiple projection lenses and inevitably enlarges the optical system. In contrast, micro-electro-mechanical systems (MEMS) laser beam scanning (LBS) displays [4,5] have become an attractive alternative due to their focus-free, highly efficient light-on-demand projection mechanisms. LBS leverages a MEMS mirror to modulate laser pulses temporally and spatially, forming individual pixels of the image sequentially. The pixel-level display of LBS provides high optical efficiency and contrast by switching the laser on/off with pixel-level precision, reducing waste heat generation. Furthermore, the unique focus-free projection of LBS negates the need for projection lenses, significantly shrinking the optical system.

 figure: Fig. 1.

Fig. 1. Schematic of (a) panel-based and (b) LBS light-engine for AR display.

Download Full Size | PDF

In addition to the efficient on-demand lighting display mechanism, achieving LBS for AR displays necessitates specific optical characteristics. These include a large optical aperture to achieve finely projected pixels and a generous etendue for the exit pupil. Furthermore, it requires the integration of high-frequency bi-axial resonance to ensure resilience against vibration disturbances and the establishment of a scanning trajectory with a consistently high fill-factor to minimize any flicker effects. In detail, human visual acuity and color detection is concentrated in the center of approximately 17 degrees of the eye's Field of View (FoV) [6]. A typical Maxwellian near-eye AR displays have been designed around this principle, using eye-tracking techniques to dynamically project a small FoV and eyebox area onto the retina, thus creating a larger pupil exit and an expansive visible view angle [1]. To meet the etendue requirements for LBS system, a mirror optical aperture of at least 2 mm and a scanning FoV exceeding 30 degrees are necessary to encompass the ∼4 mm diameter of the human pupil and the 17-degree span of the fovea at each exit pupil position [7].

For those typical AR applications, such as automotive AR head-up display (AR-HUD), display light-engine operates accompanied with 0∼2000Hz external vibrations from road conditions [8]. In contrast to the LBS with raster-mode that operate slow-axis scan at a low frequency of frame rate, LBS of Lissajous-mode operate a MEMS mirror at bi-axial high-frequency eigenmode. Their essentially rigid torsional structures enable a high mechanical stability against external shock and vibrations [9], which make them preferable as a strong candidate for automotive AR applications. Additionally, employing a MEMS laser beam scanner operating in Lissajous mode, may offer an ideal closely matched fast- and slow-axis frequencies (with a small fast-to-slow axis ratio) help ensure a uniform scanning trajectory across the entire image [10].

Nevertheless, existing Lissajous LBS usually featured a small mirror diameter less than 2 mm, while remaining large fast-to-slow axis frequency ratios of bi-axial scanning mode. Since the bi-axial frequency ratio of MEMS mirrors determines the Lissajous trajectory repetition rates and scanned line densities, they fail to deliver high frame rates and image fill factors with moderate resolution, concurrently [11]. In brief, the reported Lissajous mirrors with a combination of large fast-to-slow axis frequency ratios originate from the gimbal-type mirror design. The gimbal structure itself, as well as the need to enlarge the mirror diameter, results in a substantial increase in rotational inertia for slow-axis motion. Consequently, this leads to a lower frequency resonance in the slow axis (typically less than 2 kHz) and a significant fast-to-slow axis frequency ratio. This low-frequency slow-axis motion falls within the random vibration of automotive applications, raising the risk of sensitivity to external vibration disturbance. From a mathematical perspective of Lissajous scanning, it also leads to a non-uniform scanning trajectory and a noticeable flicker effect, particularly when there is relative motion between the viewer's eyesight and the scanning image [12,13].

On the other hand, the scanning field-of-view (FoV), brightness and heat generation control of LBS light-engine are the most critical specifications for practical AR applications [5]. Performance metrics index to in-depth evaluate these comprehensive properties of different light-engines is crucial for guiding the development of LBS display technologies towards specific scenarios of AR applications. However, to the best of our knowledge, few research has been reported on this.

In this paper, a unique gimbal-less Lissajous MEMS LBS design was proposed to deliver high fill factors and frame rates. In combination with a large mirror size of 2 mm to deliver refined pixels projection and larger exit pupil, the tailored bi-axial high-frequency resonance, by optimizing the mirror structure, results in an ideal scanning frequency ratio and thus enhanced laser scanning stability against external vibration disturbance. Mathematic model of Lissajous pixel-cells and its image reconstruction simulation have been constructed to demonstrate a great application potential for automotive AR head-up displays (AR-HUD), effectively suppressing flicker visual effects and withstanding a broad spectrum of comparable road-induced vibrations with normal operation. A figure of merit (FoM) calculation methodology was defined for AR display, in terms of FoV, resolution, optical efficiency, heat generation control, etc. Various light-engines are compared by indexing the FoM values to give a preliminary assessment of off-the-shelf display techniques and guiding future AR light-engines developments.

2. Experimental and methods

2.1 Modulation principles of Lissajous scanning

The Lissajous scanning pattern is dominated by two sinusoidal signals of different frequencies for the fast- and slow-axis, as the control equations shown in Eq. (1) [14]:

$$\left\{ {\begin{array}{{c}} {X = {A_x}\sin ({2\pi {f_x}t + {\varphi_x}} ){\; }}\\ {Y = {A_y}\sin ({2\pi {f_y}t + {\varphi_y}} )} \end{array}} \right.$$
where X and Y denote the trajectory of the laser beam at x- and y-coordinate. A is the scanning amplitude at the peak-to-peak driving voltage, f indicates the scanning frequency, t is the time, and $\varphi $ is the phase value of each axis. The image comprises numerous pixel cells, each hosting Lissajous stripes. By overlaying the Lissajous patterns onto the image, the distribution of these trajectories on each pixel cell becomes apparent. Within Fig. 2(a), certain pixel cells exhibit an absence of stripes passing through them, leading to a lack of information, while others feature multiple stripes crossing their boundaries. To address this, the time interval Δt of the Lissajous stripes closest to the center within the pixel cell was selected and replace the entire cell with this selected time-interval of line segment. Figure 2(b) demonstrates how Δt corresponds to the time taken to pass through the first pixel cell at the center velocity of the Lissajous trajectory. The pixel cells are characterized by their small size and approximate horizontal scanning direction, allowing to estimate the average scanning speed v in the x-axis direction to be utilized to approximates the speed of the scanning line as it passes through the first cell. The calculation is denoted as Eq. (2).:
$$v \approx X^{\prime}{|_{t = 0}} = 2\pi {f_x}{A_x}\textrm{cos}({2\pi {f_x}t} )\; {|_{t = 0}} = 2\pi {A_x}{f_x}$$

Therefore Δt can be calculated by Eq. (3):

$$\Delta t \approx \frac{{2{A_x}}}{{v{R_x}}} = \frac{{2{A_x}}}{{2\mathrm{\pi }{A_x}{f_x}{R_x}}} = \frac{1}{{\pi {f_x}{R_x}}}$$
where ${R_x}$ denotes the resolution of the x-direction. Furthermore, the location of the line segment nearest to the central point varies within each pixel cell, which induces the extension of the scanning line segment corresponding to Δt into the neighboring pixel point. Given the brevity of the time interval Δt, its impact on the overall image remains insignificant and can thus be disregarded.

 figure: Fig. 2.

Fig. 2. (a) Schematic of Lissajous scanning trajectory overlapping with pixel cells; (b) selection of scanning time interval and line segment for pixel cell.

Download Full Size | PDF

2.2 MEMS mirror design concepts

For a typical MEMS scanner with a circular mirror plate supported by a couple of torsion bars, the resonant frequency f can be calculated according to the following equations [15]:

$$f = \frac{1}{{2\pi }}\sqrt {\frac{K}{I}} $$
$$I = {\; }\frac{{\rho \pi {R^4}t}}{4} + \frac{{\rho \pi {R^2}{t^3}}}{{12}}$$
$$K = \frac{{2Gw{t^3}}}{L}\left[ {\frac{1}{3} - 0.21\frac{t}{w}\left( {1 - \frac{1}{{12}}\frac{{{t^4}}}{{{w^4}}}} \right)} \right]$$
Where R is the mirror radius, ρ indicates the density of silicon, G indicates the shear modulus of silicon, t, w, L correspond the thickness, width, and length of torsion bar, respectively. Based on the mirror inertia I and torsion bar spring constant K, the resonant frequency can be deduced. To attain high-frequency bi-axial resonance in Lissajous scanning, it is vital to enhance the slow-axis frequency in the torsional mode by reducing its inertial moment. As depicted in Fig. 3, a novel gimbal-less structure was adopted in the design of the proposed MEMS mirror. The interior torsion bar, which allows horizontal deflection (x-direction) of the mirror, functions as the fast axis. Conversely, the exterior torsion bar serves as the slow axis, enabling vertical deflection (y-direction). The inertia moment can be described in Eq. (5), which proportional to the cubic of rotation mass thickness and the quartic of mass radius. Generally, the presence of a gimbal is a thick and large ring-shape structure that supported by slow-axis to isolate both axes motion, result in a substantial increase in rotational inertia of the slow axis. Consequently, this leads to a lower frequency resonance in the slow axis and a significant disparity in frequency between the fast and slow axis. In contrast, the proposed gimbal-less Lissajous scanner design incorporates self-similar fractal geometric structures for both axes, utilizing them as torsional beams and integrated into a monolithic support structure. This innovative approach eliminates the need for a traditional thick gimbal structure to support and isolate the resonances of both axes. Without intermediate large ring-shape gimbal, the mass for slow-axis loading is dramatically decreased, thereby reducing rotational inertia for the slow-axis motion.

The MEMS mirror facilitates the use of a straightforward external actuator through monolithic assembly with a Lead Zirconate Titanate (PZT) bulk sheet, simplifying the driving process compared to other actuators for obtaining frequency excitation from piezoelectric thin-film deposition on both torsional axes. Moreover, the mirror with 2 mm in diameter, accommodates for refiner pixels projection and increased etendue for larger exit pupil, thereby enhancing resolution and eyebox. Furthermore, the serpentine designs of the torsion bars on both axes allow for optimal stress distribution and relief against large shear stresses at maximum scanning angles. As depicted in Fig. 3, finite element simulation of COMSOL Multiphysics was utilized to analyze the modal and frequency-domain behaviors of the piezoelectrically actuated MEMS mirror. The mirror exhibited high eigen-frequencies of fast- and slow-axis torsional modes at 12,245 Hz and 7,651 Hz respectively, with a comparatively low frequency ratio of 1.6. Despite the large mechanical scanning angles of the fast-axis (±7.7°) and the slow-axis (±3.9°), the maximum von Mises stresses for the fast- and slow-axis were still significantly lower than the single-crystal silicon limit of 2.0 GPa, measuring at 1.4 GPa and 0.5 GPa respectively [16].

 figure: Fig. 3.

Fig. 3. (a) 3D model schematic of the proposed MEMS mirror. (b) SEM image of the as-fabricated MEMS mirror. Finite-element simulation for modal analysis of (c) slow-axis, (d) fast-axis, and von Mises stress analysis of (e) slow-axis, (f) fast-axis with frequency-domain piezoelectric actuation.

Download Full Size | PDF

2.3 Device fabrication

The fabrication process of the proposed MEMS mirror, depicted in Fig. 4, comprises the following steps: (a) The process initiated with silicon-on-insulator (SOI) wafers, comprising a 100 µm-thick device layer, 1 µm-thick buried oxides (BOX), and 300 µm-thick handle layers. 500 nm thick SiO2 layer was added to both surfaces to function as a hard mask for silicon etching. (b) Photolithography was employed to define the mirror geometric region. (c) Subsequently, the oxide layer was patterned to serve as a shadow mask through BHF etching. (d) Deep reactive ion etching (DRIE) was applied to the device layer until the BOX stop layer was reached, which established the basic structures of the mirror plate and torsion bars. (e-g) A backside DRIE was carried out on the handle layer to develop the movable structure suspension of mirror. (h) The oxide layer was selectively removed to release the underlying structures. (i) Finally, a PZT sheet actuator was assemble with the mirror substrate to constitute a monolithic device.

 figure: Fig. 4.

Fig. 4. Microfabrication process flow of the proposed MEMS mirror.

Download Full Size | PDF

2.4 Measurement system

Figure 5 presents both a schematic and photograph of the laser scanning measurement system. The MEMS mirror was strategically oriented at an inclination of 45°. To ensure the reflected laser spot aligned perpendicularly with the screen, the modulated laser was directed onto the mirror plate at an angle of θ=45°. The interrelation of the parameters d, L, and θopt is elucidated through Eq. (7):

$${\theta _{opt}} = 2{\tan ^{ - 1}}\left( {\frac{L}{{2d}}} \right)$$
where $d$=160 mm indicates the vertical distance between the mirror and the screen. L denotes the length of the projected laser lines and θopt is the full optical scanning angle.

 figure: Fig. 5.

Fig. 5. (a) Schematic and (b) photograph of the scanning measurement system.

Download Full Size | PDF

3. Results and discussions

3.1 Characterization of MEMS mirror

The scanning performance of MEMS mirror was assessed through the amplification of sinusoidal drive signals by a voltage amplifier, which were subsequently loaded onto the bulk PZT actuators. As depicted in Fig. 6(c) and (d), the PZT actuator operated at the resonant frequency of 7,182 Hz for the slow axis by sweeping the actuation voltage from 1 V to 25 V. In a similar vein, the fast axis operated at a resonant frequency of 12,242 Hz as the actuation voltage was varied from 6.48 V to 58.40 V. The fast axis achieved a maximum full optical scanning angle θopt of 28.40°, while remaining a Full Width at Half Maximum (FWHM) of 65 Hz and Q-factor of 223 at a peak-to-peak actuation voltage of 58.40 V. In parallel, the slow axis reached a maximum θopt of 16.36°, while exhibiting a FWHM of 22 Hz and Q-factor of 299 at 25 V. It is noting that, manufacturing errors that occurs during practical device fabrication may lead to slight variances between the actual frequency performance of the device and the values of finite-element simulation. Both axes demonstrated appreciable linear scanning behavior across the range of drive voltages below 60 V. Further examination of the linearity was undertaken by scrutinizing the association between the bi-axial central resonant frequency and θopt across various peak-to-peak driving voltage ranges, as shown in Fig. 6(a) and (b). The maximum θopt was maintained by manually modulating the drive signal frequency at each drive voltages, as the bi-axial scanning of Lissajous trajectory were modulated as displayed in Fig. 7. As the driving voltage changed, both axes exhibited negligible shifts in central resonant frequency. It was also found that the relation between the optical scanning angle and actuation voltage broke away from the linear relation at a high voltage near 60 V. This can be owing to the torsional structure undergoes a substantial increase in torsional angle under high-voltage actuation, eventually reaching the nonlinearity region attributed to mechanical nonlinear stiffness. Mechanical structures undergoing significant transformations are susceptible to geometrical nonlinearities, often exhibiting either hardening or softening behavior. Deformation-induced softening in spring nonlinearity is a characteristic frequently observed in serpentine-type beams. This phenomenon can be attributed to the inherent ambiguity in defining the axis of rotation in serpentine-like torsional beam structures, in contrast to conventional bar-like beams. Consequently, it leads to coupling with translational stress, resulting in a reduction of apparent stiffness and an increase in the angle [17].

 figure: Fig. 6.

Fig. 6. Relationship between scan angle and resonant frequency of (a) fast-axis and (b) slow-axis under varied actuation voltages. (c) and (d) indicate the linear behavior of scanning angle increasement versus driving voltage at fixed resonant frequency.

Download Full Size | PDF

 figure: Fig. 7.

Fig. 7. Modulation of MEMS mirror with bi-axial scanning.

Download Full Size | PDF

The mechanical stability of MEMS mirror against external vibrational disturbance is crucial for its optimal functionality, particularly within the typical 0∼2000Hz random vibrational that may encounter in automotive AR applications [18]. Generally, external vibrations can significantly affect the operating frequency of MEMS mirror resonance, leading to undesirable consequences such as line splitting of scanning trajectories, inconsistencies in pixel placement, or amplitude reductions [19]. To quantitatively verify the stability of the device, we employed two distinct methods:

  • (a) Laser Stripe Projection Method: laser scanning stripes were projected onto a surface, ensuring a precise synchronization between the laser modulation frequency and the fast-/slow-axis frequency. Thus, any deviations in the MEMS operating frequency would result in disordered or varying stripe patterns. Horizontal stripes were generated by setting the laser frequency as multiples of the fast axis driving frequency value. Adjusting these multiples enables to tune the number of stripes. In the experiments, the fast-axis frequency was set at 12,240 Hz and the slow axis at 7,180 Hz. The laser was driven at 50 times of fast-axis frequency, generating precisely 25 stripes. Then, the MEMS mirror was subjected to vibration disturbances at frequencies ranging from 800 Hz to 2,000 Hz using an external bulk PZT actuator. Remarkably, the test images showed no obvious disorder or fluctuations, highlighting the robustness of proposed MEMS design against external excitation with negligible operating frequency shift.
  • (b) Structural Similarity Index (SSIM) Analysis [20]: To quantitatively assess the impact of external vibrational noise at the pixel level, comparison of SSIM for identical laser stripes in the presence and absence of vibration disturbance, are conducted as shown in Fig. 8. First, the simulations demonstrated that even a slight frequency shift of 0.002 Hz that induced by vibrational noise could lead to sawtooth-like scanning line behavior, which matched the experimental observation in laser stripe projections under vibrations exceeding 1900Hz. This behavior stemmed from the splitting of scanning trajectories at the pixel level. Quantitatively evaluation of these fluctuations was implemented by performing grayscale and denoise processing on five selected regions of different images, then calculating the SSIM to measure structural, brightness, and contrast similarity. The resulting average SSIM values, 0.978, 0.954, 0.943, 0.909, and 0.907, respectively, indicated a maximum scanning fluctuation of 9.3%. Additionally, these fluctuations have been introduced into an AR display image reconstruction to assess the impact on AR display quality. As shown in Fig. 11(d), while maximum fluctuations were present in the image reconstruction, key information of AR display remain discernible, highlighting even the 9.3% vibration noise induced fluctuations are still giving an acceptable image quality and a strong stability of our devices.

 figure: Fig. 8.

Fig. 8. (a) Setup schematic of vibration stability test of MEMS mirror. (b) Simulation of the laser stripes with frequency shift induced pattern fluctuations. (c) Flow-chat of image processing for SSIM calculation. (d) Stripe patterns projections under vibration noise of varied frequency and the corresponding SSIM values.

Download Full Size | PDF

3.2 Image reconstruction simulation of AR display

The assessment of frequency ratios’ impact on display was conducted through image simulation, leveraging the Lissajous trajectory of the proposed MEMS mirror to reconstruct an input image. Lissajous scanning generates a variety of patterns, each varying according to its respective frequency ratio. To examine the visual impact of different greatest common divisors (GCD) of bi-axial frequency on the Lissajous pattern, a spectrum of frequency ratios was devised. Figure 9 illustrates images with different linear densities, each resulting from a unique GCD. Notably, a higher GCD between frequencies correlates with a lower line density of the pattern and a higher repetition rate. Moreover, the frequency ratio significantly influences both the uniformity and fill factor of the image. Consequently, when employing the Lissajous scanning pattern as a display modality, the implications of the GCD value on the pattern must be thoroughly evaluated [8,15].

 figure: Fig. 9.

Fig. 9. Lissajous patterns with varied frequency ratios and GCD. (a) ${f_x}$: ${f_y}$= 35: 40, GCD = 5. (b) ${f_x}$: ${f_y}$= 36: 40, GCD = 4. (c) ${f_x}$: ${f_y}$= 37: 40, GCD = 1. (d) ${f_x}$: ${f_y}$= 38: 40, GCD = 2.

Download Full Size | PDF

Corresponding pixel positions are associated with distinct time intervals in the Lissajous trajectory. To examine the effects of line density and uniformity, an image with a dark background was selected for visual effect simulation, as depicted in Fig. 10. Most MEMS mirrors cited in literatures function at resonance with frequency ratios in the tens, akin to what is shown in Fig. 10(b). This high frequency ratio often leads to an inconsistent linear density and a reduced fill factor in images. To enhance the linear density and maintain stability in the face of rapid laser power variations, there are two primary strategies. The first involves maximizing the frequency ratio, achieved by increasing the fast-axis frequency and reducing the slow-axis frequency to match the video refresh frequency, as illustrated in Fig. 10(a) [12].

 figure: Fig. 10.

Fig. 10. Scanning trajectories and the corresponding images of GCD = 57 at different scanning frequencies (a) ${f_x}$: ${f_y}$= 12255: 57, frequency ratio = 215, fill factor = 50.14%. (b) ${f_x}$: ${f_y}$= 12255: 798, frequency ratio = 15.36, fill factor = 76.75%. (c) ${f_x}$: ${f_y}$= 12255: 7182, frequency ratio = 1.71, fill factor = 85.11%. (d) ${f_x}$: ${f_y}$= 12255: 12198, frequency ratio = 1.005, fill factor = 89.25%. The source resolution of the image is 640 × 360.

Download Full Size | PDF

 figure: Fig. 11.

Fig. 11. Display simulation of AR-HUD with GCD = 57. (a) The image of Lissajous scanning pattern, ${f_x}$: ${f_y}$= 12255: 798, fill factor = 76.75%. (b) The image of Lissajous scanning pattern, ${f_x}$: ${f_y}$= 12255: 7182, fill factor = 85.11%. (c) The image of raster scanning patterns, ${f_x}$: ${f_y}$= 12255: 57, fill factor = 90.14%. (d) Image of Lissajous scanning with external vibration disturbance, SSIM = 0.907. The source resolution of the image is 640 ${\times} $ 360.

Download Full Size | PDF

Another option involves minimizing the frequency ratio, such that the resonant frequencies of the slow and fast axis differ only by the refresh frequency value, as shown in Fig. 10(d). Although this low-frequency ratio enhances image display quality, the proximity of the frequencies may induce mechanical coupling between the axes. In this study, a balanced frequency ratio of 1.6 was attained via a distinct gimbal-less MEMS mirror design, as previously discussed. This resulted in meeting with the relatively high fill factor and uniformity pattern requirements of the Lissajous display. A comparison of Fig. 10(c) and (d) illustrates that a moderate uniform display without large fill-factor sacrifice can be achieved through the reasonable selection of frequency ratio.

The MEMS mirror developed in this study facilitates consistent, high fill-factor Lissajous scanning imaging, maintaining mechanical stability against external disturbances, even under vibrational perturbations of up to 2,000 Hz. This technology offers significant potential for automotive AR applications, particularly AR-HUDs, given that the random vibration spectrum from both the vehicle and road conditions typically falls below 2,000 Hz [18]. Display simulations for the AR-HUD are depicted in Fig. 11, where the color area presents basic driving information against a black background of the see-through region. While a noticeable flicker visual effect associated with the high-frequency ratio of the MEMS mirror in Fig. 11(a), both Fig. 11(b) and (c) demonstrate minimal degradation in image quality relative to the original image. Specifically, in the context of an automotive AR application that generally display dashboard information in the peripheral area of the image, the fill factors of the edge sections have been separately analyzed. Among them, the proposed device of Fig. 11(b) ranked the highest fill-factor of 88.39% in the edge section, indicating that mirrors delivered more homogeneous image trajectory. Additionally, a comparative analysis of the AR reconstruction display in the presence of external disturbance noise has also been conducted. Although with the presence of a 9.3% image fluctuation, as it has been previously discussed and shown in Fig. 11(d), critical display information remains easily discernible. Therefore, the frequency ratio design proposed in this study could offer a uniform image display and 57 Hz frame refresh rates that comparable to the raster scanning, while exhibiting greater robustness even under vibrational circumstances.

3.3 Figure-of-merit of AR display light-engines

While the integration of the proposed MEMS mirrors has indeed improved the uniformity of LBS scanning trajectories, resulting in an enhanced image display quality, it's crucial to acknowledge that the resolution and visual consistency of LBS technology still lag behind the conventional LCD and DLP techniques, primarily in terms of the pixel density. Nevertheless, it is crucial to present a systematic overview that highlights the inherent trade-offs and advantages of LBS display as it continues to solidify its competitive position within the realm of practical AR engineering. Moreover, to the best of our knowledge, a comprehensive metrics to quantify and guide the selection of augmented reality (AR) light engines remains absent. Therefore, the aim of this section is to establish an optical engine evaluation framework that comprehensively assesses various facets, facilitating the decision-making process for AR display light engine technologies.

The performance of AR display applications hinges heavily on key factors of light-engines: the FoV, optical efficiency, heat generation management. Building quantitative models to critically assess the all-round performance of various light-engines is instrumental in steering the progression of LBS display technologies toward AR applications. Consequently, a figure of merit (FoM) has been established for MEMS LBS, employing AR-HUD as a representative case. This FoM encompasses aspects like FoV, resolution, optical efficiency, and heat dissipation control, etc. The FoM is delineated in the following section, with parameters details found in Table 1. To streamline calculation and comparison, the principal parameters of DLP and LBS light-engines are normalized with referring LCD as standard.

$$FoM = \frac{{N \cdot FoV \cdot E \cdot B \cdot VID}}{H}$$
$$B = P \cdot {M_{eff}} \cdot {D_{eff}} \cdot {L_{eff}}^{{L_{num}}}$$
$$H = {T_{rise}}P \cdot ({1 - {M_{eff}}} )\cdot ({1 - {D_{eff}}} )$$
Where B denotes the brightness, H indicates heat generation of the system. Figure 12 illustrates a preliminary FoM assessment of various AR light-engines including off-the-shelf display techniques and the emerging LBS technique. Notably, the LBS exhibits a markedly higher FoM value of 760.7, significantly surpassing the 3.98 of DLP and 0.34 of LCD light engines. This suggests a superior comprehensive performance of LBS for AR-HUD applications. Such performance advantage primarily stems from the device's inherent high reflectivity, reduced lens optics requirement, and highly efficient light-on-demand display principle. Furthermore, the overall optical system of LBS has lower light absorption compared to LCDs, leading to substantially less heat generation, a vital attribute especially under direct sunlight exposure when placed on a dashboard.

Tables Icon

Table 1. Parameters for FoM calculation of varied AR display light engines a

4. Conclusion

In this study, an innovative gimbal-less Lissajous MEMS LBS design was presented, with an emphasis on achieving high fill factors and frame rates for display. Utilizing a large mirror diameter of 2 mm, the design ensures an increased optical etendue for large exit pupil and finer pixel projection. The proposed system leverages a tailored bi-axial high-frequency resonance of 12,255 Hz and 7,182 Hz, resulting in a favorable low frequency ratio that enhances operational stability, even under 2,000 Hz external vibration disturbance. The advanced LBS is capable of achieving a resolution of 640 × 360 pixels and video frame refresh rate of 57 Hz, while preserving a densely pixelated, uniform AR visual effect with an image fill factor of 85.11%. A mathematic model of Lissajous pixel-cells and its image reconstruction simulation have been constructed to indicate a great potential of the as-proposed system for automotive AR-HUD application, by effectively suppressing flicker visual effects and withstanding a broad spectrum of comparable road-induced vibrations. In a groundbreaking step, a new performance metric of FoM was formulated, accounting for FoV, resolution, optical efficiency and heat management, thereby providing a standardized approach to evaluate AR displays across various picture-generation methodologies. This has laid the groundwork for guiding future AR light-engine development.

 figure: Fig. 12.

Fig. 12. Performance metrics of varied AR display light engines. The larger FoM value represents better comprehensive performance.

Download Full Size | PDF

Funding

Nuclear Power Institute of China (HDLCXZX-2022-ZH-014).

Disclosures

The authors declare no conflict of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. J. Xiong, E.-L. Hsiang, Z. He, T. Zhan, and S.-T. Wu, “Augmented reality and virtual reality displays: emerging technologies and future perspectives,” Light: Sci. Appl. 10(1), 216 (2021). [CrossRef]  

2. Y.-H. Lee, T. Zhan, and S.-T. Wu, “Prospects and challenges in augmented reality displays,” Virtual Reality & Intelligent Hardware 1(1), 10–20 (2019). [CrossRef]  

3. E.-L. Hsiang, Z. Yang, Q. Yang, P.-C. Lai, C.-L. Lin, and S.-T. Wu, “AR/VR light engines: perspectives and challenges,” Adv. Opt. Photonics 14(4), 783 (2022). [CrossRef]  

4. L. Noui and J. Reitterer, “TriLite Addresses Challenges with Projection Display Technologies for AR Glasses,” Inf Disp 37(4), 12–16 (2021). [CrossRef]  

5. F. Fidler, A. Balbekova, L. Noui, S. Anjou, T. Werner, and J. Reitterer, “Laser beam scanning in XR: benefits and challenges,” in Optical Architectures for Displays and Sensing in Augmented, Virtual, and Mixed Reality (AR, VR, MR) II, B. C. Kress and C. Peroz, eds. (SPIE, 2021), p. 1.

6. B. Bastani, E. Turner, C. Vieri, H. Jiang, B. Funt, and N. Balram, “Foveated pipeline for AR/VR head-mounted displays,” Inf Disp 33(6), 14–35 (2017). [CrossRef]  

7. T. E. Cognard, A. Goncharov, N. Devaney, C. Dainty, and P. Corcoran, “A Review of Resolution Losses for AR/VR Foveated Imaging Applications,” in 2018 IEEE Games, Entertainment, Media Conference, GEM 2018 (Institute of Electrical and Electronics Engineers Inc., 2018), pp. 228–231.

8. G. Pettitt, J. Ferri, and J. Thompson, “47.1: Invited Paper : Practical Application of TI DLP ® Technology in the Next Generation Head-up Display System,” SID Symposium Digest of Technical Papers 46(1), 700–703 (2015). [CrossRef]  

9. Y.-H. Seo, H. Kim, S.-P. Yang, K. Hwang, and K.-H. Jeong, “Lissajous scanned variable structured illumination for dynamic stereo depth map,” Opt. Express 28(10), 15173 (2020). [CrossRef]  

10. J. Wang, G. Zhang, and Z. You, “Design rules for dense and rapid Lissajous scanning,” Microsyst. Nanoeng. 6(1), 101 (2020). [CrossRef]  

11. Y. H. Seo, K. Hwang, H. Kim, and K. H. Jeong, “Scanning MEMS Mirror for high definition and high frame rate Lissajous patterns,” Micromachines 10(1), 67 (2019). [CrossRef]  

12. U. Hofmann, J. Janes, and H.-J. Quenzer, “High-Q MEMS Resonators for Laser Beam Scanning Displays,” Micromachines 3(2), 509–528 (2012). [CrossRef]  

13. B. Xu, Y. Ji, K. Liu, and J. Li, “Piezoelectric MEMS Mirror with Lissajous Scanning for Automobile Adaptive Laser Headlights,” Micromachines 13(7), 996 (2022). [CrossRef]  

14. Q. A. A. Tanguy, O. Gaiffe, N. Passilly, J.-M. Cote, G. Cabodevila, S. Bargiel, P. Lutz, H. Xie, and C. Gorecki, “Real-time Lissajous imaging with a low-voltage 2-axis MEMS scanner based on electrothermal actuation,” Opt. Express 28(6), 8512 (2020). [CrossRef]  

15. H. Urey, “Torsional MEMS scanner design for high-resolution scanning display systems,” in Optical Scanning 2002, S. F. Sagan, G. F. Marshall, and L. Beiser, eds. (2002), 4773, p. 27.

16. A. Wolter, H. Schenk, H. Korth, and H. Lakner, “Torsional stress, fatigue and fracture strength in silicon hinges of a micro scanning mirror,” in Reliability, Testing, and Characterization of MEMS/MOEMS III, D. M. Tanner and R. Ramesham, eds. (SPIE, 2004), 5343, p. 176.

17. A. Opreni, N. Boni, R. Carminati, and A. Frangi, “Analysis of the nonlinear response of piezo-micromirrors with the harmonic balance method,” Actuators 10(2), 21 (2021). [CrossRef]  

18. T. Asari, M. Miyachi, Y. Oda, T. Koyama, H. Kurosu, M. Sakurai, M. Tani, Y. Yasuda, and H. Toshiyoshi, “Adaptive driving beam system with MEMS optical scanner for reconfigurable vehicle headlight,” J. Optical Microsystems 1(01), 1–9 (2021). [CrossRef]  

19. H. W. Yoo, R. Riegler, D. Brunner, S. Albert, T. Thurner, and G. Schitter, “Experimental Evaluation of Vibration Influence on a Resonant MEMS Scanning System for Automotive Lidars,” IEEE Trans. Ind. Electron. 69(3), 3099–3108 (2022). [CrossRef]  

20. R. Dosselmann and X. D. Yang, “A comprehensive assessment of the structural similarity index,” Signal Image Video Process 5(1), 81–91 (2011). [CrossRef]  

21. V. Thakur and J. Ferri, TI DLP Pico Technology for Aftermarket Head-up Displays Application Report TI DLP ® PicoTM Technology for Aftermarket Head-up Displays (2016), (September).

22. G. Pettitt, J. Ferri, and J. Thompson, “DLP® technology: Solving design challenges in next generation of automotive head-up display systems,” 21st International Display Workshops 2014, IDW 2014 2(December 2014), 1079–1082 (2014).

23. L. Gu, D. Cheng, Y. Liu, J. Ni, T. Yang, and Y. Wang, “Design and fabrication of an off-axis four-mirror system for head-up displays,” Appl. Opt. 59(16), 4893 (2020). [CrossRef]  

24. A. Rankin and J. Thompson, “Next-Generation Head-Up Displays,” Inf Disp 31(3), 18–21 (2015).

25. W. Rui, J. Lun, and S. Zhihua, “Optical Design of Ensemble Head-Up Display System Based on Mini-Projector,” Laser Optoelectron. Prog. 112201, 112201 (2018). [CrossRef]  

26. S. Martin and J. Watanabe, DLP ® Technology: Solar Loading in Augmented Reality Head-up Display Systems (2018), (July).

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (12)

Fig. 1.
Fig. 1. Schematic of (a) panel-based and (b) LBS light-engine for AR display.
Fig. 2.
Fig. 2. (a) Schematic of Lissajous scanning trajectory overlapping with pixel cells; (b) selection of scanning time interval and line segment for pixel cell.
Fig. 3.
Fig. 3. (a) 3D model schematic of the proposed MEMS mirror. (b) SEM image of the as-fabricated MEMS mirror. Finite-element simulation for modal analysis of (c) slow-axis, (d) fast-axis, and von Mises stress analysis of (e) slow-axis, (f) fast-axis with frequency-domain piezoelectric actuation.
Fig. 4.
Fig. 4. Microfabrication process flow of the proposed MEMS mirror.
Fig. 5.
Fig. 5. (a) Schematic and (b) photograph of the scanning measurement system.
Fig. 6.
Fig. 6. Relationship between scan angle and resonant frequency of (a) fast-axis and (b) slow-axis under varied actuation voltages. (c) and (d) indicate the linear behavior of scanning angle increasement versus driving voltage at fixed resonant frequency.
Fig. 7.
Fig. 7. Modulation of MEMS mirror with bi-axial scanning.
Fig. 8.
Fig. 8. (a) Setup schematic of vibration stability test of MEMS mirror. (b) Simulation of the laser stripes with frequency shift induced pattern fluctuations. (c) Flow-chat of image processing for SSIM calculation. (d) Stripe patterns projections under vibration noise of varied frequency and the corresponding SSIM values.
Fig. 9.
Fig. 9. Lissajous patterns with varied frequency ratios and GCD. (a) ${f_x}$: ${f_y}$= 35: 40, GCD = 5. (b) ${f_x}$: ${f_y}$= 36: 40, GCD = 4. (c) ${f_x}$: ${f_y}$= 37: 40, GCD = 1. (d) ${f_x}$: ${f_y}$= 38: 40, GCD = 2.
Fig. 10.
Fig. 10. Scanning trajectories and the corresponding images of GCD = 57 at different scanning frequencies (a) ${f_x}$: ${f_y}$= 12255: 57, frequency ratio = 215, fill factor = 50.14%. (b) ${f_x}$: ${f_y}$= 12255: 798, frequency ratio = 15.36, fill factor = 76.75%. (c) ${f_x}$: ${f_y}$= 12255: 7182, frequency ratio = 1.71, fill factor = 85.11%. (d) ${f_x}$: ${f_y}$= 12255: 12198, frequency ratio = 1.005, fill factor = 89.25%. The source resolution of the image is 640 × 360.
Fig. 11.
Fig. 11. Display simulation of AR-HUD with GCD = 57. (a) The image of Lissajous scanning pattern, ${f_x}$: ${f_y}$= 12255: 798, fill factor = 76.75%. (b) The image of Lissajous scanning pattern, ${f_x}$: ${f_y}$= 12255: 7182, fill factor = 85.11%. (c) The image of raster scanning patterns, ${f_x}$: ${f_y}$= 12255: 57, fill factor = 90.14%. (d) Image of Lissajous scanning with external vibration disturbance, SSIM = 0.907. The source resolution of the image is 640 ${\times} $ 360.
Fig. 12.
Fig. 12. Performance metrics of varied AR display light engines. The larger FoM value represents better comprehensive performance.

Tables (1)

Tables Icon

Table 1. Parameters for FoM calculation of varied AR display light engines a

Equations (10)

Equations on this page are rendered with MathJax. Learn more.

{ X = A x sin ( 2 π f x t + φ x ) Y = A y sin ( 2 π f y t + φ y )
v X | t = 0 = 2 π f x A x cos ( 2 π f x t ) | t = 0 = 2 π A x f x
Δ t 2 A x v R x = 2 A x 2 π A x f x R x = 1 π f x R x
f = 1 2 π K I
I = ρ π R 4 t 4 + ρ π R 2 t 3 12
K = 2 G w t 3 L [ 1 3 0.21 t w ( 1 1 12 t 4 w 4 ) ]
θ o p t = 2 tan 1 ( L 2 d )
F o M = N F o V E B V I D H
B = P M e f f D e f f L e f f L n u m
H = T r i s e P ( 1 M e f f ) ( 1 D e f f )
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.