Near-to-eye displays (NEDs), or also called head-mounted displays (HMDs), are major devices for augmented reality (AR) and virtual reality (VR) applications. One issue of the currently available NEDs is the vergence accommodation conflict (VAC). In the usual NED configuration, each eye of the user sees the virtual image of the micro display panel and thus the accommodation of the eye responds to the virtual image distance which is fixed by the NED optics. To the contrary, the converging distance of the optic axes of two eyes corresponds to the disparity between the left and right images, and thus it varies according to the presented contents. Therefore, the accommodation and converging distances do not match in usual NED configuration, which is called VAC. In addition to the mismatch between the vestibular and visual sensory inputs, the VAC is also thought to be a major cause of the fatigue in NED experience [1].

To solve the VAC, various methods have been proposed. Most of them have the approach to present not a two-dimensional (2D) image at a fixed virtual image distance but a three-dimensional (3D) image at an arbitrary distance to each eye of the user. For the presentation of the 3D images, the light field display [2–5] and the holographic display techniques [6–9] are used. These techniques solve the VAC by presenting 3D images with true focal cue, but usually accompany significant loss in resolution and high computational load [10].

Maxwellian display, or retina scanning display is another approach to alleviate the VAC [11]. Instead of presenting the 3D images with true focal cue, the Maxwellian display presents images which are always focused regardless of the focal length of the eye lens. Scanning the modulated beam through a single point in the eye pupil as shown in Fig. 1 [12] can be one implementation of the Maxwellian display. Because the effective eye pupil becomes a pinhole, the depth of focus of the image is much enlarged, showing clear image wherever the eye focuses. One advantage of the Maxwellian display over 3D display techniques with true focal cue is that it does not require any special computation for the image contents preparation. The target images are displayed on the display panel because they are without any modulation. The image resolution is another advantage. Although it is dependent on the desired depth of focus as will be discussed later, the displayed image usually has higher resolution than the light field displays.

 

Fig. 1. Conceptual diagram of Maxwellian NED using a scanning mirror.

Download Full Size | PDF

The configuration shown in Fig. 1, however, has drawbacks. It places a bulk concave mirror in front of the eye, and therefore the optical see-through capability which is crucial in AR applications is not realized. More importantly, because the scanning focus should be inside the eye pupil, the eyebox is limited to the eye pupil size, which is too restrictive.

Recently, several techniques have been reported to enhance the Maxwellian displays. In one method [13], the computer generated holography was used to form a spot in the eye pupil plane, realizing large depth of focus. For optical-see-through view, however, the bulk optical combiner is used, which increases the system volume. In other methods [14,15], the holographic optical element (HOE) was used as the optical combiner, and the wavelength multiplexing [15] was performed to give full color images. The use of the HOE enables the compact system configuration with optical-see-through view. In these previous works, however, the focusing spot is still fixed to a single position in the eye pupil plane and thus the eyebox is limited to the eye pupil size. Because the wide eyebox is crucial for comfortable viewing in NEDs, the eyebox extension technique for the Maxwellian displays is required.

In this Letter, a novel Maxwellian NED configuration which extends the eyebox is proposed. The proposed configuration uses a waveguide with a HOE where multiple concave mirrors are recorded. The displayed images are focused by the HOE into multiple spots in the eye pupil plane, enhancing the eyebox in the proposed configuration. The combinational use of the HOE and the waveguide enables the display of the always-focused images on top of the optical see-through view in a compact form factor.

Note that although exit pupil expansion techniques for waveguide configuration have been reported, they are not adequate for Maxwellian NED because they split a single light ray into multiple parallel rays, forming an extended exit pupil [16]. To the contrary, the proposed method forms a spot array such that the effective exit pupil at each eye position is not extended but still is a pinhole. Therefore, the proposed method can extend the eyebox while maintaining the Maxwellian NED configuration.

Figure 2 shows the conceptual diagram of the proposed Maxwellian NED when the single concave mirror is recorded in the HOE. The proposed configuration consists of collimated laser illumination, an amplitude-type SLM loaded with desired always-focused images, four-f filtering optics, a waveguide with slanted input side, and a HOE functioning as a concave mirror. The collimated laser illumination is spatially modulated by the SLM and filtered by the four-f optics to pass only a modulated lowest order term, blocking all higher order diffraction terms caused by the discrete pixel structure of the SLM. The filtered beam enters the input side of the waveguide, is delivered to the out-coupling HOE by the total internal reflections, and finally is focused by the HOE onto a single spot inside the eye pupil. The image loaded in the SLM is projected to the eye retina through a single spot at the eye pupil, and the users can see the clear image regardless of the focal length of the eye lens. Because the HOE is transparent, the displayed image appears on top of the real environmental view, achieving the optical see-through capability.

 

Fig. 2. Conceptual diagram of the proposed Maxwellian NED.

Download Full Size | PDF

The basic configuration in Fig. 2 having the HOE with single concave mirror recorded has eyebox limitation as the conventional Maxwellian displays. In the proposed method, we loosen this limitation by recording multiple concave mirrors with laterally shifted focal spots in the eye pupil plane as shown in Fig. 3. The lateral spacing of the multiplexed focal spots in the proposed method is chosen to be slightly larger than the eye pupil width. Thus, only a single focal spot enters the eye pupil wherever the eye is located within the spanning range of the multiple focal spots, presenting always-focused images without double image problem in the extended eyebox.

 

Fig. 3. Eyebox enlargement by multiplexing multiple concave mirrors.

Download Full Size | PDF

In the experiment, we built a proof-of-concept system on the optical table. The experimental setup is shown in Fig. 4. The SLM used in the experiment has $800\times 600$ pixels of 13.97 μm size with linear polarizers in its input and output planes, working as the amplitude modulator. The two lenses in the four-f optics have a 10 cm focal length, and an aperture located in the Fourier plane was used as the filter. The glass waveguide has a 9 mm thickness with input side slanted by 30° from the normal direction. The filtered beam is normally incident on the slanted input side, coupled into the waveguide, and focused by the HOE which is attached on the output side of the waveguide. The wavelength of the laser illumination is 532 nm.

 

Fig. 4. Experimental setup.

Download Full Size | PDF

We used a photopolymer as the recording medium of the HOE. The HOE was fabricated using the setup shown in Fig. 5. The photopolymer was attached on the waveguide before recording. The reference beam is the collimated laser beam which enters the waveguide from the slanted side and the beam arrives at the photopolymer. The signal beams are multiple beams illuminating the photopolymer simultaneously and converging at three different focal spots. In our experiment, this simultaneous recording was found to result in better uniformity of the diffraction efficiencies of the three focal spots than the sequential recording. The axial distance between the photopolymer and the focal spot is 8.2 cm, and the diameter of the signal beam on the photopolymer is 15 mm, which can give a $10.5°\times 10.5°$ field of view at maximum. Note that in our implementation, however, the field of view is limited to $9.2°\times 5.2°$ by the SLM size projected on the HOE plane. In the recording, the power of the reference beam was $0.55\mathrm{mW}/{\mathrm{cm}}^2$, and the power of the three signal beams were 0.409, 0.322, and $0.386\mathrm{mW}/{\mathrm{cm}}^2$, respectively. The measured diffraction efficiency is 25% (i.e., = power at a single focal spot/reference beam power), 18% and 24% for each focal spot, giving 67% in total. Note that the reference beam power was measured before it enters the waveguide, and thus the diffraction efficiency given above has the waveguide coupling loss. The spacing between the three focal spots in our experiment is 3 mm.

 

Fig. 5. HOE recording setup.

Download Full Size | PDF

Figure 6 shows the focal spots generated by the implemented HOE. The image comprising Fig. 6 was taken by locating a diffusing sheet in the eye pupil plane of the experimental setup. We can see multiple spots with the desired horizontal separation. The range of the multiple spots defines the enlarged eyebox in the proposed method, and it is measured to be approximately 9 mm in our setup. Additional weak spots and noise observed in Fig. 6 are believed to come from multiple reflections of the recording beams inside the waveguide and nonideal volume hologram property of the photopolymer. A more sophisticated recording setup would be required to reduce these noises.

 

Fig. 6. Focused spots in the eye pupil plane.

Download Full Size | PDF

Figure 7 and the associated movies show the observed image when the camera is located at the different spot positions. The camera used in the experiment is a Canon EOS 5D Mark II with 9.93 f-number and 25 mm focal length setting which gives 2.5 mm aperture diameter simulating the eye pupil. As seen in Fig. 7, the image is displayed on top of the real environment, showing the optical see-through feature of the proposed configuration. At each camera position, the displayed image remains clear and focused while the real environment is focused and blurred according to the focal length of the camera, demonstrating the presentation of the always-focused images as expected.

 

Fig. 7. Observed image with different eye focal lengths (see Visualization 1, Visualization 2, Visualization 3, and Visualization 4 for movies).

Download Full Size | PDF

Figure 8 shows the observed images when the camera position spans the extended eyebox with a small unit step. The transition of the images to neighboring focal spot happens quickly only when the camera is located exactly in the middle of neighboring spots, and the clear single image is seen in most of the area in the extended eyebox. Therefore, from the experimental results shown in Figs. 7 and 8, it is confirmed that the always-focused images can be displayed on top of the optical see-through view within the extended eyebox by the proposed waveguide with multiplexed HOE configuration.

 

Fig. 8. Observed images at different eye positions.

Download Full Size | PDF

Although the working principle of the proposed configuration was verified by the experiment, there still is large room for enhancement. Possible axial misalignment of the eye pupil and the focal spots is one limitation of the proposed configuration like other Maxwellian displays. By recording multiple concave mirrors with laterally shifted focal spots, the lateral eyebox can be extended. But still the axial movement of the eye is not adequately covered. The eye rotation can also bring small axial misalignment due to the round shape of the eyeball. In the current setup, the axial misalignment reduces the field of view, like the situation where the eye sees the images through a pinhole just before or behind the eye pupil. The axial movement of the eye, however, rarely happens in real use, and the mechanical adjustment of the system to a specific user needs to be done only once when the system is calibrated for the first time. The effect of the axial misalignment caused the eye rotation was insignificant in our experiment, and it is also expected to be overcome by recording multiple focal spots at slightly different axial distances, following the round shape of the eyeball.

The spacing between the focal spots is fixed to 3 mm in our configuration, which may cause double image or a blind area problem if the eye pupil is much different from 3 mm. One possible solution would be to slant or curve the wavefront from the SLM by superimposing an appropriate phase pattern on the SLM such that the HOE focal spots shift or their spacing is adjusted according to the detected eye pupil position and size. This is a topic of our further research.

The position of the displayed image is not fixed within the field of view but moves as the eye moves laterally. This is because the area where the image is projected on the retina changes according to the relative focal spot position with respect to the optic axis of the eye. In practical application of the proposed configuration, the eye pupil tracking system would be required, and the image in the SLM should be updated accordingly to fix the image position within the field of view.

The depth of focus of the displayed image is large in the proposed configuration, but it is not infinite. Assuming that the HOE is the ideal concave mirror of a focal length $f$, the spot size in the eye pupil is given by $w=f\lambda B$ where $\lambda$ is the wavelength and $B$ ($<1/p$ where $p$ is the pixel pitch of the SLM) is the spatial bandwidth of the image loaded in the SLM, respectively. The depth of focus enhancement of the displayed images by the proposed configuration over optical see-through view is $w_e/w=w_e/f\lambda B$ where $w_e$ is the eye pupil width. Even though this enhancement is not infinite, it is sufficiently large, so that the displayed images are clear when the camera focal distance is changed from 0.3 to 3.5 diopter in our experiment. Also note that this finite depth of focus is not a limitation of the proposed configuration but is a common limitation of the Maxwellian displays.

Finally, resolution of the current setup is not sufficiently high due to low resolution of the SLM and nonoptimized optics. Figure 9 shows an observed image when a resolution target pattern is displayed within a small field of view.

 

Fig. 9. Observed image of a resolution target pattern.

Download Full Size | PDF

In conclusion, we proposed an optical see-through Maxwellian NED configuration with an extended eyebox. The proposed method uses the transparent HOE instead of the bulk concave mirror to give optical see-through views. The use of the HOE also enables the multiplexing of multiple concave mirrors with shifted focal spots such that the eyebox can be extended without double image problems. The implemented experimental setup shows that the always-focused images can be displayed on top of the optical see-through view of the environment with $9.2°\times 5.2°$ field of view within $9\mathrm{mm}\times 3\mathrm{mm}$ eyebox by multiplexing three concave mirrors. We expect the proposed configuration can be used in the AR applications where always-focused information needs to be displayed on the real environment.