1. Introduction

Peripheral field loss (PFL) severely restricts a patient’s visual field. Homonymous hemianopia (HH; loss of visual function in the same hemifield in both eyes) can result from stroke, tumors, trauma, surgery, and neurologic disease [1], though tunnel vision (highly constricted residual central field) is mostly caused by retinal diseases such as retinitis pigmentosa, glaucoma, or choroideremia [2]. Highly restricted peripheral visual field negatively affects PFL patients’ mobility [3], and they often report increases in collisions with obstacles and other pedestrians outside of their residual visual field while walking [4]. Patients with severe PFL are prohibited from driving in most states in the US [5]. Such risks and restrictions result in a loss of independence and decreased quality of life [6].

Field expansion using peripheral prisms (PPs) is a common treatment for PFL patients [7]. The field expansion effect is achieved by peripherally mounted optical prisms, which deflect the light from the portions of a visual scene in the blind field to the residual seeing field while clearing central vision, which typically remains intact in PFL patients [8]. The optical prisms, which deflect rays of light away from the prism’s apex toward its base, are typically fitted “base-out” with the prism base lying on the side of the visual field loss (e.g., base-left for left HH or base-right for right HH). Various prism configurations have been actively developed for both patients with HH [9,10] and with tunnel vision [11]; these configurations include both horizontal PPs which shift images only laterally, and oblique configurations, which shift the images vertically to capture the area of the visual field bisected by the horizontal meridian (∼40°), the latter of which is useful in driving, where the shifted images originate from the transparent windshield [12].

This field expansion method has been shown to help the patient detect obstacles or colliding pedestrians earlier than they might otherwise [13]. However, some patients report difficulties in utilizing views through the prism [14]. Such difficulty may be due to the optical distortions in the refractive prisms caused by the differences in effective prism power (i.e., the amount of deflection) dependent on the angle of incidence and total internal reflection [15]. To eliminate these refractive prism limitations, multi-periscopic prisms (MPP) were developed, which use multi-reflections, instead of refractions, to afford high-deflection power with almost no optical distortion [10]. However, even with MPP, apparent differences still exist between the view through the prisms and the corresponding view without prisms [16]. This finding suggests that prism distortion may not be the sole source of patient difficulties.

Optical ray tracing and photographic depiction [17] confirmed that the discrepancy in viewpoints causes such apparent differences. The viewpoint for the view seen through prism (Prism viewpoint) is located outside of the eye (i.e., horizontally translated and rotated by the prism’s power), while the viewpoint for the corresponding view seen without prism (Original viewpoint) is located at the entrance pupil of the eye. As a result, objects seen from the prism viewpoint appear to be rotated when compared to the original viewpoint (e.g., when facing a person, more of the side of that person’s face is seen through the prism [10]). This suggests that the viewpoint changes may be the source of the difficulties in interpreting the view through the PP, and thus clear understanding of the viewpoint changes through PPs and possible impacts to field expansion are needed.

However, investigating the effect of viewpoint discrepancy using optical prisms presents several practical issues because PPs must be configured with predetermined specifications (e.g., size, power, and horizontal or oblique configuration [9]). One issue to consider is the apical scotoma, i.e., the gap between the ray deviated at the apex of the prism and the adjacent undeviated ray [9]. The size of the gap depends on the strength of the prism where higher power prisms result in larger apical scotomas. As a consequence, any objects present in the visual field covered by the scotoma are not visible. To mitigate this effect, PPs are often fitted unilaterally (i.e., over a single eye) so that the portion of the visual field lost to the scotoma remains visible to the fellow eye [8]. Although optical distortion could be minimized in MPPs [10], the prisms also require customized fitting for each patient based on the type and severity of their field loss (e.g., HH and tunnel vision), and the location of the residual seeing field [11]. Furthermore, the optical and mechanical characteristics (e.g., prism width, tilt, unilateral or bilateral fitting [10,11,18]) must be set for each patient to minimize diplopia and apical scotoma [9]. In addition, it is practically impossible to make a control condition, where the optical PPs work without viewpoint translation and rotation.

For these reasons, we developed an alternative approach to field expansion using digital methods. Specifically, we aimed to replicate prism field expansion via simulation in virtual reality (VR). By simulating the effects in VR, we are able to investigate the effects of viewpoint changes while avoiding these practical issues with optical PPs. Because the field expansion methods and specifications are parameterized and can be adjusted on the fly in VR, we can systematically compare the prism viewpoint and the original viewpoint in static (e.g., standing) and dynamic (e.g., walking) conditions from the patient’s perspective. Specifically, we are free to choose the prism power, size and positioning, configuration (horizontal or oblique), and fitting (unilateral or bilateral).

In this paper, we describe in detail how PPs can be simulated in a VR environment and illustrate the resulting visual differences among various prism simulation methods. Finally, we explore the theoretical consequences of how these altered viewpoints, and thus altered optic flows may affect mobility tasks such as collision detection and avoidance.

2. Materials and methods

2.1 Prism simulation via angular shift in horizontal PP field expansion

To focus on investigating the effect of viewpoint changes, we first simulate an ideal prism that provides constant deflection power across its surface with no optical distortions or aberrations, such as MPP [10]. Then, we implement the ideal prism in different VR environments to test the viewpoint effects. Figure 1 illustrates how the prism viewpoint is formed through an ideal horizontal prism with a prism power, δ_h, mounted on a spectacles’ lens (i.e., fronto-parallel plane) located at a distance, d, from the entrance pupil of the eye. Since only the rays that pass through the prism that are also deflected into the entrance pupil of the eye are visible to the observer, the prism viewpoint can be traced back by continuing these rays before the deflection. The prism viewpoint is located where the extended rays converge, and its horizontal shift amount, s_h, can be calculated by

We illustrate the prism viewpoint in fittings for patients with left HH (Fig. 1(a)) and patients with tunnel vision (Fig. 1(b)). The view from the original viewpoint through the prism is the same as the view from the prism viewpoint while looking at the translated and rotated viewing direction. Note that the illustration shows a top-down view, which does not show the PP fitting (i.e., placing prisms above or/and below the eye level) with inter-prism separation [8], through which the primary position of gaze passes. For this reason, we use the vertical meridian of the visual field as the reference point for the prism viewpoint analysis. The PPs are mounted in the base-out configuration to cover their blind visual field. For HH patients, the PPs are fitted unilaterally (i.e., placed over the left eye only for left HH) [8], while the PPs for tunnel vision patients are fitted bilaterally (e.g., one placed over the left eye to cover further left and one over the right eye to cover further right sides) [11].

 

Fig. 1. Prism viewpoint formed by PP in horizontal field expansion for patients with PFL. (a) For left HH (blind in the left side of the visual field), an ideal prism with constant prism power δ_h is fitted at a distance d on the spectacle’s lens. (b) For tunnel vision, the same principles apply. For simplicity, we are showing the left eye fitting condition here. (c) Horizontal PP field expansion effect for left HH. (d) Horizontal PP field expansion for tunnel vision.

Download Full Size | PDF

In HH (Fig. 1(a)), although the PPs’ sizes and prism powers can be specified arbitrarily, the field of view (FOV) of the prism viewpoint, which is defined by size and prism power, should be matched to the point where the apical scotoma starts, s_h, [9]. Note that, to cover natural eye scanning [19], the PP may be extended into the peripheral blind fields. However, it changes only the FOV, i.e., it does not affect the prism viewpoint. The field expansion effect for left HH is illustrated in Fig. 1(c) for unilaterally fitted base-out PPs fitted over the left eye.

In tunnel vision (Fig. 1(b)), since the residual central field is symmetric around the horizontal and vertical meridians, the PPs should be fitted to provide an expanded field abutting the residual central field [11]. Typically, δ_h for tunnel vision is selected to be the same as the residual central field size (θ_h) [11], but it may be selected differently for specific tasks (e.g., 45° for walking) [10]. Similar to the PP configuration for HH, it can be further extended into the blind side as much as to cover scanning. The field expansion effect for tunnel vision is illustrated in Fig. 1(d) for bilaterally fitted base-out PPs with the upper prism fitted over the right eye and the lower prism fitted over the left eye.

2.2 Prism simulation via angular shift in oblique PP field expansion

With the horizontal PP, the prism viewpoint is only shifted along the horizontal meridian. Since the prism is physically located with vertical offsets (peripherally), the prism viewpoints are aimed at the upper and lower directions for the upper and lower prisms, respectively. This configuration was shown to be less effective for collision detection which requires monitoring the heading direction (i.e., toward the focus of expansion, FOE) [13]. The oblique prism configuration resolves this problem by adding a vertical angular shift in addition to the horizontal angular shift by obliquely rotating the PP base [10], which results in the prism viewpoint aiming toward the FOE even if it is fitted peripherally. Note that if a single prism is peripherally added for field expansion, it should aim toward the FOE. However, if a pair of prisms are installed upper and lower periphery, their prism viewpoints should not overlap to avoid diplopia.

Figure 2 shows a side view of how the prism viewpoint is formed through an oblique PP spanning, θ_v, positioned, φ, below the primary position of gaze and placed at d from the entrance pupil of the eye. To cover the angular gap caused by inter-prism separation of the PPs, the vertical angular shift of the oblique prism, δ_v, should be the same as φ [10]. The prism FOV (orange shaded area) spans between the ray passing the edge of the prism and entering orthogonally to the prism plane and the ray passing the edge at the opposite end of the prism. The edge rays traced back to their intersection define the location of the prism viewpoint. This new viewpoint is rotated by δ_v and translated vertically from the original viewpoint by

The oblique prism’s viewpoint is the vector sum of the horizontal translation described in Section 2.1, s_h, and the above-mentioned vertical field expansion, s_v, rotated by δ_h and δ_v.

 

Fig. 2. Prism viewpoint in oblique PP for PFL (side view). Only the lower prism of the paired (upper and lower prism) configuration is illustrated.

Download Full Size | PDF

2.3 Field expansion via linear shift (image crop-and-shift) in peripheral field expansion

Although the optical prism’s viewpoint is, in fact, rotated and translated from the original viewpoint, most existing literature (e.g., Fig. 3 in [8]) illustrate the prism viewpoint as an image translation of a portion of the scene along a plane orthogonal to the primary position of gaze, which is hard to achieve in optical prism and is not reflective of the angular shift which actually happens with optical prisms. However, such image translation methods can be achieved in camera-based field expansion and thus a practical choice for head-mounted displays (HMDs) or smart glasses with a fixed forward-facing camera.

 

Fig. 3. Viewpoint simulation via linear shift (i.e., image crop-and-shift) for peripheral field expansion. (a) For left HH, an image in the blind field, γ_h, is captured by a forward-aiming camera at the entrance pupil of the patient’s eye and shifted into the residual field, θ_h. (b) For tunnel vision, the image should be cropped outside of the residual visual field and shifted to the center position. We illustrate only the left-field expansion here for simplicity, but for tunnel vision, field expansion usually supports both left and right directions.

Download Full Size | PDF

Figure 3 shows how field expansion can be achieved by translating the camera-captured image via cropping and shifting. If an image is captured by a camera whose entrance pupil occupies the same location of the entrance pupil of the eye (not simultaneously), the image captured by the camera is correctly depicted by photography as it is done by the eye [16] assuming the optical axes of the camera and eye are also aligned. Accordingly, the portion of the image cropped from the blind field by such a camera will be exactly the same as the scene projected to the eye without any perspective distortion. Then, the cropped image is displayed in the residual seeing field as the prism viewpoint on a display plane located at d from the entrance pupil of the eye (i.e., a fronto-parallel plane at the spectacles).

Note that because of the tangential relationship between the angular size of the objects and the linear size of the captured objects on the flat image plane, the linear size of the captured object at the left-far eccentricity (yellow object 1 in Fig. 3) is larger than that of the object close to the zero eccentricity (red object 3 in Fig. 3). Due to the decentration distortion, when they are projected back to the display plane, the physical size of object 1 appears to be larger than object 3, causing a size discrepancy.

In HH (Fig. 3(a)), a portion of the blind field camera image γ_h (i.e., FOV of prism viewpoint) is cropped and shifted (linearly) along the display plane and into the seeing field θ_h by

To avoid diplopia and minimize scotoma, the displayed image θ_h should be matched to the size of the cropped image γ_h. In tunnel vision (Fig. 3(b)), the displayed image spans θ_h which is equal to the horizontal extent of the residual central field, could be the same as the captured image, γ_h, but shifted from the edge of the residual seeing field (i.e., s_l = γ_h + θ_h ⁄ 2). To avoid central visual rivalry, suppression, or scotoma (where it may be most bothersome [8]), the captured images should be displayed in the upper and/or lower peripheries as is the case in the fitting of optical PPs.

Note that since the image translation and displaying are computationally handled, we can easily simulate an oblique PP configuration (which utilizes oblique prism tilting) by cropping the correct region of the image, in the same manner as it is described above. The amount of vertical linear shift for each captured image should then be one-half of the inter-prism separation to avoid diplopia and minimize scotoma resulting from the vertical shift. In the VR environment, the image translation method can be implemented using a lens shift [20], where the virtual lens axis is rotated according to the theoretical analysis detailed in 2.1.

2.4 Prism simulation parameters and field expansion testing scenarios in VR

The simulations were developed using Unity 3D (Unity Technologies, San Francisco, CA, US) with Oculus Integration for the Meta Quest 2 HMD (Meta Platforms Inc., Menlo Park, CA, US). A “primary” VR camera was set to render the subject’s perspective (original viewpoint) in the HMD (Fig. 4). Two 3D rectangular prism objects were added above and below the primary camera with a vertex distance following the optical PP fitting guidelines (i.e., 40° and 10° inter-prism separation between the prisms for HH and tunnel vision, respectively). Then, two “prism” cameras were positioned and aimed to capture the prism viewpoints following the theoretical analysis for each simulation. The prism cameras’ viewpoints were rendered on the surfaces of the corresponding prism objects so that they displayed the prism viewpoints at the correct location, thereby creating virtual PPs. The viewpoints seen through the virtual prisms could then be configured by adjusting the input parameters δ_h, δ_v, s_h, s_v, and d.

 

Fig. 4. Camera configuration for prism field expansion simulation in VR. The “subject’s” viewpoint is rendered by a “primary” VR camera which is flanked by an upper and a lower prism camera. The viewpoints of the two prism cameras were then rendered to the surfaces of two small rectangular objects (not pictured) which were placed in front of the primary camera.

Download Full Size | PDF

To reduce the impact of the apical scotoma that occurs when fitting optical prisms binocularly [9], we simulated a unilateral configuration that allows the fellow eye to cover the apical scotoma caused by the prism. The simulation also supports a bilateral see-through configuration, such as bilateral see-through prisms [18], where semi-transparent prisms are rendered in front of both eyes allowing for each eye to see both the original and prism viewpoints within the prism region simultaneously.

For HH, to cover the part of the visual field that poses the greatest hazard risk [21], the prism power was set to 45°, which translates both prism cameras horizontally (and vertically if using the oblique configuration) away from the blind field. The prism cameras are then rotated toward the blind field by 45° (and toward the horizontal meridian if oblique), thus expanding the visual field into the lateralized blind field.

For tunnel vision, to provide bilateral field expansion (i.e., on both sides of the narrow residual seeing field), the prism cameras are translated and rotated in opposing horizontal directions. The vertical translation and rotation of the prism cameras are the same as previously described. Assuming a residual central field of ∼20° diameter, to provide field expansion that covers the region of greatest hazard risk in tunnel vision, ∼15° [11], the prism power should be set to at least 5°, so that the prism viewpoint covers the 15° eccentricity. Larger prism powers may be used if discontinuities between the seeing field and the prism viewpoint can be tolerated.

To allow for normal eye-scanning behavior while still providing field expansion, the virtual prisms extend into the blind fields by at least 15° (c.f. MPP, [10]). In our simulations, the virtual prisms were set to be ±45° (width) × 20° (height) and positioned at ±20° vertical positions for HH. For tunnel vision, the prisms extended into the blind field both horizontally and vertically 15°.

We conducted simulations under lower complexity conditions in a static environment to investigate the apparent differences in viewpoint changes among the prism simulation methods. The reference objects and viewpoints were evaluated for each of the field expansion methods and each of the virtual prism configurations within a three-dimensional field diagram environment.

Next, we conducted simulations under higher complexity conditions in a dynamic environment. To investigate the impact of viewpoint changes in the dynamic environment, we analyzed the patterns of optic flow specific to the original and prism viewpoints during a simulated pedestrian collision event (i.e., walking in a virtual shopping mall populated with other pedestrians).

3. Results

3.1 Viewpoint changes in field expansion in the static condition

In the static condition, we rendered a three-dimensional field diagram with radial steps of 10° on the inner surface of a large sphere (Fig. 5(a)). The subject’s viewpoint was placed at the center of the sphere. We chose to apply the field diagram to the inner surface of the sphere because the spherical surface maintains the angular relationships seen from the center without tangential projection distortion as would occur when viewing the field diagram on a flat display. Accordingly, any translation of the subject’s viewpoint should then produce decentration distortion of the field diagram.

 

Fig. 5. Virtual environment for prism simulations in static condition. (a) Three-dimensional field diagram wrapping around the virtual subject’s viewpoint (camera) as the background. (b) Original viewpoint of the reference cubes for simulation.

Download Full Size | PDF

A set of reference objects (i.e., 0.5 m × 0.5 m × 0.5 m cubes) were aligned in a 3(H) × 5(V) × 3(D) grid in front of the virtual subject’s viewpoint (Fig. 5(b)). The cubes were placed at 3 m, 4.5 m, and 6 m depth and ±2 m above/below the primary position of gaze. The near, middle, and far-left cubes along the horizontal midline were positioned at 35°, 25°, and 20° horizontal eccentricities, respectively. The cubes on the right side were aligned at the same horizontal eccentricity as the left side. The cubes are also vertically aligned in a similar way to the horizontal alignment.

Figure 6 shows the results of the simulated viewpoint changes for left HH patients from the (virtual) subject’s point of view. As expected, the physical prism simulations via angular shift, which are exact simulations of the physical optical prisms (Fig. 6(a) & (c)), show viewpoint distortions due to the translation and rotation of the prism viewpoint, while the simulation using linear shift (image crop-and-shift) shows no rotational distortion. Since the horizontal prism power was set to 45° (Fig. 6(a)), the upper left reference cube (pink) on the nearest plane is shifted to 15° right of the vertical meridian of the upper prism. The perspective change is most apparent if we focus on the facing direction of the cube shown through the prism. The cube through the upper prism appears to be rotated clockwise horizontally compared to the original cube arrangement, which indicates the prism viewpoint is located right from the original viewpoint. Note that all reference cubes were originally facing the direction orthogonal to the fronto-parallel plane. Such rotational viewpoint distortion does not exist when the prism viewpoint is rendered by via linear shift (image crop-and-shift, Fig. 6(b)).

 

Fig. 6. Simulation results for horizontal peripheral field expansion using (a) the angular shift method (i.e., optical prisms) and (b) the linear shift method (i.e., image crop-and-shift), and for oblique peripheral field expansion using (c) the angular shift method (i.e., optical prisms) and the linear shift method (i.e., image crop-and-shift), assuming that the prisms are fitted for left HH patients.

Download Full Size | PDF

Although there is no rotational distortion observable in Fig. 6(b), it can be seen that the upper-left pink cube is now aligned with about 20° horizontal eccentricity. It is because of the decentration distortion that occurs in the captured image which is cropped and shifted from its original projection location to the optical axis. Since the linear size of the object in the far eccentricity is larger, when it is linearly shifted to the central eccentricity, the displayed object (and the space between) appears to be larger and farther apart than its ground truth (angular size mismatches). This effect is most notable when an object (e.g., upper-left pink cube in Fig. 6(b)) is shifted from a mid-range eccentricity where the difference is maximal. As a consequence, the object appears to be shifted slightly farther (e.g., 20° instead of 15°).

For oblique peripheral field expansions (Fig. 6(c) & (d)), the upper and lower prism viewpoints now cover the reference cubes (purple, cyan, and light green) aligned around the eye level (i.e., the horizontal meridian of the original viewpoint). The viewpoint distortion is less visible in the oblique prism because the vertical deflection of the oblique prisms compensates for the vertical viewpoint shift. However, rotational viewpoint distortion is still apparent in the optical prism simulation (Fig. 6(c)), while it does not exist in the prism viewpoint rendered by image crop-and-shift (Fig. 6(d)), similar to the horizontal prism conditions. If the upper and lower prism viewpoints of Fig. 6(c) were stitched together along the inner edges and translated leftward, the cubes (purple, cyan, and light green) would not match the original viewpoint (between prisms). However, it will be a perfect match if we do the same stitching with Fig. 6(d).

Figure 7 shows the field expansion fittings for tunnel vision patients where the upper prism brings the reference cubes on the right side (dark purple and dark grey) to the vertical meridian (seeing field), while the lower prism brings the reference cubes on the left side (cyan and green) to center. Although the causes of distortions and patterns are the same as the prism simulations fit for the HH patient (Fig. 6), the distortion is less visible in tunnel vision prism fitting conditions because we usually fit a smaller power (20°) prism. Note that the amount of distortion (viewpoint shift and rotation) is non-linearly proportional to the tangent of prism power (Eqs. (1) & (2)).

 

Fig. 7. Simulation results for horizontal field expansion using (a) the angular shift method (i.e., optical prisms) and (b) the linear shift method (i.e., image crop-and-shift), and for oblique prisms using (c) the angular shift (i.e., optical prisms) method and (d) the linear shift method (i.e., image crop-and-shift), assuming that the prisms are fitted for tunnel vision patients with 20° residual seeing field.

Download Full Size | PDF

3.2 Viewpoint changes in field expansion in the dynamic condition

In the dynamic condition, we simulated a pedestrian collision event in a virtual shopping mall environment (Fig. 8; Visualization 1). The subject’s perspective was set to move along a straight path at 1 m/s. A virtual pedestrian walked 1 m/s along another straight path which intersects the subject’s path at 10 m from the initial position. The virtual pedestrian approaches the subject with 35° bearing angle relative to the subject’s walking path. Since the subject and colliding pedestrian were set to walk on a straight path with constant speeds, the bearing angle is maintained until the collision [11,21].

 

Fig. 8. The scenario for field expansion simulations in dynamic conditions. (a) Schematic of a pedestrian collision event and (b) a still frame of the collision event (Visualization 1).

Download Full Size | PDF

3.3 Optic flow discrepancy between the prism and non-prism views

To understand the perceptual effect of the prism simulation methods under dynamic conditions (i.e., involving motion), we computed optic flow using the Lucas-Kanade method [22]. Detected feature trajectories were marked in green. The apparent location of the FOE was identified by tracing the tracked motion vectors to their point of intersection. Note that the FOE is specific to the direction of global self-motion [23] and invariant to local transformation (e.g., head rotation).

In this section, we will focus on the effect of PP fit for left HH patients for simplicity. The results for the tunnel vision patient fitting are fundamentally the same as the HH case. However, due to the smaller prism power, the effect is smaller as stated in the previous section. Most importantly, since tunnel vision patients do not have much of residual field in periphery, we expect the perceptual effect will be negligible. For more details, see Supplement 1 for prism simulations for tunnel vision patients.

Figure 9 and Visualization 2, Visualization 3, Visualization 4, and Visualization 5 show the resulting optic flows for the simulated collision events. Since the pedestrian’s bearing angle is 35° and the prism power is 45°, the pedestrian through the prism viewpoint is shifted to 10° on the right side of the visual field (seeing side) for all prism simulations. The pedestrian’s eccentricity is maintained throughout the scenario, but its angular size increases as the distance to the pedestrian decreases (looming, [11]).

 

Fig. 9. The optic flows for horizontal PPs using (a) the angular shift method (i.e., optical prisms) (Visualization 2) and (b) the linear shift method (i.e., image crop-and-shift) (Visualization 3). Oblique prisms using (c) the angular shift method (i.e., optical prisms) (Visualization 4) and (d) the linear shift method (i.e., image crop-and-shift) (Visualization 5), assuming that the prisms are fitted for left HH patients. Tracked feature trajectories of the were marked in green. The focus of expansion (FOE) of the original viewpoint is marked as a magenta circle, and the FOEs based on the prism viewpoint are marked as red circles.

Download Full Size | PDF

As expected, the upper prisms in the horizontal PP simulations (Figs. 9(a) & (b)) are not helpful for detecting a possible collision because the upper prisms are aiming upward. The lower prisms provide more valuable information on a possible collision with the pedestrian, but with the optical prism simulation (Fig. 9), the pedestrian appears to be approaching from the diagonal direction (i.e., see the pedestrian’s feet orientation in the lower prism viewpoint in Fig. 9(a)). Since the pedestrian is shown on the right side, the pedestrian appears to have already passed in front of the subject. Note that the orientation of the floor tiles in Figs. 9(a) & (c) clearly shows a rotated viewpoint, while the tiles in Figs. 8(c) & d show that the image is merely shifted horizontally.

Tracing the motion vectors on the prisms reveals a 45° horizontal rightward shift of the FOE (red, prism viewpoint) from the ground truth (magenta, original viewpoint) FOE (Fig. 9(a) & (b)). A similar FOE shift occurs in the oblique prism configuration (Figs. 9(b) & (d)), but the shifting is split by the upper and lower prisms. This indicates that there exists a strong optic flow discrepancy between prism and original viewpoints which may make it difficult for the subject to judge which direction from which the pedestrian approaches. Combined with the above-mentioned pedestrian’s rotated viewpoint, this directional confusion may provide a potentially misleading collision cue, incorrectly suggesting that the potentially colliding pedestrian is a non-colliding pedestrian.

3.4 Apparent non-colliding pedestrian motion through the prism view

The simulation of the prism viewpoints also revealed the reversal of the eccentric shift of non-colliding pedestrians. Figure 10(a) illustrates the schematic of a non-colliding event where the pedestrian appears with a bearing angle of 0°, but passes by the subject at the collision point. If the subject keeps looking in the forward direction while walking, the eccentricity of the pedestrian increases as the pedestrian passes by the subject. However, due to the prism shift, the initial eccentricity of the pedestrian through the prism view (Fig. 10(b)) starts from the non-zero eccentricity matching with the prism power on the right side, and then decreases as time goes by, making the pedestrian appear to move toward the vertical meridian. This reversal of the pedestrian’s apparent motion may also impact collision judgments where non-colliding pedestrians may be judged as colliding. To illustrate this effect, we simulated a non-collision scenario, in which the patient gazes through a simulated bilateral see-through prism [18] which superimposes a semitransparent prism viewpoint over the subject’s viewpoint to allow for the viewing of both viewpoints simultaneously (Fig. 10(c); Visualization 6).

 

Fig. 10. Prism field expansion eccentricity effect. (a) Schematic of a non-colliding event showing the approaching pedestrian passes by the subject, showing the bearing angle of the pedestrian increases over time. (b) Depiction of the same event through the prism, showing the apparent eccentricity decreases over time. (c) A simulated non-collision event illustrating the prism shift-eccentricity effect (Visualization 6).

Download Full Size | PDF

4. Discussion

Prism field expansion is most commonly depicted as a linear shift of visual information from the blind field to the residual seeing field (i.e., simple image translation, or crop-and-shift). This assumes that the angular relationships between the patient and their environment remain unchanged when gazing through the prism. However, our prism simulation showed that this is not the case and is not the result of optical distortions or aberrations. In fact, the prism view discrepancy is caused by the rotation and translation of the prism viewpoint, which changes the apparent locations, sizes, and orientations of objects.

Because the expanded visual field appears rotated and shifted, it also changes the patterns of optic flow, which may alter judgments about potential collision hazards seen through the prism while walking. Therefore, if the goal of prism field expansion is to enhance mobility by making potential hazards visible when they would otherwise be missed, then we must consider how the content of the prisms is modified by the change in viewpoint. For example, if patients make a collision judgment solely based on the prism viewpoint, they may erroneously judge colliding pedestrians as non-colliding (or oppositely a non-colliding pedestrian as colliding, see Visualization 6). Such a situation may result in potential risk or injuries not directly related to the actually colliding pedestrian (e.g., colliding with other pedestrians or environmental hazards). Although the standard protocol for prism fitting and usage suggests that the patients scan into the blind field when something captures attention through the prism view (e.g., possible colliding pedestrian) [8,10], our simulations showed that collision judgment through the prism viewpoint may misguide in many ways because the prism field expansion rotates the viewpoint, which may cause the patient to misperceive the approaching pedestrian's true heading direction which can serve as a cue for avoiding collisions [24].

The image crop-and-shift method provides similar field expansion, but without rotating the viewpoint. Therefore, the objects seen in the prism viewpoint appear at a different eccentricity without altering the relative angular relationships seen from the primary viewpoint. However, the process of shifting the captured image away from its original location relative to the optical axis introduces tangential/decentration distortion [25] which needs to be corrected at runtime. While such correction can be implemented using additional image processing techniques, this may constrain the usefulness of this field expansion method as it introduces further computational demands to the image processing pipeline.

Horizontally flipping the prism image before displaying it on the virtual prism object is another approach that may reduce apparent misinformation through the prism viewpoint. With the flipping of the prism viewpoint, non-colliding pedestrians passing by the patients move toward outer eccentricity, not moving toward the center of the visual field. The impact of the decentration should also be reduced as the shifted image will be tangentially stretched toward the outer eccentricity.

The simulation tool presented here offers a novel platform in VR for the evaluation of various field expansion methods, and viewpoint changes, and allows for the bespoke fitting of visual aids for various visual field deficit conditions (including but not limited to HH and tunnel vision). The tool is intuitive and simple to use, requiring simple input parameters to produce any amount of field expansion in any sort of configuration (e.g., horizontal, oblique, unilateral/bilateral, opaque, transparent/multiplexing, etc.). This is a clear advantage over physical optical prisms which require costly manufacturing and precise custom fitting. Additionally, because the simulation uses cameras and image processing instead of refraction and reflection (as in optical prisms), the image quality of the prism viewpoints is not subject to the many issues of the optical prisms such as chromatic aberration, total internal reflection, obscuration scotomata, etc. [15].

Previous works have evaluated the optical field expansion methods in simulated collision scenarios [26–28], however, these investigations did not allow for free walking, scanning, or natural avoidance behaviors. Recently, we developed a naturalistic collision detection measuring platform using a standalone VR headset to allow for natural avoidance behaviors to emerge without restrictions to locomotion or gaze [29]. Using this platform, patients navigate a virtual shopping mall populated by both colliding and non-colliding pedestrians. Patients are tasked with first detecting and responding to colliding pedestrians, then avoiding the collision in whatever manner they may do in the real world (e.g., slowing, turning, etc.). We are currently incorporating our field expansion simulation tool into the existing VR mobility evaluation platform to evaluate the impacts of various field expansion methods, including the prism image flipping method, on the detection and avoidance of possible collisions in a crowded virtual environment.

Collision avoidance capabilities and overall mobility will also benefit from our virtual field expansion simulations due to the flexibility and lack of constraints present in optical visual field expansion methods. Specifically, optical field expansion methods are limited in their effective prism power and physical configuration on the carrier lens. Our field expansion simulation is not bound by these limits and can achieve any degree of expansion in both horizontal and oblique configurations without sacrificing expansion along any axis. Along this line, we are also implementing and testing the feasibility of virtual prisms in augmented reality (AR) and mixed reality (MR) headsets as real-world assistive devices. 

5. Conclusion

We designed a novel prism field expansion simulation tool in VR to evaluate the changes in viewpoint produced by two different field expansion methods. We illustrated how optical prisms produce new viewpoints via translation and rotation, how linear field expansion can be achieved via image crop-and-shift, and how each simulation method may affect the detection of potential collision hazard under dynamic conditions such as walking. Understanding how different field expansion methods produce new viewpoints, how those changes alter the patient’s interactions with the environment, and what method may be most optimal for mobility enhancement needed steps toward developing effective real-world AR/MR applications for patients with visual field deficits.