1. Introduction

Polarization describes the path along which light’s electric field vector oscillates. The polarization state of a plane wave is fully constrained by its intensity, the ellipticity and azimuth angles of its polarization ellipse, and the degree to which it is or is not fully polarized. This latter quantity is known as the degree of polarization (DoP), a measure of the mutual coherence between orthogonal components of the electric field [1–3]. Most compactly, these four parameters are represented by the four Stokes parameters, written in vector form as $\vec {S} = \begin {bmatrix} S_0 & S_1 & S_2 & S_3 \end {bmatrix}^{T}$.

Consequently, any instrument designed to measure the Stokes vector—a polarimeter, or imaging polarimeter if this is done over a photographic scene—must take at least four projective and independent intensity measurements. These can be separated in time as polarization optics are readjusted, mechanically or otherwise (division-of-time), along multiple beam paths (division-of-amplitude), or by using a sensor whose pixels are covered with polarization analyzers (division-of-focal plane). In more recent “channeled” schemes [4], polarization information can be encoded as fringes in the spectral [5] or spatial domains [6]. Each method presents unique benefits and drawbacks [2,7].

Recently, we demonstrated diffraction gratings capable of re-directing light on the basis of its polarization state [8]. The gratings are formed from an array of subwavelength-scale pillar elements possessing form birefringence — a metasurface. Using the Jones calculus and Fourier optics, a metasurface grating can be designed to primarily direct light into a desired set of diffraction orders. Each of these orders can be made to act as a polarization analyzer sensitive to arbitrarily selected polarization states (Fig. 1(a)). Measuring the intensity of at least four such orders, given an appropriate invertible selection of these analyzer polarization states, yields sufficient information to determine the Stokes parameters $\vec {S}$ of plane-wave-like light illuminating the grating.

 

Fig. 1. General concept: (a) A periodic metasurface grating consisting of repeating, polarization-sensitive subunits can function as a diffraction grating designed to direct incident light primarily into a set of desired diffraction orders which function as polarization analyzers for desired polarization states with a Malus’ Law intensity response. If four of these are used and their intensity measured, the grating can function as a single-component full-Stokes polarimeter. (b) If the metasurface is paired with an imaging system in an infinite-conjugate scenario, i.e., for a scene infinitely far away, this can be done for many incident viewing angles in parallel, forming four images of the scene (along with the undiffracted zero order) that have each been analyzed with respect to their characteristic polarization. These can be combined to determine the Stokes vector across the scene. (c) As the system is infinite conjugate, distances on the sensor plane correspond to angular swaths. Here we define several important angles of the system, described in depth throughout Sec. 2.

Download Full Size | PDF

If light at multiple angles illuminates the grating, i.e., from an object envisioned to be at infinity, this occurs simultaneously for each angle over some angular field-of-view (FOV). If the grating is paired with imaging optics (in a way that, for now, we leave undefined) and, by some mechanism, its FOV is limited (discussed extensively below), the four orders of interest will form four copies of the image of the scene on separate regions of a sensor, each effectively having been passed through a polarizer for a polarization chosen during the grating’s design (Fig. 1(b)) [8]. If the images are subsequently co-registered and processed, an image of the scene can be formed in which $\vec {S}$ is known across the angular FOV: in other words, a polarization image.

An imaging system paired with a metasurface polarization grating is, effectively, a division-of-amplitude imaging polarimeter. This approach may present a number of unique benefits for polarimetry, among them:

Here we develop this final aspect in particular, showing that a metasurface can serve as a simple polarimetric “attachment” to an existing camera. Throughout, we seek to understand when and how this adaptation can be carried out by simply placing a metasurface grating in front of an existing imaging system. In Sec. 2, we elaborate the rules and tradeoffs governing this and then apply them to an example machine vision system in Sec. 3; in Sec. 4, we discuss what this may add to existing technologies and approaches for imaging polarimetry and conclude in Sec. 5.

2. Imaging polarimetry with a metasurface grating: principle and design considerations

In this section, we describe how an existing intensity-only imaging system can be adapted by a metasurface to create one capable of full-Stokes polarimetry. In Sec. 2.1, we briefly review key concepts of compound imaging lenses that greatly simplify our treatment throughout. Then, in Sec. 2.2, we review how the Stokes parameters can be mathematically extracted after passage through a metasurface grating. In Sec. 2.3, we describe basic geometrical considerations that govern the pairing of a grating with a given imaging system. Then, in the final two sections, we discuss important issues including the actual placement of a metasurface grating in front of given imaging lens (Sec. 2.5) and methods of limiting the FOV prior to the grating (Sec. 2.4). Both of these impose important tradeoffs which manifest as different forms of (undesired) vignetting.

2.1 Abstracting multi-element imaging systems

Any practical imaging system is a cascade of many lens elements for any number of purposes, including the correction of mono- and polychromatic aberrations. Abstractions afforded by paraxial geometrical optics can simplify matters considerably even if the precise internal details of a given lens are not known.

A multi-element lens can be simplified to a paraxial singlet model [13] using a process known as Gaussian reduction. This is a standard method of geometrical optics but for completeness we review it here. A photographic objective lens can reduced to two principal planes and two focal points (front and rear). These provide sufficient information to trace paraxial rays, thus permitting any multi-element system of lenses to be abstracted as a (thick) singlet lens.

In addition, a real multi-element imaging lens has an aperture stop, usually a deliberately placed iris (often adjustable in size) or, in its absence, one of the lens surfaces. The aperture stop’s image in object space, i.e., through the lens elements that precede it, is known as the entrance pupil. The entrance pupil may be located in front of the first surface of a compound lens or it may be virtual, located behind it. Its position, which serves as an angular reference point for any camera system, and size, which limits radiative throughput, i.e., the effective diameter collimated ray bundles imaged by the system, are also critical in specifying any imaging system.

Viewed from object space, then, a complex, multi-element lens system can be abstracted as a paraxially equivalent singlet lens (from Gaussian reduction) and a throughput-limiting entrance pupil. This abstraction is used throughout to describe simple design rules for pairing polarization-analyzing gratings with existing imaging optics; these rules are then universal and agnostic to the precise internal details of the lens system involved, under the paraxial approximation.

2.2 Polarimetry with a metasurface: basic principles

2.2.1 Metasurface polarization gratings

A metasurface is a diffractive optical element comprised of phase-shifting elements with subwavelength spacing. These elements can be anisotropic and exhibit polarization dependence. When suitably designed, metasurfaces can enable a variety of polarization-sensitive optical elements [8,14–16].

Among these are polarization analyzing gratings, i.e., gratings whose diffraction orders exhibit analyzer-like behavior for a custom-specified set of polarization states. The design of these is thoroughly described in [8]. Briefly, these gratings consist of anisotropic metasurface elements grouped into periodically repeating unit cells. For the sake of simplicity, here we consider gratings with square-shaped unit cells repeated on square lattices as in [8], though we note that this can be generalized (see App. B.). Incident light diffracts from these unit cells into discrete orders. The response of each order to incident light’s polarization state can be controlled through the design of the elements comprising the grating’s unit cell. It is possible to design a given set of diffraction orders to implement polarization analyzer-like behavior so that the intensity observed on a set of diffraction orders depends on the incident polarization itself.

2.2.2 Point polarimetry with a metasurface grating

Each such diffraction can be described by a Stokes-like vector $\vec {D}_n$ so that the intensity of order $n$ in response to incident light is given by

If the intensity of at least four such polarization-sensitive orders is measured (as in Fig. 1(a)), $\vec {S}_{\text {inc}}$ can be determined. Suppose the four intensity measurements are assembled into a vector of intensity readings $\vec {I} = [I_1, I_2, I_3, I_4]^{T}$. Then, Eq. (1) can be written simultaneously for each order as

2.2.3 Imaging polarimetry with a metasurface grating

If paired with imaging optics, these metasurface gratings can also be used for imaging polarimetry, that is, determining $\vec {S}_{\text {inc}}$ over a photographic scene. We consider here camera systems focused on an object located at or near infinity (that is, infinite-conjugate systems), the scenario in which a majority of cameras are used. Similar arguments to those given here are readily derived for finite-conjugate systems as well.

As sketched in Fig. 1(b), a metasurface grating paired with an infinite-conjugate imaging system will form several images of a photographic scene on the sensor plane. These correspond to the innermost four diffraction orders (and the zero-order, present to some degree in any diffractive optical element including these gratings). To accomplish this, the grating must be oriented so that its lattice vectors are rotated $45^{\circ }$ relative to the axes of the sensor.

This image formation process can be understood using the picture given in Fig. 1(c), wherein the infinite-conjugate system is imagined to be a singlet lens one focal length from a sensor plane (an abstraction we are entitled to make, as discussed above). Crucially, we stipulate that — by some means, left abstract for now — the angular range of ray fields permitted to impinge on the grating is limited to some range given by $\theta _{\text {FOV}}$. Figure 1(c) traces just one diffraction order (and the zero order, in gray) through the system at the extrema and center of the field-of-view (FOV). The grating’s role is to deflect incoming light in such a way that multiple images of the scene can be formed on non-overlapping regions of the sensor, i.e., so that the order shown just overlaps with the zero order at the edges of the FOV.

Each of these images will have effectively passed through a polarization analyzer corresponding to the diffraction order’s $\vec {D}_n$. If these images are then aligned and registered to each other, Eq. (3) can be applied across the FOV to yield a full-Stokes polarization image.

This simplest picture (i.e., Fig. 1(c)) has neglected two crucial issues: 1) Where, relative to a potentially multi-element lens to be adapted, the metasurface grating is placed, and 2) How the FOV of the system is limited prior to the grating. Both of these present important design considerations and tradeoffs (in particular, vignetting), and are discussed extensively in Secs. 2.5 and 2.4 below. First, however, we consider more basic geometrical design constraints.

2.3 Geometrical and field-of-view (FOV) considerations

A schematic view of an image as would be formed on the camera sensor is shown in Fig. 1(d). Again, and as discussed below, the FOV of the entire system must be limited prior to the grating such that individual sub-images do not overlap on the camera sensor (as sketched in Fig. 1(d)) which would prevent disambiguation of angular from polarimetric information. As depicted in Fig. 1(d), we assume here that the FOV is limited to a circle simply due to convenience of implementation, though the choice is not optimal as discussed further in Appendix B..

Being infinite-conjugate, distances on the sensor can be identified with angular quantities in object space. There are several important angles labeled in Fig. 1(d). First is the native diffraction angle of the square grating, given by the Bragg condition as

A second important angle is the camera system’s native diagonal FOV, $\theta _{\text {camera}}$, governed directly by the imaging optics’ focal length $f$ and the sensor’s side-dimension $d_{\text {sensor}}$ as

The usable FOV over which polarimetry can be performed with the grating without overlap of the sub-images and the zero order (i.e., the diameter of each circle in Fig. 1(c)) is, by inspection, $\theta _{\text {grating}}$ so that we may write

A field stop should be arranged to limit the incoming FOV to this value to prevent unwanted overlap of the sub-images, mixing polarization information from different channels and viewing angles. We elaborate on this point in the next section.

The sensor space occupied by the configuration of the four sub-images and zero order, from corner to corner, spans an angle of

Here, $\sim$ denotes equality in the limit of small $\theta _{\text {grating}}$. If $\theta _{\text {grating}}$ is chosen to be too small, the sensor will not be filled to its edges as depicted in Fig. 1(d). On the other hand, if $\theta _{\text {grating}}$ is too large, the subimages will spill over the sensor edges. Ideally, the subimages will just fill the allotted sensor space. An existing imaging system to be adapted by a metasurface has a native FOV $\theta _{\text {camera}}$ predefined by Eq. (5). The period of the grating, then, should be chosen so that $\theta _{\text {grating}}$ allows Eqs. (5) and (7) to agree, the situation sketched in Fig. 1(d).

2.4 Limiting the FOV before the grating

As noted above, this polarimetric scheme relies on limiting the range of incident viewing angles in the system prior to the grating. Otherwise, individual sub-images will overlap, mixing together spatial and polarization information from different channels.

How this is accomplished is a crucial design consideration unto itself. In any imaging system, the FOV imaged is limited by a field stop which could be any surface of the system. In an infinite-conjugate system, the sensor itself often forms the FOV-limiting field stop. Here, however, in order to prevent the undesired overlap of diffraction orders, the FOV must be restricted prior to the grating. One clear solution is to create an intermediate image plane before light reaches the grating. A suitably sized aperture could be placed there to restrict the FOV to the $\theta _{\text {FOV}}$ dictated by Eq. (6). Creating this intermediate image necessitates additional lens elements. In particular, at least two are needed, one to create the intermediate image and one to recollimate the light in an afocal telescope arrangement as shown in Fig. 2(a) that, depending on the application, may or may not enact magnification. This solution, however, requires elongating the proposed attachment by a minimum of $f_1+f_2$ if two lenses are used. Additional third and fourth lenses could shorten the attachment at the expense of added weight and complexity. The complexity inherent in creating an intermediate image, then, tends to defeat the compactness sought here.

 

Fig. 2. Limiting the field-of-view before the grating: (a) Ideally, an intermediate image would be created before the metasurface grating in which an aperture could be placed to limit the range of viewing angles. This would require at least two additional lenses. (b) In the absence of this, an aperture of size $D_A$ placed some distance $d_{\text {g}, \text {A}}$ from the metasurface grating can be used instead. This simpler approach has vignetting as a tradeoff. Three cases are possible, depending on $D_A$, $d_{\text {g}, \text {A}}$, and the size of the metasurface grating $D_g$. The aperture diameter $D_A$ should be set to limit the viewing angles beyond $\theta _1$ to avoid image overlap. In (i), $D_A>D_g$ and viewing angles between $\theta _2$ and the desired cutoff $\theta _1$ are vignetted by the aperture, while other angles pass through the system fully. In (ii), $D_A<D_g$ and all viewing angles are at least partially vignetted. The limit case of this is in (iii), when the aperture is closed and no viewing angles can be imaged. A schematic of how vignetting would appear in the raw image obtained is given in (c).

Download Full Size | PDF

In this work, we take a more minimal approach, using an aperture placed some distance in front of the grating to limit the FOV entering the system (with no additional lenses). This choice leads to considerably less bulk, but the aperture—which is not located at a true intermediate image—will additionally induce vignetting, the tradeoff for its simplicity. The extent of this vignetting depends on the relative size of the metasurface grating $D_g$, the aperture diameter $D_A$, and the distance between them $d_{\text {g}, \text {A}}$, discussed here and shown schematically in Fig. 2(b).

The aperture will assure that all viewing angles beyond some maximum angle (here denoted as $\theta _1$) will be completely obstructed and cannot form an image. This angle is given by

The parameters of the system ($D_A$, $D_g$, and $d_{\text {g}, \text {A}}$) should be chosen so that $\theta _1 = \theta _{\text {FOV}}$ to avoid image overlap. However, since the aperture is not placed at a true image plane, this angular cutoff is not immediate. The first viewing angle $\theta _2$ able to traverse this aperture field stop without attenuation is given by

Viewing angles between $\theta _1$ and $\theta _2$ experience vignetting that increases linearly from $\theta _2$ to the cutoff at $\theta _1$. This situation, and the definitions of $\theta _1$ and $\theta _2$, are shown in case (i) of Fig. 2(b). As the aperture is placed farther and farther away (i.e., as $d_{\text {g}, \text {A}}\rightarrow \infty$, and, accordingly, the size of $D_A$ will have to increase to continue limiting the FOV to $\theta _1$), $\theta _2$ approaches $\theta _1$ and the vignetted region disappears.

On the other hand, as the aperture size $D_A$ becomes smaller and approaches the size of the metasurface grating $D_g$, eventually $\theta _2 = 0$ when $D_A=D_g$ so that only on-axis light is unvignetted. If $D_A<D_g$, no viewing angle can pass unvignetted and the entire FOV is at least partially vignetted (case (ii) in Fig. 2(b)). In this case, the aperture itself—though intended as a field stop—also becomes the system’s throughput-limiting aperture stop. Eventually, if $D_A$ approaches 0 (case (iii)), no light can traverse the system unvignetted.

In the implementation of Sec. 3, we take this aperture-only approach to limit the FOV prior to the grating, keeping in mind these important tradeoffs.

2.5 Placement of grating relative to imaging optics

A second important design consideration is where the metasurface grating is placed relative to the existing imaging optics. This question is the subject of Fig. 3, where we trace a single diffraction order (i.e., polarization channel) and presume an existing camera system focused at infinity under different grating placement conditions. The imaging lens, abstracted in accordance with Sec. 2.1, is depicted as a co-located entrance pupil and effective thin lens, drawn as an arrow.

 

Fig. 3. Placement of the metasurface polarization grating in front of a system focused at infinity: In each case (a)-(d), a schematic ray trace of a parallel ray bundle from the extreme (left and rightmost columns) and center (center column) viewing angles of the FOV is shown for one of the grating’s diffracted orders (similar arguments apply for the other three). In each case, the trajectory of the undiffracted zero order is shown in gray. A schematic of each case’s impact on the raw image obtained is shown at right. Throughout the figure, it is assumed that the FOV has been limited prior to the grating by some means (such as by an aperture in this work), but the vignetting effect of this choice on each field is not shown. (a) Ideally, the grating would be placed in the imaging system’s entrance pupil (shown here as a black aperture) or a conjugate plane thereof, filling it completely. In this ideal case, all parts of the FOV receive equal illumination without vignetting, and the system’s light throughput is not limited by the grating’s inclusion. (b) If, however, the grating cannot be placed in the entrance pupil (e.g., because it is inaccessible) and must be pulled back somewhat, vignetting will necessarily occur for all viewing angles except $0^{\circ }$. (c) As the grating’s separation from the entrance pupil increases, eventually we find that all viewing angles past a certain point are completely vignetted and contribute no signal to the image. (d) The vignetting of situation (b) can, however, be eliminated by reducing the size of the metasurface so that it becomes the entrance pupil, evening out the radiance seen at the sensor allowing full use of the detector’s dynamic range at the expense of reduced illumination overall. (Note that in this figure an infinite-conjugate imaging system is abstracted as a co-located entrance pupil and singlet thin lens. While not true in general, this depiction simplifies this discussion without affecting its conclusions).

Download Full Size | PDF

Ideally, the grating would be placed in the entrance pupil and made just large enough to fill it, as shown in Fig. 3(a). Then, as dictated by the entrance pupil’s definition, all incident light would pass through the imaging system and inclusion of the grating would not further limit the system’s light throughput. If the grating angle is chosen as described in Sec. 2.3, and the FOV limited suitably, an image of the scene can be formed over the entire FOV, from its two extreme angles (blue, left, and green, right) and on-axis rays (red, center). Rays of the undiffracted zero-order are drawn in gray.

The grating could also be placed in the exit pupil, aperture stop, or any conjugate plane thereof; the decision of which is optimal would depend on the case. Each image of the aperture stop will be of a different transverse size, and thus easier to fill with a small grating. On the other hand, the rays passing through these demagnified copies will approach at steeper angles (by the Conservation of Étendue), a regime in which the grating may not perform as well. In another approach, the grating could be placed anywhere and, given suitable relay optics (e.g., a 4f system of two lenses), imaged into one of these stops. This, however, necessitates additional optics and propagation length. See Appendix A.

Given an existing imaging system whose design cannot be modified, placement of the grating in the entrance pupil, aperture stop, or an image thereof may be impossible. These planes could be inside the objective lens, in between the objective and sensor, or otherwise inaccessible without dismantling the system, contrary to the ideal of a nondestructive polarimetric attachment. The best we can do, then, is to place the grating somewhere in front of the optical system, accepting that this location is not the entrance pupil. This situation is shown in Fig. 3(b). If the grating remains the same size as the entrance pupil, there will inevitably be vignetting as it backs away. All rays from one extremal ray field (blue, left) will be imaged, but as the viewing angle changes, some rays will be lost, producing the vignetting effect schematically shown at right. If the grating is further displaced from the entrance pupil, this effect can become so severe that, eventually, some viewing angles cannot contribute at all to the final image, leading to a sharp cutoff (Fig. 3(c)). This cutoff can be eliminated if the grating is made larger, with the necessary grating aperture increasing with distance from the entrance pupil.

For many imaging systems, however, we can place the grating just before the lens housing and still realize the situation of Fig. 3(b) in which the entire FOV is used avoiding a sharp cutoff, albeit with unequal sensor illumination. Reducing the size of the grating, however, can serve as a compromise: As shown in Fig. 3(d), if the grating’s size is reduced so that it becomes the system’s aperture stop, extremal viewing angles can still fully pass through the imaging optics’ entrance pupil, eliminating the vignetting effect. This is the regime in which we operate here, though we note that this choice comes at the expense of diminished illumination in areas of the image that were not vignetted before. This is the price to pay for “evening out” the image so that the dynamic range of the sensor can be fully exploited.

In analogy with the previous section (and Fig. 2(b)), the viewing angle $\theta _2$ where vignetting just begins is given by

It is important to note that the vignetting induced by an imperfect field stop (discussed above in Sec. 2.4) and that induced by the grating placement manifest in fundamentally different ways. The former affects all polarization channels (diffraction orders) equally and is centered on the orders (as in Fig. 2(c)), while the latter is centered on the final image plane, affecting a different part of each sub-image (as in Fig. 3).

3. Implementation

Here, we demonstrate the design rules of Sec. 2 through a practical example showing how a metasurface grating can serve as an “attachment” capable of converting an existing, intensity-only imaging system into one capable of full-Stokes polarimetry.

3.1 Choice of imaging system

Our chosen example system consists of a CMOS imaging sensor with a 1/1.8" format, roughly 7 mm $\times$ 5 mm in size (sensor: Sony IMX265, board and housing: IDS UI-3270LE). Since we design around the assumption of a square sensor, we work the smaller dimension (5 mm) in what follows. The sensor is paired with a C-mount, fixed focal-length photographic objective lens with $f=16$ mm (Edmund Optics, part no. 59-870). This pair represents a relatively generic imaging system for, e.g., machine vision applications. By Eq. (5), the system’s FOV is $\theta _{\text {camera}}=24.9^{\circ }$.

3.2 Grating design

Given $\theta _{\text {camera}}=24.9^{\circ }$, Eq. (7) mandates that $\theta _{\text {grating}} = 7.3^{\circ }$ to fully utilize the camera’s FOV. If $\lambda =530$ nm, assumed throughout this work, then Eq. (4) gives the grating period $D = 4.17$ µm $\sim 8\lambda$.

A metasurface polarization grating can then be fabricated to meet this specification. The unit cell of the grating used here is a $12\times 12$ array of TiO$_2$ pillars (each with a separation of $420$ nm and a height of $600$ nm) whose high index contrast with air in the visible ($\sim$2.3) and low loss enable shape-tunable, birefringent phase control [14,15]. The parameters of each pillar (length, width, and orientation angle) in the grating’s periodic unit cell are optimized to diffract as much light as possible into the innermost four diffraction orders of the grating ($(\pm 1, 0$) and $(0, \pm 1)$) while constraining these orders to act as polarizers for desired polarization states. This design procedure is described in depth in [8] and its supplement, while the fabrication of these TiO$_2$ metasurfaces with electron beam lithography and atomic layer deposition has been extensively described elsewhere [17].

The parameters given above — a $12\times 12$ array of pillars with 420 nm separation — give $D=5.04$ µm and $\theta _{\text {grating}} = 6.0^{\circ }$. To be conservative and assure good fit of the four sub-images on the sensor, this is somewhat smaller than the grating angle calculated above, slightly under-utilizing the sensor. The repeating unit cell of the grating is shown in Fig. 4(a), with a scanning electron micrograph (SEM) of the grating in Fig. 4(b).

 

Fig. 4. Adaptation of an intensity-only imaging system with a metasurface polarization grating: (a) Schematic of the periodic unit cell of the metasurface design, composed of $12\times 12$ TiO$_2$ pillars 600 nm in height, each separated by 420 nm (yielding a diffraction angle compatible with the system being adapted). This unit cell is repeated $\sim 300$ times in $x$ and $y$ to form a diffraction grating (chosen to be circular in profile) with a diameter of 1.5 mm. (b) A scanning electron micrograph (SEM) of the grating’s unit cell. (c) The sample is fabricated on a 1” fused silica substrate and is surrounded by a metallic (gold) aperture to assure only light passing through the grating enters the imaging system. (Photo courtesy of Leah Burrows, Harvard SEAS). (d) The grating is mounted just in front of the front face of our chosen objective lens in a tube that also houses a bandpass filter centered at 530 nm and an adjustable iris functioning as a field stop. This is shown implemented (top), as a schematic (middle), and as a ray trace in which the lens is represented abstractly by its entrance pupil and principal planes (as discussed in Sec. 2.1). Simple trigonometry verifies that the extreme of the angular FOV can pass through the lens without vignetting.

Download Full Size | PDF

3.3 Integration with imaging system

Next, we consider the polarimetric adapation of the chosen lens with the designed metasurface grating and any tradeoffs involved. Figure 4 shows the adapted camera prototype and clarifies the discussion that follows (from top to bottom: as a photograph, as a schematic, and as a minimal optical schematic with the camera lens abstracted in terms of its principal planes and entrance pupil).

The camera lens’ entrance pupil location is determined from an encrypted “black box” ray-tracing file provided by the lens’ manufacturer (this could also be determined, albeit more roughly, by looking through the lens from the front and measuring the apparent location of its stop). It is located $24.4$ mm from the front face of the first lens. In other words, the grating lies $24.4$ mm from the entrance pupil when placed directly in front of the lens housing, the closest accessible point. The entrance pupil’s size is governed by an adjustable aperture stop with $\textit {f}/1.6$ when wide open. Given that $\textit {f}/\#=f/D_{\text {EP}}$ and $f=16$ mm, $D_{\text {EP}} = 10$ mm with the stop at its widest.

The grating angle $\theta _{\text {grating}} = 6.0^{\circ }$ gives the FOV $\theta _{\text {grating}} = \theta _{\text {FOV}} = \pm 3^{\circ }$ over which imaging polarimetry can occur. The grating in this work is $1.5$ mm in diameter, a reasonable footprint for electron beam lithography ($\sim$3 hours of writing time on a 125kV e-beam system at Harvard University using the positive-tone e-beam resist ZEP520A). A photograph of the grating — surrounded by a stray light-blocking metallic aperture — is shown in Fig. 4(c).

Following the discussion of Sec. 2.5 above, these parameters confirm that a parallel ray bundle from the edge of the FOV diffracted by the grating will fully clear the camera lens’ entrance pupil and be imaged. This would remain true (retaining the situation of Fig. 3(d)) up to a grating diameter of $2.27$ mm, after which point vignetting would set in at the FOV’s edges as in Fig. 3(b). As described above, reduced throughput is a consequence of imaging through the grating: Including the 1.5 mm metasurface grating changes the camera’s f-number from $\textit {f}/1.6$ to roughly $\textit {f}/10.7$.

As shown in Fig. 4(d), the polarimetric “attachment” consists of a grating, chromatic bandpass filter, and an adjustable aperture. As discussed in Sec. 2.4, the aperture is a particularly important component, acting as a field stop to prevent crosstalk between quadrants of the image. The aperture should be set to limit the FOV to $\theta _{\text {FOV}} = \pm 3^{\circ }$. It is placed approximately 3 cm from the metasurface grating, so that its diameter $D_A$ should be set to about 1.6 mm. This is larger than the grating diameter but only slightly (as in Fig. 2(b), case (i)): only a small part of the center of the FOV between $0^{\circ }$ and $\sim \pm 0.15^{\circ }$ experiences no vignetting from the aperture. This could be improved by moving the aperture farther away from the grating at the expense of increasing the overall size of the prototype.

The chromatic bandpass filter is also a component of crucial importance. The metasurface grating naturally disperses wavelengths in addition to its intended role as a parallel polarization analyzer. Light incident over a broad bandwidth from the same viewing angle would smear over the sensor due to this angular dispersion, decreasing the resolution of the final image. This effect is worsened by the fact that each channel is affected differently, since the chromatic smear occurs in a different direction pointing outwards from the zero-order. Roughly, we can write the angular resolution of the final polarimetric image $\Delta \theta$ (neglecting any registration error) as

In the prototype of Fig. 4, $\Delta \lambda = 3$ nm. From Eq. (13) and the camera specifications above, this results in a 2-3 pixel blurring effect even given perfect registration. Thus, use of the metasurface grating presents an important tradeoff between radiative throughput and resolution.

In Supplement 1 (Sections S1 and S2) we discuss further this important limitation and present possible strategies for its mitigation.

3.4 Calibration

3.4.1 Grating alone at normal incidence

Multiple effects conspire to make the fabricated metasurface deviate in performance from its ideal, perfectly homogeneous counterpart whose behavior can be simulated ahead-of-time. Determining the real, ground-truth properties of the grating is important for determining the polarimetric data reduction matrix $\boldsymbol {A}$, through which intensity measurements are converted to polarization data (Eq. (2), referred to as demodulation in some communities).

This is known as polarimetric calibration. A first attempt at calibration can be made by testing the metasurface grating alone, independent of the imaging system of which it becomes a part. We perform Stokes polarimetry to find the characteristic four-element Stokes parameters $\{\vec {D}_n\}$ analyzed by each of the four orders of interest. The grating is illuminated with many ($\geq 4$) well-calibrated polarization states with known Stokes parameters and measuring the power response of each order with a calibrated photodiode so that its characteristic Stokes parameters $\vec {D}_n$ can be determined in a least-squares sense (see, e.g., [8]). This was done with a laser at $\lambda =532$ nm normally incident on the grating that passes in succession through a polarizer, half-, and quarter-waveplate, with the waveplates in variable rotation mounts. This process is shown schematically in Fig. 5(a). Importantly, the calibration in this work is referenced to a commercial rotating-waveplate polarimeter. This is suitable for the largely qualitative nature of the work here, but for precision applications a more traceable reference would be necessary.

 

Fig. 5. Normal-incidence calibration: (a) Each order of the grating has a characteristic Stokes vector $\vec {D}_n$ governing its intensity response to incident polarization. By illuminating the grating with a laser at normal incidence whose polarization can be controlled and measuring the power response of each order, the $\vec {D}_n$ are experimentally determined. (b) shows the diattenuation (top) and efficiency (bottom) for each order. Efficiency here is defined as the fraction of incident light directed to an order for unpolarized incident light. Perfection represents each of the four orders having an efficiency of 25%. The polarization ellipse corresponding to each order’s $\vec {D}_n$ are drawn below the plots with red and blue indicating opposite right- and left-handedness, respectively. (c) The $\vec {D}_n$ are plotted on the Poincaré sphere alongside the target regular tetrahedron. In (b) and (c), results from FDTD simulation are shown in red and measurements in green.

Download Full Size | PDF

The first element $D_{0,n}$ of each $\vec {D}_n=\begin {bmatrix} D_{0,n} & D_{1, n} & D_{2,n} & D_{3,n} \end {bmatrix}^{T}$ encodes the efficiency of the order, the fraction of incident of power in the order $n$ when unpolarized light is incident. The diattenuation of the order is

The results of this grating-alone measurement, cast in these terms, are shown in Fig. 5(b) and (c). In Fig. 5(b) and (c), results derived from a full-wave, finite-difference time domain (FDTD) simulation of the grating design are shown in red, while measured results (derived from the measurement depicted in Fig. 5(a)) are shown in green. Beneath the plot, the index of each diffraction order measured is given along with the polarization ellipse corresponding to its measured state-of-polarization $\hat {s}_n$ (blue and red denote the handedness of the ellipse). In Fig. 5(b), the diattenuation and efficiency of each grating order is plotted. The diattenuation of each order exceeds 85% and in some cases reaches the high 90s. Notably, the highest performing diffraction order is the one designed to analyze for circular polarization. Its function may thus rely primarily on geometric phase effects, which are more immune to fabrication shape errors. Figure 5(b) shows the efficiency of each order from the first element of each $\vec {D}_n$. For a given input polarization state (that is not unpolarized), the efficiency of a given order is at most twice this efficiency and at minimum 0, as governed by the order’s diattenuation as

Finally, in Fig. 5(c), we show the analyzer states $\{\hat {s}_n\}$ on the Poincaré sphere, alongside the target regular tetrahedron. These effects are accounted for in calibration.

We attribute the difference between the simulated and measured results evident in Fig. 5(b) and (c) to fabrication errors, in particular the discrepancy between the designed profile of the TiO$_2$ and those realized in fabrication. Size differences are already qualitatively evident by comparison of Fig. 4(a) and (b).

3.4.2 Whole instrument, whole FOV calibration

The grating characterization described above provides a baseline understanding but neglects other polarization-dependent effects that can happen inside the imaging system (e.g., Fresnel reflections off lens surfaces) and is only valid for normal incidence; however, the metasurface grating’s and the system’s polarization response may vary as a function of viewing angle across the FOV.

Both of these issues were addressed by a calibration scheme that fills the FOV of the camera with light of known polarization states as shown in Fig. 6(a). The exit port of an integrating sphere (IS) illuminated with green LED light is imaged using a 1:1 4$f$ relay system of two lenses of equal focal length which provides a collimated space for the linear polarizer and rotating quarter- and half-waveplates to reside. The IS’ exit port, and its image created after the 4$f$ system, provide an extended source of uniform radiance that, when passed through a final lens one $f$ away, filled the adapted camera’s entrance pupil (now the metasurface grating). This final lens is selected to fill the instrument’s $\pm 3^{\circ }$ FOV in angle space.

 

Fig. 6. System calibration over FOV: (a) The camera can be calibrated over its whole FOV simultaneously with light from an extended source, here the exit port of an integrating sphere (IS). Using a 4f setup, light from the IS is imaged to the focal plane of a lens just before the prototype, providing a collimated space for rotating polarization optics to reside. The final 50 mm lens provides uniform illumination over the camera’s FOV. Exposures are taken as the polarization optics rotate, switching between known incident states. (b) A sample camera frame from the calibration where the illuminating light’s polarization is very nearly orthogonal to the Stokes vector of the upper right order. (c) The relative efficiency of each part of each order over the FOV, showing vignetting from the aperture serving as a field stop. (d) The diattenuation variation for each order over the FOV. (e) Polarization ellipses corresponding to the $\vec {D}_n$ over the angular FOV. Blue indicates right and red left-handed $\vec {D}_n$. In (c)-(e), a threshold has been applied to isolate each polarization channel from the background.

Download Full Size | PDF

The sensor plane is conjugate to the integrating sphere exit port and four sub-images plus the zero order are formed on the sensor. A sample frame of the calibration is shown in Fig. 6(b): Here the incident polarization state is nearly orthogonal to the analyzer state of the upper right order which appears dark. By analyzing the intensity response of each pixel in each subimage to a set of known polarization states generated by rotating the waveplates, a $\vec {D}_n$ can be calculated in the least-squares sense for each pixel. This enables simultaneous calibration across the whole FOV with the entire instrument as-assembled.

Here, we used 25 rather arbitrarily generated (and relatively uniformly distributed) polarization states. However, the extensive analysis dedicated to optimizing such calibration schemes by others in the literature ([18], to name just one example) should be mentioned here.

Figure 6(c)-(e) summarize the results of this calibration. Figure 6(c) shows the efficiency of each point in each channel in the FOV, the quantity $D_{0, n}$. Again, this efficiency represents the transmission of unpolarized incident light, but unlike in Fig. 5(b), the plot here shows a relative efficiency, normalized by the highest value in the figure rather than the incident intensity. For each channel, including the zero-order, a fall-off is visible from the center of the FOV to the edges. This fall-off, as described in Secs. 2.4 and 3.3 above, is caused by the field-limiting aperture in front of the camera system.

This calibration data was normalized to the maximum pixel and cut-off below a threshold of 0.1; this results in each thresholded order being of a different size and shape. Imaging is only carried out at pixels where calibration information is available for each of the four orders, i.e., the “smallest” order after thresholding limits the size of the ultimate polarization image.

In Fig. 6(d), diattenuation is plotted across the field-of-view. The measurements are consistent with Fig. 5(b). Closer to the center of the sensor, diattenuation decreases which we attribute to the zero order bleeding slightly from the center into the inner corner of each channel, serving as an unwanted background flux. Figure 6(e) shows the $\vec {D}_n$ over the FOV as polarization ellipses (blue and red again denote the handedness of the ellipse’s rotation).

3.5 Imaging examples

Once calibrated, the camera can perform polarization imaging of objects. An example frame of raw data prior to data reduction is shown in Fig. 7(a). First, the sub-images must be registered to one another. A checkerboard pattern provides a useful spatial calibration target. By selecting one quadrant as a reference and applying autocorrelation, we determined the amount by which each image should be shifted to to achieve (ideally subpixel) alignment (see, e.g., [19]). The goal here is to avoid strong polarization artifacts around edges and regions of strong image gradients. A more sophisticated registration procedure would also take distortion into account. Each raw exposure is segmented which constitutes entries of the vector $\vec {I}$ in Eq. (2). Then, the pixel-wise data reduction matrices $\boldsymbol {A}^{-1}$ are formed from calibration data, enabling straightforward computation of the Stokes vector image.

 

Fig. 7. Polarization imaging with adapted prototype: (a) A raw exposure is processed by selecting the four sub-images using a pre-characterized image registration. Using the pixel-wise instrument matrix from calibration, these can be converted into the four-element Stokes vector at each pixel. (b) Four imaging examples, two indoor (left) and two outdoor (right) are shown. These examples primarily explore the polarized nature of specular (Fresnel reflection), in which unpolarized ligth is partially polarized perpendicular to the plane of incidence upon reflection. In (i), a soccer ball illuminated with LED light shows an azimuth angle that varies around its edge, equal at diametrically opposite points, revealing its 3D shape. In (ii), light passing through a wheel of polarizing films is observed to be highly polarized relative to the background, with its polarization azimuth angle revealing the orientation of the filters. In (iii), a parked car and a picnic bench show smooth surfaces that polarize incident sunlight, revealing their orientation (such as the top surface of the bench or the curved rear windshield of the car). DoP reveals that light from these specular surfaces is polarized relative to the background. Finally, in (iv), similar effects are observed in a row of parked cars where differently oriented panels of the car’s bodies exhibit different azimuth signatures and high DoP.

Download Full Size | PDF

From this polarization image, three scalar quantities are derived and presented for visualization. These are $S_0$ (merely the first entry of the Stokes vector, equivalent to a monochrome intensity-only image), the azimuth angle of the polarization ellipse’s major axis (given by $\frac {1}{2}\arctan \frac {S_2}{S_1}$), and the degree of polarization (DoP, given by $\sqrt {S_1^{2}+S_2^{2}+S_3^{2}}/S_0$).

Figure 7(b) shows four polarimetric image examples. We note here that applications of imaging polarimetry are wide-ranging, for applications as disparate as inspection of liquid crystal display panels during manufacturing, target detection, stress mapping, thin film thickness and refractive index measurement, solar magnetic field studies, and aerosol measurements. Consequently, the use of data from imaging polarimeters can vary from highly quantitative analysis (especially in remote sensing) to simple, qualitatively discerned visualization.

The examples here (with the exception of Fig. 7(ii)) show the polarized nature of specular, Fresnel reflection. Unpolarized light reflected from a specular surface becomes partially polarized in a direction perpendicular to the plane of incidence. Thus, polarization aids in identifying the azimuthal orientation of a specular surface’s surface normal vector [20], that is, the surface normal’s projection in the viewing plane (with circular polarization information the zenith angle of the surface normal [21] can also be determined).

Figures 7(i) and (ii) were taken indoors under diffused LED illumination. In Fig. 7(i), the light polarized upon reflection from a soccer ball exhibits a major axis azimuth angle (often called ‘AoP’ — angle of polarization) that sweeps between $-90^{\circ }$ and $+90^{\circ }$ from the perspective of the observer, passing through $0^{\circ }$ on the top and bottom of the ball. The DoP shows that this light is only weakly polarized. Figure 7(ii) shows an assembly of Polaroid films with radially oriented transmission axes, easily seen in the azimuth image of the filters, while the DoP shows that light passsing through the filters is highly polarized.

Figures 7(iii) and (iv) are images taken outside under direct sunlight, including cars (often featured as demonstration targets in imaging polarimetry works). Smooth surfaces exhibit different orientations, such as the flat bench top in (iii) ($0^{\circ }$ azimuth) and the different car body panels in (iv). Variations in azimuth reveal changing surface features such as the curved back windshield in (iii). Here the DoP is strongest from the highly specular automobile glass surfaces.

3.6 Field of view (FOV) considerations — using fisheye optics to expand the system’s FOV

Here we consider means by which the FOV can be increased. The rules detailed in Sec. 2.3 govern the relation between the camera’s FOV, the grating angle, and the FOV over which polarization imaging can subsequently be performed. Notably, if we seek to perform polarimetry over a wide FOV, a large grating angle (not to mention a wide FOV lens/sensor combination) is required since $\theta _{\text {FOV}} = \theta _{\text {grating}}$. Consequently, as $\theta _{\text {FOV}}$ grows larger the grating period $D$ shrinks (Eq. (4)). For large FOVs, this mandates grating periods that are quite small. For example, imaging over a $50^{\circ }$ FOV (which thus mandates a $50^{\circ }$ grating angle) would require $D=0.69$ µm at $\lambda =530$ nm. If TiO$_2$ pillars are spaced by 420 nm, as in this work, this does not even permit sampling with an array of $2\times 2$ pillars. Moreover, this is far outside of the paraxial regime of validity for the design strategy used in this work [8]. Consequently, unless some more advanced design strategy is used [22], function and efficiency will suffer at higher angles.

 

Fig. 8. Extending the angular FOV with an afocal fisheye extender: A fisheye extender is shown mounted on the prototype of this work (top left). A raytrace shows that the fisheye extender takes light over a wide angular FOV and recollimates it with reduced angle in the aperture stop, where the metasurface can be placed to experience a smaller range of incident angles (bottom left, the traytraced fisheye is for illustration and does not correspond directly to the fisheye used here). In this way, imaging over a wider FOV is possible without increasing the grating angle. Imaging with the fisheye extender shows a widened FOV.

Download Full Size | PDF

This limitation can be circumvented if we are willing to add additional fore-optics. An afocal system can de-magnify a wide FOV into a small FOV at the plane of the grating. This could be as simple as a telescope comprised of two converging lenses (i.e., the reverse of Fig. 9(b)), whose size can be reduced by the inclusion of both diverging and converging lenses. In essence, this is the function performed by a fisheye lens. So-called “fisheye adapters” – afocal systems which expand the FOV of existing camera lenses – are widely available for amateur photography applications. The FOV of the camera prototype of this work can be expanded simply in this manner (with care taken so that the exit pupil of the fisheye and the grating are not too far apart to avoid possible vignetting resembling Fig. 3). Figure 8 shows the the metasurface-adapted camera of Fig. 4 with an off-the-shelf 0.25x fisheye adapter attached, which expands its FOV by a factor of 4 from $6^{\circ }$ to $24^{\circ }$. In the same figure, images with and without this fisheye are shown. Without the fisheye adapter, a single house nearly fills the FOV; with it, the house is seen to be only a part of a larger scene.

4. Discussion and technology comparison

In this work we have shown how a properly-designed metasurface grating can convert an intensity-only imaging system into one capable of full-Stokes imaging polarimetry. Given the many approaches and technologies that exist for polarimetry, what advantages might metasurfaces hold?

Compactness is the most compelling advantage of metasurfaces for polarimetric application. What we have shown here is a division-of-amplitude configuration with a single optical element performing all polarization analysis and beamsplitting enabling streamlined optical systems along a single axis with a single sensor. A metasurface imaging polarimeter is thus simple and easily calibrated. The use case of this work, in which a metasurface adapts a pre-existing imaging optics, is just one possibility. It is also possible to design the metasurface itself to impart some imaging power, and hybrid metasurface/refractive (or completely metasurface-based designs) should also be possible, affording even greater compactness.

Bulk, non-absorptive polarization analyzing devices can be fabricated from birefringent media in the form of, e.g., polarization beam splitter cubes and Wollaston prisms. As individual elements, these can only separate light into orthogonally polarized channels. Optical systems comprised of many such elements, however, are not subject to this constraint. For instance, a setup consisting of three polarization beamsplitters (one of which is partially polarizing) combined with two retarders (one quarter- and one half-waveplate) can produce four channels which analyze for a configuration of polarization states tracing out a tetrahedron inscribed in the Poincaré sphere (see Fig. 2 of [23] and Fig. 7.12 of [2]). This setup is then a bulk (non-absorptive) analogue of the metasurface gratings presented here, one that could exceed the metasurface in overall efficiency. It is, however, considerably bulkier. In a sense, a metasurface grating is a generalization of Wollaston prisms or traditional polarization beamsplitter cubes with multiple customizable output channels that need not be orthogonal. Metasurfaces allow the advantages of the division-of-amplitude approach (namely, simultaneity and immunity from transients) to be accessed without this considerable bulk as a tradeoff.

Some of these advantages – namely, compactness and simultaneity – are also offered by division-of-focal plane (DoFP) polarization sensors, such as Sony’s Polarsens^TM. However, this approach has several key drawbacks to which a metasurface is not subject. First, circular polarization ($S_3$) is not usually measured. This is not a fundamental limitation, but it requires an additional liquid crystal patterning step [9] and, as-of-this-writing, full-Stokes division-of-focal plane sensors are not produced at industrial-scale. With a metasurface, the inclusion of circular polarization adds no complexity. Additionally, being based on absorptive polarizers, a division-of-focal plane sensor has a hard efficiency limit at 50%, most likely lower in practice (though these details from commercial DoFP sensors are not publicly available). This is a limit that, with carefully controlled fabrication, a metasurface can surmount. We note that metasurfaces have also been proposed as focal plane polarimetric components [24].

One notable drawback of metasurface gratings for polarimetry is their dispersion. The configuration presented here ideally works for a single wavelength. The grating, by its inherent nature, separates different color components, blurring the resulting image and decreasing effective resolution. Consequently, all the images of this work were taken through a 3 nm bandpass filter as described above in Sec. 3.3. This is further discussed in Supplement 1 (Sections S1 and S2).

5. Conclusion

We have shown how a metasurface diffraction grating may be used as a polarimetric “attachment” to a traditional, intensity-only imaging system, enabling full-Stokes imaging polarimetry. Simple design rules governing this adaptation were presented and then illustrated by turning a lens/sensor combination typical of a machine vision application into a polarization camera. This same approach can be applied to a wide variety of other imaging systems as well. Adapting existing systems, however, is just one use of metasurfaces in polarimetry. More generally, metasurfaces may serve as a new component in the toolkit of polarization optics for the design of polarization-sensitive optical systems for machine vision and remote sensing applications. The appendices below describe additional design considerations for the integration of metasurface polarization gratings into optical systems.

Appendices

Our primary concern for this manuscript is a methodology to adapt existing imaging systems with a metasurface grating and field-limiting aperture to create simple imaging polarimeters. In these appendices several topics receive further consideration.

A. Increasing the system’s entrance pupil diameter

As discussed in Sec. 2.5, if the grating cannot be placed exactly in the imaging optics’ entrance pupil (or a conjugate plane thereof), vignetting of some parts of the FOV will occur (Fig. 3(c)). This results in unequal illumination over the FOV. In Sec. 2.5, the limited aperture of the metasurface was proposed as a solution to this, evening out the illumination. The metasurface itself then becomes the aperture stop and the system’s entrance pupil as well (with diameter $D_{\text {EP}}$). Its finite size thus limits the system’s radiative throughput. The inclusion of the metasurface mandates longer exposure times or decreases the system’s shot noise-limited signal-to-noise ratio.

If we are willing to add components to the system, optical means exist to increase $D_{\text {EP}}$. Two scenarios which keep the metasurface grating in a collimated space are shown in Fig. 9. In scenario (a), it is assumed that a metasurface can be fabricated whose size equals that of the camera lens’ entrance pupil. Then, an afocal, 1:1 telescope can make the metasurface conjugate to that entrance pupil, eliminating the vignetting of Fig. 3(b) while utilizing the full $D_{\text {EP}}$ afforded by the lens. A Keplerian telescope is shown in Fig. 9(a) for clarity, though a Galilean or mixed design could be used to reduce overall length, so long as the camera lens’ housing permits placement of the second telescope lens one $f$ from its entrance pupil. In the example of this work, the camera lens’ entrance pupil diameter was 10 mm, and the metasurface grating’s was 1.5 mm. The latter’s size was limited by the serial nature of electron beam lithography. However, several techniques have emerged that promise centimeter-scale and larger mass-produced metasurfaces (see, e.g., [25] and [26]) so that Fig. 9(a) may prove a viable strategy for non-destructive adaptation of existing systems.

 

Fig. 9. Increasing the system’s entrance pupil diameter with additional optics: (a) If the metasurface grating can be made equal in size to that of the lens’ entrance pupil, an afocal, 1:1 telescope after the grating can make the lens’ entrance pupil and the grating conjugate, keeping the grating in a collimated space without limiting the system’s radiative throughput. (b) If the grating cannot be made large enough to fill the system’s entrance pupil, an afocal telescope can be placed before the grating can magnify the grating as seen from object space, at the expense of a larger range of incident angles on the grating.

Download Full Size | PDF

If, however, the metasurface cannot be matched in size to the lens’ entrance pupil, Fig. 9(b) presents an alternative scheme. Here, the metasurface’s size is magnified from the perspective of object space by an afocal telscope, effectively increasing $D_{\text {EP}}$. Again, a Keplerian design is shown for clarity, though a different design could be employed (though the Keplerian design has the advantage of offering a plane for a true field stop). In order to conserve Étendue, however, the metasurface must receive a wider range of incident angles. To satisfy the design rules of Sec. 2.3, then, the metasurface must implement a larger grating angle and be paired with a lens/sensor with a larger $\theta _{\text {camera}}$ than would otherwise be required by the FOV imaged.

B. Grating geometry and sensor allocation

In this work, we have limited ourselves to diffraction gratings whose periodicity is square. That is, gratings whose two lattice vectors are of equal magnitude and strictly perpendicular. As a result, the diffraction pattern — the reciprocal lattice — is also of square symmetry. If the grating is turned $45^{\circ }$ so that its lattice vectors point toward the corners of a square, the inner diffraction orders, plus the zero order, will land (when considered over a circular FOV) as originally sketched in Fig. 1(c). Limiting the FOV is necessary to prevent overlap of different sub-images from different diffraction orders. A circular FOV is convenient, matching the azimuthal symmetry of most optical components about the optical axis, easily implemented in this work with an adjustable iris. But it is not optimal in terms of sensor allocation. Some space can be recovered in between sub-images by using a square, diamond shaped FOV, as shown in Fig. 10; this would be the optimum field stop shape in a custom-designed system. In this optimum scenario, the geometry of Fig. 10 mandates that 37.5% of a square sensor’s space go unused to prevent images from overlapping with each other and higher diffraction orders (not shown) which are an undesired loss channel (this number jumps to 50% if the zero order is not considered usable signal).

Most commonly, however, sensor formats are rectangular. In this case, a square grating wastes sensor space. Indeed, all the raw data acquisitions of this work have been cropped to a square aspect ratio. However, this is not a fundamental design limitation. Instead, a metasurface grating whose lattice vectors are not perpendicular and not equal in magnitude can be designed. In that case, the diffraction pattern will have a rectangular symmetry which, upon an appropriate rotation of the grating, can be made to make optimal use of the sensor depending on its aspect ratio, a lengthy analysis we omit here (for the mathematics governing the diffraction pattern in this case see, e.g., [27], Secs. 2 and 3).

Finally, we note that a zero order’s presence has been assumed throughout this work. The grating, as a diffractive element, will possess zero order light that the designer would be hard pressed to eliminate completely. However, if the system were constructed to operate off-axis, that is, if the sensor intercepted higher diffraction orders and was not centered with the grating, it could be avoided. In this case, with no accomodations for the zero order necessary, the square sensor of Fig. 10 could be used in its entirety.

 

Fig. 10. Sensor allocation: To maximize sensor allocation, the system’s field stop would be diamond-shaped. Here, cross-hatch areas are sensor areas that may not be used to avoid overlap with other diffraction orders. The grating’s lattice vectors are shown in red, illustrating that the grating must be rotated $45^{\circ }$ to match its diffraction pattern to the symmetry of the square sensor.

Download Full Size | PDF

C. Noise and DoP error

In Fig. 7(b), several real-world targets were imaged with the camera prototype and images of $S_0$, azimuth angle, and degree of polarization (DoP) were presented. A striking feature appears in the DoP image of the ball, where the regions of highest DoP coincide with dark areas in the $S_0$ image. The formula for DoP contains a normalization by $S_0$, so in some sense this trend is unsurprising.

While a full polarimetric error analysis is outside the scope of this work, we consider this issue further in Fig. 11. Figure 11 centers on an example frame of a sports car which, like the ball of Fig. 7, contains dark areas. In Fig. 11(a), the intensity, azimuth, and DoP are again shown. The computed DoP has been clipped at 1, its highest allowable physical value. Pixels with unphysical values of the DoP are shown in red at the bottom of Fig. 11(a) and congregate in dark regions of the image, such as the car’s underside shadow. Also shown is a plot comparing $S_0$, the intensity, to the computed DoP at each pixel which reveals a clear correlation between low intensity and unphysical DoP.

Ideally, a perfect sensor with no noise (neither thermal, electronic, nor photon) would faithfully report the incident intensity and the DoP would always remain physical. However, at low intensity, noise (particularly from the sensor’s amplifiers and ADCs) can overwhelm the signal. This noise is reduced by averaging in space or time. In Fig. 11(b) data is first pre-processed through a Gaussian blur kernel of the indicated size (3, 5, or 7 pixels to a side). It can be seen that locations with unphysical DoP disappear rapidly with blurring. Blurring here averages noise from groups of adjacent pixels at the cost of lower spatial resolution.

 

Fig. 11. Empirical effect of noise on computed DoP in dark regions of image: (a) A representative example is found in this image of a sports car containing shadows. A plot of DoP vs. $S_0$ (intensity) shows unphysical DoP values, labeled with red pixels, correlate with low intensity. (b) Applying blur of different kernel sizes averages together adjacent pixels, sampling random noise distributions and eliminating these unphysical DoP values (while sacrificing image resolution).

Download Full Size | PDF

 

Fig. 12. Adaptation of a mobile phone camera with a metasurface grating: (a) A metasurface polarization grating is inserted in a 3D printed mount that holds it in place at the entrance to a mobile phone’s camera objective. The grating is mounted in a tube that also contains a color filter and an adjustable iris functioning as a field stop. (b) An image of a wheel containing film polarizers (the same object as Fig. 7(b), second column) is captured on the mobile phone. While the image is saturated (due to an inability to adjust the sensor’s exposure time), polarization contrast is evident.

Download Full Size | PDF

D. Adaptation of mobile phone camera

Very briefly, we show how a metasurface grating can be similarly integrated with a mobile phone camera. The grating is placed in a tube held by a 3D printed mount (Fig. 12(a)) right against the front of the camera. The tube also contains a color filter and an aperture. Figure 12(b) shows an image taken on the mobile phone of a wheel of polarizers (the same object as in Fig. 7(b), second column). The image is saturated, but still polarimetric contrast can be observed.

Mobile phone cameras differ in many respects from the machine vision system adapted in Sec. 3. Most notably, they feature large FOVs (70$^{\circ }$ and higher). This poses a challenge for the scheme here, since by Eq. (7) the necessary grating angle increases with the FOV of the camera. Large angle polarization gratings are a challenge for the metasurface platform here, since at high angles the grating unit cell may consist of only a few pillars. Consequently, in the example of Fig. 12, no effort has been made to fill the FOV (only the central portion of the image is shown as a proof-of-concept).

Funding

NASA Earth Science Technology Office; National Science Foundation (1541959, DGE1144152); Air Force Office of Scientific Research (FA9550-18-P-0024).

Acknowledgments

The authors wish to thank Wei-Ting Chen (Harvard), Phil Oakley (Ball Aerospace), Roberto Casini (National Center for Atmospheric Research), Lisa Li (U. Arizona and Harvard), and Prof. Meredith Kupinski (U. Arizona). This work was supported by the Earth Science Technology Office (ESTO) of the National Aeronautics and Space Administration (NASA) under a grant from the Instrument Incubator Program (IIP). This work was also supported by the U.S. Air Force Office of Scientific Research under grant no. FA9550-18-P-0024. Additionally, NAR acknowledges support from the National Science Foundation (NSF) Graduate Research Fellowship Program (GRFP) under grant no. DGE1144152. This work was performed in part at the Center for Nanoscale Systems (CNS), a member of the National Nanotechnology Coordinated Infrastructure (NNCI), which is supported by the National Science Foundation under NSF award no. 1541959. CNS is part of Harvard University.

Disclosures

The subject of this work has been filed as a provisional patent with the US Patent & Technology Office.

Author disclosures: NAR: P, Metalenz, Inc. (C), PC: P, RC: Airy Optics, Inc. (I, E), FC: P, Metalenz, Inc. (I, S).

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Supplemental document

See Supplement 1 for supporting content.

References

1. W. Shurcliff, Polarized Light: Production and Use (Harvard University, 1962).

2. R. A. Chipman, W.-S. T. Lam, and G. Young, Polarized Light and Optical Systems (CRC, 2019).

3. M. Born and E. Wolf, Principles of Optics (Pergamon, 1999).

4. A. S. Alenin and J. S. Tyo, “Generalized channeled polarimetry,” J. Opt. Soc. Am. A 31(5), 1013 (2014). [CrossRef]  

5. F. Snik, T. Karalidi, and C. U. Keller, “Spectral modulation for full linear polarimetry,” Appl. Opt. 48(7), 1337–1346 (2009). [CrossRef]  

6. M. W. Kudenov, M. J. Escuti, E. L. Dereniak, and K. Oka, “White-light channeled imaging polarimeter using broadband polarization gratings,” Appl. Opt. 50(15), 2283–2293 (2011). [CrossRef]  

7. J. S. Tyo, D. L. Goldstein, D. B. Chenault, and J. A. Shaw, “Review of passive imaging polarimetry for remote sensing applications,” Appl. Opt. 45(22), 5453–5469 (2006). [CrossRef]  

8. N. A. Rubin, G. D’Aversa, P. Chevalier, Z. Shi, W. T. Chen, and F. Capasso, “Matrix Fourier optics enables a compact full-Stokes polarization camera,” Science 365(6448), eaax1839 (2019). [CrossRef]  

9. W.-L. Hsu, G. Myhre, K. Balakrishnan, N. Brock, M. Ibn-Elhaj, S. Pau, K. M. Twietmeyer, R. A. Chipman, A. E. Elsner, Y. Zhao, and D. VanNasdale, “Full-Stokes imaging polarimeter using an array of elliptical polarizer,” Opt. Express 22(3), 3063–3074 (2014). [CrossRef]  

10. R. Azzam, I. Elminyawi, and A. El-Saba, “General analysis and optimization of the four-detector photopolarimeter,” J. Opt. Soc. Am. A 5(5), 681–688 (1988). [CrossRef]  

11. D. S. Sabatke, M. R. Descour, E. L. Dereniak, W. C. Sweatt, S. A. Kemme, and G. S. Phipps, “Optimization of retardance for a complete Stokes polarimeter,” Opt. Lett. 25(11), 802–804 (2000). [CrossRef]  

12. J. S. Tyo, “Design of optimal polarimeters: maximization of signal-to-noise ratio and minimization of systematic error,” Appl. Opt. 41(4), 619–630 (2002). [CrossRef]  

13. J. Greivenkamp, Field Guide to Geometrical Optics (SPIE Press, 2004).

14. A. Arbabi, Y. Horie, M. Bagheri, and A. Faraon, “Dielectric metasurfaces for complete control of phase and polarization with subwavelength spatial resolution and high transmission,” Nat. Nanotechnol. 10(11), 937–943 (2015). [CrossRef]  

15. J. P. B. Mueller, N. A. Rubin, R. C. Devlin, B. Groever, and F. Capasso, “Metasurface polarization optics: independent phase control of arbitrary orthogonal states of polarization,” Phys. Rev. Lett. 118(11), 113901 (2017). [CrossRef]  

16. N. A. Rubin, A. Zaidi, A. H. Dorrah, Z. Shi, and F. Capasso, “Jones matrix holography with metasurfaces,” Sci. Adv. 7(33), eabg7488 (2021). [CrossRef]  

17. R. C. Devlin, M. Khorasaninejad, W.-T. Chen, J. Oh, and F. Capasso, “Broadband high-efficiency dielectric metasurfaces for the visible spectrum,” Proc. Natl. Acad. Sci. 113(38), 10473–10478 (2016). [CrossRef]  

18. R. M. A. Azzam, “Photopolarimetric measurement of the Mueller matrix by Fourier analysis of a single detected signal,” Opt. Lett. 2(6), 148–150 (1978). [CrossRef]  

19. M. Guizar-Sicairos, S. T. Thurman, and J. R. Fienup, “Efficient subpixel image registration algorithms,” Opt. Lett. 33(2), 156–158 (2008). [CrossRef]  

20. S. Rahmann and N. Canterakis, “Reconstruction of specular surfaces using polarization imaging,” in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, (2001), pp. 149–155.

21. N. M. Garcia, I. de Erausquin, C. Edmiston, and V. Gruev, “Surface normal reconstruction using circularly polarized light,” Opt. Express 23(11), 14391 (2015). [CrossRef]  

22. D. Sell, J. Yang, S. Doshay, R. Yang, and J. A. Fan, “Large-angle, multifunctional metagratings based on freeform multimode geometries,” Nano Lett. 17(6), 3752–3757 (2017). [CrossRef]  

23. V. L. Gamiz, “Performance of a four-channel polarimeter with low-light-level detection,” in Polarization: Measurement, Analysis, and Remote Sensing, vol. 3121 (SPIE, 1997), p. 35.

24. E. Arbabi, S. M. Kamali, A. Arbabi, and A. Faraon, “Full-Stokes Imaging Polarimetry Using Dielectric Metasurfaces,” ACS Photonics 5(8), 3132–3140 (2018). [CrossRef]  

25. J.-S. Park, S. Zhang, A. She, W. T. Chen, P. Lin, K. M. A. Yousef, J.-X. Cheng, and F. Capasso, “All-Glass, Large Metalens at Visible Wavelength Using Deep-Ultraviolet Projection Lithography,” Nano Lett. 19(12), 8673–8682 (2019). [CrossRef]  

26. X. Yin, J.-S. Park, K. K. Stensvad, R. L. Brott, N. Rubin, M. B. Wolk, and F. Capasso, “Roll-to-roll dielectric metasurfaces,” in Metamaterials, Metadevices, and Metasystems 2020, vol. 11460N. Engheta, M. A. Noginov, and N. I. Zheludev, eds., International Society for Optics and Photonics (SPIE, 2020).

27. L. A. Romero and F. M. Dickey, “Theory of optimal beam splitting by phsae gratings. II. Square and hexagonal gratings,” J. Opt. Soc. Am. A 24(8), 2296–2312 (2007). [CrossRef]