1. Introduction

The search for an order out of disorder has fascinated physicists for many decades. The interaction of waves with a disordered medium has provided a testing ground for this quest, which led to interesting observations. Examples include Anderson localization [1], coherent backscattering [2], random lasing [3], the effectiveness of time-reversal/phase conjugation [4], and the existence of perfect-transmission eigenchannels [5]. In recent years, the seminal works of optical phase conjugation [6] and the single-channel optimization with the scattering media [7] reminded the research community of the feasibility of applying these concepts to the real practices. In conjunction with the technological advances in wavefront shaping devices and imaging techniques for the wavefront recording, a variety of studies have been followed to deal with more practical problems of interrogating targets located either at the opposite side or within scattering media [8]. The methodologies include feedback-optimization of intensity at a target spot [7], the use of so-called memory effect to scan the spot [9, 10], the transmission matrix of scattering media [11–13], acoustic guide star in conjunction with optical phase conjugation [14, 15], and spatio-temporal focusing combined with nonlinear excitation [16].

In this review paper, we will mainly cover the study of the transmission matrix of scattering media and its use for biophotonics applications, which our group has pursued for a few years. The interaction of a scattering medium with light wave is highly complicated due to the multiple light scattering, which makes it extremely difficult to interrogate target objects within the medium. However, when it comes to finding the input-output relation, it is straightforward. If we confine our interest to the linear interaction and also consider that the coherence length of light source is much longer than the transit time broadening of the media, then the scattering medium is a simple linear system with respect to the electric field of the light wave. Then the input-output response of a scattering medium can be described by a so-called transmission matrix, which relates free modes at the input to those at the output. The experimental measurement of transmission matrix is rather time-consuming because it requires both the scanning of incident waves in such a way to cover input free modes and the recording of complex field, i.e. amplitude and phase, at various output channels. However, it is the complete way of describing the response of scattering media to the incident light waves. The first transmission matrix measurement in optics was made by Gigan’s group [13], where they used the self-interference of two different input modes generated by the same spatial light modulator. About the same time, we reported the measurement of the transmission matrix by the combination of the scanning the angle of incident wave and off-axis interference microscope [12].

The knowledge of the transmission matrix of a scattering medium provides us with two important opportunities. At first, we can find the eigenchannels of the scattering medium with which we can control the energy transmission through the medium. According to the random matrix theory (RMT) developed in 1980s, a particular incident wave with specific pattern can propagate through a disordered medium in the waveguide geometry without undergoing any energy loss [5]. Mathematically, this special wave is the eigenvector of t⁺t, where t stands for a transmission matrix and ⁺ for a conjugate transpose operation, with maximum eigenvalue, and often called as an open eigenchannel. The underlying physical explanation for the unusual transmittance of the open eigenchannel is that a proper choice of eigenchannel as an incident wave can induce strong constructive interference of scattered waves at the opposite side of the medium. For the scattering medium with slab geometry, open eigenchannels do not exist due to the loss of energy leaking away from the detection area. However, the coupling of light to the eigenchannels with maximum eigenvalues guarantees the maximal energy transport through the medium.

The second opportunity is to make use of a scattering medium as an imaging optics. With the knowledge of the transmission matrix, we can reconstruct original object image from the image distorted by the scattering layer. The multiplication of the matrix inversion to the complex field map of a distorted wave results in an object wave at the upstream of the scattering layer. This is more general approach than the deconvolution process typically used in the conventional imaging system where shift invariance is valid. Therefore, there are unexpected advantages of using a matrix inversion over conventional imaging. It allows us to make use of the optical elements such as multimode optical fibers and fiber bundles as an aberration-free imaging optics. In addition, it is possible in principle to use scattering media made of sub-wavelength particles as a superlens to capture the near-field waves for high-resolution imaging.

In our previous studies, we explored the two opportunities of the transmission matrix approach mentioned above. For the rest of this paper, we will introduce some of them in detail. In section 2, we will cover the numerical/experimental studies for investigating the physical properties of eigenchannels [17–19] and experimental coupling of light to individual transmission eigenchannels for enhancing wave penetration through a scattering layer [20]. In section 3, we will introduce the studies of imaging with scattering media. Our work on the enlargement of numerical aperture with the use of a scattering medium will be covered, with which the enhancement of spatial resolution of an imaging system was demonstrated [12]. We then present aberration-free endoscopic imaging with a multimode optical fiber and a fiber bundle in both reflectance and fluorescence modes of detection [21, 22]. In section 4, we conclude this paper by discussing the future perspective of the transmission matrix approach.

2. Controlling light energy delivery through a scattering medium

2.1 Internal field distribution of transmission eigenchannels

Studies have long been conducted to understand the wave propagation in the disordered media. One of the striking predictions was the existence of particular incident waves, which can transmit through scattering media with near unity transmittance. In 1984, Dorokhov used RMT to calculate the statistical distribution of transmission eigenvalues and predicted the existence of this so-called open eigenchannels [5]. In this section we introduce our numerical study in which we investigated the internal field distribution of various transmission eigenchannels and developed our understanding knowledge on the mechanism of perfect transmission [17]. For this purpose, we have solved electromagnetic wave propagation through a disordered medium using the finite-difference time-domain (FDTD) method, numerically constructed a transmission matrix in an optical regime, and obtained both eigenvalues and their associated eigenchannels. We observed that open eigenchannels enhance the energy stored inside a disordered medium.

For this study, we numerically prepared 2D disordered media in open slab geometry (Fig. 1(a)). To generate a disorder, absorption-free square particles with a side length of 200 nm and a fixed refractive index, n_p = 2.0, are randomly distributed in the vacuum. For the filling factor of about 50%, the scattering mean free path was calculated to be about 370 nm. The thickness of the slab was set to be l_th = 8, 12, and 16 μm. The vertical width of the disordered medium in the x direction is set as 130 μm for the computation, but the output field is recorded at the 90 μm width in the middle (red dashed-dotted line).

 

Fig. 1 Construction of a transmission matrix of a disordered medium and analysis of its eigenvalues. (a) Schematic of numerical simulation. The disordered medium is 130 μm wide in x-direction. The electric field is recorded at the output side of the medium indicated as a red dashed-dot line and then numerically Fourier transformed to obtain the field at the wave vector space, $k_{out}=\left(k_x^{\text{'}},k_z^{\text{'}}\right)$. The lens represents numerical Fourier transform. The inset is the magnified image of the disordered medium. Black squares are the particles. (b) Comparison between FDTD computation and random matrix theory (RMT). Eigenvalue distribution of the transmission matrix sorted in descending order. Red curves are calculated from the FDTD method while blue dashed curves are obtained by RMT. Three different curves account for three different disordered media with thickness of 8, 12, and 16 μm, respectively, for the same n_p = 2.0. For each of the FDTD result and RMT, the uppermost curve is from the 8 μm sample, the middle one is from the 12 μm sample, and the curve at the very bottom is from the 16 μm sample (modified from Ref [17].).

Download Full Size | PDF

We prepare a set of angular plane waves constituting complete set for the width of 90 μm and compute their propagation through a disordered medium. At the steady state, we record electric fields along a line parallel to the x axis, which is located 50 nm behind the medium indicated by a red dashed-dotted line in Fig. 1(a). Recording is done at four successive times with a time interval of 1/(4ck₀), one-quarter of optical oscillation. By applying a phase-shifting interferometry algorithm, we extract the amplitude and phase of the transmitted electric field, E_out(x,z = z_d). To use the wave vector space for a basis of the transmission matrix, we perform numerical Fourier transform of E_out(x) and obtain the electric field in k space, E_out(k_x′). This process is illustrated as an insertion of a virtual lens behind the medium, as shown in Fig. 1(a). Then, we construct a transmission matrix, t(k_x′, k_x), by the following relation:

We compare the numerically calculated transmission matrix with RMT in their singular values. We perform the singular value decomposition of the transmission matrix, t,

For the disordered media with l_th = 8, 12, and 16 μm with the same n_p = 2.0, we calculate their transmission matrices. For each medium, we plot eigenvalues after arranging them in descending order [solid red curves in Fig. 1(b)]. With the increase of thickness, the area under the curve is reduced and thus the average eigenvalue decreases. Considering that an arbitrary input is a random superposition of eigenchannels, this decrease in average eigenvalue agrees well with the reduction in total transmission. In accordance with RMT, there always exists an eigenchannel whose transmission is unity, which is called an open eigenchannel, regardless of the thickness. This is because the thickness of the sample is much smaller than the width of the medium. We compared these numerical results with RMT. We constructed a transmission matrix following the RMT in which only the transmittance of any given medium determines the statistics of the eigenvalue distribution. The dashed blue curves in Fig. 1(b) show the eigenvalue distributions obtained from RMT. The distributions are in excellent agreement with our FDTD results. This confirms that the open slab geometry of the given disordered media can be modeled by the RMT based on the waveguide geometry at least for the considered media [5, 23].

The FDTD method differs from RMT in that it can provide the field distribution inside a disordered medium. This enables us to explore how eigenchannels propagate through the medium and how much energy they deposit into the medium. As a baseline, we first compute the propagation of a plane wave through a disordered medium whose thickness and index of particles are 16 μm and n_p = 2.0, respectively. The field amplitude throughout the medium is displayed in Fig. 2(a) when the incident angle is 11.5° with respect to the z axis. Inside the medium, there is a linear decrease in the amplitude of the field as the wave propagates through the medium, as is shown in Fig. 2(d), which plots the intensity averaged along the x axis as a function of the depth (z axis). This agrees well with previous studies [24, 25] conducted in the context of diffusion theory in which the transmission decreases linearly when the thickness of the medium is much larger than the transport mean free path.

 

Fig. 2 Field distributions of eigenchannels inside medium. (a)-(c) Field distribution of a plane wave whose incident angle is 11.5°, open eigenchannel and closed eigenchannel, respectively. The incident field is subtracted on the left-hand side of the medium. Here, the amplitude normalized to the input wave. Scale bar: 10 μm. (d) Average intensity along the x direction as a function of the depth in the z direction. The disordered medium fills the space between 0 and 16 μm in depth. The intensity is normalized to that of a normally incident plane wave (modified from Ref [17].).

Download Full Size | PDF

For the same disordered medium, we obtain eigenchannels from singular value decomposition of the transmission matrix, and solve their propagation through the medium. Figure 2(b) displays the propagation of an open eigenchannel whose eigenvalue is 0.955. We find that the field strength inside the medium is enhanced such that its average intensity is higher than that of the input [blue curve in Fig. 2(d)]. This is in analogy with a Fabry-Perot cavity in which internal field strength increases at the resonance condition due to the constructive interference. Likewise, the constructive interference enhances the internal energy in open eigenchannels, which leads to strongly enhanced transmission.

In the case of a closed mode whose eigenvalue is close to zero [Fig. 2(c)], a steep decrease of intensity is observed along the z direction as soon as the wave enters the medium. This suggests strong destructive interference as the wave reaches the end of the medium. The intensity profile follows the exponentially decaying curve [red curve in Fig. 2(d)]. Overall, the location of the peak intensity inside the medium shifts from the center to the input side with the decrease of the eigenvalue. The internal energy of the single-channel optimizing mode is close to the open eigenchannel in connection with its enhanced transmission.

The efforts to directly observe open and closed eigenchannels have been under way, and a study shows their existence for the elastic waves [26]. However, their observation in optics has so far been unsuccessful due to the difficulty in preparing a waveguide containing scattering media and also in covering all the optical modes.

2.2. Open eigenchannels in the Anderson localization regime

The interference of the multiple-scattered waves in the highly disordered media leads to interesting phenomena such as Anderson localization [1, 27–29]. Unlike in the diffusion regime where the transmission decreases linearly over the thickness, it decreases exponentially when the Anderson localization takes place. This is due to the dominance of local waves by the constructive interference over the incoherently interfering propagating waves [28]. However, there also exist extended modes of high transmittance that accounts for the most of the average transmittance of the medium [5, 30]. Such extended modes, called necklace states or non-localized modes, were predicted in 1D random media [31] and were subsequently observed in the experiment [32]. The underlying physics is that the propagation of energy is mediated by the coupling of neighboring localized modes, which results in the extended modes. But the existence of the extended modes in 2D or 3D disordered media and the mechanism of transmittance enhancement, if they exist, are yet to be explored.

Main challenge in the higher dimensional media is that the probability of such modes is too small to find them by chance. Arbitrary incident wave mostly couples to the localized modes under which rare contribution of extended modes is buried. Therefore finding the pure contribution of extended modes is of critical concern. We demonstrated that coupling light to open eigenchannels can be a systematic way of searching for the extended modes. Open eigenchannels in the strongly scattering media may lead to finding extended states of high transmittance. In section 2.1, we introduced the procedure of finding open eigenchannels for a medium in the diffusion regime [17]. But as to be shown in the following, the propagating waves that used to be prevalent in the diffusion regime disappear in the Anderson localization regime and non-propagating waves start to dominate. Therefore, the mechanism of the constructive interference is expected to be dramatically different in forming open eigenchannels.

In this section, we show the internal field distribution of eigenchannels for the 2D disordered medium in the Anderson localization regime. We numerically prepare a highly disordered 2D medium whose localization length is shorter than the thickness of the medium. Using FDTD method, we compute wave propagation through the medium, and observe the localization in the internal energy distribution and the disappearance of propagating waves within the medium. We then construct a transmission matrix, extract eigenchannels and acquire their field distribution inside the medium. We found that the spatial distribution is highly localized in the open eigenchannels within a narrow channel that connects the shortest passage from input to the output planes of the medium. Our study strongly suggests that open eigenchannels induce coupling of non-propagating localized waves, which results in the extended modes of high transmittance.

For the two 16 μm-thick disordered media of n_p = 1.6 and 2.5, we obtain the spatial intensity map of the electromagnetic wave within the disordered media (Fig. 3(a) and 3(b)). The illumination source is a plane wave normally incident to the input plane. We observe that internal intensity decreases exponentially for n_p = 2.5 medium while it decreases linearly in n_p = 1.6 medium. In particular, prominent intensity-enhanced spots exist for n_p = 2.5 while the intensity is rather uniformly distributed over the space for n_p = 1.6 medium. These suggest that the n_p = 2.5 medium be in the strong localization regime while the n_p = 1.6 medium in the weak localization regime.

 

Fig. 3 Internal field distribution. (a), (b) Intensity maps inside the random media of n_p = 1.6 and 2.5 respectively. Incident wave is a plane wave propagating in z-direction. Color bar of amplitude is in arbitrary unity. Scale bar is 5 μm. (c) and (d) are angular spectrum maps for (a) and (b), respectively. Horizontal and vertical axes are represented in terms of the wave number, k₀, in free space. Color bar of amplitude is in arbitrary unit (modified from Ref [18].).

Download Full Size | PDF

Since the FDTD method computes the complex amplitude of the field inside the medium, we can survey the properties of the field as well as the intensity in the Anderson localization regime. To this end, we take the 2D discrete Fourier transform with respect to the spatial coordinates for the field within the medium and obtain an angular spectrum in the spatial frequency. Figures 3(c) and (d) are the field amplitude in k-space for n_p = 1.6 and 2.5, respectively. The bright ring in the Fig. 3(c) indicates that propagating wave is dominant in the weakly scattering medium. The presence of the ring represents that waves propagate to all different directions. The radius of the ring is measured to be 1.2k₀, which is equal to the effective wave vector inside the medium. Hence, it is bigger than k₀, the wave number in the free space, because of the high index particles. In the strongly scattering medium (Fig. 3(d)), on the other hand, the ring pattern disappears. This indicates that the propagating waves are dominated by the non-propagating evanescent waves. Spatial Fourier transform of the evanescent waves typically exhibits broadened spectra due to the abrupt decay of the field amplitude. This is the analogue of the waves in the stop band of photonic crystal. This observation goes well with the theory of Anderson localization that the locally interfering waves dominate over the incoherently interfering propagating waves. But our analysis provides direct observation of the disappearance of propagating waves and the prevalence of the local evanescent waves in the Anderson localization regime. This also supports very well that the medium in the strong localization regime is on the mobility edge of photon.

For each singular value, we obtain a corresponding eigenchannel in the form of the coefficients of incident channels. We then solve the propagation of the constructed eigenchannel and record the map of the field within the medium. Figures 4(a)-(e) shows the intensity map inside the strongly scattering medium for the eigenchannels of transmittance corresponding to 80, 12, 2.3, 0.39, and 2.0 × 10^-7%, respectively. We find that there exists a striking difference between the Anderson localization regime and the diffusion regime. In the diffusion regime, the field energy of the open eigenchannel is spread over the entire transverse extent [17]. In the Anderson localization regime, however, it is highly concentrated in space such that a dense cluster of localized modes appears within the disordered medium (Fig. 4(a)). Moreover, the clustering is found to exist only for the eigenchannels of high transmittance. We repeatedly observe this appearance of the cluster for other media whose spatial organizations of constituting particles are different but with the same particle size and index, and the fill factor. As the transmittance becomes low, the internal waves gradually spread along the transverse direction (Fig. 4(b)-(e)). For the eigenchannels of sufficiently low transmission, the internal wave covers the entire transverse range. These suggest that the dense cluster be formed in the process of enhancing transmission through a disordered medium where non-propagating localized waves are dominant.

 

Fig. 4 Internal field distribution of eigenchannels for the disordered medium of n_p = 2.5 (a)-(e): eigenchannels indices are 1, 11, 21, 31, and 291, respectively, and their transmittances are 80, 12, 2.3, 0.39, and 2.0 × 10^-7% respectively. Color bar indicates natural logarithm of the amplitude. Aspect ratio is set to different between horizontal and vertical axes for better visibility. Both scales bars indicate 5 μm (modified from Ref [18].).

Download Full Size | PDF

The existence of the dense cluster of internal modes in the Anderson localization regime can be explained as follows. According to Fig. 4(d), most waves exist in the form of non-propagating waves in the strongly scattering medium. In this case, the propagation of energy can mainly be mediated by the neighboring localized modes and by the coupling among them. This can be thought of as a kind of frustrated total internal reflection in which a neighboring pair of prisms induces the coupling of non-propagating evanescent waves and therefore enhances wave propagation. Studies in the 1D media also showed that neighboring of localized modes enhances the transmission [31, 32]

2.3. Enhancing wave transport through disordered media

In this section, we briefly introduce an experimental method to maximize energy transport through disordered media [20]. Although various techniques are developed in the wavefront sensing and recording, the injection of waves into single eigenchannels was still a challenge. Several studies have been made to calculate individual eigenchannels in optics [13] and in microwave [33] from the recorded transmission matrix, but the injection of waves into single eigenchannels had not been realized before. Coupling waves into unique eigenchannels is difficult because of two stringent requirements. First, the transmission matrix of the disordered medium must be recorded in the short time before the medium is perturbed. Second, complex wavefronts should be generated accurately corresponding to the eigenchannels of highly complicated wavefronts derived from the measured matrix.

To overcome such difficulties, the experiment has been carried out using the experimental setup shown in Fig. 5. A disordered medium is placed between the input plane (IP) and output plane (OP). As a disordered medium, a 27 μm thick ZnO nanoparticle layer is used. The average transmittance of this medium is measured to be 0.79% at the collection angle corresponding to 0.32 NA. For the purpose of recording a transmission matrix, scanning mirrors (GM) are installed in the sample beam path. The angle, (θ_ξ, θ_η), of the plane wave incident to the medium was scanned, and transmission image was recorded at each incident angle. Figure 6(b) shows the phase maps of the transmitted waves at representative incident angles. The complex field at the input plane was measured for the same set of incident angles (Fig. 6(a)). This was simply done by measuring the field after removing the disordered medium. From the two sets of complex field maps, a transmission matrix was constructed. The amplitude and phase of the transmission matrix are shown in Figs. 6(c) and 6(d), respectively. If there were no disordered medium, the matrix would be diagonal. However multiple scattering processes cause the incident beam to spread in a disordered medium. Therefore non-zero off-diagonal elements are generated in the matrix.

 

Fig. 5 Experimental schematic for recording a transmission matrix and generating each transmission eigenchannel. The main frame of the setup is an off-axis interference microscope with scanning mirrors (GM, Cambridge Technology) installed in the sample beam path. The laser output from a He-Ne laser is split by a beam splitter (BS1), and one of the beams (sample beam) is sent to the sample and the other (reference beam) through free space. The two beams are recombined by another beam splitter (BS2) to form an interference image at the camera (RedLake M3, 500 fps). A spatial light modulator (SLM, Hamamatsu Photonics, X10468-06) that can generate an arbitrary wavefront is installed in the sample beam path. A disordered medium is placed between the input plane (IP) and the output plane (OP), and the transmitted image at the OP is delivered to the camera by an objective lens (Olympus UPLSAPO) and a tube lens (modified from Ref [20].).

Download Full Size | PDF

 

Fig. 6 Construction of a transmission matrix for a disordered medium. a, Phase maps of the recorded waves acquired in the absence of a disordered medium as a function of incident angle. Scale bar, 10 μm. b, Phase maps of the transmitted waves through a disordered medium as a function of incident angle. c, Amplitude part of a transmission matrix constructed from (a) and (b). Color bar, amplitude in arbitrary unit. (d), Phase part of the transmission matrix. The same color bar, which represents the phase in radians, applies to those phase maps in (a), (b) and (d) (modified from Ref [20].).

Download Full Size | PDF

Singular value decomposition was performed for the measured transmission matrix in order to derive the transmission eigenchannels and their corresponding transmission eigenvalues. An optical wave corresponding to the eigenchannel was experimentally generated by using a spatial light modulator (SLM). The SLM used in this experiment operated in the phase-only mode. In order to generate eigenchannels having complicated distributions in both amplitude and phase, we used a modified method in which multiple SLM pixels act as a single unit at the target wave [34].

After that, using the wavefront shaping technique, the optical wave of the first transmission eigenchannel was implemented and sent to the medium. Then complex field map at the output plane was recorded (Fig. 7(b)). Figure 7(a) is the transmitted image when the uncontrolled wave (plane wave) was sent to the medium. The enhancement factor, which is defined as the ratio of the transmittance of the first eigenchannel to that of the uncontrolled wave, was 2.8 for this specific sample. When the same experimental procedure was repeated for various samples, the observed transmission enhancement was highly reproducible. The enhancement factor varies somewhat in different samples, typically in the range of 2 to 4.

 

Fig. 7 Comparison between the uncontrolled wave and the first eigenchannel. (a), (b), Transmitted images of the uncontrolled wave and the first eigenchannel through the disordered medium, respectively (modified from Ref [20].).

Download Full Size | PDF

Various eigenchannels are sequentially generated and their transmittances are observed (red circles in Fig. 8). The transmittance is monotonically decreased with the increase of the eigenchannel index, and the first eigenchannel has the maximum transmittance. The experimentally measured transmittance was compared with the eigenvalues (blue squares in Fig. 8) acquired from the recorded transmission matrix after accounting for the limited accuracy of wavefront shaping. Experimentally generated eigenchannels faithfully follow the prediction from measured transmission matrix. The residual discrepancy aside from the wavefront shaping inaccuracy might come from mechanical perturbation of the sample and imperfect repeatability of the GM scanning. This method was compared with the single-channel optimizing modes [30] in which intensity was optimized at a single point at the far side of the medium. Although the previous method shows increase in the transmittance, the transmittance (green line in Fig. 8) is well below that of the first eigenchannel. This is because the single-channel optimizing mode couples light into other eigenchannels as well as the eigenchannel with maximum eigenvalue [17].

 

Fig. 8 Measured transmittance of individual eigenchannels. Red circles: measured transmittance when the optical wave of each eigenchannel is generated and illuminated onto the disordered medium. Blue squares: corrected eigenvalues. Black line: mean transmittance of the medium measured under illumination by a plane wave. Green line: mean transmittance when the point optimization process is performed (modified from Ref [20].).

Download Full Size | PDF

3. Imaging through a scattering medium

3.1 Overcoming the diffraction limit with the use of scattering media

Image information transmitted through a disordered medium is scrambled because of multiple light scattering caused by the turbidity. However, if the transmission matrix of the medium is known, then the image distortion can be removed using the relationship deterministically connecting the output image and the transmission matrix. In this respect, a scattering medium can be a unique optical element, not just a hurdle for imaging, which possesses extra benefits that usual optics is devoid of. In the following section, we briefly introduce the basic concept of imaging with a scattering medium.

Let us consider that an input wave E_IP(θ_x, θ_y) carrying image information in terms of angular spectrum is sent to a scattering medium from the input plane (x, y), which is indicated as an IP in Fig. 5. Then the transmitted wave E_OP(ξ, η) at the output plane (ξ, η) is determined as

 

Fig. 9 Reconstruction of 2-D object images. (a) The USAF target-like pattern used as an object. (b) The distorted image of the structure in (a) by the ZnO layer. (c) Angular spectrum of the object extracted by the projection operation. Corresponding phase components are also acquired. Scale bar: 0.5 mm⁻¹. (d) The reconstructed target image by using the angular spectrum in (c). (e) An image of the emblem of Korea University positioned before the ZnO layer. (f) Reconstructed emblem image from the distorted image (modified from Ref [12].).

Download Full Size | PDF

Once TLI converts a turbid medium into a scattering lens, it provides a counter-intuitive benefit for optical imaging: enhancement of a spatial resolution for an imaging system. A conventional lens can capture only the light directing into the solid angle extended by the NA of the lens. Therefore, the achievable resolution of the imaging system is determined by the NA in general. If a scattering medium is placed between the object and the imaging system, the light with a propagation angle greater than NA can be redirected into the capture range of the lens by the multiple scattering. Thus if the acquired image through the scattering medium is carefully processed, the spatial resolution of the imaging system can be enhanced.

To demonstrate the benefit from TLI, a test object composed of 1-D stripes is generated on the SLM. We take the image with a high NA objective lens (1.0 NA) in a normal imaging configuration (Fig. 10(a)). With the diffraction limited resolution of 0.77 μm, every single structure up to the finest lines indicated by the red box (spatial period is 2.5 μm at OP) is clearly resolved as shown in Fig. 10(c). Next, the NA of the objective lens is reduced intentionally by decreasing the aperture size in the back focal plane of the objective lens which corresponds to 0.15 NA. Then some of fine structures indicated by red box in Fig. 10(c) disappear due to the lack of resolving power (Fig. 10(d)). Then, a ZnO nano-particle layer as a scattering lens is introduced between the low NA objective lens and the object as shown in Fig. 10(b). A transmission matrix for the scattering lens can be recorded up to 0.85 NA although the low NA of the objective lens because the turbid medium redirects a high-angle light to an adoptable low-angle input. With the help of the scattering lens, the transmission matrix is measured beyond the limit of the low NA objective lens of 0.15 NA. An image of the 1-D object is taken through the fully characterized scattering lens. Although it looks like a speckle pattern shown in Fig. 10(e), it contains information ranging from low- to high-angle. As the previous experiments, TLI retrieves the angular spectrum of the object embedded in the distorted image, and then reconstructs the object image. The result is presented in Fig. 10(f) and the finest lines are clearly as imaged by the high NA objective lens. For the TM recording, the illumination angle is steered from −53° to 53° (0.85 NA) in 5,000 steps along the direction orthogonal to the lines in the object. This means that the retrieved angular spectrum from the image taken by the scattering lens, which is obtained by the low NA objective lens (0.15 NA), has components up to 0.85 NA. As a consequence, the actual detection NA is increased by more than 5 fold and the spatial resolution is enhanced by the same factor. Note that the value of 0.85 NA is not a fundamental limit, which can be further improved by covering a wider angular range.

 

Fig. 10 (a) Conventional imaging configuration. ${\theta }_{max}$ is the maximum angle that the imaging setup can collect. (b) Scattering lens imaging configuration. Scattered wave whose angle ${\theta }_r$ exceeding ${\theta }_{max}$ can be captured with a disordered medium. (c) Imaging with high NA (1.0 NA), and (d) low NA (0.15 NA), respectively. Red arrows indicate invisible structures. Scale bar: 10 μm. (e) A distorted object image through a ZnO layer (T = 6%). The image is taken with low NA. (f) The object image recovered from the distorted image in (e) (modified from Ref [12].).

Download Full Size | PDF

3.2. Wide-field endoscopic imaging through a single multimode optical fiber

Endoscopy has been playing a crucial role in many areas such as industrial, military and medical fields because it visualizes structures placed at a position where conventional imaging optics cannot reach. Nowadays, a fiber bundle has been widely used and thus became a standard for commercial endoscopes. However, the requirement for a large number of fibers for high-resolution imaging has posed constraints on the diameter of the endoscope, thereby causing considerable limitation on the accessibility of the device. A single multimode optical fiber can be a good alternative to the fiber bundle because its mode density is two or three orders of magnitude higher than the pixel density of the fiber bundle. However, the realization of a single-fiber endoscope has been elusive for several decades. Endoscopy using a single-fiber has a requirement that an object image be taken with a reflection configuration. Two folds distortion (light in and light out) should be addressed for imaging an object in a reflection mode. In this section, we review the recently developed endoscopic imaging, operating in a wide-field reflection mode, with a single multimode optical fiber. This method is called lensless microendoscopy by a single fiber (LMSF).

The setup for LMSF is schematically shown in Fig. 11 (for more details, see [21]). A He-Ne laser (λ = 633 nm) illuminates an input end (at IP) of a multimode fiber via two beam splitters (BS1 and BS2) and a 2-axis galvanometer scanning mirror (GM). The laser beam couples into the fiber and subsequently propagates toward the sample plane (SP) located at the exit of the fiber to illuminate a target object. Here we use a 1 meter-long multimode optical fiber with 200 μm core diameter, 15 μm clad thickness (230 μm in total diameter) and 0.48 NA. The total number of guiding modes for this fiber is on the order of 10⁵. In our case, we measured a transmission matrix up to 0.22 NA. Therefore, the number of modes that we dealt with was about 20,000. For a test object we used either a USAF target or a biological tissue. The light reflected from the object is collected by the same optical fiber. The collected light is guided backward through the fiber to IP, and then the output reflection image is captured by the camera. The laser beam reflected by BS1 is combined with the beam from the fiber to form an interference image at the camera. Using an off-axis digital holography algorithm, both amplitude and phase of the image from the fiber are retrieved [35, 36].

 

Fig. 11 Experimental scheme for LMSF. The setup is based on an interferometric phase microscope. The interference between the reflected light from the object located at the opposite side of a single multimode fiber and the reference light is recorded by a camera. GM: 2-axis galvanometer scanning mirror. BS1, BS2 and BS3: beam splitters. OL: objective lens. IP: illumination plane of a multimode optical fiber. SP: sample plane (modified from Ref [21].).

Download Full Size | PDF

Figure 12(c) shows typical reflection images of the USAF target recorded by the camera. Two distortion processes scramble the object image. First one is the distortion of illumination light on the way in (IP → SP) and the second one is the mixing of the reflected light (from the object) on the way out (SP → IP). In order to deal with both the problems, TLI and the speckle imaging method [37–39] are applied sequentially.

 

Fig. 12 Image reconstruction process. (a) Representative images of the measured TM following the scheme shown in (b). (b) Scheme of the separate setup for measuring a TM of the single fiber from SP to IP. (c) Reflected object images taken by the setup shown in Fig. 14 at different illumination angles ${({\theta }_x,{\theta }_y)}_S$. (d) Reconstructed object images by applying TLI method on the images in (c). (e) Averaging of all reconstructed images in (d) ([21]).

Download Full Size | PDF

For applying TLI, a transmission matrix for the multimode fiber, from SP to IP, is measured with 15,000 images up to 0.22 NA while scanning the incident light with the angle ${({\theta }_{\xi },{\theta }_{\eta })}_T$. Some representative elements of measured transmission matrix are presented in Fig. 12(a). The transmitted patterns are random speckles confined in the fiber core and the average size of the speckle becomes finer as increasing the incident angle. Because light impinging at a high angle mostly couples to the high order modes of the fiber, which have more complicated spatial distribution.

Once we measure the transmission matrix of the fiber, TLI deals with the distortion SP → IP. The object image (each image in Fig. 12(c)) is reconstructed by TLI. Using Eq. (4), the distortion from SP to IP is reversed. More details about the inversion algorithm can be found in Refs [20, 21]. The recovered images at SP are shown in Fig. 12(d). The images still remain speckled because of the distortion in the illumination light due to the propagation from IP to SP.

The speckle imaging method resolves the distortion of illumination light at SP. When imaging an object, the angle of incident light, ${({\theta }_x,{\theta }_y)}_S$, at IP is scanned as shown in Fig. 12. The scanning of the incident angle at IP causes the realization of various speckle illuminations at SP. Different speckle field is generated as a function of the angle ${({\theta }_x,{\theta }_y)}_S$. According to the speckle imaging method, a clean object image can be obtained if we average a sufficient number of images recorded at different speckle illuminations [19]. Then the complex speckle patterns are averaged out, leaving a clean object image. To implement this, object images are taken at 500 different ${({\theta }_x,{\theta }_y)}_S$, some of which are shown in Fig. 12(c). This is done in 1 second, the actual image acquisition time of LMSF. After applying TLI method to those images, the reconstructed results at SP (each in Fig. 12(d)) contain the object information even though they still look like speckle patterns. By adding all the intensity images in Fig. 12(d), the final clear object image shown in Fig. 12(e) is produced. Therefore the combination of speckle imaging and TLI method enables a single fiber microendoscopic imaging.

LMSF can provide a surveying operation as done by conventional diagnostic endoscopy. Although the change of a transmission matrix involved with a fiber modification prohibits LMSF from supporting a fully flexible endoscopic operation, we found that this operation is attainable up to a partially flexible degree. To demonstrate the endoscopic searching operation for LMSF, we translated the fiber end facing to the sample (Fig. 13(a)). A transmission matrix is measured only at the initial position of the fiber end and 500 distorted object images are acquired at different positions of the sample. Only the initial transmission matrix is repeatedly used for reconstructing the multiple object images. Figure 13(b) represents the composite image of the USAF target by stitching multiple images together. Structures ranging in a wide area, where the fiber end is translated about 800 μm in horizontally, about 1000 μm in vertically, are all clearly visible.

 

Fig. 13 Scanning operation of LMSF. (a) The fiber end is translated to take images at different sites of the sample. Each image is reconstructed with the same TM measured at the initial position of the fiber. (b) Reconstructed images are stitched to extend the field of view. Scale bar: 100 μm ([21]).

Download Full Size | PDF

Since LMSF captures complex field maps, $E_{SP}(\xi , \eta )$, of an object containing both amplitude and phase of reflected light, it can acquire 3-D information of the sample without depth scanning of the fiber end. The wave propagation can be numerically solved to make a focus at a different depth [39]. Figure 14(c) is the result of the numerical refocusing from the image in Fig. 14(b) by 40 μm toward the fiber end. The villus indicated by A that used to be blurred in Fig. 14(b) comes to a focus and its boundary becomes sharp. On the other hand, the villus pointed by B gets blurred due to the defocusing. By comparing Figs. 14(b) and 14(c), we can clearly distinguish relative depths of multiple villi in the view field. This 3-D imaging ability of LMSF will eliminate the need for the physical depth-scanning such that it will dramatically enhance the speed of volumetric imaging.

 

Fig. 14 Image of villi in a rat intestine tissue taken by LMSF. (a) A composite bright field image taken by a conventional transmission microscope. Scale bar, 100 μm. (b) A composite image by LMSF at the same site. (c) The same image as (b) but after numerical refocusing by 40 μm toward the fiber end. The arrows indicated by (A) point to the villus in focus at (c) while those indicated by (B) refer to the villus in focus at (b) ([21]).

Download Full Size | PDF

3.3. Fluorescence endomicroscopy

A drawback of using fiber bundles for endoscopy is an image pixilation artifact. An image delivered through a fiber bundle is highly pixelated because individual fibers act as image pixels. We note that the mode mixing along each fiber in a bundle can be used to eliminate the pixilation artifact. For this purpose, we measured the transmission matrix of the fiber bundle and implemented scanning fluorescence imaging. In the previous studies, fluorescence mode of endoscopic imaging was demonstrated by generating a focused spot at the opposite end of a fiber and scanning the focus to acquire an object image. In this process, a transmission matrix was used for identifying the proper incident wave to form a focus and liquid crystal type SLM (LC-SLM) was used for wavefront control [40, 41]. A digital phase conjugation was also used to calibrate the image distortion by a multimode fiber [42, 43]. In those studies, the speed of an LC-SLM limited the image acquisition rate.

To solve this problem, a digital micromirror device (DMD) was used instead of LC-SLM for the purpose of high-speed wavefront shaping [44, 45]. It was used to generate a focused spot through a scattering medium or a multimode fiber in previous studies. In this section, we review our fluorescence endoscopic imaging method working with a fiber bundle. A transmission matrix for a fiber bundle is measured using a binary control of a DMD. From the transmission matrix, a specific binary pattern that generates a focused spot at an arbitrary position on the far side of the fiber bundle. We experimentally demonstrate near real-time fluorescence imaging through a fiber bundle, which is free from the pixelation artifact.

The setup is depicted in Fig. 15. The core component of the setup is an interferometric microscope equipped with a digital micromirror device (DMD, D4100, Texas Instruments) placed in the sample beam path. A fiber bundle (FIGH-06-300S, Fujikura), which has 6000 ± 600 individual fibers with average diameter of 3.45 μm and the total outer diameter of 300 ± 25 μm, is used as a fluorescence endoscopic device. The average core-to-core distance is about 3.8 μm. We record a complex field image of the wave at the sample plane (SP) located 350 μm away from the fiber bundle exit. The field produced at SP was the mixture of each output field from the individual fiber and then is a fully developed speckle pattern as a consequence. The DMD has a 1024 × 768 micromirror array and is positioned at the conjugate plane to the input plane (IP) of the fiber bundle via the 4-f imaging system composed of a tube lens (TL1) and an objective lens (OL1). We measure a TM for the fiber bundle along the direction of IP → SP. We make a single macropixel using 7 × 7 micromirrors and create M = 10,000 macropixels in total. Using 1/111 demagnification from the DMP plane to IP, the size of an individual macropixel at IP is 860 nm × 860 nm.

 

Fig. 15 Schematic for recording a TM and fluorescence endoscopy. The output beam from a He-Ne laser is divided into a sample and a reference beam using a beam splitter, BS1. Another beams splitter, BS2, recombines the two beams. OL1 and OL2, objective lenses (PLN 40 × , Olympus); DS, dichroic beam splitter (T647lpxr, Chroma); TL1, TL2, and TL3, tube lenses (LA1979-A, Thorlabs); LF, long wavelength fiber (FELH0650, Thorlabs); DG, diffraction grating (830 Grooves); D, aperture; PMT, photomultiplier tube (H5784-20, Hamamatsu); camera (M3, IDT, 500 fps); IP, input plane of the fiber bundle; SP, sample plane. x, y, and z are the spatial coordinates in SP. The magnification from the DMD to IP 1/111 and that from SP to the camera is 55.6 (modified from Ref [22].).

Download Full Size | PDF

In order to maximize the signal-to-noise ratio (SNR) of the detection, the superposed response of the fiber bundle is measured using a DMD pattern composed of randomly chosen M/2 macropixels. The measured output response, O, at SP is expressed by

The binary pattern that generates a focus at SP can be predicted from the measured TM. To generate a focus at position i' at SP, the complex conjugate of a row vector t_i’j is implemented as an input wave. By selectively turning on the input segments which have phase values within the range between 0 and π, the required pattern can be realized on the DMD. Figure 16(c) shows the binary pattern on the DMD for generating a focused spot at the center of SP. As shown in Fig. 16(d), a clean focused spot can be seen at the target position. The local intensity enhancement compared to that of the background is 343, which is about half the theoretical prediction [46]. The discrepancy comes from the cross talk among neighboring segments on the DMD which effectively reduces the independent segments. The focus is robust under the displacement of the fiber tip up to 6.5 mm.

 

Fig. 16 Transmission matrix recording and diffraction-limited focusing. (a) Representative maps of the binary speckle basis, i.e., the binary sequence S in the main text, displayed on the DMD. White (black) pixels indicate that mirrors are in an on (off) state. (b) Intensity maps at SP corresponding to the individual binary speckle basis in (a). x and y are the coordinates at SP, and p is the index of the speckle basis. Scale bar, 10 μm. (c) A binary input pattern on the DMD identified from the measured TM for focusing the laser beam at SP. (d) Output intensity image at SP when the binary pattern in (c) was displayed on the DMD. The inset figure is 3.33 times zoom-view and 10 times over saturated display of the focus. Scale bar, 5 μm (modified from Ref [22].).

Download Full Size | PDF

Next fluorescence imaging for test objects as well as living cells is demonstrated. We scan the focus along x-y plane at SP by displaying the binary patterns at a frame rate of 22,727 Hz. The fluorescence excited by the generated focus is collected by the same fiber bundle and detected by the photomultiplier tube (PMT). As a test objet, fluorescent beads with 2 μm diameter are used. The sample is placed at SP and a transmission image is taken for a comparison (Fig. 17(a)). As shown in Fig. 17(a), the pixilation artifact deteriorates fine details of the sample. A fluorescence endoscopic image is acquired using our method. The result is presented in Fig. 17(b). The pixelation is removed and the gap where there used to be no information is filled out. Each bead, whose diameter is smaller than the size of the individual fibers, is now clearly distinguishable. For this particular imaging, 70 × 70 points are scanned covering 58 μm × 58 μm area at SP and the image acquisition speed is 4.64 Hz. A stream of fluorescence images taken while translating the fiber shows the capability of real-time imaging of our method.

 

Fig. 17 Pixelation-free endoscopic imaging through fiber bundle. (a) and (c) Conventional transmission images of 2 μm fluorescence beads (Skyblue, Spherotech) and a cancer cell line (SNU-1074), respectively, recorded with an LED illumination through the fiber bundle. (b) and (d) Endoscopic fluorescence images of the same samples as in (a) and (c), respectively, recorded by our method. Scale bar, 20 μm (modified from Ref [22].).

Download Full Size | PDF

We also demonstrated fluorescence imaging of living cells. A stained human cell line (SNU-1074) with a fluorescence dye SRfluor 680 Carboxylate (Polyscience) is used as a sample. Figure 17(c) shows a usual transmission image of a cell taken with the fiber bundle. The image taken by our method is presented in Fig. 17(d). It verifies that the image is free from the pixelate artifact and has no loss of information.

4. Conclusion

Throughout this review, we introduced the physical properties of the transmission matrix of a scattering medium and its applicability to light energy delivery and optical imaging. While these previous studies showed conceptual advances, they are still at the proof-of-concept level and further efforts are necessary for these early studies to be useful in the real practices. For example, the study of enhancing light energy delivery works only for the slab geometry of a scattering medium. For the semi-infinite samples like human body, the concept of eigenchannels described in RMT becomes meaningless in the sense that any incident wave will be totally reflected. In order to deliver light energy deep inside the semi-infinite samples, time-resolved studies can potentially be a useful approach [47]. In the case of endoscopic imaging, the data acquisition speed should be improved and it is necessary for making the transmission matrix stable throughout the measurements. The attempt to use a scattering medium as a far-field superlens should also deal with the measurement of transmission matrix covering the near-field waves. Finally, the transmission matrix approach relates input and output waves and it itself is not applicable to interrogating targets embedded within a scattering medium. In order to investigate targets inside a scattering medium, we will need to solve inverse problem rather than inversion problem. This challenging task is yet to be explored in the future.