Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group
Journal Home About Issues in Progress Current Issue All Issues Feature Issues

Polarization differential interference contrast microscopy with physics-inspired plug-and-play denoiser for single-shot high-performance quantitative phase imaging

Mariia Aleksandrovych, Mark Strassberg, Jonathan Melamed, and Min Xu

Open Access Open Access

Abstract

We present single-shot high-performance quantitative phase imaging with a physics-inspired plug-and-play denoiser for polarization differential interference contrast (PDIC) microscopy. The quantitative phase is recovered by the alternating direction method of multipliers (ADMM), balancing total variance regularization and a pre-trained dense residual U-net (DRUNet) denoiser. The custom DRUNet uses the Tanh activation function to guarantee the symmetry requirement for phase retrieval. In addition, we introduce an adaptive strategy accelerating convergence and explicitly incorporating measurement noise. After validating this deep denoiser-enhanced PDIC microscopy on simulated data and phantom experiments, we demonstrated high-performance phase imaging of histological tissue sections. The phase retrieval by the denoiser-enhanced PDIC microscopy achieves significantly higher quality and accuracy than the solution based on Fourier transforms or the iterative solution with total variance regularization alone.

© 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

Quantitative phase imaging (QPI) is an emerging imaging modality that enables the examination of transparent biological samples for biomedical applications by probing alterations in the amplitude and phase of the illumination light [1]. Quantitative phase imaging can be largely categorized into two classes: interferometry-based holography [2] and optical diffraction tomography [3,4], and intensity-only methods such as those based on phase contrast, differential interference contrast [5,6], the transport of the intensity equation (TIE) [7,8], and ptychography [9–12]. QPI has become a powerful tool for measuring biomass distribution and time-lapse imaging of cellular dynamics for biomedical monitoring and diagnosis [13–16] with recent advances in spatial resolution [17], imaging speed [18–21], high content imaging [22], polarization-sensitive imaging [3,23], and 3D imaging [24–27]. Multiplexing approaches in polarization [28] and color [29] have also been developed for single-shot phase imaging.

QPI belongs to the class of image restoration (IR) problems, traditionally approached using classical methods such as Fourier transforms [30] and variational methods [31]. Recently, machine learning (ML) techniques have garnered significant attention in the field of IR, finding applications in QPI and microscopy for superresolution imaging [32], lensless computational imaging [33,34], single-molecule localization [35–38], protein localization at a cellular level [39], particle tracking [40–42], and biomedical applications such as digital pathology [43–45] and label-free virtual staining of tissue [46,47].

One particular ML approach, Plug-and-Play (PnP) learning-based computational imaging [48,49], departs from a purely computational or data-driven algorithm. Instead, utilizing a denoiser, it can uniquely combine a forward model (physics constraints) and data-driven priors on the solution distribution. Unlike traditional model-based methods, which need to specify the explicit and hand-crafted image priors, plug-and-play IR implicitly defines the prior via the denoiser, offering the possibility of leveraging a deep convolutional neural network (CNN) denoiser to improve effectiveness [50]. Similarly, physics-based learning facilitates designing optimized coded illumination patterns for quantitative phase imaging, reducing the number of illumination patterns required while maintaining the image quality [51]. These developments underscore the growing interest in Physics-inspired machine learning to overcome existing challenges and enhance imaging performance.

Quantitative phase imaging strives to enhance four figures of merit: space-bandwidth product, time-bandwidth product, spatial phase sensitivity, and temporal phase sensitivity [1]. While it is difficult to maximize each of these parameters in one single system, we recently introduced a single-shot quantitative phase imaging technique using polarization differential interference contrast (PDIC) [52], which performs well in all four figures of merit with single snapshot broadband illumination and common-path geometry. This PDIC microscopy is capable of high spatial resolution phase imaging in real-time, limited only by the camera frame rate. A total variance (TV) penalty was introduced in the PDIC phase recovery to compensate for the lack of sensitivity to the phase gradient along the direction orthogonal to the DIC shear. In this work, we contribute to the evolving field of Physics-inspired machine learning and quantitative phase imaging and demonstrate the superior image quality and accuracy in phase recovery achieved by a Physics-inspired Plug-and-Play Denoiser-enhanced PDIC microscopy, which incorporates a pre-trained deep Dense Residual U-net (DRUNet) denoiser [50,53] on top of the total variance regularization for polarization differential interference contrast microscopy. This new physics-inspired approach significantly outperforms the solution based on Fourier transforms or the iterative solution with total variance regularization alone, promising single-shot, high spatial resolution, and high-quality quantitative phase imaging for vast applications in biomedicine.

The paper is organized as follows. We first introduce polarization differential interference contrast microscopy and its integration with a plug-and-play pre-trained Dense Residual U-net denoiser. Then, we provide implementation details utilizing the alternating direction method of multipliers (ADMM) in section 2. In section 3, we validate the effectiveness of this DRUNet-enhanced PDIC microscopy through simulated data and phantom experiments. Finally, we demonstrate the high-performance phase imaging of histological tissue sections. We evaluate the phase recovery quality and accuracy improvement using various image quality metrics.

2. Method and materials

2.1 Polarization differential interference contrast microscopy

The PDIC microscope we recently introduced [52] is briefly outlined here. The PDIC microscopy is implemented on a Nomarski DIC microscope (Fig. 1). The incident wavefront and its replica with orthogonal polarization directions pass through the same sample at an offset (DIC shear s). The incident wavefront ${{\boldsymbol E}_1}$ and its replica ${{\boldsymbol E}_2}$ have equal amplitudes |${E_1}$ | = |${E_2}$ | = ${E_0}$ and orthogonal polarization directions.

 figure: Fig. 1.

Fig. 1. A schematic diagram of an inverted PDIC microscope.

Download Full Size | PDF

The two waves then recombine and interfere, producing an interference pattern recorded by a polarization-sensitive camera. Inside the reference frame (viewed facing the impugning beam) of the polarization camera, the net electric field is expressed as

$$\hat{x}{E_s}' - \hat{y}{E_p}' = \hat{x}{E_0}\frac{{1 + \exp \left( {i{\Delta }} \right)}}{{\sqrt 2 }} - \hat{y}r{E_0}\frac{{1 - \exp \left( {i{\Delta }} \right)}}{{\sqrt 2 }}\exp \left( {i\delta } \right)$$
where $\hat{x}\; $ and $\hat{y}\; $ are the unit vectors along the x and y directions, $\Delta \equiv {\boldsymbol s} \cdot \nabla \phi + {\phi _b}$, $\phi $ is the phase profile of the specimen, s is the DIC shear, ${\phi _b}$ is the DIC bias, and r and δ are the reflectivity ratio and phase difference between the s and p polarization components by the deflection mirror in the microscope [52]. The unfiltered interference pattern produced by Eq. (1) is recorded, yielding ${\boldsymbol I} = ({{I_{{0^ \circ }}},{I_{{{45}^ \circ }}},{I_{{{90}^ \circ }}},{I_{{{135}^ \circ }}}} )$ for the intensity of light linearly polarized along ${0^o}$, ${45^o}$, ${90^o}$, and ${135^o}$ directions, respectively, and the Stokes vector
$${\boldsymbol S}(\boldsymbol\rho )= \left( {\begin{array}{{c}} {{S_0}(\boldsymbol\rho )}\\ {{S_1}(\boldsymbol\rho )}\\ {{S_2}(\boldsymbol\rho )} \end{array}} \right) = \left( {\begin{array}{{c}} {{I_{{0^ \circ }}}(\boldsymbol\rho )+ {I_{{{90}^ \circ }}}(\boldsymbol\rho )}\\ {{I_{{0^ \circ }}}(\boldsymbol\rho )- {I_{{{90}^ \circ }}}(\boldsymbol\rho )}\\ {{I_{{{45}^ \circ }}}(\boldsymbol\rho )- {I_{{{135}^ \circ }}}(\boldsymbol\rho )} \end{array}} \right)$$
in a single snapshot where ${\boldsymbol \rho } = ({x,y} )$ denotes the lateral spatial coordinates. The PDIC microscopy relates the measured Stokes vector to the optical properties of the specimen via
$$\begin{array}{l} {S_0}' = 2{r^2}{I_0}\\ {S_1}' = 2{r^2}{I_0}\cos {\Delta }\\ {S_2} = - 2r{I_0}\sin \delta \sin {\Delta } \end{array}$$
where ${S_{0,1}}^{\prime} = [{(1 + {r^2}){S_{0,1}} - (1 - {r^2}){S_{1,0}}} ]/2$ and ${I_0} = E_0^2$ [52]. We obtain the phase map by solving ${\mathbf s} \cdot \nabla \phi = \Delta - {\phi _b} = \textrm{atan2(}{S_2}, - S_1^{\prime}\sin \delta ) - {\phi _b}$ under the optimal measurement condition for phase retrieval (${\phi _b} = {\pi / 2}$). For convenience, the phase retrieval problem can be rewritten as the determination of the phase profile $u({x,y} )$ given the measured phase gradient ${{\partial u} / {\partial x}} \equiv {\partial _x}u = {\phi _x}({x,y} )$ assuming the DIC shear along the $x$-axis. A total variance regularized solution strategy has been described [52].

2.2 Alternating direction method of multipliers for PDIC microscopy with a plug-and-play denoiser

We use the alternating direction method of multipliers [54] to integrate the denoising prior into a physical forward model to achieve high-quality image restoration results. We consider phase retrieval incorporating total variance and deep denoiser as the following optimization problem:

$$\begin{array}{{c}} {\mathop {\min }\limits_u \int {d\boldsymbol{\rho} } \left( {\frac{\mu }{2}{{|{{\partial_x}u - {\phi_x}} |}^2} + \alpha |w |+ \gamma \Phi (z )+ \frac{\tau }{2}{{|{u - z} |}^2}} \right)}\\ {\textrm{subject to }{\partial _y}u - w = 0} \end{array}$$
where $\mu $ is a positive constant that controls the weight of the fidelity term, $\alpha $ is a positive constant that controls the weight of TV regularization, $\Phi (z)$ denotes the denoiser, $\gamma $ is a positive constant that controls the degree of the denoiser activation, and $\tau $ is a penalty parameter. The denoiser prior is introduced in Eq. (4) by Half Quadratic Splitting (HQS) technique [55] following [50].

This ADMM formalism combines the advantages of dual decomposition and augmented Lagrangian methods, allowing for the incorporation of the dual update of the phase with TV regularization. The Eq. (4) is solved by the introduction of an augmented Lagrangian function [56,57]:

$${L_A} = \int {d{\boldsymbol \rho }\left( {\alpha |w |+ \frac{\beta }{2}{{|{w - {\partial_y}u} |}^2} - \lambda ({w - {\partial_y}u} )+ \frac{\mu }{2}{{|{{\partial_x}u - {\phi_x}} |}^2} + \gamma \Phi (z )+ \frac{\tau }{2}{{|{u - z} |}^2}} \right)}$$
and with the iterations:
$$\begin{array}{l} {w^{({k + 1} )}} = \mathop {\min }\limits_w \int {d{\boldsymbol \rho }\left( {\alpha |w |+ \frac{\beta }{2}{{|{w - {\partial_y}{u^{(k )}}} |}^2} - {\lambda^{(k )}}w} \right)} \\ {u^{({k + 1} )}} = \mathop {\min }\limits_u \int {d{\boldsymbol \rho }\left( {\frac{\beta }{2}{{|{{w^{({k + 1} )}} - {\partial_y}u} |}^2} + {\lambda^{(k )}}{\partial_y}u + \frac{\mu }{2}{{|{{\partial_x}u - {\phi_x}} |}^2} + \frac{\tau }{2}{{|{u - {z^{(k )}}} |}^2}} \right)} \\ {z^{({k + 1} )}} = \mathop {\min }\limits_z \int {d{\boldsymbol \rho }\left( {\Phi (z )+ \frac{\tau }{{2\gamma }}{{|{z - {u^{({k + 1} )}}} |}^2}} \right)} \\ {\lambda ^{({k + 1} )}} = {\lambda ^{(k )}} - \beta ({{w^{({k + 1} )}} - {\partial_y}{u^{({k + 1} )}}} ). \end{array}$$
where the superscript $(k )$ means the quantities at $k$-th iteration. The first three equations in (6) are the minimization of w, u, and z, respectively, and the fourth is for the dual variable update. In particular, the intermediate phase ${u^{(k + 1)}}$ is input to the denoiser $\Phi (z )$ to obtain the denoised phase ${z^{(k + 1)}}$.

This set of iterations reduces to

$$\begin{array}{l} {w^{({k + 1} )}} = \left( {{v^{(k )}} - \frac{\alpha }{\beta }\frac{{{v^{(k )}}}}{{|{{v^{(k )}}} |}}} \right)\textrm{H}\left( {|{{v^{(k )}}} |- \frac{\alpha }{\beta }} \right)\\ {{\tilde{u}}^{({k + 1} )}} = \frac{{ - i\mu {k_x}{{\tilde{\phi }}_x} - i\beta {k_y}{{\tilde{w}}^{({k + 1} )}} + i{k_y}{{\tilde{\lambda }}^{(k )}} + \tau {{\tilde{z}}^{(k )}}}}{{\mu k_x^2 + \beta k_y^2 + \tau }}\\ {z^{({k + 1} )}} = \mathop {\min }\limits_z \int {d\rho \left( {\Phi (z )+ \frac{\tau }{{2\gamma }}{{|{z - {u^{({k + 1} )}}} |}^2}} \right)} \\ {\lambda ^{({k + 1} )}} = {\lambda ^{(k )}} - \beta ({{w^{({k + 1} )}} - {\partial_y}{u^{({k + 1} )}}} ). \end{array}$$

Here ${v_k} \equiv {\partial _y}{u^{(k )}} + \frac{{{\lambda ^{(k )}}}}{\beta }$, $\textrm{H}$ is the Heaviside function, ${\tilde{\phi }_x},\tilde{u},\tilde{w},\tilde{z}$ and $\tilde{\lambda }$ are the Fourier transform of ${\phi _x},u,w,z$ and $\lambda $, respectively, and ${k_x}$, ${k_y}$ are the x and y components of the wavenumber inside the Fourier space.

The beauty of the ADMM formalism is that each primal sub-problem has its physical meaning. Updating ${u^{(k + 1)}}$ projects the solution onto the data fidelity domain. It has a closed-form solution in the Fourier domain. Updating ${w^{(k + 1)}}$ and ${z^{(k + 1)}}$ projects the solution to the image prior domain, minimizing the total variance and DRUNet deep denoising, respectively. These steps blend data fidelity, total variance regularization, and denoising in a unified framework for a versatile and state-of-art recovery.

For the TV-regularization part, we follow Mead [58] and set $\mu = {1 / {\textrm{Var}}}({{\phi_x}} )$ and initialize $\alpha = 20$ and $\beta \textrm{ = 20}\alpha$. For the denoiser part, we follow Zhang et al. [50] and set $\gamma = 0.23$ and $\sqrt {{1 / \tau }} $ taking a series of values ${\sigma _k}$ decreasing from ${\xi _0}$ to $\sqrt {\textrm{Var}(u )} $, i.e.,

$$\begin{array}{l} {\sigma _k} = {\xi _0}\textrm{ to }\sqrt {\textrm{Var}(u )} \textrm{ geometrically }({k = 0,1,\ldots ,K - 1} )\\ \gamma = 0.23\\ \tau = {1 / {\sigma _k^2}} \end{array}$$

An adaptive method for updating β is adopted for faster convergence by forcing the primary and dual residues to converge to zero at a similar rate. The primary and dual residues for ADMM are given by [54]

$${r^{({k + 1} )}} = {\partial _y}{u^{({k + 1} )}} - {w^{({k + 1} )}}$$
and
$${s^{({k + 1} )}} ={-} \beta \textrm{ }{\partial _y}^T({{w^{({k + 1} )}} - {w^{(k )}}} )$$
with their feasibility tolerance given by
$$\begin{array}{l} {\varepsilon ^{\textrm{pri,(}k\textrm{)}}} = {\varepsilon ^{\textrm{rel}}}\max \{{|{{\partial_y}{u^{(k )}}} |,|{{w^{(k )}}} |} \}\\ {\varepsilon ^{\textrm{dual,(}k\textrm{)}}} = {\varepsilon ^{\textrm{rel}}}|{{\partial_y}^T{\lambda^{(k )}}} |. \end{array}$$

The update rule is formulated as

$${\beta ^{({k + 1} )}} = \left\{ \begin{array}{l} 2{\beta^{(k )}}\\ \frac{1}{2}{\beta^{(k )}}\\ {\beta^{(k )}} \end{array} \right.\textrm{ }\begin{array}{{c}} {\textrm{if }|{{r^{(k )}}} |/{\varepsilon ^{\textrm{pri,(}k\textrm{)}}} > \eta |{{s^{(k )}}} |/{\varepsilon ^{\textrm{dual,(}k)}}}\\ {\textrm{if }|{{r^{(k )}}} |/{\varepsilon ^{\textrm{pri,(}k)}} < {\eta ^{ - 1}}|{{s^{(k )}}} |/{\varepsilon ^{\textrm{dual,(}k)}}}\\ {\textrm{otherwise}} \end{array}$$
with a typical choice $\eta = 2$ and ${\varepsilon ^{\textrm{rel}}} = 0.1$. This choice of update rule was experimentally chosen to accelerate the ADMM algorithm.

The iteration starts by setting.

$${u^{(0 )}} = {\mathbf 0},\textrm{ }{w^{(0 )}} = {\mathbf 0},\textrm{ }{z^{(0 )}} = {\mathbf 0},\textrm{ }{\lambda ^{(0 )}} = {\mathbf 0}$$
where 0 is a zero matrix of the same shape as ${\phi _x}$. The error of ${\phi _x}$ is $\textrm{Var}({{\phi_x}} )= {({{d / s}} )^2}{({2{\gamma_0}{I_0}} )^{ - 1}}$ where d is the pixel size after converting to ${\partial _x}u = {\phi _x}$ over the discrete space of unit grids and ${\gamma _0}$ is the photon electron conversion factor. The error in u is found from experiments to be $\textrm{Var}(u )\simeq {({2{\gamma_0}{I_0}} )^{ - 1}}$ [52]. The value ${\xi _0}$ sets the weakest denoiser penalty at the start of the iterations. It controls the overall balance between total variance regularization and denoising. It is set ${\xi _0} = 0.1$ for tissue specimens. The total number of denoiser strength levels is set at $K = 10$. Multiple iterations at each denoiser strength level are conducted to reduce the total residue ratio $({|{{r^{(k )}}} |/{\varepsilon^{\textrm{pri,(}k\textrm{)}}} + |{{s^{(k )}}} |/{\varepsilon^{\textrm{dual,(}k\textrm{)}}}} )/2$ below a given threshold ($= 1$). The final solution is reached when the relative change in u is less than a threshold (${10^{ - 3}}$) or the image quality (for example, sharpness) drops.

2.2 Denoiser architecture

While CNN denoisers hold potential, current methods are often constrained by specific choices that underutilize CNN capabilities [59]. We note that it is crucial to accommodate varied noise levels in plug-and-play techniques for denoising. We adopt a plug-and-play deep denoiser DRUNet proposed by Zhang [50,60], a deep learning model composed of a downsampling tower (capturing global information) and an upsampling tower (capturing local information). DRUNet uses the prevalent U-Net [61] with a combination of the ResNet [62] architecture to achieve high-performance and versatile image restoration (see Fig. 2). The denoiser takes a noisy image with an additional noise map as input. The denoiser is trained under varying noise levels, which not only allows the denoiser to be aware of the noise level in denoising but also increases its robustness. The denoiser is imposed upon the iterative phase recovery with a steadily increasing strength and decreasing noise level until reaching the measurement noise. The UNet part includes four scales of 64, 128, 256, and 512 channels, respectively, and each scale has skip connections between 2 × 2 stride convolution (SConv) downscaling and 2 × 2 transposed convolution (TConv) upscaling layers. Downscaling and upscaling have four residual blocks in each scale. No activation function is allowed in the SConv, TConv, first, and last convolutional layers. Each residual block contains one activation function.

 figure: Fig. 2.

Fig. 2. DRUNet architecture. Each green box represents four consecutive Residual Blocks. Each blue box represents a Stride Convolutional layer. Each pink box represents a Transposed Convolutional layer.

Download Full Size | PDF

2.3 DRUNet training details

We trained the DRUNet denoiser on a set of natural scene images, leveraging the benefits of a large training dataset for convolutional networks. The training set consists of 400 grayscale images of size 180 × 180 (Trian400 [50]). The training process follows Zhang and Zuo [50]. A stochastic optimization algorithm (ADAM) [63] minimizes the L1 loss between the denoised images and the ground truth. The learning rate for the training starts from ${10^{ - 4}}$ and adaptively reduces to ${10^{ - 6}}$ during epochs. For each iteration, 16 patches from the training data were randomly selected. The training process continues until the loss function reaches the minimum. The training takes 2000 epochs and requires approximately 20 hours on a GPU (Nvidia GeForce GTX 1050). Figure 3 shows the training and validation loss of pre-trained DRUNet where the noise level varies between 0 and 50 in training and is set at 25 in validation.

 figure: Fig. 3.

Fig. 3. The training and validation losses of the pre-trained DRUNet.

Download Full Size | PDF

While pretraining on a biological dataset could be advantageous, it is worth noting that biological phase images lack a definitive ground truth. Training using natural scenes proves more beneficial [64] as natural scenes’ diverse structures and styles facilitate the denoiser's ability to learn complex images of varying types. As a result, the denoiser can generalize its performance for diverse images.

2.4 Experimental setup

Experiments were performed on an inverted epifluorescence DIC microscope (IX73, Olympus). The light source was a Halogen 100W lamp filtered by a green filter (wavelength 0.545µm). The numerical aperture for the condenser was 0.55, and the numerical apertures for the 40x dry and 60x water immersion objectives were 0.6 and 1.2, respectively. The Stokes vector was recorded by a polarization camera (Blackfly S Polarized, FLIR). Linear interpolation was used to fill in the missing data of the polarization images (light intensity linearly polarized along 0, 45, 90, and 135 degrees) captured by the polarization camera. The histopathologic specimens were hematoxylin and eosin (H&E) stained prostate cancer tissue sections (PR632, PR243c, Biomax). Three methods were employed to recover phase images: the Fourier transform-based solution, an iterative solution with a total variance regularization, and the Plug-and-Play deep denoiser-enhanced approach proposed above.

3. Results

3.1 Phase retrieval of simulated data and phantom experiments

The proposed phase retrieval method was first validated using a simulated target of Siemens star with a modified version of microlith [65] under the identical condition of the experimental PDIC microscope using a 60x objective. Figure 4 shows the ground truth, the reconstructed phase images by Fourier transform (FT), total variance regularized, and the TV-DRUNet denoiser methods.

 figure: Fig. 4.

Fig. 4. Phase retrieval of simulated Siemens star. (a) Ground truth, and the reconstructed phase using (b) the Fourier Transform solution, (c) the total variance regularized solution, and (d) the TV + DRUNet solution.

Download Full Size | PDF

Phase profiles at various distances (blue: 5 µm, red: 3 µm, green: 2.3 µm, and magenta: 2 µm) from the center are plotted in Fig. 5 to compare the performance of the three methods. The TV + DRUNet outperforms the other two methods, reaching the theoretical resolution limit (0.31 µm) under this condition (see Fig. 5(f)).

 figure: Fig. 5.

Fig. 5. (a, c, e) show the reconstructed phase images by the Fourier transform, total variance regularized, and TV + DRUNet methods. Examples of the corresponding phase profiles are shown in (b), (d), and (f) at the radius of 5 µm (blue), 3 µm (red), 2.3 µm (green), and 2 µm (magenta) from the center together with the ground truth (black). The phase profiles are shown for the angular range between 0 and π.

Download Full Size | PDF

For the simulated Siemens star, root-mean-squared error (RMSE) and multiscale structural similarity (MS-SSIM) between the recovered phase image by different approaches and the ground truth were calculated (see Table 1). The TV + DRUNet method yields the smallest RMSE value (0.0731) and the largest MS-SSIM (0.688) compared to the other two techniques. The phase of Siemens star could be resolved at 2 microns from the center by the TV + DRUNet method (the diffraction limit). In contrast, the Fourier Transform solution loses resolution in the vertical direction at 3 microns from the center. The TV-regularized solution partially loses resolving power at 2.3 microns from the center.

Tables Icon

Table 1. Performance of Fourier Transform (FT), Total Variance (TV) regularized, and Total Variance plus DRUNet denoiser (TV + DRUNet) Solutions for Siemens Star

View Table | View all tables in this article

Figure 6 displays the SSIM histogram and map for the recovered Siemens Star. The SSIM score spans between -1 and 1, with a value of 1 denoting a perfect match, while -1 represents a complete mismatch. The histograms illustrate most SSIM scores clustering around 1 for the TV + DRUNet solution and peaking at ∼0.8 for the TV solution. These SSIM distributions align with their mean MS-SSIM values, i.e., 0.620 for FT, 0.611 for TV, and 0.688 for TV + DRUNet.

 figure: Fig. 6.

Fig. 6. (a, c, e) Histograms of per-pixel SSIM scores of the reconstructed phase images by the Fourier transform, total variance regularized, and TV + DRUNet methods. (b, d, e) The corresponding SSIM maps.

Download Full Size | PDF

Furthermore, we performed measurements of the phase of a 3.85µm silica sphere (Bangs Laboratories, Catalog No. SS05002, refractive index: 1.43) embedded in glycerol (refractive index: 1.47) under the water immersion 60x objective. Horizontal and vertical phase profiles were plotted for three methods to show the improvements of the TV + DRUNet method. Horizontal and vertical profiles (DIC shear direction) agree well with the ground truth for the TV + DRUNet method (see Fig. 7(f)).

 figure: Fig. 7.

Fig. 7. The negative phase images of a 3.85µm silica sphere embedded in glycerol recovered by the Fourier transform solution in (a), (d), the TV regulation method in (b), (e), and the TV + DRUNet method in (c), (f). The horizontal and vertical profiles along the red and green dashed lines in (a) – (c) are shown in (d) – (f), respectively.

Download Full Size | PDF

3.2 Phase retrieval of histological tissue specimens

After validating PDIC microscopy with a plug-and-play denoiser for the simulated data and phantom experiments, we demonstrated its performance on histological tissue sections. Figures 8 and 9 show the retrieved phases for prostate cancer tissue (adenocarcinoma, stage III) measured with 40x objective and prostate cancer adjacent normal tissue measured with 60x objective, respectively. Compared to the other two methods, the improvement in the phase recovery quality by TV + DRUNet can be recognized in increased human perception of histological tissue features such as fiber formation and cellular structure with yellow arrows pointing to representative regions.

 figure: Fig. 8.

Fig. 8. Prostate cancer tissue (adenocarcinoma, stage III) measured with 40x objective. (a) H&E-stained histopathological image and reconstructed phase images using (b) Fourier transform, (c) total variance regularized, and (d) total variance plus DRUNet denoiser methods. Yellow arrows point to representative regions.

Download Full Size | PDF

 figure: Fig. 9.

Fig. 9. Prostate cancer adjacent normal tissue measured with 60x objective. (a) H&E-stained histopathological image and reconstructed phase images using (b) Fourier transform, (c) total variance regularized, and (d) total variance plus DRUNet denoiser methods. Additionally, the region marked by the yellow rectangle in the H&E-stained histopathological image was enlarged in (e)-(h). Yellow arrows point to representative regions.

Download Full Size | PDF

As evident in Fig. 9, TV regularization in phase retrieval effectively eliminates artifacts. Nonetheless, some artifacts persist along the vertical direction, which is orthogonal to the shear direction of the PDIC microscope. However, the residue artifacts observed in the method above are almost gone by integrating the denoiser into the phase retrieval process, as depicted in Fig. 9(h).

Owing to the lack of ground truth for the tissue phase maps, we adopt a no-reference quality metric for superresolution images based on human perception [64] to measure the quality of phase recovery without ground truth (see Table 2). This metric is observed to correlate closely with the human perception of the image quality and outperforms other no-reference metrics. The score ranges between 0 and 10. A low score indicates high perceptual quality and a high one suggests low perceptual quality. An alternative metric is to compare the structural similarity between the phase image and the corresponding H&E-stained image as they all reveal the underlying tissue structure (see Table 3). The TV + DRUNet method outperforms the other two methods in both metrics.

Tables Icon

Table 2. Phase recovery perception score of Fourier transform (FT), Total Variance (TV) regularized, and Total Variance plus DRUNet denoiser (TV + DRUNet) Solutions

View Table | View all tables in this article
Tables Icon

Table 3. Phase recovery Multiscale Structural Similarity Index (MS-SSIM) of Fourier transform (FT), Total Variance (TV) regularized, and Total Variance plus DRUNet denoiser (TV + DRUNet) Solutions

View Table | View all tables in this article

3.3 Discussions

Quantitative phase imaging by polarization differential interference contrast microscopy inherits significant advantages of DIC microscopy, including no need for phase unwrapping, tolerance to scattering, and the optical sectioning much sharper than other QPI methods [23]. Comparing the three approaches for solving the phase for PDIC microscopy, the TV regularization strategy improves the phase image quality by alleviating the imbalance in sensitivities along the DIC shear and its orthogonal directions for phase retrieval from the partial derivatives along one single axis alone. The artifacts in Fourier transform solutions [29,30] largely disappear with the TV regularization strategy. Integrating total variance regularization and a deep denoiser (DRUNet) further suppresses the residue artifacts. It achieves the highest phase image quality and accuracy performance among the three methods. This outperformance originates from the complementary priors of total variance as a locally smoothing internal prior, which captures the overall structure and serves as a good starting point, and DRUNet denoiser as a deep global external prior, which refines the details and retains the global structure [66].

The TV + DRUNet method yielded the lowest root-mean-square error (RMSE) and the highest multiscale structural similarity index (MS-SSIM) values for Siemens star images. It also recovers silica spheres in the best agreement with the ground truth. Furthermore, the phase images recovered for histological tissue sections by TV + DRUNet are superior to other methods measured by the score of super-resolution images based on human perception and structural similarity to the H&E-stained image without ground truth. Traditional image quality metrics such as MSE [67] require ground truth and do not match human perception. Ma et al. [64] showed that images containing smoother content tend to exhibit higher perceptual scores, particularly when lacking prominent edge textures and details. Therefore, a no-reference image quality assessment for superresolution images based on human perception was evaluated for phase imaging of histological tissue sections. The structural similarity between the H&E-stained images further assesses how well the phase recovery maintains the tissue texture.

The superior performance of DRUNet denoiser in improving the phase retrieval quality and accuracy is related to its ability to adapt to the underlying data structure and eliminate noise while preserving fine details and updating the data each iteration of image restoration. It avoids over-smoothing and reduces the impact of irrelevant degradation caused by the noise. In addition, we find that the denoiser should be selected and customized according to the underlying physics. DRUNet was modified to use the Tanh activation function instead of the ReLU function. This choice of the activation function is essential to guarantee the symmetry requirement for phase retrieval (odd function near zero). Combining total Variance and DRUNet with the ReLU activation function leads to distorted phase recovery.

Figure 10 and Table 4 show that the utilization of the Tanh activation function resulted in improved quality of the phase recovery compared to that with a ReLU activation function using the Siemens star as an example. The artifact of the phase recovery using the ReLU activation function is evident. Furthermore, the Tanh activation function outperforms the ReLU activation function regarding RMSE and MS-SSIM for phase reconstruction quality (see Table 4).

 figure: Fig. 10.

Fig. 10. The recovered phase of Siemens star using the proposed algorithm using a denoiser with (a) ReLU and (b) Tanh activation functions.

Download Full Size | PDF

Tables Icon

Table 4. Performance of the Recovered Siemens Star by TV + DRUNet with ReLU or Tanh Activation Function

View Table | View all tables in this article

The proposed adaptive algorithm achieved rapid convergence. Figure 11 shows one typical behavior of phase recovery convergence using the Siemens star as an example. The relative error between the recovered phase and the ground truth quickly decreased within the first ten rounds of TV and DRU iterations. The following TV and DRU iterations enhance phase image quality, with the relative error converging to around 6.2%. Although the relative errors do not drop significantly after the 10th iteration, the quality metrics steadily improve with further iterations. For example, the MS-SSIM score increases from 0.659 at the 10th iteration to 0.688 when the iterations end.

 figure: Fig. 11.

Fig. 11. Relative errors between the recovered phase and the ground truth for the Siemens star. Blue marks indicate total variance iterations, while red marks indicate denoising iterations.

Download Full Size | PDF

Finally, it is worth pointing out several limitations. Quantitative phase imaging by PDIC microscopy assumes sample birefringence is weak and absorption variation is slow, as detailed earlier [52]. Using a polarization camera, although providing advantages in speed and reliability, constrains the choice of imaging devices. The extension of PDIC microscopy for quantitative phase and birefringence imaging is currently under development.

4. Conclusions

We have presented polarization differential interference contrast microscopy with Physics-inspired plug-and-play denoiser for single-shot high-performance quantitative phase imaging. The details of the iterative algorithm, which balances total variance regularization and a deep denoiser based on the alternating direction method of multipliers, are provided. The pre-trained DRUNet denoiser was modified to use the Tanh activation function to guarantee the symmetry requirement for phase retrieval. In addition, we have introduced an adaptive strategy for accelerating measurement noise-aware convergence. From validation on simulated data and phantom experiments and demonstration of histological tissue sections, the phase retrieval by the denoiser-enhanced PDIC microscopy achieved significantly higher quality and accuracy than the solution based on Fourier transforms or the iterative solution with total variance regularization alone. Deep denoiser-enhanced PDIC microscopy may find vast applications in biomedicine for single-shot, high spatial resolution, and high-quality quantitative phase imaging.

Funding

National Science Foundation (1607664).

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request. The code for the TV+DRUNet algorithm will be publicly available at [68].

References

1. Y. K. Park, C. Depeursinge, and G. Popescu, “Quantitative phase imaging in biomedicine,” Nat. Photonics 12(10), 578–589 (2018). [CrossRef]  

2. D. Donnarumma, A. Brodoline, D. Alexandre, and M. Gross, “4D holographic microscopy of zebrafish larvae microcirculation,” Opt. Express 24(23), 26887 (2016). [CrossRef]  

3. A. Saba, J. Lim, A. B. Ayoub, E. E. Antoine, and D. Psaltis, “Polarization-sensitive optical diffraction tomography,” Optica 8(3), 402 (2021). [CrossRef]  

4. S. Kang, R. Zhou, M. Brelen, H. K. Mak, P. T. C. So, and Z. Yaqoob, “Reflection-mode optical diffraction tomography for label-free imaging of thick biological specimens,” arXiv, arXiv:2202.13668 (2022). [CrossRef]  

5. M. Shribak, J. LaFountain, D. Biggs, and S. Inouè, “Orientation-independent differential interference contrast microscopy and its combination with an orientation-independent polarization system,” J. Biomed. Opt. 13(1), 014011 (2008). [CrossRef]  

6. M. Shribak, K. G. Larkin, and D. Biggs, “Mapping optical path length and image enhancement using quantitative orientation-independent differential interference contrast microscopy,” J. Biomed. Opt. 22(1), 016006 (2017). [CrossRef]  

7. D. Paganin and K. A. Nugent, “Noninterferometric phase imaging with partially coherent light,” Phys. Rev. Lett. 80(12), 2586–2589 (1998). [CrossRef]  

8. L. Waller, Y. Luo, S. Y. Yang, and G. Barbastathis, “Transport of intensity phase imaging in a volume holographic microscope,” Opt. Lett. 35(17), 2961–2963 (2010). [CrossRef]  

9. A. M. Maiden, J. M. Rodenburg, and M. J. Humphry, “Optical ptychography: a practical implementation with useful resolution,” Opt. Lett. 35(15), 2585–2587 (2010). [CrossRef]  

10. T. M. Godden, R. Suman, M. J. Humphry, J. M. Rodenburg, and A. M. Maiden, “Ptychographic microscope for three-dimensional imaging,” Opt. Express 22(10), 12513 (2014). [CrossRef]  

11. C. Shen, A. C. S. Chan, J. Chung, D. E. Williams, A. Hajimiri, and C. Yang, “Computational aberration correction of VIS-NIR multispectral imaging microscopy based on Fourier ptychography,” Opt. Express 27(18), 24923 (2019). [CrossRef]  

12. S. Zhou, J. Li, J. Sun, N. Zhou, Q. Chen, and C. Zuo, “Accelerated Fourier ptychographic diffraction tomography with sparse annular LED illuminations,” J. Biophotonics 15, 1–15 (2022). [CrossRef]  

13. Z. Wang, G. Popescu, K. V. Tangella, and A. Balla, “Tissue refractive index as marker of disease,” J. Biomed. Opt. 16(11), 1 (2011). [CrossRef]  

14. M. Xu, “Scattering-phase theorem: anomalous diffraction by forward-peaked scattering media,” Opt. Express 19(22), 21643 (2011). [CrossRef]  

15. L. Zheng, K. Yu, S. Cai, Y. Wang, B. Zeng, and M. Xu, “Lung cancer diagnosis with quantitative DIC microscopy and a deep convolutional neural network,” Biomed. Opt. Express 10(5), 2446 (2019). [CrossRef]  

16. X. Chen, M. E. Kandel, and G. Popescu, “Spatial light interference microscopy: principle and applications to biomedicine,” Adv. Opt. Photonics 13(2), 353 (2021). [CrossRef]  

17. C. Zheng, D. Jin, Y. He, H. Lin, J. Hu, Z. Yaqoob, P. T. C. So, and R. Zhou, “High spatial and temporal resolution synthetic aperture phase microscopy,” Adv. Photonics 2(06), 1–8 (2020). [CrossRef]  

18. N. Lue, J. W. Kang, T. R. Hillman, R. R. Dasari, and Z. Yaqoob, “Single-shot quantitative dispersion phase microscopy,” Appl. Phys. Lett. 101(8), 084101 (2012). [CrossRef]  

19. M. Trusiak, V. Mico, J. Garcia, and K. Patorski, “Quantitative phase imaging by single-shot Hilbert–Huang phase microscopy,” Opt. Lett. 41(18), 4344 (2016). [CrossRef]  

20. N. Hagen, S. Shibata, W. Takano, M. Matsuda, and Y. Otani, “Video-rate quantitative phase analysis by a DIC microscope using a polarization camera: errata,” Biomed. Opt. Express 10(6), 2967 (2019). [CrossRef]  

21. H. Kwon, E. Arbabi, S. M. Kamali, M. S. Faraji-Dana, and A. Faraon, “Single-shot quantitative phase gradient microscopy using a system of multifunctional metasurfaces,” Nat. Photonics 14(2), 109–114 (2020). [CrossRef]  

22. L.-H. Yeh, S. Chowdhury, and L. Waller, “Computational structured illumination for high-content fluorescence and phase microscopy,” Biomed. Opt. Express 10(4), 1978 (2019). [CrossRef]  

23. P. Bouchal, L. Štrbková, Z. Dostál, R. Chmelík, and Z. Bouchal, “Geometric-Phase Microscopy for Quantitative Phase Imaging of Isotropic, Birefringent and Space-Variant Polarization Samples,” Sci. Rep. 9(1), 3608 (2019). [CrossRef]  

24. P. M. S. Roma, L. Siman, F. T. Amaral, U. Agero, and O. N. Mesquita, “Total three-dimensional imaging of phase objects using defocusing microscopy: Application to red blood cells,” Appl. Phys. Lett. 104(25), 251107 (2014). [CrossRef]  

25. S. Chowdhury, W. J. Eldridge, A. Wax, and J. A. Izatt, “Structured illumination multimodal 3D-resolved quantitative phase and fluorescence sub-diffraction microscopy,” Biomed. Opt. Express 8(5), 2496 (2017). [CrossRef]  

26. A. Descloux, K. S. Grußmayer, E. Bostan, T. Lukes, A. Bouwens, A. Sharipov, S. Geissbuehler, A. L. Mahul-Mellier, H. A. Lashuel, M. Leutenegger, and T. Lasser, “Combined multi-plane phase retrieval and super-resolution optical fluctuation imaging for 4D cell microscopy,” Nat. Photonics 12(3), 165–172 (2018). [CrossRef]  

27. P. Ledwig and F. E. Robles, “Quantitative 3D refractive index tomography of opaque samples in epi-mode,” Optica 8(1), 6 (2021). [CrossRef]  

28. N. Hai, R. Kumar, and J. Rosen, “Single-shot TIE using polarization multiplexing (STIEP) for quantitative phase imaging,” Opt. Lasers Eng. 151, 106912 (2022). [CrossRef]  

29. Z. F. Phillips, M. Chen, and L. Waller, “Single-shot quantitative phase microscopy with color-multiplexed differential phase contrast (cDPC),” PLoS One 12(2), e0171228 (2017). [CrossRef]  

30. M. R. Banham and A. K. Katsaggelos, “Digital image restoration,” IEEE Signal Process. Mag. 14(2), 24–41 (1997). [CrossRef]  

31. M. Trusiak, M. Cywinska, V. Mico, J. A. Picazo-Bueno, C. Zuo, P. Zdankowski, and K. Patorski, “Variational Hilbert Quantitative Phase Imaging,” Sci. Rep. 10(1), 13955 (2020). [CrossRef]  

32. V. Sitzmann, S. Diamond, Y. Peng, X. Dun, S. Boyd, W. Heidrich, F. Heide, and G. Wetzstein, “End-to-end optimization of optics and image processing for achromatic extended depth of field and super-resolution imaging,” ACM Trans. Graph. 37(4), 1 (2018). [CrossRef]  

33. A. Sinha, J. Lee, S. Li, and G. Barbastathis, “Lensless computational imaging through deep learning,” Optica 4(9), 1117 (2017). [CrossRef]  

34. K. Moanakhova, J. Yurtsever, G. Kuo, N. Antipa, K. Yanny, and L. Waller, “Learned reconstructions for practical mask-based lensless imaging,” Opt. Express 27(20), 28075 (2019). [CrossRef]  

35. M. Lelek, M. T. Gyparaki, G. Beliu, F. Schueder, J. Griffié, S. Manley, R. Jungmann, M. Sauer, M. Lakadamyali, and C. Zimmer, “Single-molecule localization microscopy,” Nat. Rev. Methods Prim. 1(1), 39 (2021). [CrossRef]  

36. L. von Diezmann, Y. Shechtman, and W. E. Moerner, “Three-Dimensional Localization of Single Molecules for Super-Resolution Imaging and Single-Particle Tracking,” Chem. Rev. 117(11), 7244–7275 (2017). [CrossRef]  

37. L. Möckl, A. R. Roy, and W. E. Moerner, “Deep learning in single-molecule microscopy: fundamentals, caveats, and recent developments [Invited],” Biomed. Opt. Express 11(3), 1633 (2020). [CrossRef]  

38. A. Speiser, L. R. Müller, P. Hoess, U. Matti, C. J. Obara, W. R. Legant, A. Kreshuk, J. H. Macke, J. Ries, and S. C. Turaga, “Deep learning enables fast and dense single-molecule localization with high accuracy,” Nat. Methods 18(9), 1082–1090 (2021). [CrossRef]  

39. H. Kobayashi, K. C. Cheveralls, M. D. Leonetti, and L. A. Royer, “Self-supervised deep learning encodes high-resolution features of protein subcellular localization,” Nat. Methods 19(8), 995–1003 (2022). [CrossRef]  

40. B. Midtvedt, J. Pineda, F. Skärberg, E. Olsén, H. Bachimanchi, E. Wesén, E. K. Esbjörner, E. Selander, F. Höök, D. Midtvedt, and G. Volpe, “Single-shot self-supervised object detection in microscopy,” Nat. Commun. 13(1), 7492 (2022). [CrossRef]  

41. V. Ulman, M. Maška, K. E. G. Magnusson, et al., “An objective comparison of cell-tracking algorithms,” Nat. Methods 14(12), 1141–1152 (2017). [CrossRef]  

42. J. M. Newby, A. M. Schaefer, P. T. Lee, M. G. Forest, and S. K. Lai, “Convolutional neural networks automate detection for tracking of submicron-scale particles in 2D and 3D,” Proc. Natl. Acad. Sci. U. S. A. 115(36), 9026–9031 (2018). [CrossRef]  

43. E. Bengtsson, H. Danielsen, D. Treanor, M. N. Gurcan, C. MacAulay, and B. Molnár, “Computer-aided diagnostics in digital pathology,” Cytometry, Part A 91(6), 551–554 (2017). [CrossRef]  

44. K. Kartasalo, W. Bulten, B. Delahunt, P. H. C. Chen, H. Pinckaers, H. Olsson, X. Ji, N. Mulliqi, H. Samaratunga, T. Tsuzuki, J. Lindberg, M. Rantalainen, C. Wählby, G. Litjens, P. Ruusuvuori, L. Egevad, and M. Eklund, “Artificial Intelligence for Diagnosis and Gleason Grading of Prostate Cancer in Biopsies—Current Status and Next Steps,” Eur. Urol. Focus 7(4), 687–691 (2021). [CrossRef]  

45. P. Ström, K. Kartasalo, H. Olsson, et al., “Artificial intelligence for diagnosis and grading of prostate cancer in biopsies: a population-based, diagnostic study,” Lancet Oncol. 21(2), 222–232 (2020). [CrossRef]  

46. Y. Rivenson, H. Wang, Z. Wei, K. De Haan, Y. Zhang, Y. Wu, H. Günaydın, J. E. Zuckerman, T. Chong, A. E. Sisk, L. M. Westbrook, W. D. Wallace, and A. Ozcan, “Virtual histological staining of unlabelled tissue- autofluorescence images via deep learning,” Nat. Biomed. Eng. 3(6), 466–477 (2019). [CrossRef]  

47. Y. Zhang, K. de Haan, Y. Rivenson, J. Li, A. Delis, and A. Ozcan, “Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue,” Light: Sci. Appl. 9(1), 78 (2020). [CrossRef]  

48. S. V. Venkatakrishnan, C. A. Bouman, and B. Wohlberg, “Plug-and-Play priors for model based reconstruction,” 2013 IEEE Glob. Conf. Signal Inf. Process. Glob. 2013 - Proc.945–948 (2013).

49. R. Ahmad, C. A. Bouman, G. T. Buzzard, S. Chan, S. Liu, E. T. Reehorst, and P. Schniter, “Plug-and-Play Methods for Magnetic Resonance Imaging: Using Denoisers for Image Recovery,” IEEE Signal Process. Mag. 37(1), 105–116 (2020). [CrossRef]  

50. K. Zhang, Y. Li, W. Zuo, L. Zhang, L. Van Gool, and R. Timofte, “Plug-and-Play Image Restoration With Deep Denoiser Prior,” IEEE Trans. Pattern Anal. Mach. Intell. 44(10), 6360–6376 (2022). [CrossRef]  

51. M. R. Kellman, E. Bostan, N. A. Repina, and L. Waller, “Physics-Based Learned Design: Optimized Coded-Illumination for Quantitative Phase Imaging,” IEEE Trans. Comput. Imaging 5(3), 344–353 (2019). [CrossRef]  

52. M. Strassberg, Y. Shevtsova, D. Kamel, K. Wagoner-Oshima, H. Zhong, and M. Xu, “Single-shot quantitative phase imaging with polarization differential interference contrast,” Appl. Phys. Lett. 119(17), 173702 (2021). [CrossRef]  

53. R. G. Gavaskar, C. D. Athalye, and K. N. Chaudhury, “On Plug-and-Play Regularization Using Linear Denoisers,” IEEE Trans. on Image Process. 30, 4802–4813 (2021). [CrossRef]  

54. S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, “Distributed optimization and statistical learning via the alternating direction method of multipliers,” Found. Trends Mach. Learn. 3(1), 1–122 (2010). [CrossRef]  

55. D. Geman and C. Yang, “Nonlinear Image Recovery with Half-Quadratic Regularization,” IEEE Trans. on Image Process. 4(7), 932–946 (1995). [CrossRef]  

56. C. Wu and X. C. Tai, “Augmented Lagrangian Method, Dual Methods, and Split Bregman Iteration for ROF, Vectorial TV, and High Order Models,” SIAM J. Imaging Sci. 3(3), 300–339 (2010). [CrossRef]  

57. Y. Wang, W. Yin, and Y. Zhang, “Rice University, CAAM Technical Report TR07-10 A Fast Algorithm for Image Deblurring with Total Variation Regularization,” Image (Rochester, N.Y.)1–19 (2007).

58. J. L. Mead, “Chi-squared test for total variation regularization parameter selection,” Inverse Probl. Imaging 14(3), 401–421 (2020). [CrossRef]  

59. K. Zhang, W. Zuo, S. Gu, and L. Zhang, “Learning deep CNN denoiser prior for image restoration,” Proc. - 30th IEEE Conf. Comput. Vis. Pattern Recognition, CVPR 2017 2017-Janua,2808–2817 (2017).

60. K. Zhang, W. Zuo, and L. Zhang, “Deep plug-and-play super-resolution for arbitrary blur kernels,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 2019-June, 1671–1681 (2019).

61. W. Weng and X. Zhu, “INet: Convolutional Networks for Biomedical Image Segmentation,” IEEE Access 9, 16591–16603 (2021). [CrossRef]  

62. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 2016-Decem, 770–778 (2016).

63. Z. Zhu, H. Sun, and C. Zhang, “Effectiveness of Optimization Algorithms in Deep Image Classification,” arXiv, arXiv:2110.01598 (2021). [CrossRef]  

64. C. Ma, C. Y. Yang, X. Yang, and M. H. Yang, “Learning a no-reference quality metric for single-image super-resolution,” Comput. Vis. Image Underst. 158, 1–16 (2017). [CrossRef]  

65. S. B. Mehta and R. Oldenbourg, “Image simulation for biological microscopy: microlith,” Biomed. Opt. Express 5(6), 1822 (2014). [CrossRef]  

66. Z. Zha, B. Wen, X. Yuan, J. T. Zhou, J. Zhou, and C. Zhu, “Triply Complementary Priors for Image Restoration,” IEEE Trans. on Image Process. 30, 5819–5834 (2021). [CrossRef]  

67. B. Girod, “What’s Wrong with Mean-squared Error?” Digit. Images Hum. Vis.1991, 207–220 (1993).

68. M. Aleksandrovych, M. Strassberg, J. Melamed, and M. Xu, “Polarization differential interference contrast microscopy with physics-inspired plug-and-play denoiser for single-shot high-performance quantitative phase imaging: code: PDIC-DRUNet,” Github2023, https://github.com/Biomedical-Photonics-Lab.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request. The code for the TV+DRUNet algorithm will be publicly available at [68].

68. M. Aleksandrovych, M. Strassberg, J. Melamed, and M. Xu, “Polarization differential interference contrast microscopy with physics-inspired plug-and-play denoiser for single-shot high-performance quantitative phase imaging: code: PDIC-DRUNet,” Github2023, https://github.com/Biomedical-Photonics-Lab.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (11)
Fig. 1.
Fig. 1. A schematic diagram of an inverted PDIC microscope.

View in Article | Download Full Size | PDF

Fig. 2.
Fig. 2. DRUNet architecture. Each green box represents four consecutive Residual Blocks. Each blue box represents a Stride Convolutional layer. Each pink box represents a Transposed Convolutional layer.

View in Article | Download Full Size | PDF

Fig. 3.
Fig. 3. The training and validation losses of the pre-trained DRUNet.

View in Article | Download Full Size | PDF

Fig. 4.
Fig. 4. Phase retrieval of simulated Siemens star. (a) Ground truth, and the reconstructed phase using (b) the Fourier Transform solution, (c) the total variance regularized solution, and (d) the TV + DRUNet solution.

View in Article | Download Full Size | PDF

Fig. 5.
Fig. 5. (a, c, e) show the reconstructed phase images by the Fourier transform, total variance regularized, and TV + DRUNet methods. Examples of the corresponding phase profiles are shown in (b), (d), and (f) at the radius of 5 µm (blue), 3 µm (red), 2.3 µm (green), and 2 µm (magenta) from the center together with the ground truth (black). The phase profiles are shown for the angular range between 0 and π.

View in Article | Download Full Size | PDF

Fig. 6.
Fig. 6. (a, c, e) Histograms of per-pixel SSIM scores of the reconstructed phase images by the Fourier transform, total variance regularized, and TV + DRUNet methods. (b, d, e) The corresponding SSIM maps.

View in Article | Download Full Size | PDF

Fig. 7.
Fig. 7. The negative phase images of a 3.85µm silica sphere embedded in glycerol recovered by the Fourier transform solution in (a), (d), the TV regulation method in (b), (e), and the TV + DRUNet method in (c), (f). The horizontal and vertical profiles along the red and green dashed lines in (a) – (c) are shown in (d) – (f), respectively.

View in Article | Download Full Size | PDF

Fig. 8.
Fig. 8. Prostate cancer tissue (adenocarcinoma, stage III) measured with 40x objective. (a) H&E-stained histopathological image and reconstructed phase images using (b) Fourier transform, (c) total variance regularized, and (d) total variance plus DRUNet denoiser methods. Yellow arrows point to representative regions.

View in Article | Download Full Size | PDF

Fig. 9.
Fig. 9. Prostate cancer adjacent normal tissue measured with 60x objective. (a) H&E-stained histopathological image and reconstructed phase images using (b) Fourier transform, (c) total variance regularized, and (d) total variance plus DRUNet denoiser methods. Additionally, the region marked by the yellow rectangle in the H&E-stained histopathological image was enlarged in (e)-(h). Yellow arrows point to representative regions.

View in Article | Download Full Size | PDF

Fig. 10.
Fig. 10. The recovered phase of Siemens star using the proposed algorithm using a denoiser with (a) ReLU and (b) Tanh activation functions.

View in Article | Download Full Size | PDF

Fig. 11.
Fig. 11. Relative errors between the recovered phase and the ground truth for the Siemens star. Blue marks indicate total variance iterations, while red marks indicate denoising iterations.

View in Article | Download Full Size | PDF

Tables (4)

Tables Icon

Table 1. Performance of Fourier Transform (FT), Total Variance (TV) regularized, and Total Variance plus DRUNet denoiser (TV + DRUNet) Solutions for Siemens Star

View Table | View all tables in this article
Tables Icon

Table 2. Phase recovery perception score of Fourier transform (FT), Total Variance (TV) regularized, and Total Variance plus DRUNet denoiser (TV + DRUNet) Solutions

View Table | View all tables in this article
Tables Icon

Table 3. Phase recovery Multiscale Structural Similarity Index (MS-SSIM) of Fourier transform (FT), Total Variance (TV) regularized, and Total Variance plus DRUNet denoiser (TV + DRUNet) Solutions

View Table | View all tables in this article
Tables Icon

Table 4. Performance of the Recovered Siemens Star by TV + DRUNet with ReLU or Tanh Activation Function

View Table | View all tables in this article

Equations (13)

Equations on this page are rendered with MathJax. Learn more.

x ^ E s ′ − y ^ E p ′ = x ^ E 0 1 + exp ⁡ ( i Δ ) 2 − y ^ r E 0 1 − exp ⁡ ( i Δ ) 2 exp ⁡ ( i δ )
S ( ρ ) = ( S 0 ( ρ ) S 1 ( ρ ) S 2 ( ρ ) ) = ( I 0 ∘ ( ρ ) + I 90 ∘ ( ρ ) I 0 ∘ ( ρ ) − I 90 ∘ ( ρ ) I 45 ∘ ( ρ ) − I 135 ∘ ( ρ ) )
S 0 ′ = 2 r 2 I 0 S 1 ′ = 2 r 2 I 0 cos ⁡ Δ S 2 = − 2 r I 0 sin ⁡ δ sin ⁡ Δ
min u ⁡ ∫ d ρ ( μ 2 | ∂ x u − ϕ x | 2 + α | w | + γ Φ ( z ) + τ 2 | u − z | 2 ) subject to  ∂ y u − w = 0
L A = ∫ d ρ ( α | w | + β 2 | w − ∂ y u | 2 − λ ( w − ∂ y u ) + μ 2 | ∂ x u − ϕ x | 2 + γ Φ ( z ) + τ 2 | u − z | 2 )
w ( k + 1 ) = min w ⁡ ∫ d ρ ( α | w | + β 2 | w − ∂ y u ( k ) | 2 − λ ( k ) w ) u ( k + 1 ) = min u ⁡ ∫ d ρ ( β 2 | w ( k + 1 ) − ∂ y u | 2 + λ ( k ) ∂ y u + μ 2 | ∂ x u − ϕ x | 2 + τ 2 | u − z ( k ) | 2 ) z ( k + 1 ) = min z ⁡ ∫ d ρ ( Φ ( z ) + τ 2 γ | z − u ( k + 1 ) | 2 ) λ ( k + 1 ) = λ ( k ) − β ( w ( k + 1 ) − ∂ y u ( k + 1 ) ) .
w ( k + 1 ) = ( v ( k ) − α β v ( k ) | v ( k ) | ) H ( | v ( k ) | − α β ) u ~ ( k + 1 ) = − i μ k x ϕ ~ x − i β k y w ~ ( k + 1 ) + i k y λ ~ ( k ) + τ z ~ ( k ) μ k x 2 + β k y 2 + τ z ( k + 1 ) = min z ⁡ ∫ d ρ ( Φ ( z ) + τ 2 γ | z − u ( k + 1 ) | 2 ) λ ( k + 1 ) = λ ( k ) − β ( w ( k + 1 ) − ∂ y u ( k + 1 ) ) .
σ k = ξ 0  to  Var ( u )  geometrically  ( k = 0 , 1 , … , K − 1 ) γ = 0.23 τ = 1 / σ k 2
r ( k + 1 ) = ∂ y u ( k + 1 ) − w ( k + 1 )
s ( k + 1 ) = − β   ∂ y T ( w ( k + 1 ) − w ( k ) )
ε pri,( k ) = ε rel max { | ∂ y u ( k ) | , | w ( k ) | } ε dual,( k ) = ε rel | ∂ y T λ ( k ) | .
β ( k + 1 ) = { 2 β ( k ) 1 2 β ( k ) β ( k )   if  | r ( k ) | / ε pri,( k ) > η | s ( k ) | / ε dual,( k ) if  | r ( k ) | / ε pri,( k ) < η − 1 | s ( k ) | / ε dual,( k ) otherwise
u ( 0 ) = 0 ,   w ( 0 ) = 0 ,   z ( 0 ) = 0 ,   λ ( 0 ) = 0
Select as filters
Select Topics Cancel
Top
       
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.
Privacy | Terms of Use