Open Access
Add to BibSonomy
Abstract
Quantitative phase imaging (QPI) has emerged as a new digital histopathologic tool as it provides structural information of conventional slide without staining process. It is also capable of imaging biological tissue sections with sub-nanometer sensitivity and classifying them using light scattering properties. Here we extend its capability further by using optical scattering properties as imaging contrast in a wide-field QPI. In our first step towards validation, QPI images of 10 major organs of a wild-type mouse have been obtained followed by H&E-stained images of the corresponding tissue sections. Furthermore, we utilized deep learning model based on generative adversarial network (GAN) architecture for virtual staining of phase delay images to a H&E-equivalent brightfield (BF) image analogues. Using the structural similarity index, we demonstrate similarities between virtually stained and H&E histology images. Whereas the scattering-based maps look rather similar to QPI phase maps in the kidney, the brain images show significant improvement over QPI with clear demarcation of features across all regions. Since our technology provides not only structural information but also unique optical property maps, it could potentially become a fast and contrast-enriched histopathology technique.
© 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Histological tissue staining is commonly used to color and enhance contrast of tissues and thus to observe their microscopic features. Commonly, thin tissue slices of 2-10 µm thickness are treated with specific dyes that bind to specific targets of proteins or cellular structures, enabling differential visualization of microstructures under a brightfield microscope. The most commonly used dyes in histological tissue staining are hematoxylin and eosin (H&E), which produce a characteristic purplish blue and pink coloration, respectively [1]. In some cases, co-staining with multiple dyes is performed to collect the necessary range of information from a tissue section. For example, co-staining of H&E and Luxol fast blue (LFB) is used to identify cells and myelin, respectively, in mouse brain tissue [2]. Although considered the gold standard, the staining process can be time consuming, costly, and labor-intensive. For example, Masson’s Trichrome staining takes ∼2-3 hours, costs $\$$16-35, and often requires process monitoring by an expert [3].
Introduction of quantitative phase imaging (QPI) essentially bypasses the aforementioned issues by skipping the cumbersome staining process [4]. It allows for the quantitative analysis of the optical phase delays introduced by a sample, providing valuable information about the sample's morphological and structural properties. At any given point, phase delay is linearly proportional to refractive index difference and thickness of a sample as follows:
where ${k_0} = 2\pi /\lambda $ and $\lambda $ is the average wavelength, h is the thickness, and $\varDelta n$ is the difference between the refractive index of the sample and the media. Due to nanoscale sensitivity, absence of exogenous contrast agents and low phototoxicity, QPI techniques are being widely adopted in cell biology [5–7], neuroscience [8], and hematology [9–11]. They have also been utilized for examination of unstained thin tissue sections. Diffraction phase microscopy (DPM) [12], which is a common-path, off-axis interferometry-based QPI technique, has been recently used to image rat kidney, liver, brain [13], mouse acute kidney disease (AKI) [14], and mouse Alzheimer’s brain tissue [15]. In addition, spatial light interference microscopy (SLIM) [16], a temporal phase shifting-based technique, has been employed for imaging healthy mouse brain [17], rat liver [18], malignant tissues taken from prostate [19] and breast cancer [20] biopsies. In addition, results obtained by Soltaninezhad et al. [21] demonstrate significant statistical deviation of scattering properties, i.e., scattering coefficient and anisotropy, between normal and abnormal (cancerous) tissue sections. These scattering properties can also be obtained from QPI by means of scattering-phase theorem [22]. In many research papers, including the abovementioned ones [13–15,17–19], scattering coefficient and anisotropy have been used for tissue classification. These studies indicate a correlation between tissue architecture and scattering properties. However, the scope of use cases is limited to a few organs and thus translation to practice remains a challenge.Despite the advantages that QPI can offer to the field of digital histopathology, raw phase images might not be suitable for trained physicians who are accustomed to the gold standard of histological examination, i.e., brightfield imaging of labeled samples [3]. One potential way to overcome this issue would be utilizing deep learning to digitally stain phase images to histologically stained-like state. This approach brings several advantages. First, utilization of virtual staining removes the inconsistencies of image quality introduced by differences in staining protocols, technician skills, imaging hardware, etc. [23], so that the stain contrast on the generated images remains uniform from sample to sample [3,24]. The normalization of image conditions, in its turn, can potentially bring benefits for further tissue analysis using other models for segmentation, classification, etc. [24]. Second, deep learning models can be trained for various types of stains so that sample section areas can be selectively stained with different histological dyes providing broader scope of information for more accurate diagnosis [1,25]. This virtual staining approach can eventually reduce the amount of tissue required for a complete histopathological analysis, hence, optimizing biopsy acquisition process. Current advances in virtual staining approaches demonstrate highly accurate image inference not only from QPI [3], but also from other imaging modalities, e.g., tissue-autofluorescence [24], multi-photon microscopy [26], Fourier transform infrared imaging [27], and even conventional brightfield imaging [1,23].
In this study, we have examined tissue sections from 10 major mouse organs using DPM, analyzed their scattering coefficient and anisotropy distribution. Furthermore, we utilized deep learning model based on generative adversarial network (GAN) architecture for virtual staining of phase delay images to a H&E-equivalent state. To our knowledge, this is the first study to investigate major mouse organs in QPI and brightfield imaging, while simultaneously analyzing their scattering properties and providing a high-performance virtual staining model. As a result, for a single label-free tissue section, we can provide information with different contrasts and visibilities, i.e., QPI, scattering properties maps and virtually-stained brightfield imaging.
2. Materials and methods
2.1 Sample preparation
All animal procedures in this study were carried out in strict accordance with the approved guidelines of IACUC (UNISTIACUC-17-07) using 8 weeks old male mice of wild type. The mouse organs used in the experiments were brain, kidney, pancreas, liver, stomach, lung, heart, spleen, small intestine, and bone. Tissue sections of 5 µm thickness for the abovementioned organs were prepared according to traditional histological procedure without the staining step. The paraffin was completely removed by washing with xylene and graded ethanol before covering glass mounting for imaging with DPM system. After DPM image analysis, the cover glasses were removed by immersing the samples in xylene for 48 hours and the sections were stained with H&E dyes followed by secondary cover glass mounting.
2.2 Imaging
The schematic of wide-field DPM and detailed beam path are illustrated in Fig. 1. For the illumination source, a CW visible fiber laser having center wavelength of 560 nm with bandwidth of 6-14 nm (mpbc.ca, Canada) was employed. The scattered light from a tissue was collected by objective lens (OL, Olympus, LUMPLFLN 40XW, NA 0.8). The long-range linear stage (X-Y ST, Marzhauser, Germany) was equipped for scanning large area. To avoid the defocusing error, we precisely calibrated the tilt of the linear stage by adjusting for 4 points on the left, right, top and bottom of the edge of the sample surface and repeating until all points were well focused. A pair of achromatic lenses (L1 and L2) was placed after objective and tube lenses (Thorlabs, ITL200). The light was split into multiple beams by diffraction grating (DG, Edmond, 92 grooves/mm) that was inserted at the back focal plane of L2. The two diffraction orders (0th and 1st) were transmitted through the lens (L3) and spatially filtered through the pinhole at the back focal plane of L3. 0th order was used as the reference beam after passing through the 50 µm pinhole, while the 1st order passed without any field interruption which makes it the sample beam. Two light beams were focused by achromatic lens (L4), and their resulting interference pattern was detected by EMCCD (Andor, iXon3). The measured pattern, that includes amplitude and phase information, was used to retrieve phase difference between specimen and its surrounding medium. The transverse resolution was 1.3 µm and field-of-view was 150 × 150 µm (512 × 512 pixels). The phase temporal and spatial stability of the system were measured to be 60 mrad and 50 mrad, respectively. A set of DPM images was acquired and digitally stitched together for wide-field imaging. The acquisition time for a single mosaic image with 35% overlap was 0.2 seconds, which resulted in a 5 frame/sec acquisition rate. Acquisition time for whole-slide imaging varied depending on the tissue slice dimensions. After DPM image analysis, all samples were stained with H&E dyes and imaged with virtual microscope slide scanning system (OLYMPUS) for side-by-side comparison.
Fig. 1. Schematic of wide-field diffraction phase microscopy (DPM). L, lens; OL, objective lens; TL, tube lens; G, grating; P, pinhole for spatial filtering; S, sample; X-Y ST, xy linear stages; IP, image plane; FP, Fourier plane.
2.3 Scattering coefficient and anisotropy factor calculation
The scattering coefficient and anisotropy factor maps were derived from raw phase delay maps based on scattering-phase theorem [22] as follows:
2.4 Deep learning model
Compared to brightfield images from H&E-stained samples, raw stain-free DPM images look relatively blurry and contain less intracellular structure information such as nuclei locations. To achieve better detailed imaging qualities in DPM comparable to H&E-stained image, meticulous dataset preparation and appropriate deep neural network architecture with adequate loss design are crucial. We employed the generative adversarial network (GAN) [28] to implement virtual staining that transforms a DPM image to a corresponding H&E-stained image. To maximize the advantage of adversarial learning using GAN, we designed a model architecture and loss in the following manner: a generator, G, is composed by combination of residual blocks [29] and U-net structure [30] with separated dataflow as shown in Fig. 2. The reason for such model architecture is to achieve learning ambiguous features from DPM images to match details and reconstruct equivalent brightfield textures through residual blocks while learning appropriate coloring from features across different dimensions through the U-net path. For the discriminator D, we employed the PatchGAN discriminator [31] which is apposite to support learning of generator about low and high frequency details together efficiently with its architecture design for prediction real/fake along the patches. The loss function for the training consists of the adversarial loss with two regularization terms; the mean absolute error (MAE, L1 loss) and the mean squared error (MSE, L2 loss) for less blurry effect and effective colorization. Specifically, we formulated the loss function as follows: the generative loss for the generator is:
and the discriminative loss for the discriminator is: where x is the DPM input image, G(x) is the generative model output, D(·) is the discriminative model prediction of an image (ground truth BF or virtually stained), and y is the ground truth brightfield image. The regularization constants $\lambda $ and $\mu $ were set to be 1–10% of the generative loss $L({G;D} )$ during the training.Fig. 2. Architecture of the deep learning model for virtual staining. Generator consists of parallel paths through residual blocks and U-net structure (upper). Discriminator is composed of layers of convolution, batch normalization, leaky ReLU and discrimination patches (lower).
The virtual staining deep-learning model was trained using acquired DPM and brightfield sample images. The damaged tissue areas during staining were excluded to obtain clear and precise dataset. At first, we attempted to use the scale-invariant feature transform (SIFT) algorithm to register image pairs, but it was unsuccessful because of the changes in the structure caused by the staining process. As a result, we manually registered specific parts of the samples using a Matlab script to address magnification, rotation, and translation issues. Appropriate conditions were chosen to maximize SSIM values between the image pairs. As a result, the raw data consists of dozens to hundreds of pairs between brightfield and corresponding DPM images for each organ type. The datasets for training and validation were augmented from the raw data by applying random cropping, mirroring, and rotation to be at least 50,000 and 10,000 patches of size 256 × 256 (see Table 1), respectively. The number of epochs was set to 100. For an efficient learning schedule, we split training steps into pretraining and adversarial training. In the pretraining step, we trained the generator based on MAE (L1 loss) only for a single round of the training set. After that adversarial training was initiated. For the optimization, we employed the Adam optimizer [32] with learning rate of 10-4. After 105 steps of training, the learning rate decreased to 10-5. All trainable parameters in the architectures were initialized with a normal distribution with zero mean and standard deviation of 0.02. The batch size was fixed to 12 for all training steps. The PatchGAN discriminator output matrix size was set to 32 × 32. Virtual staining deep neural network for each organ type was trained separately.
Table 1. Dataset details for the virtual staining at different tissue types. For each tissue type, 50,000 and 10,000 augmented patches were used as training and validation datasets, respectively
To apply the trained networks for whole-slide imaging of the tissue sections, each whole-slide DPM image was divided into patches of 256 × 256 pixels with 50% overlap of neighboring patches. DPM patches are virtually stained by a tissue-specific network and stitched into a whole-slide virtually stained image. For seamless stitching, overlapping regions were linearly blended along neighboring patches.
3. Results
We obtained DPM and H&E-stained images of mouse tissue samples extracted from 10 major organs as shown in Fig. 3. In histology, internal structures of cells were stained by H&E dyes, and in DPM, all biological features are characterized by different phase delay values. For the DPM case, the image contrast was adjusted to visualize both macrostructure and microstructure in tissue samples. In case of wide-field images, the range was set as -2σ to 2σ (σ: standard deviation), but in magnified subfigures, range was set as -0.9 to 1 rad to show more details. When the phase delay and H&E-stained images are compared side-by-side, significant structural correlation of morphological features are observed. Particularly in subfigures of Fig. 3 the specific cell shape of each organ shown in the H&E image is also noticeable in the DPM; for example, granule cells in hippocampus (a), cardiac muscle cells and fibers (b), alveoli (c), hepatocytes (d), epithelial cells in mucosa (e), renal tubules in cortex (f), acinar cells (g), lymphoid follicle (h), Paneth cells (i), and osteoblasts (j). The reason for this similar contrast between the DPM and H&E-stained images is that the respective regions have different functional cell groups, and their structures change accordingly, which causes distinct attachment rates of H&E dyes in histology and different phase delays in QPI.
Fig. 3. Comparison between H&E-stained digital image and quantitative phase imaging of various organ tissue. The small regions of tissue indicated by red box are enlarged and shown next to the wide-field images. Scale bar: 1 mm (50 µm for inset images).
µs and g were calculated for each organ using scattering-phase theorem as shown in Fig. 4. The window size for scattering properties calculation was 20 × 20 µm2. In Fig. 4(a), the results show the tissues have a 26 rad2/mm and 0.965 for average values of µs and g, respectively. However, the bone, pancreas, liver, and brain showed different trends. Analyzing the results with regards to the morphological properties of the tissue, the pancreas has relatively high µs and g values because they contain tightly packed acinar cells. The brain and liver have the lowest average values of µs and g since both consist of locally homogeneous tissue components. The bone tissue on the other hand is composed of dense and heterogeneous osteoblasts, resulting in high µs values. To investigate the potential of using scattering property values for histological contrast, we visualized scattering coefficient and anisotropy maps for the cases of brain and kidney (Fig. 4(b1,2) and 4(c1,2)), which are the organs with the most distinct structural and functional differences. Whereas the contrast is rather similar to QPI modality in the kidney images, the brain image shows significant improvement over QPI with clear demarcation of features across all regions. We also plotted the distribution of scattering coefficient side-by-side with phase values (see Fig. 4(b3, c3)) for three different tissue regions. The graphs show that while the mean phase values are relatively equivalent across the regions, scattering coefficients are distinguishably more variant.
Fig. 4. (a) The plot of scattering properties (scattering coefficient and anisotropy factor) for various organs. Statistical independence of organ types to the measurement were confirmed by the analysis of variance, ANOVA (p < 10-5). (b1, b2) scattering coefficient and anisotropy maps of the brain section, respectively. (b3) statistical distribution of phase values (left) and scattering coefficient (right). (c1, c2) scattering coefficient and anisotropy maps of the kidney section, respectively. (c3) statistical distribution of phase values (red) and scattering coefficient (blue). Scale bar: 1 mm.
Figure 5 demonstrates the side-by-side comparison of virtual staining results with the H&E-stained images. The trained network takes a raw QPI image with the range of -0.8 to 2 rad as an input and converts it to a virtually stained H&E image. Visibly, the results are well matched to the histologically stained images with all the major features observed in histology images being reflected in the virtually stained images. To further analyze the structural similarities between virtually stained and H&E histology images in a quantitative manner, spatial windows of all organs were numerically compared using the structural similarity (SSIM) index method [33]. The SSIM index was calculated using SSIM function in MATLAB and ranging from 0 to 1 and is defined as:
Fig. 5. SSIM index maps between Virtual Stained and H&E Histology images of brain, heart, kidney, liver, pancreas, intestine and lung. For areas with strong correlation, the SSIM index is close to 1, and vice versa, for areas with low correlation, it approaches 0.
Table 2. Mean and variance values of SSIM index for 500 random validation dataset patches
4. Discussion and conclusion
This paper presents wide-field DPM system as a versatile imaging tool that offers not only morphological information of unstained tissue samples with phase-based virtual staining, but also scattering properties, such as µs and g, as additional contrast to characterize pathological tissues. To evaluate the feasibility of DPM for histological studies, we generated high-resolution phase delay maps of various mouse organ sections and compared the results to H&E-stained images. The experiment demonstrates that the contrast in phase delay images alone is sufficient to visualize morphological features as detailed as the H&E-stained images in each organ. Thus, we hypothesized that scattering map could be utilized as an additional contrast in histopathology as it may be able to delineate disease-related structural alterations in tissue. For example, diagnosis of fibrosis which usually requires co-staining and precise sample preparation, can tremendously benefit from our proposed approach of using optical scattering properties. In our previous work, we found that the progression of acute kidney injury was significantly correlated with scattering coefficient and anisotropy, which was not as evident on the raw phase delay maps alone [14]. To our knowledge, this is the first study to investigate control groups of mouse organs in terms of whole-slide QPI and optical scattering properties. Further research in diseased tissues is necessary to extend our knowledge in this direction. In addition, we implemented a deep learning module to digitally stain the raw phase delay maps to compare the H&E-stained images. The results are well matched to the ground truth with the average similarity index of 0.93. Our virtual staining model successfully replicated all the major features of the respective organ tissue sections. However, there remains experimental challenge to fully utilize the effect of virtual staining on label-free images. The major source of error in virtual staining is the unconformity between QPI and H&E-stained images due to structural alternation of tissues caused by the staining process itself. Since the QPI & BF images are sequentially acquired from separate imaging systems, corresponding images do not often overlap with each other, thus it requires tedious image pair registration process for the model training. We noticed that this issue is more severe in intestine, pancreas, and lung due to the vast abundance of spatially separated microstructures, which led to mismatches observed on the SSIM maps. To overcome the co-registration issue, image set for training must be obtained by single body optical imaging system which equips both functionality of QPI and BF. In addition, the fast and robust staining process is essential. We believe that further research into this issue is fully justified as it will reduce the complexity of image registration process, and thus improve the virtual staining accuracy and optimize the overall process workflow.
In summary, wide-field quantitative phase imaging with derived µs and g maps, when complemented with deep-learning-enabled virtual staining, can be a very promising tool for practical using in histopathology research. Our future studies will focus on their diagnostic capabilities by delineating altered cellular structures of diseased tissues.
Funding
National Research Foundation of Korea (2020R1A2B5B02001987); Korea Medical Device Development Fund (1711174449, RS-2020-KD000024); Korea Health Industry Development Institute (HP20C0032030020).
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. K. de Haan, Y. Zhang, J. E. Zuckerman, T. Liu, A. E. Sisk, M. F. Diaz, K.-Y. Jen, A. Nobori, S. Liou, S. Zhang, R. Riahi, Y. Rivenson, W. D. Wallace, and A. Ozcan, “Deep learning-based transformation of H&E stained tissues into special stains,” Nat. Commun. 12(1), 4884 (2021). [CrossRef]
2. J. Weickenmeier, R. de Rooij, S. Budday, T. C. Ovaert, and E. Kuhl, “The mechanical importance of myelination in the central nervous system,” J. Mech. Behav. Biomed. Mater. 76, 119–124 (2017). [CrossRef]
3. Y. Rivenson, T. Liu, Z. Wei, Y. Zhang, K. de Haan, and A. Ozcan, “PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning,” Light: Sci. Appl. 8(1), 23 (2019). [CrossRef]
4. Y. Park, C. Depeursinge, and G. Popescu, “Quantitative phase imaging in biomedicine,” Nat. Photonics 12(10), 578–589 (2018). [CrossRef]
5. M. Mir, Z. Wang, Z. Shen, M. Bednarz, R. Bashir, I. Golding, S. G. Prasanth, and G. Popescu, “Optical measurement of cycle-dependent cell growth,” Proc. Natl. Acad. Sci. U. S. A. 108(32), 13124–13129 (2011). [CrossRef]
6. F. Charrière, A. Marian, F. Montfort, J. Kuehn, T. Colomb, E. Cuche, P. Marquet, and C. Depeursinge, “Cell refractive index tomography by digital holographic microscopy,” Opt. Lett. 31(2), 178–180 (2006). [CrossRef]
7. F. Merola, P. Memmolo, L. Miccio, R. Savoia, M. Mugnano, A. Fontana, G. D’ippolito, A. Sardo, A. Iolascon, A. Gambale, and P. Ferraro, “Tomographic flow cytometry by digital holography,” Light: Sci. Appl. 6(4), e16241 (2016). [CrossRef]
8. M. Mir, T. Kim, A. Majumder, M. Xiang, R. Wang, S. C. Liu, M. U. Gillette, S. Stice, and G. Popescu, “Label-free characterization of emerging human neuronal networks,” Sci. Rep. 4(1), 4434 (2014). [CrossRef]
9. Y. Park, M. Diez-Silva, G. Popescu, G. Lykotrafitis, W. Choi, M. S. Feld, and S. Suresh, “Refractive index maps and membrane dynamics of human red blood cells parasitized by Plasmodium falciparum,” Proc. Natl. Acad. Sci. U. S. A. 105(37), 13730–13735 (2008). [CrossRef]
10. G. Popescu, K. Badizadegan, R. R. Dasari, and M. S. Feld, “Observation of dynamic subdomains in red blood cells,” J. Biomed. Opt. 11(4), 040503 (2006). [CrossRef]
11. G. Popescu, T. Ikeda, C. A. Best, K. Badizadegan, R. R. Dasari, and M. S. Feld, “Erythrocyte structure and dynamics quantified by Hilbert phase microscopy,” J. Biomed. Opt. 10(6), 060503 (2005). [CrossRef]
12. G. Popescu, T. Ikeda, R. R. Dasari, and M. S. Feld, “Diffraction phase microscopy for quantifying cell structure and dynamics,” Opt. Lett. 31(6), 775–777 (2006). [CrossRef]
13. H. Ding, F. Nguyen, S. A. Boppart, and G. Popescu, “Optical properties of tissues quantified by Fourier-transform light scattering,” Opt. Lett. 34(9), 1372–1374 (2009). [CrossRef]
14. S. Ban, E. Min, S. Baek, H. M. Kwon, G. Popescu, and W. Jung, “Optical properties of acute kidney injury measured by quantitative phase imaging,” Biomed. Opt. Express 9(3), 921–932 (2018). [CrossRef]
15. M. Lee, E. Lee, J. Jung, H. Yu, K. Kim, J. Yoon, S. Lee, Y. Jeong, and Y. Park, “Label-free optical quantification of structural alterations in Alzheimer’s disease,” Sci. Rep. 6(1), 31034 (2016). [CrossRef]
16. Z. Wang, L. Millet, M. Mir, H. Ding, S. Unarunotai, J. Rogers, M. U. Gillette, and G. Popescu, “Spatial light interference microscopy (SLIM),” Opt. Express 19(2), 1016–1026 (2011). [CrossRef]
17. E. Min, M. E. Kandel, C. J. Ko, G. Popescu, W. Jung, and C. Best-Popescu, “Label-free, multi-scale imaging of ex-vivo mouse brain using spatial light interference microscopy,” Sci. Rep. 6(1), 39667 (2016). [CrossRef]
18. H. Ding, Z. Wang, X. Liang, S. A. Boppart, K. Tangella, and G. Popescu, “Measuring the scattering parameters of tissues from quantitative phase imaging of thin slices,” Opt. Lett. 36(12), 2281–2283 (2011). [CrossRef]
19. S. Sridharan, V. Macias, K. Tangella, A. Kajdacsy-Balla, and G. Popescu, “Prediction of prostate cancer recurrence using quantitative phase imaging,” Sci. Rep. 5(1), 9976 (2015). [CrossRef]
20. H. Majeed, T. H. Nguyen, M. E. Kandel, A. Kajdacsy-Balla, and G. Popescu, “Label-free quantitative evaluation of breast tissue using Spatial Light Interference Microscopy (SLIM),” Sci. Rep. 8(1), 6875 (2018). [CrossRef]
21. M. Soltaninezhad, A. Bavali, Z. Nazifinia, and V. Soleimani, “Optical anisotropy measurement in normal and cancerous tissues: backscattering technique,” Biomed. Opt. Express 11(6), 2996–3008 (2020). [CrossRef]
22. Z. Wang, H. Ding, and G. Popescu, “Scattering-phase theorem,” Opt. Lett. 36(7), 1215–1217 (2011). [CrossRef]
23. D. Li, H. Hui, Y. Zhang, W. Tong, F. Tian, X. Yang, J. Liu, Y. Chen, and J. Tian, “Deep learning for virtual histological staining of bright-field microscopic images of unlabeled carotid artery tissue,” Molecular imaging and biology 22(5), 1301–1309 (2020). [CrossRef]
24. Y. Rivenson, H. Wang, Z. Wei, K. de Haan, Y. Zhang, Y. Wu, H. Günaydın, J. E. Zuckerman, T. Chong, A. E. Sisk, L. M. Westbrook, W. D. Wallace, and A. Ozcan, “Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning,” Nat. Biomed. Eng. 3(6), 466–477 (2019). [CrossRef]
25. Y. Zhang, K. de Haan, Y. Rivenson, J. Li, A. Delis, and A. Ozcan, “Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue,” Light: Sci. Appl. 9(1), 78 (2020). [CrossRef]
26. N. Borhani, A. J. Bower, S. A. Boppart, and D. Psaltis, “Digital staining through the application of deep neural networks to multi-modal multi-photon microscopy,” Biomed. Opt. Express 10(3), 1339–1350 (2019). [CrossRef]
27. M. Lotfollahi, S. Berisha, D. Daeinejad, and D. Mayerich, “Digital staining of high-definition Fourier transform infrared (FT-IR) images using deep learning,” Appl. Spectrosc. 73(5), 556–564 (2019). [CrossRef]
28. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial networks,” Commun. ACM 63(11), 139–144 (2020). [CrossRef]
29. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778 (2016).
30. O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, 234–241 (2015).
31. U. Demir and G. Unal, “Patch-based image inpainting with generative adversarial networks,” arXiv, arXiv:1803.07422 (2018). [CrossRef]
32. D. P. Kingma and J. Ba, “Adam: a method for stochastic optimization,” arXiv, arXiv preprint arXiv:1412.6980 (2014).
33. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. on Image Process. 13(4), 600–612 (2004). [CrossRef]
