Open Access
Add to BibSonomy
Abstract
Acoustic resolution photoacoustic microscopy (AR-PAM) is a major modality of photoacoustic imaging. It can non-invasively provide high-resolution morphological and functional information about biological tissues. However, the image quality of AR-PAM degrades rapidly when the targets move far away from the focus. Although some works have been conducted to extend the high-resolution imaging depth of AR-PAM, most of them have a small focal point requirement, which is generally not satisfied in a regular AR-PAM system. Therefore, we propose a two-stage deep learning (DL) reconstruction strategy for AR-PAM to recover high-resolution photoacoustic images at different out-of-focus depths adaptively. The residual U-Net with attention gate was developed to implement the image reconstruction. We carried out phantom and in vivo experiments to optimize the proposed DL network and verify the performance of the proposed reconstruction method. Experimental results demonstrated that our approach extends the depth-of-focus of AR-PAM from 1mm to 3mm under the 4 mJ/cm2 light energy used in the imaging system. In addition, the imaging resolution of the region 2 mm far away from the focus can be improved, similar to the in-focus area. The proposed method effectively improves the imaging ability of AR-PAM and thus could be used in various biomedical studies needing deeper depth.
© 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Photoacoustic imaging (PAI) is a hybrid imaging modality that combines the advantage of high contrast of optical imaging and high penetration depth of ultrasound imaging [1–4]. PAI has been found to have great potential in preclinical and clinical studies, such as whole-body imaging of small animals, breast cancer diagnosis, and detection of cardiovascular disease [5–12]. The acoustic-resolution photoacoustic microscopy (AR-PAM) is a major implementation of PAI, which has wide applications in molecule imaging, small-animal imaging, and brain function studies [13–15]. While the image generated is high resolution at the focal plane, the imaging resolution of AR-PAM decreases rapidly for the targets away from the focal plane of the transducer, limiting its applications in biomedical studies.
Many studies have been conducted in recent years to improve the out-of-focus resolution of AR-PAM. Initially, Wang et al. developed a virtual detector technique to enhance the quality of PAM images in the scattered focus region [16–18]. However, this technique results in limited resolution improvement since it assumes ‘point-like’ detection, which is difficult to achieve in AR-PAM with a relatively long focal zone. A coherence weighting factor (C.F.) was introduced to address the ‘point-like’ detection problem and improve the reconstructed image quality. Still, photoacoustic signals are lost during reconstruction, leading to signal distortions [19]. Meng et al. proposed a compressed sensing-based method to improve the resolution of the out-of-focus region in AR-PAM [20]. This method improved the signal-to-noise ratio and resolution of reconstructed images with sparse sampling. Methods based on the virtual-detector technique were also developed for improving the reconstruction quality of out-of-focus regions in AR-PAM [21–23].
All existing methods depend upon a small focal point for their performance improvement, which is not the case in AR-PAM, where the focal point is long. Thus, achieving high-resolution imaging at the out-of-focus plane in the AR-PAM system is limited using the current methods. Here, we propose a new method based on deep learning technology to improve the out-of-focus planes imaging.
Deep learning (DL) is a popular method in the signal processing field and now has been used in photoacoustic imaging as well [24–27], e.g., sparse-sampling reconstruction exhibited dramatic performance using DL [28–31]. Recently, DL was also applied for the out-of-focus image reconstruction in AR-PAM [32]. Here, a traditional U-Net network was employed to show the feasibility and effectiveness of DL in improving the imaging depth of AR-PAM with high resolution. However, they only used the simulated data from k-wave to train the U-Net model. As a result, the reconstruction quality of photoacoustic images in the out-of-focus region was sub-optimal and suffered from discontinuity of signals and loss of many weak signals. In addition, the evaluation was only limited to the surface of the biological tissues; thus, the large-depth high-resolution imaging of AR-PAM was not explored. Moreover, the ground truth was not listed in this work, affecting the evaluation to the accuracy of the reconstructed results.
Therefore, we propose a two-stage reconstruction framework using a novel DL network to recover signals at different out-of-focus depths in AR-PAM. The first-stage DL network is used to reconstruct the region far away from the focus, and the second stage reconstructs the region near the focus. We specifically designed a residual U-Net structure with attention gate (A.G.) (ResUnet_AG) to implement the image reconstruction. We also designed phantom and in vivo experiments to acquire the training data to optimize the ResUnet_AG for in vivo applications. We conducted phantom and in vivo imaging experiments on mice to show the effectiveness of the method in improving the reconstruction of the out-of-focus images in AR-PAM at large depth. We believe with such enhancement, the use of AR-PAM can be extended to fulfill more biomedical studies. In the following few sections, we present our experimental method setup, followed by results and a conclusion.
2. Method
2.1 Two-stage reconstruction
High resolution imaging can be achieved in the focal zone of the focused transducer equipped in AR-PAM. However, the out-of-focus region becomes progressively blurred with the increasing distance away from the focal site. Thus, the image quality at different imaging depths is different. In previous work [32], the out-of-focus images at different depths were reconstructed using a single DL network, ignoring the difference of image quality at different depths, making it difficult to converge and thus limiting the image quality. Here we developed a two-stage reconstruction strategy for AR-PAM, as illustrated in Fig. 1, to recover high-resolution photoacoustic images at different depths. In the first stage, images far away from the focal zone are recovered, achieving the resolution of images near the focal zone. For this stage, images at far away and near focal zones are combined as data pairs to train the first-stage DL network. In the second stage, the images from at-focus are used as ground truth to recover the images near the focal zone. Thus, the images near the focal zone and at-focus zones are combined as data pairs to train the second-stage network. In this figure, the near focus images in the training stage come from the initially acquired data by AR-PAM, and in the reconstruction stage, they are the reconstructed results of the first stage network. We developed a residual U-Net network structure with attention gate, named ResUnet_AG, to obtain the reconstruction of photoacoustic images at each stage, presented in the next section. As a result, our method can obtain improved high-resolution images at larger depth of AR-PAM compared to previous method.
2.2 Residual U-Net with attention gate
The U-Net has been verified to have good performance in data classification and signal prediction [33,34]. Thus, using U-Net as the backbone, we developed a residual U-Net with an attention gate (ResUnet_AG) network to reconstruct out-focus imaging planes, as shown in Fig. 2.
The proposed method has three major components of U-Net, i.e., contraction path, expansion path, and concatenation path. In theory, the processing results are better when the level of the network is deeper. However, with deeper network, the parameters to be trained increase dramatically, resulting in the problems of gradient disappearance and degradation. Thus, in the proposed network structure, residual blocks are inserted after the convolution blocks in the contraction path to address the above problems [35]. In addition, for the out-of-focus reconstruction of photoacoustic images, it is necessary for the network to ignore background and distribute more weight to learn signals. Thus, to improve the performance of our proposed network, the attention gates (A.G.) are integrated into the concatenation path to automatically suppress unrelated backgrounds and distribute more weightage to useful salient features [36]. The detailed structure of Res-block and A.G. are listed in Fig. 2. In the A.G. block, g is the feature maps from the decoding part (expansion path), and x represents the feature maps from the encoding part (contraction path). Using the information from x and g, the attention matrix α is calculated after a series of operations shown in the figure. The feature map x will be weighted by this matrix to implement the attention to valuable signals. In the ResUnet_AG, batch normalization (BN) is inserted before the nonlinear activation function in conv. and A.G. blocks. It can pull the distribution of the input data back to the standard normal distribution, and guarantee the input data of the nonlinear function falls into a reasonable range to avoid the gradient disappearance in the training process of the DL network.
3. Imaging system and experiment
Photoacoustic imaging of phantom and in vivo mice were performed using our customized acoustic-resolution photoacoustic microscopy (AR-PAM) system. Major components of the system include: (1) a tunable pulsed OPO laser (SpitLight EVO S OPO-100, InnoLas, Munich, Germany) to excite PA signals; (2) a focused ultrasound transducer (V324-SU, Olympus IMS, Waltham, USA; central frequency: 25 MHz; fractional bandwidth: 14MHz; N.A.: 0.25) to detect PA signals; (3) a precision movable scanner (PSA2000-11, Zolix, Beijing, China) to move the imaging head for data acquisition of 3D imaging; (4) a two-channel data acquisition (DAQ) card (CS1422, Gage Applied Technologies Inc., Lockport, USA) to digitize the PA signals via a 200-MS/s sampling rate. The depth of focus (DOF) of our AR-PAM is about 1 mm. The DOF is calculated by $DOF = 4\lambda {(F/D)^2}$, where $F = 12.7mm$ is the focal length, $D = 6.35mm$ is the dimension of the ultrasound transducer, and $\lambda = 61.6\mu m$ is the acoustic wavelength at central frequency. Further details of the imaging system can be found in our earlier publication [20].
To validate the performance of the proposed method for the out-of-focus imaging and to acquire training data pairs for ResUnet_AG, we fabricated three phantoms. The phantoms were made to mimic the biological tissues by mixing the agarose gel and water thoroughly at 1:100 mass ratio. Tungsten wires and carbon fibers were embedded within the mixture to mimic the vessel structures. Phantom 1 was designed by placing 90 μm diameter tungsten wires in agarose gel at varying depths, and phantom 2 was designed by placing 30 µm, 60 μm, and 90 µm tungsten wires with three different diameters. Both phantoms are divided into three layers with each layer having 1 mm thickness. The tungsten wires at varying depths are arranged in different directions. For simulating a more realistic vascular structure, various diameter tungsten wires and carbon fibers were placed irregularly and randomly in three layers to form the phantom 3. The photoacoustic imaging process of the phantoms was as follows: the ultrasonic transducer probe was first focused on the top layer (about 0.5mm deep below the surface) of the phantom and scanned; then, the scanner was controlled to shift the probe focus down by 1 mm and 2 mm and scanned again; finally, three groups of data were collected for each phantom.
The above data acquisition process is illustrated in Fig. 3. Figures 3(a) – 3(c) each represents the three cases that the focal plane is located at 0.5 mm, 1.5 mm and 2.5 mm deep below the surface. In every case, we obtained one photoacoustic image at focus and two images at out-of-focus (i.e. 2mm-far away focus and 1mm-near focus or two 1mm-near focus, as shown in the figure). Hence, training data pairs were obtained to optimize the proposed two-stage DL network. Specifically, the 2mm-far away focus and 1mm-near focus data pairs were used to optimize the 1st-stage network, and the 1mm-near focus and at-focus data pairs were used to train the 2nd-stage network.
Fig. 3. Illustration on data acquisition and constructing of data pairs. (a) Focus at the first layer. (b) Focus at the second layer. (c) Focus at the third layer.
Before the training data (B-scans) are fed into the network, they are first processed by Hessian matrix operation to enhance the weak vasculature, and then normalized to the range of [0, 1] and axially cut into three 1mm-thick patches, as indicated in Fig. 3. In the reconstruction process, B-scans are also axially cropped into three 1mm-thick regions (each containing 66${\times} $240 pixels) and then fed into the optimized network to reconstruct the high-resolution images. In our work, the data size of network input and output is 66${\times} $240 pixels.
In all experiments, the mean absolute error is used as the loss function to train the DL network and Adam is adopted as the optimization algorithm. The learning rate is set to 1e-4, and the epoch is set to 300. All programs are written in Python with Keras, and when the patch size fed into the network is 66${\times} $240 pixels, it takes about 60 seconds for one epoch. The network model was performed on a PC equipped with Intel Core i9-10900 K central processing unit (3.50 GHz CPU and 16 GB memory) and a NVIDIA GeForce GTX 1080 card (8GB memory).
For in vivo imaging, several male BALB/c nude mice (4-6 weeks old, weighing 18-20 g) were purchased from Beijing Vital River Laboratory Animal Technology Co., Ltd. (Beijing, China). They were used for photoacoustic imaging of abdominal, cerebral blood vessels and tumor planted on the back. The mice were initially anesthetized with 2% isoflurane in oxygen at a 150 ml/min flow rate and then positioned on an imaging bracket with a heating pad to maintain body temperature. The oxygen mask was placed on the mouth to ensure breathing. The in vivo data collection process was similar to the phantoms. The ultrasonic transducer was initially focused on the skin surface and scanned; then, the imaging probe was moved down by 1 mm and 2 mm and scanned again. So, we acquired the at-focus and different out-of-focus photoacoustic images for each 1mm-thick imaging block. These data were used to optimize and verify the proposed DL network. During experiments, 780 nm wavelength was used to illuminate the biological tissues, and the optical energy per pulse remained at approximately 4mJ/cm2, well below the ANSI safety limit (ANSIZ136.3-2005). Therefore, the achieved imaging depth demonstrated in our work is 3 mm with 4 mJ/cm2 laser fluence used in the imaging system. All animal experiments were performed in accordance with the protocol approved by the Animal Research Committee of the Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences.
4. Experimental results
4.1 Phantom
To train and verify the effectiveness of two-stage ResUnet_AG, we first performed phantom experiments. The phantom imaging strategy discussed in Section 3 was used to acquire at-focus and different out-of-focus depth photoacoustic images for each layer of each phantom. Using the at-focus and out-of-focus data pairs from phantom 1 and 2, we optimized the two-stage ResUnet_AG network. Specifically, the 1st-stage network was trained using data pairs composed of 2mm-far away and 1mm-near focus data, i.e., 1200 data pairs. The 2nd-stage network was trained using another 1200 data pairs from the 1mm-near focal and at-focus regions. The validation experiments were performed on phantom 3, and the reconstructed results are shown in Fig. 4. Figure 4(A) shows the initial maximum-amplitude-projection (MAP) photoacoustic image of phantom 3 acquired by AR-PAM with the focal point located at the third layer, i.e. focus at 2.5 mm deep below the surface. In this case, the second layer is considered as the 1mm-near focal region and the first layer as the 2mm-far focal region. Compared to the photoacoustic image at the third layer, the images at the first and second layers are more blurred. The resolution analysis on selected signals (indicated by a-1 and a-2) at the two layers is shown in Figs. 4(a-1) and 4(a-2). Using the proposed two-stage reconstruction strategy, we first reconstruct the photoacoustic images of the first layer using the 1st-stage DL network, and the result is shown in Fig. 4(B). The resolution of the first layer is improved to the level of the second layer (Figs. 4(b-1)). We then reconstruct the second and first layer images by the 2nd-stage network, and the result is listed in Fig. 4(C). Compared to the at-focus image shown in Fig. 4(D), the resolution of the recovered photoacoustic images in the first and second layer (Figs. 4(c-1) and 4(c-2)) is improved and is almost similar to the at-focus image (Figs. 4(d-1) and 4(d-2)). Here, the at-focus photoacoustic image in Fig. 4(D) is obtained by combining the at-focus images of three layers in phantom 3. The phantom experiments confirmed that the resolution of images located at different out-of-focus depths is recovered almost similar to the at-focus level, thus verifying the effectiveness of our proposed two-stage DL reconstruction strategy.
Fig. 4. Phantom imaging of the tungsten wires and carbon fibers. (A) Acquired image with focus at 2.5 mm depth. (B) Reconstructed results of (A) after the 1st-stage network. (C) Reconstructed results of (B) after the 2nd-stage network. (D) Ground truth. (a-1) – (d-1) Resolution analysis of the selected signals indicated by a-1 – d-1 in (A) – (D). (a-2) – (d-2) Resolution analysis of the selected signals indicated by a-2 – d-2 in (A) – (D).
4.2 In vivo imaging
In vivo imaging of the abdomen and brain of two nude mice was conducted to evaluate the performance of our proposed reconstruction method. In experiments, three photoacoustic images corresponding to three different focus positions, i.e., 0.5 mm, 1 mm, 2 mm below the surface, were acquired from the brain and abdomen of the mouse. The acquired images from the nude mouse were then used to extract the training data pairs to optimize our ResUnet_AG network. Specifically, 120 data pairs of 2mm-far away focus and 1mm-near focus from the abdominal and brain were used to optimize the 1st-stage network. The same amount of data pairs at 1mm-near focus and at the focus was employed to train the 2nd-stage network.
Figure 5 shows the reconstructed results of abdomen vasculature from a different nude mouse. Figure 5(A) is the original MAP image of the vasculature with focus at 2.5mm deep below the surface. Compared to the ground truth shown in Fig. 5(F), the resolution of vessel signals decreases significantly, and many tiny vessels cannot be seen. After the 1st-stage reconstruction using ResUnet_AG, as shown in Fig. 5(B), the image resolution of large vessels improved dramatically, and the network of small vessels became visible. After the 2nd-stage reconstruction of ResUnet_AG, the imaging resolution improved for all vessels, and the vascular network with small vessels became clearer (Fig. 5(D)). In Fig. 5, we also show the 1mm-near focus reconstruction results. Figure 5(C) is the original near-focus (with focus at 1.5mm deep below the surface) photoacoustic image, and Fig. 5(E) is the reconstructed result using the 2nd-stage network of ResUnet_AG. Although the difference between near-focus and the at-focus image is not so dramatic like in (A), our method still improves the image quality. Many small-vessel networks that blurred in the defocus region can be observed clearly in the reconstructed image. To quantitatively analyze the improvement, the resolution analysis for selected signals indicated by lines in Figs. 5(A) – 5(F) are shown in Figs. 5(a) – 5(f). Here, the dotted lines are the plots of selected signals, and the solid lines represent their profile after Gaussian fitting. Here, the resolution is computed using full width at half maximum (FWHM) on the fitted plots. This figure shows that our method effectively improves the spatial resolution of photoacoustic images located at different out-of-focus depths. The resolution of images far away from the focal zone can be improved similar to the at-focus image, and the originally blurred blood vessels are now clearly visible.
Fig. 5. Reconstructed abdominal photoacoustic images from a nude mouse. (A) MAP image with focus at 2.5 mm deep below the surface (out-of-focus above). (B) Intermediate results of the first stage. (C) MAP image with focus at 1.5 mm deep below the surface. (D) Reconstructed results of (A) after two stages. (E) 2nd-stage reconstructed results of C. (F) Ground truth. (a) – (f) resolution analysis for the selected signals indicated by lines in (A) – (F).
In the abdomen imaging, most of the vessels in MAP images are distributed near the surface. Thus, to show the depth imaging ability of our proposed method, we show the 3mm-depth brain imaging results of a mouse and their depth encoded images in Fig. 6. Figure 6(A) shows the photoacoustic brain image acquired by AR-PAM with focus at 2.5 mm below the surface. Compared to the ground truth shown in Fig. 6(D), most vessels are blurred and cannot be distinguished. Figure 6(B) shows the reconstructed result of Fig. 6(A) using 1st-stage network. Here (i.e., Fig. 6(B)), many vessels become clear with the resolution improvement. Figure 6(C) is the final recovered result after two stages of reconstruction of ResUnet_AG, and this image exhibits high imaging quality close to the ground truth. For better illustrating the depth imaging ability of our proposed method, the two representative B-scans are shown in Figs. 6(a-1) – 6(g-2). These B-scans show that the signals at 2mm-far away focus can be reconstructed with the imaging resolution similar to the ground truth. The comparisons are shown better in the regions indicated by the rectangular boxes in these B-scans.
Fig. 6. Reconstructed brain photoacoustic images of a nude mouse. (A) Images with focus at 2.5 mm deep below the surface (out-of-focus above). (B) 1st-stage reconstructed results of A. (C) reconstructed results of A after two stages. (D) Ground truth. (E) Images with focus at 0.5 mm deep below the surface (out-of-focus below). (F) 1st-stage reconstructed results of E. (G) Reconstructed results of E after two stages. (a1) – (g2) two representative B-scan images. The color images are depth encode MAP images. MAP: maximum amplitude projection.
The above discussed reconstructed results are for out-of-focus regions above the focal point. Next, recovered images for out-of-focus regions below the focal point are also listed in this figure. Figure 6(E) shows the initially acquired photoacoustic image with the focal point at about 0.5mm below the surface. Figure 6(E) shows more vessels than Fig. 6(A), but signals located far away from the focus are still blurred. After the 1st- and 2nd-stage reconstruction, signals at the large depth are recovered with high imaging resolution similar to the ground truth (Fig. 6(D)). Thus, these in vivo imaging experiments confirm that high-resolution imaging can be reconstructed at large depth for out-of-focus regions using our proposed two-stage reconstruction network.
To verify the generalization ability of the proposed network, a new imaging experiment on a tumor model planted on the back of a mouse was conducted. The DL model based on training and reconstructing mouse brain was applied to the tumor data. The reconstructed MAP images of 3mm-thick tumor tissue, their depth encoded images and representative B-scan images are shown in Fig. 7. Figure 7(A) are the initially acquired AR-PAM images with the focal plane placed at about 0.5 mm deep below the tumor surface. It can be seen the signals in defocus regions (green and blue signals) are blurred, and the signals below 2 mm depth are totally submerged in the background noises. After the 1st-stage reconstruction, the resolution and image quality within 2 mm defocus region (Fig. 7(B)) are significantly improved, and the vessels become visible. After the 2nd-stage reconstruction, all defocus regions were recovered with significantly improved resolution (Fig. 7(C)) comparable to the ground truth (Fig. 7(D)). These improvements were further quantitatively analyzed by selecting typical vessels in Figs. 7(A) – 7(D). These results demonstrate the effectiveness and generalization ability of our proposed two-stage DL model to extend the DOF of AR-PAM.
Fig. 7. Reconstructed photoacoustic images of a tumor planted on the back of a nude mouse. (A) Images with focus at 0.5 mm deep below the surface (out-of-focus below). (B) 1st-stage reconstructed results of A. (C) Reconstructed results of A after two stages. (D) Ground truth.
4.3 Comparative experiments
The above section verified the high-resolution out-of-focus imaging ability of our proposed two-stage reconstruction strategy with the ResUnet_AG network. Here, we compare the superiority of our method against other reconstruction techniques, e.g., Ref. 32, traditional U-Net, and one-stage network. The comparison results for the imaging of abdomen, brain and tumor, are shown in Fig. 8. With these results, it can be seen that other methods can recover photoacoustic images of 2mm-far away focus with limited resolution improvement, and weak signals are lost and discontinuous. Our proposed two-stage ResUnet_AG network recovered the highest-quality photoacoustic images with almost complete reconstruction of small vessels and significant resolution improvement.
Fig. 8. Comparative imaging experiments among different methods. First row: reconstructed results of Abdomen. Second row: reconstructed results of brain. Third row: reconstructed results of tumor.
To compare the reconstructed results of different methods quantitatively, two parameters of peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) are imported to evaluate the quality of reconstructed images. The computed values of the two parameters for recovered MAPs in Fig. 8 are listed in Table 1. With this table, it can be concluded: (1) all deep learning based reconstruction methods can improve the PSNR and SSIM of the recovered images; (2) the quantitative indexes of two-stage reconstruction are superior to the one-stage reconstruction; (3) compared with other methods, our proposed model provides better PSNR and SSIM, e.g. 30% improvement compared to Ref. 32.
Table 1. Quantitative comparisons of reconstructed images with different methods
5. Discussion and conclusions
In this work, we propose a reconstruction method based on deep learning to improve the image quality in the out-of-focus region of AR-PAM imaging. Compared to the previous methods, this proposed two-stage method fully considers the characteristics of images at different out-of-focus depths. It thus effectively extends the high-resolution imaging depth of AR-PAM without the additional requirements on the system design, such as the virtual-point concept. To further improve upon our work, several issues still need to be addressed, and they are discussed below.
- (1) The acquisition of training data. In our work, to optimize the two-stage deep learning network, we acquired many data pairs formed by at-focus and out-of-focus images at different depths. We had to design layered phantoms, and the imaging probe was moved up or down through a controlled mechanical process to obtain the focused photoacoustic images at different layers. The fabrication of the phantom is complex, and the movement of the imaging probe is limited by the accuracy of the scanner. Thus, minor errors exist in the consistency of the imaging area of data pairs and focus positioning at different layers, affecting the DL network optimization. This results in the decrease of image reconstruction quality to a certain extent. Improving the phantom manufacturing process and accuracy of auto-controlled mechanics would certainly improve the image quality further.
- (2) The extended imaging depth. In the in vivo experiments, limited by the light intensity of our current AR-PAM system and its design, our work just showed the reconstructed results for the areas located less than 2 mm away from the focal zone. In practical application, deeper tissue signals can be detected if the system uses a laser with higher energy. Thus, the high-resolution photoacoustic images for deeper tissues can be recovered using our proposed DL framework.
- (3) Optimization of network structure. A more elaborate and well-optimized deep learning network will further improve image reconstruction quality. For example, the number and location of the remaining blocks in the network can be adjusted; more jump connections can be added to the network to identify features better.
While there is room for further improving the image quality of defocus region, our proposed method clearly shows the improvement in the reconstruction of defocus images in AR-PAM and would lead to new applications. Thus, our work should help extend the application of AR-PAM such as in the study of tumor angiogenesis and brain functions.
Funding
Natural Science Foundation of Shandong Province (ZR2020MF105); Guangdong Provincial Key Laboratory of Biomedical Optical Technology (2020B121201010).
Acknowledgments
J. Meng will thank N. Chen for helping the generation of depth maps in our work.
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. L. V. Wang and J. Yao, “A practical guide to photoacoustic tomography in the life sciences,” Nat. Methods 13(8), 627–638 (2016). [CrossRef]
2. P. F. Hai, T. Imai, S. Xu, R. Zhang, R. L. Aft, J. Zou, and L. V. Wang, “High-throughput, label-free, single-cell photoacoustic microscopy of intratumoral metabolic heterogeneity,” Nat. Biomed. Eng. 3(5), 381–391 (2019). [CrossRef]
3. M. Du, Z. J. Chen, and D. Xing, “Spectral interferometric depth-resolved photoacoustic viscoelasticity imaging,” Opt. Lett. 46(7), 1724–1727 (2021). [CrossRef]
4. H. H. Yang, T. Zhang, C. Tao, and X. J. Liu, “Multispectral photoacoustic holography of elastomers from a bright background,” Opt. Lett. 46(19), 5071–5074 (2021). [CrossRef]
5. S. M. Schoustra, T. J. P. M. op’t Root, R. P. P. van Meerdervoort, L. Alink, W. Steenbergen, and S. Manohar, “Pendant breast immobilization and positioning in photoacoustic tomographic imaging,” Photoacoustics 21, 100238 (2021). [CrossRef]
6. J. Xia and L. V. Wang, “Small-animal whole-body photoacoustic tomography: A Review,” IEEE Trans. Biomed. Eng. 61(5), 1380–1389 (2014). [CrossRef]
7. B. X. Lan, W. Liu, Y. C. Wang, J. H. Shi, Y. Li, S. Xu, H. X. Sheng, Q. F. Zhou, J. Zou, U. Hoffmann, W. Yang, and J. J. Yao, “High-speed widefield photoacoustic microscopy of small-animal hemodynamics,” Biomed. Opt. Express 9(10), 4689–4701 (2018). [CrossRef]
8. M. U. Arabul, H. M. Heres, M. Rutten, M. S. Van, F. D. V. Van, and R. Lopata, “Investigation on the effect of spatial compounding on photoacoustic images of carotid plaques in the in vivo available rotational range,” IEEE Trans. Ultrason., Ferroelect., Freq. Contr. 65(3), 440–447 (2018). [CrossRef]
9. P. Suwannasom, Y. Sotomi, Y. Miyazaki, E. Tenekecioglu, Y. Onuma, and P. W. Serruys, “Multimodality imaging to detect vulnerable plaque in coronary arteries and its clinical application,” Eur. Heart J. Cardiovasc. Imaging 18(6), 613–620 (2018). [CrossRef]
10. L. Lin, P. Hu, J. H. Shi, C. M. Appleton, K. Maslov, L. Li, R. Y. Zhang, and L. V. Wang, “Single-breath-hold photoacoustic computed tomography of the breast,” Nat. Commun. 9(1), 2352 (2018). [CrossRef]
11. Y. A. Sun, Y. Q. Liang, W. B. Dai, B. He, H. Zhang, X. Q. Wang, J. C. Wang, S. H. Huang, and Q. Zhang, “Peptide-drug conjugate-based nanocombination actualizes breast cancer treatment by maytansinoid and photothermia with the assistance of fluorescent and photoacoustic images,” Nano Lett. 19(5), 3229–3237 (2019). [CrossRef]
12. S. Iskander-Rizk, A. F. W. van der Steen, and G. van Soest, “Photoacoustic imaging for guidance of interventions in cardiovascular medicine,” Phys. Med. Biol. 64(16), 16TR01 (2019). [CrossRef]
13. E. Vienneau, W. Liu, and J. Yao, “Dual-view acoustic-resolution photoacoustic microscopy with enhanced resolution isotropy,” Opt. Lett. 43(18), 4413–4416 (2018). [CrossRef]
14. J. Yao, L. Wang, J. M. Yang, K. I. Maslov, T. T. Wong, L. Li, C. H. Huang, J. Zou, and L. V. Wang, “High-speed label-free functional photoacoustic microscopy of mouse brain in action,” Nat. Methods 12(5), 407–410 (2015). [CrossRef]
15. J. W. Baik, J. Y. Kim, S. Cho, S. Choi, J. Kim, and C. Kim, “Super wide-field photoacoustic microscopy of animals and humans in vivo,” IEEE Trans. Med. Imaging 39(4), 975–984 (2020). [CrossRef]
16. M. L. Li, H. F. Zhang, K. Maslov, G. Stoica, and L. V. Wang, “Improved in vivo photoacoustic microscopy based on a virtual-detector concept,” Opt. Lett. 31(4), 474–476 (2006). [CrossRef]
17. C. Li and L. V. Wang, “High-numerical-aperture-based virtual point detectors for photoacoustic tomography,” Appl. Phys. Lett. 93(3), 033902 (2008). [CrossRef]
18. X. Yang and L. V. Wang, “Photoacoustic tomography of a rat cerebral cortex with a ring-based ultrasonic virtual point detector,” J. Biomed. Opt. 12(6), 060507 (2007). [CrossRef]
19. Z. Deng, X. Yang, H. Gong, and Q. Luo, “Adaptive synthetic-aperture focusing technique for microvasculature imaging using photoacoustic microscopy,” Opt. Express 20(7), 7555–7563 (2012). [CrossRef]
20. J. Meng, C. Liu, J. Zheng, R. Lin, and L. Song, “Compressed sensing based virtual-detector photoacoustic microscopy in vivo,” J. Biomed. Opt. 19(3), 036003 (2014). [CrossRef]
21. J. Park, S. Jeon, J. Meng, L. Song, J. S. Lee, and C. Kim, “Delay-multiply-and-sum-based synthetic aperture focusing in photoacoustic microscopy,” J. Biomed. Opt. 21(3), 036010 (2016). [CrossRef]
22. D. Cai, Z. Li, Y. Li, Z. Guo, and S. L. Chen, “Photoacoustic microscopy in vivo using synthetic-aperture focusing technique combined with three-dimensional deconvolution,” Opt. Express 25(2), 1421–1434 (2017). [CrossRef]
23. S. Jeon, J. Park, R. Managuli, and C. Kim, “A novel 2-D synthetic aperture focusing technique for acoustic-resolution photoacoustic microscopy,” IEEE Trans. Med. Imaging 38(1), 250–260 (2019). [CrossRef]
24. C. Yang, H. Lan, F. Gao, and F. Gao, “Review of deep learning for photoacoustic imaging,” Photoacoustics 21, 100215 (2021). [CrossRef]
25. H. Deng, H. Qiao, Q. Dai, and C. Ma, “Deep learning in photoacoustic imaging: A review,” J. Biomed. Opt. 26(04), 1–32 (2021). [CrossRef]
26. J. Gröhl, M. Schellenberg, K. Dreher, and L. Maier-Hein, “Deep learning for biomedical photoacoustic imaging: A review,” Photoacoustics 22, 100241 (2021). [CrossRef]
27. H. Andreas and T. C. Ben, “Deep learning in photoacoustic tomography: current approaches and future directions,” J. Biomed. Opt. 25(11), 1–46 (2020). [CrossRef]
28. R. Manwar, X. Li, S. Mahmoodkalayeh, E. Asano, D. Zhu, and K. Avanaki, “Deep learning protocol for improved photoacoustic brain imaging,” J. Biophotonics 13(10), e202000212 (2020). [CrossRef]
29. X. Zhang, F. Ma, Y. Zhang, J. Wang, C. Liu, and J. Meng, “Sparse-sampling photoacoustic computed tomography: deep learning vs. compressed sensing,” Biomedical Signal Processing and Control 71(B), 103233 (2022). [CrossRef]
30. H. Zhang, H. Li, N. Nyayapathi, D. Wang, A. Le, L. Ying, and J. Xia, “A new deep learning network for mitigating limited-view and under-sampling artifacts in ring-shaped photoacoustic tomography,” Computerized Medical Imaging and Graphics 84, 101720 (2020). [CrossRef]
31. H. Zhao, Z. Ke, F. Yang, K. Li, N. Chen, L. Song, C. Zheng, D. Liang, and C. Liu, “Deep learning enables superior photoacoustic imaging at ultralow laser dosages,” Adv. Sci. 8(3), 2003097 (2021). [CrossRef]
32. A. Sharma and M. Pramanik, “Convolutional neural network for resolution enhancement and noise reduction in acoustic resolution photoacoustic microscopy,” Biomed. Opt. Express 11(12), 6826–6839 (2020). [CrossRef]
33. Y. Nishitani, R. Nakayama, D. Hayashi, A. Hizukuri, and K. Murata, “Segmentation of teeth in panoramic dental X-ray images using U-Net with a loss function weighted on the tooth edge,” Radiol Phys Technol 14(1), 64–69 (2021). [CrossRef]
34. M. Nishio, S. Noguchi, and K. Fujimoto, “Automatic pancreas segmentation using coarse-scaled 2D model of deep learning: usefulness of data augmentation and deep U-Net,” Appl. Sci. 10(10), 3360 (2020). [CrossRef]
35. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2016), PP. 770–778.
36. J. Schlemper, O. Oktay, M. Schaap, M. Heinrich, B. Kainz, B. Glocker, and D. Rueckert, “Attention gated networks: learning to leverage salient regions in medical images,” Med. Image Anal. 53, 197–207 (2019). [CrossRef]
