Open Access
Add to BibSonomy
Abstract
Polarization-sensitive optical coherence tomography (PS-OCT) measures the polarization states of the backscattered light from tissue that can improve angiography based on conventional optical coherence tomography (OCT). We present a feasibility study on PS-OCT integrated with deep learning for PS-OCT angiography (PS-OCTA) imaging of human cutaneous microvasculature. Two neural networks were assessed for PS-OCTA, including the residual dense network (RDN), which previously showed superior performance for angiography with conventional OCT and the upgraded grouped RDN (GRDN). We also investigated different protocols to process the multiple signal channels provided by the Jones matrices from the PS-OCT system to achieve optimal PS-OCTA performance. The training and testing of the deep learning-based PS-OCTA were performed using PS-OCT scans collected from 18 skin locations comprising 16,600 B-scan pairs. The results demonstrated a moderately improved performance of GRDN over RDN, and of the use of the combined signal from the Jones matrix elements over the separate use of the elements, as well as a similar image quality to that provided by speckle decorrelation angiography. GRDN-based PS-OCTA also showed ∼2-3 times faster processing and improved mitigation of tissue motion as compared to speckle decorrelation angiography, and enabled fully automatic processing. Deep learning-based PS-OCTA can be used for imaging cutaneous microvasculature, which may enable easy adoption of PS-OCTA for preclinical and clinical applications.
© 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Optical coherence tomography angiography (OCTA) comprises a family of techniques for label-free, in vivo imaging of the blood microvasculature with micrometer-scale resolutions [1]. It is based on optical coherence tomography (OCT) which measures the near-infrared light backscattered from tissue. OCTA further uses the temporal changes of OCT signal from the same tissue locations, to identify the microvessels with markedly high changes induced by the motion of the scatterers in the blood flow. Multiple OCTA variants have been developed, depending on the algorithm used to quantify the temporal changes of OCT signal and the type of OCT signal (i.e., intensity, phase or complex OCT signal) used for analysis [2–5]. These variants find a growing number of applications in ophthalmology [6–8], neurology [9] and dermatology [10–12]. Many tissue types in these imaging applications present birefringence due to the fibrous structures, such as the Henle’s fiber in the retina [13], nerves in the brain [9], and collagen fibers in the skin [14,15]. Imaging birefringent tissue with conventional OCT usually leads to polarization-induced artifacts in the OCT signal, which could subsequently compromise the OCTA signal, as demonstrated by Tang et al. [16].
Polarization-sensitive OCT (PS-OCT) is another extension of OCT, which images polarization properties of tissue, such as birefringence, degree of polarization uniformity and optic axis orientation [17–19]. To do this, PS-OCT measures the polarization states of the backscattered light from tissue, which, after combination also mitigates the polarization-induced artifacts observed in conventional OCT. As such, the benefits of PS-OCT for angiography have been demonstrated in recent studies. Tang et al. proposed PS-OCT angiography (PS-OCTA) for imaging human skin with optical microangiography (OMAG) using the complex signal from a PS-OCT system [16]. They demonstrated the superior performance of PS-OCTA which eliminated polarization-related artifacts, as compared to conventional OCTA. In another earlier study [20], Jones matrices provided by PS-OCT were used for PS-OCTA imaging with a speckle decorrelation algorithm, based on the intensity signal. With the optimized processing protocol, PS-OCTA showed improved vessel contrast and effective imaging depth in human skin as compared to conventional OCTA. However, one drawback of PS-OCTA is the elevated computational workload, particularly when a time-consuming OCTA algorithm (e.g., speckle decorrelation with moving window processing) was directly applied to the individual Jones matrix elements [20].
Recently, deep learning has been adopted to construct OCTA images using OCT scans, showing improved processing speed and vessel contrast [21]. Lee et al. used U-Net to generate OCTA images using single OCT B-scans of the retina, based on a large training set (401,098 OCT B-scans) with expert-generated vessel label images [22]. Although their method eliminated the requirement for repeated B-scans from the same locations and thus could be less impacted by motion artifacts, the small vessels could not be reliably imaged. Jiang et al. assessed multiple networks for generating OCTA images using repeated OCT B-scans from the same tissue locations [23]. To create high-quality vessel label images, 48 repeated OCT B-scans from each location were acquired and then processed by the split-spectrum amplitude and phase-gradient angiography (SSAPGA) algorithm. They demonstrated that the residual dense network (RDN) outperformed other networks. To date, no deep learning has been adapted to PS-OCT for angiography, which has multiple signal channels and requires an optimal processing protocol to improve the vessel contrast and the processing speed.
In this study, we present an integration of Jones matrix-based PS-OCT with deep learning for PS-OCTA imaging. Three-dimensional (3-D) PS-OCT scans from 18 human skin regions comprising 16,600 B-scan pairs (i.e., 2 repeated B-scans from each lateral location) were acquired with the vessel labels generated by the custom speckle decorrelation algorithm [20]. Training and validation of the deep learning-based PS-OCTA method was performed on 3000 B-scan pairs with the remaining used for testing, both with RDN and the upgraded grouped RDN (GRDN). We assessed the performance of RDN and GRDN for PS-OCTA imaging, as compared to the vessel labels. Subsequently, using the identified superior GRDN, we assessed different processing protocols of the Jones matrices for PS-OCTA imaging, and assessed the overall imaging performance of deep learning-based PS-OCTA with the optimal protocol. The results demonstrated that deep learning-based PS-OCTA can provide a comparable imaging capability to the speckle decorrelation algorithm, with ∼2-3 times faster processing and mitigated motion artifacts.
2. Methods
2.1 PS-OCT scan acquisition
A home-built PS-OCT system, previously described in Ref. [19], was used to collect 3-D PS-OCT scans. In brief, a wavelength-swept light source (Axsun Technologies, Billerica, USA) provided an A-line rate of 50 kHz, a center wavelength of 1302 nm and a measured axial resolution of 11 µm in air. The light passed through a free-space polarization delay unit (PDU) to delay one of the two orthogonal polarization states of the incident light on the tissue surface. This delay together with the extended total imaging depth range of 10 mm, enabled depth-encoded polarization multiplexing [24]. The polarization-delayed light illuminated the tissue via an objective lens (LSM03, Thorlabs Inc., Newton, USA), providing a lateral imaging resolution of 13 µm. The backscattered light from tissue was collected by the objective lens, combined with the reference light and subsequently input into a polarization-diverse optical mixer (PDOM-1310, Finisar, Sunnyvale, USA) for balanced detection in two orthogonal polarization directions. After data reconstruction, a complex-valued full Jones matrix, $\left[ {\begin{array}{c} {{J_{11}}}\\ {{J_{21}}} \end{array} \begin{array}{c} {{J_{12}}}\\ {{J_{22}}} \end{array}} \right]$, was provided for each voxel in the 3-D PS-OCT scans [17].
Human skin was scanned in vivo using the PS-OCT system with approval from the Human Research Ethics Committee at The University of Western Australia. Prior to scanning, written consent was obtained from each subject. In total, 18 different skin locations on the forearm of 3 healthy subjects were scanned to acquire 3-D PS-OCT scans using the protocol as reported in Ref. [20]. To reduce bulk tissue motion, the PS-OCT probe was gently placed on the skin for contact scanning. The scanning first incorporated a fiducial marker (i.e., a thin metal square with a 5-mm-diameter hole to image through) to assess the residual motion, which was consistently shown to be minor [2]. Therefore, the subsequent scanning did not use the fiducial mark. The scanning field of view (FOV) varied from 3 × 3 mm to 9.9 × 9.9 mm, depending on the local surface geometry. Each 3-D scan comprised either 800 B-scan pairs with 800 A-scans per B-scan or 1000 B-scan pairs with 1000 A-scans per B-scan. The B-scan pair referred to the two repeated B-scans acquired at the same lateral location but at different time points, to enable imaging of the blood vessels by PS-OCTA. In total, 16,600 B-scan pairs were collected within the 18 3-D scans.
2.2 Generation of vessel labels
The vessel label images were generated by applying a custom speckle decorrelation algorithm to the PS-OCT scans [20]. First, the combined linear OCT amplitude signal was calculated from the Jones matrix as the square root of the determinant (i.e., $\sqrt {|{{J_{11}}\cdot {J_{22}} - {J_{12}}\cdot {J_{21}}} |} $). The decorrelation of the combined signal between each B-scan pair was then calculated using a moving window (size: 5 × 5 pixels) based on the formula reported in Ref. [25]. The values in the resulting decorrelation volume ranged from 0 to 2, where high values indicated high changes of the OCT signal over time and thus were corresponding to vessels. However, noise in the combined OCT signal also led to artificially high decorrelation [26]. To reduce such artifacts, we further weighted the decorrelation signal by the corresponding logarithm of the combined OCT signal after setting the low OCT signal below an empirically chosen threshold (12 dB) to zero. The tissue surface in each B-scan of the combined OCT signal was detected, and subsequently used to flatten B-scan pairs of the combined OCT signal and the corresponding B-scans of the weighted decorrelation.
The weighted decorrelation still had artifacts due to residual tissue motion, which could negatively impact the training of deep learning-based PS-OCTA. Such artifacts are usually shown as lines in the projection image of the vessels [27]. To mitigate these artifacts, we generated an en face mask of the residual motion by taking the maximum projection of the weighted decorrelation within the epidermis (from the skin surface to 50 µm deep), which comprised only motion artifacts without any vessels. The mask was then subtracted from each en face image in the flattened, weighted decorrelation volume. The resulting decorrelation B-scans as the vessel labels and the corresponding OCT B-scan pairs of the four Jones matrix elements (i.e., $|{{J_{11}}} |$, $|{{J_{12}}} |$, $|{{J_{21}}} |$ and $|{{J_{22}}} |$) and the combined signal (i.e., $\sqrt {|{{J_{11}}\cdot {J_{22}} - {J_{12}}\cdot {J_{21}}} |} $), were subsequently used for deep learning-based PS-OCTA imaging. The calculation of speckle decorrelation, weighting and motion reduction were all implemented in MATLAB R2019b (The MathWorks, Inc., Natick, USA), using an Intel Core i7-9750 H processor with 16-GB memory.
2.3 Deep learning networks and processing protocols
Two neural networks, RDN and GRDN, were explored for PS-OCTA imaging in this study, as shown in Fig. 1(a). RDN was selected due to its superior OCTA performance reported in the previous comparative study [23]. Based on RDN, GRDN was proposed by Kim et al., showing improved image denoising in comparison to RDN [28], but its performance for OCTA has not been investigated. We first assessed these two networks to determine the superior network for PS-OCTA imaging. Both networks used the residual dense block (RDB) illustrated in Fig. 1(b) as their basic module. GRDN further used a grouped RDB (GRDB) structure, which fused feature maps from 4 cascaded RDBs, as shown in Fig. 1(c). With the same number of RDB modules, GRDN reduced the distance between the deep and shallow image features through staged cascade to simplify the training. For the comparison of GRDN and RDN, the two networks comprised the same number of RDBs (n = 16) in this study. In addition, a convolutional block attention module (CBAM) was incorporated into GRDN after the up-sampling layer to enhance important image features from the up-sampled data.
Fig. 1. Neural networks and basic modules used for deep learning-based PS-OCTA. (a) Architectures of GRDN (top row) and RDN (bottom row). (b) and (c) Structures of RDB and GRDB modules, respectively. CBAM: convolutional block attention module; Concat: concatenate layer; Conv: convolutional layer; ConvDown: convolutional down-sampling layer; ConvUp: convolutional up-sampling layer; ReLU: rectified linear unit layer.
2500 and 500 decorrelation B-scans and the corresponding OCT B-scan (i.e., image showing the logarithm of the OCT amplitude) pairs from three 3-D PS-OCT scans were used for the training and validation of the two networks, respectively. Prior to training, the decorrelation and OCT B-scans were cropped from skin surface to a physical depth of 650 µm, where most of the useful signal was located. The cropped B-scans were then resized into 448 × 448 pixels with three RGB channels for each pixel in order to fit for the square kernel (size: 3 × 3) used in the networks. During training, the SmoothL1 loss function was used to calculate the loss [29]:
The PS-OCT scans provided OCT B-scan pairs of the four individual Jones matrix elements and the combined signal from the four elements. We assessed three processing protocols as shown in Fig. 2 to optimize the performance of deep learning-based PS-OCTA. The first protocol (P1) used OCT B-scan pairs (at time ${\textrm{t}_1}$ and ${\textrm{t}_2}$) of the combined signal as the inputs and output one PS-OCTA B-scan corresponding to each input OCT B-scan pair. The advantage of the combined signal is that it is free from polarization-induced artifacts seen in conventional OCT, and thus can improve the PS-OCTA image quality as compared to conventional OCTA [20]. The second protocol (P2) used OCT B-scan pairs of each of the four Jones matrix elements as separate inputs and output four corresponding vessel images, which were subsequently averaged to generate one final PS-OCTA B-scan. Although the individual Jones matrix elements were still affected by polarization fading, this protocol allowed the averaging of four vessel images to improve the quality. In the third protocol (P3), OCT B-scan pairs of all four Jones matrix elements were jointly used as the inputs, which led to one output PS-OCTA B-scan. The complete detection of polarized signals jointly provided by the four elements could potentially improve the vessel image quality.
Fig. 1. Processing protocols for deep learning-based PS-OCTA. The three protocols (P1, P2 and P3) are marked in yellow, red and blue, respectively.
2.4 Evaluation of PS-OCTA image quality
To qualitatively assess the image quality, B-scans and en face projection images of vessels generated by speckle decorrelation and multiple deep learning-based PS-OCTA methods were compared. As the PS-OCTA data generated by different methods had distinct value ranges (i.e., 0-2 for speckle decorrelation and 0-255 for deep learning-based methods), the color map was customized for each image to optimize the image quality. In particular, a lower and upper display threshold were selected for each image to optimize the vessel contrast, which equivalently normalized the PS-OCTA data in the image. Therefore, the color bars of the PS-OCTA images in the Results all had a normalized value range from 0 to 1.
The quality of B-scans and en face projection images of vessels generated by deep learning-based PS-OCTA was also quantified by calculating the peak signal-to-noise ratio (PSNR), structural similarity (SSIM) and vessel area density (VAD). PSNR assesses the degree of image distortion and noise with large values indicating high quality, calculated as:
where $MA{X_x}$ is the maximum intensity in the vessel label image x, and $MSE$ is the mean square error of the output image y to the label image x, with x and y referring to the PS-OCTA images generated by speckle decorrelation and deep learning-based methods, respectively. SSIM proposed by Wang et al. [30] was calculated to assess the similarity of two vessel images based on human visual perception by combining three terms, including luminance, contrast and structure, defined as:In this formula, $\mu$ and $\sigma $ are the mean and the standard deviation of the label image x and output image y, respectively and ${\sigma _{xy}}$ is the cross-variance for x and y. ${C_1}$ and ${C_2}$ are the regulation constants to avoid instability of regions with local mean or standard deviation close to zero. To test whether PSNR and SSIM were statistically different between different methods, Student’s t-test was performed.
In addition, VAD in the en face projection images was quantified by applying a threshold to the image and then calculating the ratio of the vessel area to the total tissue area. We optimized the threshold for each projection image to make sure that the vessels were well maintained after thresholding.
3. Results
3.1 Assessment of RDN and GRDN
Figures 3 and 4 show PS-OCTA imaging with RDN and GRDN on two forearm skin regions using Protocol 1, as compared to the vessel labels. B-scans from the same lateral location within the first forearm skin region are shown in Figs. 3(a)-(c) for the vessel label, RDN-based and GRDN-based PS-OCTA, respectively. RDN and GRDN generate very similar vessel images (Fig. 3(b) vs 3(c) in the tissue region (between the two dashed lines), which are also highly consistent with the label image (Fig. 3(a)). However, in the metal maker area where there is only strong surface reflection in the OCT images (not shown), artifacts are generated by RDN-based PS-OCTA in Fig. 3(b). Such artifacts are observed in most B-scans with the metal marker from RDN, and lead to noise in the projection image (from skin surface to 650 µm deep) in Fig. 3(e). In contrast, GRDN-based PS-OCTA is less impacted by the strong surface reflection, as shown in the B-scan (Fig. 3(c)) and projection image (Fig. 3(f)). In the tissue region, RDN and GRDN also generate similar vessel structures to those from the vessel label in Figs. 3(d)–3(f). In the projection images, the calculated PSNR and SSIM are 19.88 dB and 0.624 for Fig. 3(e), and 20.69 dB and 0.667 for Fig. 3(f), respectively, suggesting good performance of both networks while GRDN moderately outperforms RDN.
Fig. 3. PS-OCTA imaging of a forearm skin region with RND and GRDN. (a)-(c) B-scans from the same location generated using speckle decorrelation (i.e., vessel label), RDN and GRDN, respectively. Dots and dashed lines mark the lateral locations of the inner edges of the metal marker. (d)-(f) Projection images (from skin surface to a depth of 650 µm) corresponding to (a)-(c), respectively. Dashed lines and dots mark the locations of the B-scans in (a)-(c), and inner edges of the metal marker, respectively. Scale bars: 500 µm.
Fig. 4. PS-OCTA imaging of a forearm skin region with RND and GRDN. (a)-(c) B-scans from the same location generated using speckle decorrelation (i.e., vessel label), RDN and GRDN, respectively. Blue squares and dashed lines mark the regions with and without prominent tissue motion as indicated by the directions of the green and white arrows, respectively. (d)-(f) Projection images (from skin surface to a depth of 650 µm) corresponding to (a)-(c), respectively. Squares and the arrows mark the same positions and directions as those in (a)-(c), respectively. Scale bars: 500 µm.
Similar to Fig. 3, PS-OCTA imaging of the second forearm skin region is shown in Fig. 4. Figures 4(a)–4(c) are B-scans from the vessel label, RDN-based and GRDN-based PS-OCTA at the same location with residual tissue motion (to the left of the dashed line as marked by the direction of the green arrows). The motion is not prominent in the vessel label image in Fig. 4(a) due to the use of the motion reduction method described in Section 2.2. Although the training was performed using label images after motion reduction, RDN-based PS-OCTA still generated the artifacts due to motion, which are prominent in the B-scan (Fig. 4(b)) and also visible as a horizontal line in the projection image in Fig. 4(e) (marked by the green arrow). In comparison, GRDN-based PS-OCTA could mitigate the motion artifacts in the B-scan (Fig. 4(c)) and projection image (Fig. 4(f)), although the vessel contrast seems to be decreased in the right half of the B-scan in Fig. 4(c) (marked by the direction of the white arrow). In the left half in Fig. 4(c), more regions of high signal were observed in comparison to Fig. 4(a). As these regions showed good correspondence with the vessel segments in Fig. 4(f), GRDN-based PS-OCTA presented an improved capability to maintain true vessel signal while mitigating motion, as compared to speckle decorrelation. Other than the locations with tissue motion, both RDN and GRDN demonstrate very comparable performance to speckle decorrelation, observed in most B-scans and the projection images (Figs. 4(d)–4(f)).
To quantitatively assess the performance of RDN and GRDN, we further calculated PSNR and SSIM using 3-D scans from nine skin locations comprising 8400 B-scan pairs. These selected scans did not contain the fiducial marker in order to avoid the impact of the noise in RDN-based PS-OCTA, induced by the strong reflection from the metal surface as shown in Fig. 3(b). The results are summarized by the box plots in Figs. 5(a) and 5(b) for PSNR and SSIM, respectively, where the mean values are also marked by the dots. The mean PSNR of RDN and GRDN is 28.39 dB and 28.67 dB, and the mean SSIM of RDN and GRDN is 0.727 and 0.729, respectively. Statistical testing indicated that the PSNR and SSIM of RDN are significantly lower than those of GRDN (p < 0.05 and p < 0.0001, respectively). GRDN also shows higher median values and shorter box plot whiskers of PSNR and SSIM than RDN, suggesting that the PS-OCTA images generated using GRDN present a higher similarity to the label images. In addition, the training time of each epoch for GRDN is only 1/4 of that for RDN, and GRDN also shows an improved capability to mitigate artifacts due to motion and strong surface reflection as shown in Figs. 3 and 4. Therefore, we selected GRDN for PS-OCTA imaging in the following sections.
Fig. 5. Summary of PSNR (a) and SSIM (b) for PS-OCTA B-scans generated using RDN and GRDN. The horizontal lines in the boxes mark the medians. The box edges represent interquartile ranges. Black dots mark the corresponding mean values.
3.2 Assessment of processing protocols
Figure 6 shows PS-OCTA imaging with GRDN of another forearm skin region using the three deep learning processing protocols. The projection images of vessels in the predicted PS-OCTA B-scans using the three protocols are shown in Figs. 6(d), 6(g) and 6(j), respectively, and compared to the projection from the vessel labels in Fig. 6(a). All three processing protocols show overall similar vessel networks to that from the vessel labels. However, differences between the three protocols are also observed in local areas, such as in the two areas outlined by the blue and green squares. In particular, some deep vessels (≥ 550 µm below skin surface) are less visible in Protocol 2 than the other two protocols, with one example indicated by the blue arrowhead in Fig. 6(h). In addition, the region marked by the green arrowhead in Fig. 6(i) from Protocol 2 also shows poorer vessel connectivity than those from Protocol 1 and 3. The moderately inferior performance of Protocol 2 can be attributed to the fact that each separate input contains polarization-induced artifacts, which may degrade PS-OCTA image quality and cannot be simply eliminated by averaging the four vessel images from the Jones matrix elements. Overall, Protocol 1 and 3 present a more similar performance to the speckle decorrelation method in Fig. 6.
Fig. 6. PS-OCTA imaging of a forearm skin region with GRDN using three processing protocols. (a), (d), (g) and (j) Projection images (from skin surface to a depth of 650 µm) generated using speckle decorrelation (i.e., vessel label), and GRDN with Protocol 1-3, respectively. (b), (e), (h), and (k) Magnified images of the area outlined by the blue squares in (a), (d), (g) and (j), respectively. (c), (f), (i), and (l) Magnified images of the area outlined by the green squares in (a), (d), (g) and (j), respectively. Blue and green arrowheads mark corresponding vessels. White scale bars: 500 µm. Cyan scale bars: 150 µm.
To quantify the performance of the three processing protocols, the same collection of B-scans from nine skin locations as described in Section 3.1 was used to calculate PSNR and SSIM. The VAD in the resulting projection images was also calculated. Figure 7(a) shows that PSNRs of Protocols 1 and 2 are higher than that of Protocol 3. However, the SSIM of Protocol 2 is much lower than those of the other two protocols in Fig. 7(b). These two parameters show significant differences among the three protocols (p < 0.0001 between each of the two protocols) based on the statistical test. VADs of vessel labels (green dots in Fig. 7(c)) are generally higher than those from the three protocols. Among the three processing protocols, Protocol 1 led to the highest VADs (red dots in Fig. 7(c)) while Protocol 2 led to the lowest VADs (blue dots in Fig. 7(c)). Additionally, the processing time of the three protocols to generate one PS-OCTA volume comprising 1000 B-scans is shown in Table 1, as compared to the speckle decorrelation method. The processing time of Protocol 1 is approximately 1/5 of that of Protocol 2, and less than half of that of Protocol 3, due to the different numbers of input images for processing. Protocol 1 is also ∼2.3 times faster than the speckle decorrelation method. Overall, considering both the quality of PS-OCTA images and the processing time, Protocol 1 outperforms the other two processing protocols.
Fig. 7. Assessment of the three processing protocols (P1-P3). (a) and (b) Box plots of PSNR and SSIM of the B-scans generated using P1-P3, respectively. Black dots mark the corresponding mean values. (c) VAD of projection images generated using vessel labels and P1-P3 from nine 3-D scans (S1-S9).
Table 1. Processing time of different PS-OCTA methods to generate 1000 OCTA B-scans
3.3 Assessment of GRDN with Protocol 1
Based on the advantages of GRDN with Protocol 1 for PS-OCTA imaging identified in Sections 3.1 and 3.2, we performed a detailed analysis of its performance for imaging vessels in different depth ranges in the skin. One example is presented in Fig. 8, showing the projection images in Figs. 8(d)–8(f) as compared to those from the vessel labels in Figs. 8(a)–8(c). The high similarity between the corresponding images, demonstrating deep learning-based PS-OCTA’s ready capability for resolving the vessels in both the superficial and deep plexuses. In the superficial tissue, the magnified image in Fig. 8(h) from deep learning-based PS-OCTA reduces the noise at the cost of slightly reduced vessel connectivity, as compared to vessels labels in Fig. 8(g). In the deep layers, images from deep learning-based PS-OCT appear smoother with clearer vessel edges in Figs. 8(j) and 8(l) than those from the vessel labels in Figs. 8(i) and 8(k). Generally, deep learning-based PS-OCTA demonstrates a very comparable imaging performance to speckle decorrelation in Fig. 8 and on other scans in this study (not shown), which is promising to become an alternative to the traditional speckle decorrelation method for imaging skin microvasculature. Additionally, similar to Table 1, to generate 1000 PS-OCTA B-scans within a 3-D scan, the average processing time of GRDN with Protocol 1 is approximately 1/3 to 1/2 of that of the speckle decorrelation algorithm. The processing time of speckle decorrelation would increase significantly when the moving window size (i.e., 5 × 5 pixels) is increased to reduce the noise in decorrelation [31].
Fig. 8. PS-OCTA imaging of a forearm skin region using GRDN with Protocol 1. (a)-(c) Projection images of vessel labels from skin surface to a depth of 200 µm, 375 µm and 650 µm, respectively. (d)-(f) Projection images generated by GRDN with Protocol 1 corresponding to (a)-(c), respectively. (g)-(l) Magnified images of the area outlined by the blue and green squares in (a), (d), (b), (e), (c) and (f), respectively. White scale bars in (a)-(f): 500 µm. Cyan scale bars in (g)-(l): 200 µm.
4. Discussions
This study presented the first adoption of deep learning to PS-OCT for angiography and demonstrated the feasibility for imaging human cutaneous microvasculature. Although there are already multiple conventional OCTA methods available, they are still subject to limitations in terms of vessel contrast, processing speed, artifacts caused by tissue motion and strong surface reflection, among others [1]. Deep learning-based OCTA, as a new OCTA variant, has demonstrated improved processing speed and vessel contrast in recent studies [21,23,32]. Using the more informative Jones matrices provided by PS-OCT, deep learning-based PS-OCTA is promising to further improve the imaging performance.
RDN and GRDN were investigated with different protocols to process the Jones matrix elements, which demonstrated that optimal performance was obtained with GRDN using the combined signal from the four elements. The process to generate the vessel contrast with deep learning can be considered as treating the static tissue in the OCT images as noise for removal, which explains why neural networks effective for denoising, including RDN and GRDN, can work for angiography [22,23]. GRDN’s better performance than RDN for PS-OCTA imaging can also be sustained by its improved denoising capability as previously reported [28]. Overall, GRDN provided very similar image quality to speckle decorrelation with faster and fully automatic processing, which can improve the practicality of PS-OCTA for clinical applications.
The vessel label images were generated using speckle decorrelation, which has been adopted for human skin imaging in multiple studies [2,25,33]. The advantage of this algorithm is the demonstrated robustness even when only two repeated B-scans were acquired at the same lateral locations. This would then help to reduce the motion and the 3-D scan acquisition time, which is beneficial for in vivo imaging. We note that vessel labels might be improved using other OCTA algorithms in combination with a larger number of repeated B-scans, such as the SSAPGA algorithm applied to 48 repeated B-scans [23]. However, an improved imaging setup to further reduce tissue motion would need to be developed first for skin imaging, such as by revising the previously reported solutions [34]. Faster scanning systems could also be used as an alternative method to reduce tissue motion [35–37]. Future work could explore alternative OCTA algorithms based on more repeated B-scans to improve the vessel labels, which could be subsequently fit into the deep learning framework proposed in this study to further enhance the imaging performance of PS-OCTA.
The motion subtraction method, as a straightforward approach, mitigated the artifacts due to residual motion, shown as the reduced lines in the projection image in Fig. 9(b), as compared to Fig. 9(a). However, a close examination also indicated the undesired breaks of the vessels in several local regions, such as those marked by the arrowheads in the magnified image in Fig. 9(d). As our scans only presented minor motion, such breaks were not prominent and thus not expected to create major issues to deep learning-based PS-OCTA. However, when there is strong tissue motion, additional compensation for such breaks will be required for use in conjunction with motion subtraction. For example, Li et al. proposed a two-stage neural network to improve vessel connectivity with the second stage predicting and compensating for the missing vessels in the first stage [38]. Alternatively, more sophisticated methods could be used for more effective reduction of the tissue motion [27,39–42].
Fig. 9. PS-OCTA imaging of a forearm skin region with speckle decorrelation. (a) and (b) Projection images of vessels from skin surface to a depth of 650 µm before and after motion subtraction, respectively. (c) and (d) Magnified images of the area marked by the blue outlines in (a) and (b), respectively. Blue and green arrowheads mark the motion artifacts in (c) and the corresponding breaks of vessels in (d). Scale bars: 500 µm.
The full Jones matrices from PS-OCT in this study provided a complete detection of the backscattered polarized light. When integrated with deep learning, multiple protocols could be used to process the Jones matrices. The results in this study demonstrated that the joint use of the information (Protocols 1 and 3) provided better vessel contrast than the use of the four elements separately (Protocol 2). This is consistent with the previous study using the speckle decorrelation algorithm [20]. Although this finding was shown only for the GRDN network in Section 3.2, it was also observed for the RDN network (results not shown). Furthermore, the lack of corresponding data by a conventional OCT system from the same skin regions impeded a comparison between deep learning-based OCTA and PS-OCTA in this study. Future work could seek to perform such a comparison which may further demonstrate the benefits of PS-OCTA [16,20]. In addition, the three processing protocols for RDN or GRDN used the OCT signal variances in time scale (i.e., repeated B-scans), and thus still required care to minimize tissue motion. Apart from the variance in time, blood flow has also been reported to have a signature in the polarization scale [43,44]. Following work could explore the use of the polarization information captured by the Jones matrix elements with custom neural networks to image vessels using B-scans at a single time point, which would then help to ease the requirement for motion reduction.
5. Conclusion
In conclusion, this study presented the feasibility of deep learning-based PS-OCTA for imaging human cutaneous microvasculature. We assessed two neural networks including RDN and GRDN, and different protocols to process the PS-OCT scans. The results from normal human skin indicated similar performance of the deep learning-based PS-OCTA to that of speckle decorrelation. Deep learning-based PS-OCTA also showed ∼2-3 times faster processing and reduced motion artifacts as compared to speckle decorrelation, and a capability of fully automatic processing, which promises to translate PS-OCTA to preclinical and clinical applications, providing improved vessel contrast and imaging depth over conventional OCTA.
Funding
Fundamental Research Funds for the Central Universities (226-2023-00091); National Natural Science Foundation of China (51875510); Australian Research Council (2020/RA/1/486/2465).
Acknowledgments
This research was funded by the National Natural Science Foundation of China under Grant 51875510. The authors would also like to thank Xiongkai Group Co., Ltd for the financial support. Yuxing Wang acknowledges funding through the Key Research and Development Program of Zhejiang Province of China 2021C02037, Development of Digital Elderly Disability Risk Prevention and Intervention Technologies for Family and Community Scenarios 2022YFC3602603 and National Key Research and Development Program of China 2022YFC3602601. Barry Cense acknowledges funding through the Brain Pool Program from the National Research Foundation of Korea (NRF) funded by the Ministry of Science and ICT under Grant 2022H1D3A2A0109644511 and funding through the Australian Research Council Discovery Project under Grant 2020/RA/1/486/2465.
Disclosures
The authors declare no conflicts of interest related to this article.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. C. L. Chen and R. K. Wang, “Optical coherence tomography based angiography [Invited],” Biomed. Opt. Express 8(2), 1056–1082 (2017). [CrossRef]
2. Y. M. Liew, R. A. McLaughlin, P. Gong, F. M. Wood, and D. D. Sampson, “In vivo assessment of human burn scars through automated quantification of vascularity using optical coherence tomography,” J. Biomed. Opt. 18(6), 061213 (2012). [CrossRef]
3. G. Liu, Y. Jia, A. D. Pechauer, R. Chandwani, and D. Huang, “Split-spectrum phase-gradient optical coherence tomography angiography,” Biomed. Opt. Express 7(8), 2943–2954 (2016). [CrossRef]
4. X. Wei, T. T. Hormel, and Y. Jia, “Phase-stabilized complex-decorrelation angiography,” Biomed. Opt. Express 12(4), 2419–2431 (2021). [CrossRef]
5. I. Gorczynska, J. V. Migacz, R. J. Zawadzki, A. G. Capps, and J. S. Werner, “Comparison of amplitude-decorrelation, speckle-variance and phase-variance OCT angiography methods for imaging the human retina and choroid,” Biomed. Opt. Express 7(3), 911–942 (2016). [CrossRef]
6. G. Querques, E. Borrelli, M. Battista, R. Sacconi, and F. Bandello, “Optical coherence tomography angiography in diabetes: focus on microaneurysms,” Eye 35(1), 142–148 (2021). [CrossRef]
7. C. E. Mendoza-Santiesteban, A. Gonzalez-Garcia, T. R. Hedges, Y. Hernandez-Silva, Y. Columbie-Garbey, L. Fernndez-Cherkasova, R. Santiesteban-Freixas, and S. V. Casali, “Optical coherence tomography for neuro-ophthalmologic diagnoses,” Semin. Ophthalmol. 25(4), 144–154 (2010). [CrossRef]
8. Q. Li, P. Gong, P. H. Ho, B. F. Kennedy, D. A. Mackey, F. K. Chen, and J. Charng, “Evaluating distribution of foveal avascular zone parameters corrected by lateral magnification and their associations with retinal thickness,” Ophthalmol. Sci. 2(2), 100134 (2022). [CrossRef]
9. T. Li, C. J. Liu, and T. Akkin, “Contrast-enhanced serial optical coherence scanner with deep learning network reveals vasculature and white matter organization of mouse brain,” Neurophotonics 6(03), 1 (2019). [CrossRef]
10. M. Liu and W. Drexler, “Optical coherence tomography angiography and photoacoustic imaging in dermatology,” Photochem. Photobiol. Sci. 18(5), 945–962 (2019). [CrossRef]
11. A. J. Deegan, W. Wang, S. Men, Y. Li, S. Song, J. Xu, and R. K. Wang, “Optical coherence tomography angiography monitors human cutaneous wound healing over time,” Quant. Imaging Med. Surg. 8(2), 135–150 (2018). [CrossRef]
12. P. Gong, S. Es, F. M. Wood, D. D. Sampson, and R. A. Mclaughlin, “Optical coherence tomography angiography for longitudinal monitoring of vascular changes in human cutaneous burns,” Exp. Dermatol. 25(9), 722–724 (2016). [CrossRef]
13. B. Cense, Q. Wang, S. Lee, L. Zhao, A. E. Elsner, C. K. Hitzenberger, and D. T. Miller, “Henle fiber layer phase retardation measured with polarization-sensitive optical coherence tomography,” Biomed. Opt. Express 4(11), 2296–2306 (2013). [CrossRef]
14. X. Zhou, S. Maloufi, D. C. Louie, N. Zhang, Q. Liu, T. K. Lee, and S. Tang, “Investigating the depolarization property of skin tissue by degree of polarization uniformity contrast using polarization-sensitive optical coherence tomography,” Biomed. Opt. Express 12(8), 5073–5088 (2021). [CrossRef]
15. P. Gong, F. M. Wood, D. D. Sampson, and R. A. Mclaughlin, “Imaging of skin birefringence for human scar assessment using polarization-sensitive optical coherence tomography aided by vascular masking,” J. Biomed. Opt. 19(12), 126014 (2014). [CrossRef]
16. P. Tang and R. K. Wang, “Polarization sensitive optical coherence tomography for imaging microvascular information within living tissue without polarization-induced artifacts,” Biomed. Opt. Express 11(11), 6379–6388 (2020). [CrossRef]
17. J. F. de Boer, C. K. Hitzenberger, and Y. Yasuno, “Polarization sensitive optical coherence tomography – a review [Invited],” Biomed. Opt. Express 8(3), 1838–1873 (2017). [CrossRef]
18. B. Baumann, “Polarization sensitive optical coherence tomography: a review of technology and applications,” Appl. Sci. 7(5), 474 (2017). [CrossRef]
19. Q. Li, K. Karnowski, P. B. Noble, A. Cairncross, A. James, M. Villiger, and D. D. Sampson, “Robust reconstruction of local optic axis orientation with fiber-based polarization-sensitive optical coherence tomography,” Biomed. Opt. Express 9(11), 5437–5455 (2018). [CrossRef]
20. P. Gong, Q. Li, Q. Wang, K. Karnowski, and D. D. Sampson, “Jones matrix-based speckle-decorrelation angiography using polarization-sensitive optical coherence tomography,” J. Biophotonics 13(9), e202000007 (2020). [CrossRef]
21. G. Kim, J. Kim, W. J. Choi, C. Kim, and S. Lee, “Deep learning enables accelerated optical coherence tomography angiography,” Proc. SPIE 11971, 1197108 (2022). [CrossRef]
22. C. S. Lee, A. J. Tyring, Y. Wu, S. Xiao, A. S. Rokem, N. P. DeRuyter, Q. Zhang, A. Tufail, R. K. Wang, and A. Y. Lee, “Generating retinal flow maps from structural optical coherence tomography with artificial intelligence,” Sci. Rep. 9(1), 5694 (2019). [CrossRef]
23. Z. Jiang, Z. Huang, B. Qiu, X. Meng, Y. You, X. Liu, G. Liu, C. Zhou, K. Yang, A. Maier, Q. Ren, and Y. Lu, “Comparative study of deep learning models for optical coherence tomography angiography,” Biomed. Opt. Express 11(3), 1580–1597 (2020). [CrossRef]
24. B. Baumann, W. Choi, B. Potsaid, D. Huang, J. S. Duker, and J. G. Fujimoto, “Swept source/Fourier domain polarization sensitive optical coherence tomography with a passive polarization delay unit,” Opt. Express 20(9), 10229–10230 (2012). [CrossRef]
25. P. Gong, S. Es’haghian, K. A. Harms, A. Murray, S. Rea, B. F. Kennedy, F. M. Wood, D. D. Sampson, and R. A. McLaughlin, “Optical coherence tomography for longitudinal monitoring of vasculature in scars treated with laser fractionation,” J. Biophotonics 9(6), 626–636 (2016). [CrossRef]
26. G. Chen, W. Wang, and Y. Li, “Comparative study of OCTA algorithms with a high-sensitivity multi-contrast Jones matrix OCT system for human skin imaging,” Biomed. Opt. Express 13(9), 4718–4736 (2022). [CrossRef]
27. D. W. Wei, A. J. Deegan, and R. K. Wang, “Automatic motion correction for in vivo human skin optical coherence tomography angiography through combined rigid and nonrigid registration,” J. Biomed. Opt. 22(6), 066013 (2017). [CrossRef]
28. D. W. Kim, J. R. Chung, and S. W. Jung, “GRDN:Grouped residual dense network for real image denoising and GAN-based real-world noise modeling,” in 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019), pp. 2086–2094.
29. R. Girshick, “Fast R-CNN,” in IEEE International Conference on Computer Vision (ICCV) (2015), pp. 1440–1448.
30. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Trans. Image Process. 13(4), 600–612 (2004). [CrossRef]
31. Q. Wang, P. Gong, B. Cense, and D. D. Sampson, “Short-time series optical coherence tomography angiography and its application to cutaneous microvasculature,” Biomed. Opt. Express 10(1), 293–307 (2019). [CrossRef]
32. Z. Jiang, Z. Huang, Y. You, M. Geng, X. Meng, B. Qiu, L. Zhu, M. Gao, J. Wang, C. Zhou, Q. Ren, and Y. Lu, “Rethinking the neighborhood information for deep learning-based optical coherence tomography angiography,” Med. Phys. 49(6), 3705–3716 (2022). [CrossRef]
33. Y. Zhang, H. Li, T. Cao, R. Chen, H. Qiu, Y. Gu, and P. Li, “Automatic 3D adaptive vessel segmentation based on linear relationship between intensity and complex-decorrelation in optical coherence tomography angiography,” Quant. Imaging Med. Surg. 11(3), 895–906 (2020). [CrossRef]
34. Y. M. Liew, R. A. McLaughlin, F. M. Wood, and D. D. Sampson, “Motion correction of in vivo three-dimensional optical coherence tomography of human skin using a fiducial marker,” Biomed. Opt. Express 3(8), 1774–1786 (2012). [CrossRef]
35. T. Klein, W. Wieser, L. Reznicek, A. Neubauer, A. Kampik, and R. Huber, “Multi-MHz retinal OCT,” Biomed. Opt. Express 4(10), 1890–1908 (2013). [CrossRef]
36. E. Auksorius, D. Borycki, P. Wegrzyn, B. L. Sikorski, K. Lizewski, I. Zickiene, M. Rapolu, K. Adomavicius, S. Tomczewski, and M. Wojtkowski, “Spatio-temporal optical coherence tomography provides full thickness imaging of the chorioretinal complex,” iScience 25(12), 105513 (2022). [CrossRef]
37. D. J. Fechtig, B. Grajciar, T. Schmoll, C. Blatter, R. M. Werkmeister, W. Drexler, and R. A. Leitgeb, “Line-field parallel swept source MHz OCT for structural and functional retinal imaging,” Biomed. Opt. Express 6(3), 716–735 (2015). [CrossRef]
38. A. Li, C. Du, and Y. Pan, “Deep-learning-based motion correction in optical coherence tomography angiography,” J. Biophotonics 14(12), e202100097 (2021). [CrossRef]
39. T. Zhang, K. Zhou, H. R. Rocliffe, A. Pellicoro, J. L. Cash, W. Wang, Z. Wang, C. Li, and Z. Huang, “Windowed eigen-decomposition algorithm for motion artifact reduction in optical coherence tomography-based angiography,” Appl. Sci. 13(1), 378 (2022). [CrossRef]
40. Y. Zhang, W. Gao, and C. Xie, “Fourier spatial transform-based method of suppressing motion noises in OCTA,” Opt. Lett. 47(17), 4544–4547 (2022). [CrossRef]
41. J. Fan, Y. He, P. Wang, G. Liu, and G. Shi, “Interplane bulk motion analysis and removal based on normalized cross-correlation in optical coherence tomography angiography,” J. Biophotonics 13(7), e202000046 (2020). [CrossRef]
42. R. Zuo, Y. Wang, K. Irsch, and J. U. Kang, “Higher-order regression three-dimensional motion-compensation method for real-time optical coherence tomography volumetric imaging of the cornea,” J. Biomed. Opt. 27(06), 066006 (2022). [CrossRef]
43. B. G. de Grooth, L. W. M. M. Terstappen, G. J. Puppels, and J. Greve, “Light-scattering polarization measurements as a new parameter in flow cytometry,” Cytometry 8(6), 539–544 (1987). [CrossRef]
44. A. Julin and J. Xia, “Polarization enhanced laser speckle contrast imaging for vascular dynamic study,” Proc. SPIE 10063, 1006315 (2017). [CrossRef]
