Fluorescence imaging is well suited to provide image guidance during resections in oncologic and vascular surgery. However, the distorting effects of tissue optical properties on the emitted fluorescence are poorly compensated for on even the most advanced fluorescence image guidance systems, leading to subjective and inaccurate estimates of tissue fluorophore concentrations. Here we present a novel fluorescence imaging technique that performs real-time (i.e., video rate) optical property corrected fluorescence imaging. We perform full field of view simultaneous imaging of tissue optical properties using Single Snapshot of Optical Properties (SSOP) and fluorescence detection. The estimated optical properties are used to correct the emitted fluorescence with a quantitative fluorescence model to provide quantitative fluorescence-Single Snapshot of Optical Properties (qF-SSOP) images with less than 5% error. The technique is rigorous, fast, and quantitative, enabling ease of integration into the surgical workflow with the potential to improve molecular guidance intraoperatively.
© 2017 Optical Society of America
Fluorescence image-guided surgery has demonstrated a strong potential for providing improved benefit to patients . However, the current implementation in clinical fluorescence imaging systems does not account for the distorting effects of tissue optical properties. As a result, the imaging of fluorophore levels is qualitative in nature and therefore highly prone to subjectivity [1–3]. In turn, subjectivity can lead to intraoperative assessments of ‘no fluorescence present’ in areas of higher attenuation, and as such surgeons are prone to leaving significant levels of non-visually fluorescent tumor tissue unresected [4, 5]. This can be particularly crucial near the end of a resection when the surgeon is surveying the surgical cavity for identifying tumor margins. Even more important to the field of fluorescence-guided surgery in general, the qualitative nature of fluorescence signals makes their “measurement” not repeatable within the same patient, from patient to patient, surgeon to surgeon, and hospital to hospital. Consequently, this lack of quantification makes fluorescence levels not interpretable and consensus cannot be defined as to the meaning of a particular fluorescence value. This alone has profound implications in terms of clinical value and translation.
The varying effects of tissue optical properties significantly impact light tissue interactions during fluorescence imaging of tissue, which lead to qualitative, relative assessments of the emitted fluorescence. Two broad categories of methodologies for addressing this non-linear phenomenon are: model based corrections or empiric ratiometric techniques . The former provides direct estimates of tissue optical properties and applies them in a model of the fluorescence, and the latter, uses a ratio of the raw fluorescence over the tissue reflectance to correct the raw fluorescence. Model based techniques to date have been mostly successful with single point probes. Recent studies have demonstrated some advances in model based methods whereas the ratiometric techniques, owing to their simplicity, have had more success translating into imaging techniques [6–8].
The raw fluorescence undergoes attenuation due to the heterogeneous effects of tissue optical properties at both the excitation and emission wavelengths. A significant effort has been dedicated at developing strategies that account for these effects to produce either attenuation-corrected, semi-quantitative, or quantitative (i.e., intrinsic, optical property corrected) fluorescence estimates [4, 8–10]. The basic concept for estimation of the quantitative fluorescence, fxm, assumes it is a function of the raw fluorescence, Fxm, and an unknown correction factor, X,Eq. (1) uses an empiric correction of the fluorescence as ratio of the raw fluorescence over the reflectance (Rx) at the excitation wavelength, Fxm/Rx [11–13]. More rigorous techniques use model-based correction methods that depend on a prior estimates of the diffuse reflectance and tissue optical properties to derive a correction factor, X, and ultimately, the quantitative fluorescence, fxm. Here we used a previously validated algorithm  noted in Eq. (2) that has been shown to correct for the distorting effects of tissue optical properties to produce quantitative estimates of tissue fluorescence,Equation (2) is a function of the raw fluorescence, the absorption coefficient at the excitation wavelength (µa,x), and the diffuse reflectance at the excitation (Rx) and emission (Rm) wavelengths. Estimation of the diffuse reflectance and derived tissue optical properties provides the necessary information to derive the correction factor, X.
An ideal optical property corrected fluorescence technique would provide real-time, i.e. video rate (> 10 fps), images of the full surgical field of view; would collect the necessary information to estimate tissue optical property maps of the surgical cavity to implement a rigorous model based fluorescence correction method; and would estimate optical property corrected fluorescence images co-registered in both space and time. A point-based correction method has been readily adapted to imaging and has had recent success using spatial frequency domain imaging (SFDI) [8, 10, 15]. SFDI offers a powerful way to acquire the tissue reflectance properties simultaneously for every pixel of an image, supplying the required parameters to calculate the correction factor X. However, while the raw fluorescence image can be acquired with a single frame, standard SFDI processing requires typically 6 images to generate tissue property maps , slowing down acquisition rate possible for acquiring the fluorescence correction factor [8, 10].
Here we present a technique using single snapshot of optical properties (SSOP) imaging  to image the tissue optical properties in real time. Single snapshot of optical properties (SSOP) is an acquisition method based on SFDI that allows for video-rate widefield acquisition of tissue optical properties, including sample profile acquisition and correction . This technique can therefore match the acquisition speed of fluorescence imaging while also providing quantitative reflectance parameters necessary for calculating correction factor X for real-time imaging . Our imaging system was modified to accommodate simultaneous acquisition of the tissue fluorescence to derive optical property corrected maps of fluorescence. We call this technique quantitative fluorescence Single Snapshot of Optical Properties (qF-SSOP) imaging. qF-SSOP represents a significant advancement in fluorescence image guidance as it enables real-time optical property corrected fluorescence imaging, thereby allowing its integration within the surgical workflow and the investigation of the benefits of quantitative fluorescence imaging in terms of clinical decision making.
2. Materials and methods
2.1 Correction method
As explained in the introduction, we used a previously validated algorithm  noted in Eq. (2) that has been shown to correct for the distorting effects of tissue optical properties to produce quantitative estimates of tissue fluorescence. The added correction terms for this algorithm include the absorption coefficient at the excitation wavelength, the loss of diffuse reflectance at the excitation wavelength, and the diffuse reflectance of the emission wavelength. Estimation of the diffuse reflectance and derived tissue optical properties provides the necessary information to derive the correction factor.
2.2 Imaging system
In this work, we designed and validated an imaging system enabled for simultaneous patterned illumination and fluorescence imaging. Patterned illumination is required for SSOP imaging at one predetermined frequency. Previous work by our group [17–19], validated the use of single patterned illumination with subsequent line processing in the frequency domain for extraction of optical properties in real-time. This was a key improvement arising from the standard spatial frequency domain imaging (SFDI) technique which uses patterned illumination at multiple frequencies and multiple phases (typically 6 total images) for estimation of the diffuse reflectance and tissue optical properties . SSOP imaging uses one single frequency without need of multiple phases. As such, it requires only one (1) image to extract the tissue optical properties, in comparison to the standard 6 images required for SFDI. Our system performs SSOP imaging with patterned illumination at the excitation (λx = 760 nm) and emission (λm = 808 nm) wavelengths, and simultaneously excites tissue to collect spatially and temporally co-registered fluorescence emissions (λ > 815 nm) (Fig. 1(a), 1(b)). We can subsequently use SSOP techniques to estimate the diffuse reflectance (Rx, Rm), and derive the tissue optical properties – absorption (μa,x, μa,m) and reduced scattering (μs,x’, μs,m’) - at both wavelengths (Fig. 1(b)) by using a rapid lookup Table .
2.3 Validation experiments
We fabricated tissue simulating phantoms using silicone oil and one tenth part methanol as the medium. India ink (Blick Art Materials, Boston MA) was used as the main absorber, and titanium oxide (Atlantic Equipment Engineers, Bergenfield NJ) as the main scatterer. ZW800-1 was used as the main fluorophore and functions as an NIR fluorophore that typifies other similar clinical NIR compounds such as Indocyanine Green (ICG) [21, 22]. Fifteen (15) phantoms at varying absorption and scattering properties were fabricated in the range of μa,x = 0.05 – 0.20 mm−1and μs,x’ = 1.0 – 2.1 mm−1 (Table 1). Each phantom was made with a fluorophore concentration of 5 μM ZW800-1. The set of 15 phantoms of 5 μM ZW800-1 were imaged sequentially and used to assess imaging methods across a large range of optical properties.
For a video demonstration, two 8 μM ZW800-1 phantoms were made with contrasting albedos, a light phantom with absorption and reduced scattering of [0.05, 2.0] mm−1 and a dark phantom of [0.15, 1.0] mm−1. These phantoms were translated and rotated while video was acquired. For each experiment, raw fluorescence, F/R, and qF-SSOP methods were compared in their accuracy and precision for retrieving a constant fluorescence signal while phantom optical properties varied.
Figure 2 presents phantom results across all combinations of scattering and absorption for both the raw (Fig. 2(a)), the F/R (Fig. 2(b)), and qF-SSOP (Fig. 2(c)). Optical property measurements were made with a spatial frequency pattern of 0.2 mm−1, a high enough frequency to successfully perform SSOP imaging  and enable correction of fluorescence from attenuating effects of optical properties in this study [15, 23]. The depth to which tissue optical properties can be estimated will depend on the spatial frequency used and the depth to which tissue fluorophores can be excited will depend on the excitation light (e.g., the longer wavelength the deeper the penetration of excitation light), fluorescence emissions, and varying tissues optical properties. In Fig. 2(a) the raw fluorescence signal intensity of 15 phantoms without correction for varying optical properties demonstrates a qualitatively clear difference in the fluorescence signal in phantoms with highest absorption and lowest scattering (lower left corner) compared to phantoms with lowest absorption and highest scattering (upper right corner). The difference between the highest and lowest fluorescence signal phantoms varied by 6.9x (st.dev 2.82 a.u., range: 1.7 - 11.7 a.u.) using the raw fluorescence estimates for ZW800-1 levels. Figure 2(b) shows the same 15 phantoms after using a standard literature empiric correction of F/R for variation in tissue optical properties. The F/R results demonstrate a reduced the 6.9x range of fluorescence signal to 2.3x in estimated levels (st.dev 1.39 a.u., range: 3.63 – 8.39 a.u.). Figure 2(c) shows the same 15 phantoms after using qF-SSOP. In comparison to both the raw and F/R fluorescence estimates, the qF-SSOP results demonstrate a minimal difference of ~10%, or 0.1x between lowest and highest levels and the true concentration in estimated levels (st.dev 0.32 μM, range: 4.55 – 5.66 μM).
We analyzed each phantom to quantify the error in fluorescence estimates. Figure 3(a) shows the raw fluorescence, Fig. 3(b), the F/R fluorescence, and Fig. 3(c) qF-SSOP. A dotted red line denotes the known fluorescence level of 5 μM. The mean percentage error (mPE) for the raw fluorescence estimates was 43.0% with a coefficient of variation of 80.1% (st.dev: 34.5%, range: 1.2% - 134%). A moderate improvement of approximately 50% in the estimates is noted with the standard F/R fluorescence estimates of 21.6% with a coefficient of variation of 75.7% (st.dev: 16.3%, range: 2.1% - 67.9%). Meanwhile, for qF-SSOP a much larger improvement was produced with a mPE of 4.8% and a coefficient of variation of 6.4% (st.dev: 4.0%, range: 0.1% - 13.2%).
Figure 4 presents a single frame from the real-time acquisition of two phantoms with distinct optical properties but the same level of fluorophore (8 μM), acquired with 0.24 mm−1 spatial frequency and flat-fielded. The video shows a real-time image of (a) the raw fluorescence, (b) F/R fluorescence, and (c) qF-SSOP. As expected, Fig. 4(a) and 4(b) show a marked difference in the detected fluorescence, whereas Fig. 4(c) shows no clear difference in the estimated fluorophore levels. Note that both (a) and (b) have arbitrary units with max fluorescence set to 85%, and (c) qF-SSOP enables units of molecular concentration .
These promising results demonstrate a real-time imaging technique that acquires maps co-registered in space and time of tissue optical properties and raw fluorescence emissions followed by a model-based correction to estimate the quantitative fluorescence. qF-SSOP provides a means to collect video rate images that are corrected for the distorting effects of tissue optical properties on the fluorescence emissions while concurrently collecting the absorption and scattering images for the same tissue. Our study uses one particular correction method noted in Eq. (2), but can be applied to a number of other rigorous model-based correction methods that depend on a priori knowledge of the tissue optical properties. We chose this model for its simplicity and successful implementation in previous studies for brain tumor surgery .
This study used flat phantoms without the use of profilometry, or curvature correction methods. We have previously described a technique for curvature correction , and our group is actively optimizing such methods to implement in our next iteration of this technology for ex vivo and in vivo studies on rodent glioma models and clinical surgeries having demonstrated the potential to benefit from quantitative fluorescence imaging. Our technique uses SSOP, which has previously been validated to provide estimates equivalent in accuracy to the standard SFDI method . Further, the current qF-SSOP system employs single band pass and long pass filters for collection of the reflectance and emitted fluorescence, and can be adapted to other fluorescent contrast agents if necessary . In this work we used a hydrophilic and zwitterionic NIR fluorophore, ZW800-1, which is similar to other NIR compounds such as ICG. Our system is built for NIR imaging, but in principle, any fluorophore could be used, including PpIX, which is currently used for brain tumor surgery.
There are several confounding factors for determining the limits of qF-SSOP technique which are factors to consider when doing SFDI and/or fluorescence imaging in conjunction or separately. For the model used, Eqs. (1) and (2) fail in the limit of the raw fluorescence Fxm approaching 0 or in the limit of the corrective factor X approaching infinity. This model will fail with excitation leakage, specular reflection, and signal in the noise floor. With imaging and illumination parameters held constant, extending Fig. 3 to even lower albedo phantoms results in Fxm reaching the noise floor. For the SSOP processing considerations, previous studies have demonstrated the extension of the SFDI model into low albedo phantoms , but phantom #13 in Fig. 2(c) shows that signal reaching the noise floor can result in demodulation artifacts and should be avoided.
Further considerations must be given to acquisition speed, spatial resolution, and depth sensitivity. Fundamental limits on acquisition speed and spatial resolution are governed by the fluorescence imaging system (and the diffusive sample), which means high-definition video is possible. Wide-field SFDI measurements have demonstrated optical property resolution below 0.5 mm , and as such, in a heterogeneous biological sample the accuracy of our technique would be limited to 0.5 mm. We have previously demonstrated improved diagnostic accuracy and utility with a quantitative intraoperative probe in heterogeneous in vivo samples with a diameter of ~1 mm . Increasing the spatial frequency used enables imaging of smaller fields of view, increasing the fluorescence contrast [15, 23], but limiting depth sensitivity, lowering overall signal, and slowing acquisition time. In this study, all data was collected at 500ms exposure time and all processing was performed offline with hardware and software that was not optimized given the proof of concept nature of this study. Exposure time can be decreased and is a function of the hardware (e.g., camera sensitivity), excitation light source power, varying tissue optical properties, and fluorophore concentrations which can be optimized for improved frame rate acquisition. Further, current algorithm speeds for demodulation (~10ms), optical property generation (~10ms), and image correction (<1ms) show potential for video-rate display for an optimized system.
The surgeon utilizes multiple adjuncts (e.g., microscope, MRI navigation, ultrasound) in addition to their clinico-anatomic knowledge. The fluorescent biomarker diagnostic thresholds used will be dependent on the biological process under study, the fluorophore under investigation, and the surgical/clinical decision-making regarding the need to increase sensitivity at the cost of specificity or vice-versa when using qF-SSOP as a surgical adjunct. For example, we have previously shown that quantitative estimates of protoporphyrin IX can be used for improved intraoperative tissue diagnostics, and by using different cut-off values can vary specificity, sensitivity and overall classification accuracy .
This technique is limited in that it uses a voxel average prediction of the optical properties. This is a fundamental property of using SFDI without doing tomographic reconstructions, which requires many acquisition frequencies and extensive computation time . With the topographical averaging, a recently developed rapid lookup table can be utilized to potentially provide real-time feedback for the proposed real-time qF-SSOP imaging method . Any extensive time requirements greatly inhibit a technique’s use for surgical guidance and its potential for aiding intraoperative intervention. The current implementation of this technique does not offer spectrally resolved imaging, and as such is not capable of separating the contributions of multiple fluorophores. In future studies we plan to integrate prior spectrally resolved acquisition methodologies to enable spectral unmixing of multiple fluorophores.
In conclusion, with the use of SSOP for real-time estimates of tissue optical properties, we demonstrate a novel imaging system and technique for rigorous and model-based quantitative fluorescence imaging. Unlike previous studies, qF-SSOP provides co-registered tissue optical property maps and quantitative fluorescence images based on a rigorous model based method and in real-time. This novel technique can be easily integrated into the surgical workflow to further guide tumor resection, and enable more accurate molecular guidance.
The authors declare that there are no conflicts of interest related to this article.
National Institute of Health (NIH) (K01-DK-093603, F31-DK-105839, R01-EB-011523); National Science Foundation (NSF) (DGE-1247312); France Life Imaging; University of Strasbourg IdEx and ICube Laboratory.
Pablo A. Valdes is supported through the Harvard Medical School Department of Neurosurgery at Brigham and Women’s/Boston Children’s Hospitals.
References and links
1. W. Stummer, U. Pichlmeier, T. Meinel, O. D. Wiestler, F. Zanella, H. J. Reulen, and ALA-Glioma Study Group, “Fluorescence-guided surgery with 5-aminolevulinic acid for resection of malignant glioma: a randomised controlled multicentre phase III trial,” Lancet Oncol. 7(5), 392–401 (2006). [CrossRef] [PubMed]
2. C. M. Gardner, S. L. Jacques, and A. J. Welch, “Fluorescence spectroscopy of tissue: recovery of intrinsic fluorescence from measured fluorescence,” Appl. Opt. 35(10), 1780–1792 (1996). [CrossRef] [PubMed]
4. P. A. Valdés, F. Leblond, V. L. Jacobs, B. C. Wilson, K. D. Paulsen, and D. W. Roberts, “Quantitative, spectrally-resolved intraoperative fluorescence imaging,” Sci. Rep. 2, 798 (2012). [CrossRef] [PubMed]
5. P. A. Valdés, F. Leblond, A. Kim, B. T. Harris, B. C. Wilson, X. Fan, T. D. Tosteson, A. Hartov, S. Ji, K. Erkmen, N. E. Simmons, K. D. Paulsen, and D. W. Roberts, “Quantitative fluorescence in intracranial tumor: implications for ALA-induced PpIX as an intraoperative biomarker,” J. Neurosurg. 115(1), 11–17 (2011). [CrossRef] [PubMed]
6. S. D. Konecky, C. M. Owen, T. Rice, P. A. Valdés, K. Kolste, B. C. Wilson, F. Leblond, D. W. Roberts, K. D. Paulsen, and B. J. Tromberg, “Spatial frequency domain tomography of protoporphyrin IX fluorescence in preclinical glioma models,” J. Biomed. Opt. 17(5), 056008 (2012). [CrossRef] [PubMed]
7. P. A. Valdes, V. L. Jacobs, B. C. Wilson, F. Leblond, D. W. Roberts, and K. D. Paulsen, “System and methods for wide-field quantitative fluorescence imaging during neurosurgery,” Opt. Lett. 38(15), 2786–2788 (2013). [CrossRef] [PubMed]
8. M. Sibai, I. Veilleux, J. T. Elliott, F. Leblond, and B. C. Wilson, “Quantitative spatial frequency fluorescence imaging in the sub-diffusive domain for image-guided glioma resection,” Biomed. Opt. Express 6(12), 4923–4933 (2015). [CrossRef] [PubMed]
9. B. Yang, M. Sharma, and J. W. Tunnell, “Attenuation-corrected fluorescence extraction for image-guided surgery in spatial frequency domain,” J. Biomed. Opt. 18(8), 080503 (2013). [CrossRef] [PubMed]
10. R. B. Saager, D. J. Cuccia, S. Saggese, K. M. Kelly, and A. J. Durkin, “Quantitative fluorescence imaging of protoporphyrin IX through determination of tissue optical properties in the spatial frequency domain,” J. Biomed. Opt. 16(12), 126013 (2011). [CrossRef] [PubMed]
11. G. M. van Dam, G. Themelis, L. M. Crane, N. J. Harlaar, R. G. Pleijhuis, W. Kelder, A. Sarantopoulos, J. S. de Jong, H. J. Arts, A. G. van der Zee, J. Bart, P. S. Low, and V. Ntziachristos, “Intraoperative tumor-specific fluorescence imaging in ovarian cancer by folate receptor-α targeting: first in-human results,” Nat. Med. 17(10), 1315–1319 (2011). [CrossRef] [PubMed]
12. V. Ntziachristos, G. Turner, J. Dunham, S. Windsor, A. Soubret, J. Ripoll, and H. A. Shih, “Planar fluorescence imaging using normalized data,” J. Biomed. Opt. 10(6), 064007 (2005). [CrossRef] [PubMed]
13. G. Themelis, J. S. Yoo, K. S. Soh, R. Schulz, and V. Ntziachristos, “Real-time intraoperative fluorescence imaging system using light-absorption correction,” J. Biomed. Opt. 14(6), 064012 (2009). [CrossRef] [PubMed]
14. A. Kim, M. Khurana, Y. Moriyama, and B. C. Wilson, “Quantification of in vivo fluorescence decoupled from the effects of tissue optical properties using fiber-optic spectroscopy measurements,” J. Biomed. Opt. 15(6), 067006 (2010). [CrossRef] [PubMed]
16. D. J. Cuccia, F. Bevilacqua, A. J. Durkin, F. R. Ayers, and B. J. Tromberg, “Quantitation and mapping of tissue optical properties using modulated imaging,” J. Biomed. Opt. 14(2), 024012 (2009). [CrossRef] [PubMed]
20. S. Gioux, A. Mazhar, B. T. Lee, S. J. Lin, A. M. Tobias, D. J. Cuccia, A. Stockdale, R. Oketokoun, Y. Ashitate, E. Kelly, M. Weinmann, N. J. Durr, L. A. Moffitt, A. J. Durkin, B. J. Tromberg, and J. V. Frangioni, “First-in-human pilot study of a spatial frequency domain oxygenation imaging system,” J. Biomed. Opt. 16(8), 086015 (2011). [CrossRef] [PubMed]
21. H. S. Choi, S. L. Gibbs, J. H. Lee, S. H. Kim, Y. Ashitate, F. Liu, H. Hyun, G. Park, Y. Xie, S. Bae, M. Henary, and J. V. Frangioni, “Targeted zwitterionic near-infrared fluorophores for improved optical imaging,” Nat. Biotechnol. 31(2), 148–153 (2013). [CrossRef] [PubMed]
22. H. S. Choi, K. Nasr, S. Alyabyev, D. Feith, J. H. Lee, S. H. Kim, Y. Ashitate, H. Hyun, G. Patonay, L. Strekowski, M. Henary, and J. V. Frangioni, “Synthesis and in vivo fate of zwitterionic near-infrared fluorophores,” Angew. Chem. Int. Ed. Engl. 50(28), 6258–6263 (2011). [CrossRef] [PubMed]
23. A. Mazhar, D. J. Cuccia, S. Gioux, A. J. Durkin, J. V. Frangioni, and B. J. Tromberg, “Structured illumination enhances resolution and contrast in thick tissue fluorescence imaging,” J. Biomed. Opt. 15(1), 010506 (2010). [CrossRef] [PubMed]
24. T. A. Erickson, A. Mazhar, D. Cuccia, A. J. Durkin, and J. W. Tunnell, “Lookup-table method for imaging optical properties with structured illumination beyond the diffusion theory regime,” J. Biomed. Opt. 15(3), 036013 (2010). [CrossRef] [PubMed]