Open Access
Add to BibSonomyAbstract
This paper presents a two-frequency binary phase-shifting technique to measure three-dimensional (3D) absolute shape of beating rabbit hearts. Due to the low contrast of the cardiac surface, the projector and the camera must remain focused, which poses challenges for any existing binary method where the measurement accuracy is low. To conquer this challenge, this paper proposes to utilize the optimal pulse width modulation (OPWM) technique to generate high-frequency fringe patterns, and the error-diffusion dithering technique to produce low-frequency fringe patterns. Furthermore, this paper will show that fringe patterns produced with blue light provide the best quality measurements compared to fringe patterns generated with red or green light; and the minimum data acquisition speed for high quality measurements is around 800 Hz for a rabbit heart beating at 180 beats per minute.
© 2013 Optical Society of America
1. Introduction
Visualizing 3D geometry of the heart with the corresponding optical cardiac mapping of the electrical activities with voltage- and/or calcium-sensitive dyes is a very powerful tool for studying complex rhythms and arrhythmias. Recent studies demonstrated that by immobilizing the heart to capture one panoramic 3D model, and then map the time-varying fluorescence images onto this static 3D model is very useful [1]. However, it is impossible to know the mechanical function if the heart is immobilized. Therefore, accurate measure of the 3D dynamic surface geometry of the beating heart is vital to improving our understanding of mechanics in health and disease. Because the optical properties of heart surfaces (i.e., partially shiny and highly absorbing), it is quite challenging for an optical method to perform such measurement with minimum error. The measurement becomes even more challenging when the heart is rapidly beating (e.g., around 180 beats/min, or 3 Hz, for an explanted rabbit heart) since a very high-speed 3D shape measurement system is required. This paper presents a novel 3D shape measurement technique to conquer these challenges.
Optical methods have been extensively adopted in measuring various objects (e.g., human faces [2], honeybee [3]). Phase-shifting-based fringe analysis techniques have the advantage of measuring challenging surfaces (e.g. low surface contrast) while achieving high spatial and temporal resolution. Digital fringe projection (DFP) methods have been extensively adopted due to the flexibility of system development and high achievable speed [2]. However, it is extremely difficult for conventional DFP techniques to achieve faster than 120 Hz which is the maximum refresh rate of a digital video projector. Therefore, it is difficult for such techniques to precisely measure beating hearts, like that of the rabbit.
Binary defocusing has enabled tens of kHz rate 3D shape measurement speed because it only requires 1-bit structured patterns rather than 8-bit grayscale patterns [4]. However, measurement error is large if the projector is not properly defocused [5]. Unfortunately, due to the low contrast of the heart surface, the projector has to be nearly focused to achieve the highest possible fringe contrast. Thus, measurement accuracy and determination of mechanical parameters like surface strain becomes compromised. Techniques based on 1-D pulse width modulation (PWM) [6, 7] have been proposed to improve the binary defocusing technique. They are proved to be successful when the fringe stripes are narrow, yet fail when fringe stripes are wide. The PWM technique has been successfully applied to measure beating rabbit hearts, but requires a gray-coding method and employs a computational framework to recover absolute 3D shape [8]. As a result of the gray-coding method, more than six (ten in our previous study) images are required to recover one 3D frame. For high-speed applications, it is desirable to use fewer number of fringe patterns to reduce motion induced artifacts.
Dithering techniques have been successfully used in printing technology to represent grayscale images with binary structured images [9]. Recently, we have introduced the ordered-Bayer dithering technique to the 3D shape measurement field to generate high quality fringe patterns even for wide fringe stripes [10]. We found that the dithering technique cannot substantially improve measurement quality when the fringe stripe is narrow, yet it provides the potential to generate high-quality sinusoidal fringe patterns with low spatial frequency using the binary defocusing method.
This paper proposes to combine the PWM technique with the dithering technique to obtain absolute phase with only six fringe patterns. Specifically, we utilize the optimal pulse width modulation (OPWM) technique [6] to generate three high-frequency sinusoidal fringe patterns, and the dithering technique to produce three low-frequency fringe sinusoidal fringe patterns. By adopting a two-frequency phase-shifting technique, the absolute phase is determined. Instead of using the ordered-Bayer dithering technique, a better technique called error-diffusion dithering is introduced to generate better low-frequency fringe patterns. Since measuring beating rabbit hearts is the ultimate goal of this study, this paper also addresses some practical considerations for measuring the cardiac deformation. First, we investigation the effect of wavelength of the projected fringe patterns on measurement quality. Optical properties of cardiac tissue (i.e., absorption, reflectance, and transmission) play an important role in measurement error. Experimental data will be presented to show the differences for beating hearts and formalin-fixed hearts. Additionally, we investigate the the minimum measurement speed to minimize both data size and motion induced artifacts.
Section 2 explains the principles of proposed techniques. Section 3 shows some experimental results, and Section 4 summarizes the paper.
2. Principle
2.1. Two-frequency phase-shifting technique for absolute phase retrieval
Phase-shifting methods have been extensively adopted in optical metrology because of their measurement speed and accuracy. Over the years, a variety of phase-shifting algorithms have been developed, that include three-step, four-step, and least-square algorithms [11]. For high-speed 3-D shape measurement, a three-step phase-shifting algorithm with a phase shift of 2π/3 is commonly used. The three fringe images can be described as:
Where I′(x, y) is the average intensity, I″(x, y) the intensity modulation, and ϕ(x, y) the phase to be solved for. Simultaneously solving Eq. (1)–(3), the phase can be obtained This equation provides the wrapped phase with 2π discontinuities. A spatial or temporal phase unwrapping algorithm can be applied to obtain continuous phase.We utilized a two-frequency temporal phase-unwrapping algorithm to unwrap the phase. Essentially, two wrapped phase maps, low frequency phase ϕl and high-frequency ϕh(x, y). ϕl(x, y) is obtained from wide fringe patterns with a single fringe covering the whole measuring range, such that no phase unwrapping is required. By referring to ϕl(x, y) point by point, ϕh(x, y) is unwrapped to obtain a continuous phase map, Φ(x, y). Because 3D information is carried on by the phase, 3D shape can be reconstructed from the unwrapped phase Φ(x, y) using a phase-to-height conversion algorithm [5].
2.2. Optimal pulse width modulation (OPWM) technique to modulate high-frequency patterns
The recently proposed binary defocusing technique can greatly improve the measurement speed of DFP methods. However, when the projector is nearly focused, serious phase errors will arise due to the effects of high-order harmonics [12]. Therefore, in this paper, OPWM is adopted to generate high-frequency patterns. This technique selectively eliminates undesired frequency components by inserting different types of notches in a conventional binary square wave. With slight projector defocusing, ideal sinusoidal fringe patterns can then be generated.
Figure 1 illustrates a general quarter-wave symmetric OPWM pattern. The square wave is chopped n times per-half cycle. For a periodic waveform with a period of 2π, because it is an odd function, only the sine terms are left with the coefficients being described as:
The n chops in the waveform provide n degrees of freedom to eliminate n − 1 selected harmonics while keeping the fundamental frequency component within a certain magnitude. Due to the ability to eliminate undesired high-order harmonics, OPWM waveform could become sinusoidal after applying a small low-pass filter, which is similar to a small degree of defocusing. Our previous research has shown the success of OPWM technique [12].
2.3. Error-diffusion-based dithering technique to modulate low-frequency patterns
Due to the high-order harmonics, it is very difficult to achieve low-frequency patterns for the binary defocusing technique. We have recently introduced the dithering technique to generate high-quality fringe patterns for wide fringe stripes [10]. Binary dithering is one of the techniques extensively used to render color or grayscale images with only 1-bit images. It has been used to process both digital audio and video data. There are numerous dithering techniques developed including random dithering [13], ordered dithering [14] and error-diffusion dithering [15]. Among these dithering techniques, the sophisticated error-diffusion dithering techniques have been most extensively adopted because they can represent the original image or signal with higher fidelity compared to simpler dithering algorithms. In this method, the pixels are quantized in a specific order, and the error of quantization for the current pixel is propagated forward to local unprocessed pixels. By this means, the local average of the converted image is close to the original one.
The process of modifying an input pixel by prior quantization errors can be mathematically described as,
Here, f(i, j) is the original image, and error e(i, j) is the difference between the quantized image pixel and the diffused image pixel including the prior processed pixel influences. The error e(i, j) of quantizing the current pixel is further diffused to its neighboring pixels by means of a two-dimensional weighting function h(i, j), known as the diffusion kernel. There are numerous error-diffusion dithering algorithms differing on the diffusion kernel selection. In this paper, we use the kernel proposed by Stucki [16],Here, – represents the previously processed pixels, and * represents the pixel in processing. It should be noted that the kernel coefficients sum to one, and thus the local average value of the quantized image will be equal to the local average of the original one.
In this paper, we introduce this dithering technique to convert 8-bit sinusoidal fringe pattern into 1-bit binary patterns. By combining with the defocusing technique, high-quality sinusoidal patterns with wide fringe stripes can be achieved. Figure 2 illustrates a simulation example with generating a sinusoidal fringe pattern with a fringe pitch (number of pixels per fringe period) of 150 pixel. The original sinusoidal fringe pattern is dithered with the ordered-Bayer dithering technique [10] and the error-diffusion dithering technique with the kernel described in Eq. (7). This simulation results show that the resultant image (Fig. 2(d)) from the error-diffusion dithering appears better than that (Fig. 2(a)) from the ordered-Bayer dithering before defocusing (or blurring). Three phase-shifted fringe patterns are generated for each dithering technique, and these patterns are smoothed by a small Gaussian filter (9 × 9 pixels in dimension with a standard deviation of 3 pixels) to emulate the effect introduced by a small amount of defocusing. After smoothing, the cross sections of these patterns are shown in Fig. 2(b) and Fig. 2(e). These smoothed patterns are further processed to obtain the phase maps using a three-step phase-shifting algorithm. The phase errors can be determined by comparing with the ideal phase obtained from the ideal sinusoidal fringe patterns. Figure 2(c) and Fig. 2(f) show the cross sections of the phase errors. It clearly shows that the error-diffusion dithering method is significantly better than the ordered-Bayer dithering method. Therefore, instead of using the ordered-Bayer dithering technique, we utilize the error-diffusion dithering technique to generate the low-frequency sinusoidal fringe patterns.
Fig. 2 Comparison of Bayer dithering and error-diffusion dithering. (a) Bayer-dithering pattern; (b) Cross section of (a) after Gaussian smoothing; (c) Phase error (rms error of 0.012 rad); (d) Error-diffusion dithered pattern; (e) Cross section of (c) after Gaussian smoothing; (f) Phase error (rms 0.008 rad).
3. Experiments
3.1. Experimental system setup
Figure 3 shows a photograph of the system we developed. It is composed of a digital-light-processing (DLP) projector (DLP LightCrafter, Texas Instruments, TX), a high-speed CMOS camera (Phantom V9.1, Vision Research, NJ). The RGB LEDs used in this projector are (R: OSRAM LEAS2W 617 nm; G: OSRAM LCG H9RN 529 nm; B: OSRAM LEBS2W 464 nm). The camera is triggered by an external electronic circuit that senses timing signal of the DLP projector. For all experiments conducted in this research, the camera was set to capture images with a resolution of 576 × 576, and both the the projector and the camera were focused and remained fixed. The camera lens aperture was adjusted to ensure that the brightest possible fringe patterns were captured.
3.2. Live and fixed rabbit heart preparation
We imaged both formalin-fixed (10% formalin, n = 1) and Langendorff-perfused hearts (n = 2) isolated from New Zealand white rabbits. All studies were approved by the Institutional Animal Care and Use Committee of Washington University in St. Louis. The formalin-fixed heart was used for experimental validation of a stationary heart as described below. Langendorff-perfused hearts, conversely, were used to verify motion-induced artifacts. While this ex vivo, Langendorff-perfused model does not completely capture in vivo cardiac mechanics due to unloading during removal, it nevertheless provides a means to validate our system on a dynamically deforming heart. Each heart was removed via a mid-sternal thoracotamy and immediately perfused with oxygenated (95% O2, 5% CO2) Tyrode’s solution at a constant pressure of 60 ± 5 mmHg and temperature of 37 °C. Insulin (200 mg/dL) was also added to the perfusate to improve glucose utilization by the heart. Once cannulated, hearts were placed in a custom superfusion chamber with an optical window for structured light imaging. Ag/AgCl electrodes were placed near the atrioventricular (AV) groove to record ECG measurements during experimentation.
The formalin-fixed and Langendorff-perfused hearts were isolated from two different rabbits. Anatomical differences between the two hearts are minimal as a result of breeding control in our rabbit supplier.
3.3. Experimental verification
3.3.1. Formalin-fixed rabbit heart measurement
Figure 4 illustrates the phase-shifting algorithms we adopted by measuring a formalin-fixed rabbit heart. Figures 4(a)–4(c) show three high-frequency phase-shifted fringe patterns that used the OPWM patterns. It is important to note that these patterns have very obvious binary structures, but we will show that high-quality 3D shape measurement can still be achieved due to the use of OPWM technique. Figures 4(e)–4(g) show three low-frequency fringe patterns that used the dithered patterns. Applying Eq. (4) to three phase-shifted fringe patterns, the wrapped phase of both frequencies can be obtained. Figure 4(d) shows the high-frequency wrapped phase map, which possesses 2π phase discontinuities. The low-frequency wrapped phase, as shown in Fig. 4(h), is continuous without 2π discontinuities. By referring to the phase map obtained from the low frequency patterns, the high-frequency wrapped phase can be unwrapped point by point. Figure 5(a) shows the unwrapped phase. In this research, we used the approximate phase-to-height conversion approach introduced in Reference [5] to calibrate the system. After calibration, the 3D shape can be recovered. Figure 5(b) shows the 3D shape measurement results of the heart surface.
Fig. 4 Wrapped phase maps for dead heart measurement. (a)–(c) High-frequency fringe patterns; (d) Wrapped phase map of high frequency patterns; (e)–(g) Low-frequency fringe patterns; (h) Wrapped phase map of low frequency pattern
Fig. 5 Unwrapped phase map for dead heart measurement. (a) Unwrapped phase map; (b) 3D reconstructed shape of the heart surface
Figure 6 shows a comparison of different projection wavelengths on the formalin-fixed heart with blue, green, and red lighting sources. The contrast of Fig. 6(a) is better than Fig. 6(b) and Fig. 6(c). Figures 6(d)–6(f) show the corresponding reconstructed 3D shapes of the fixed heart. There are slight differences among these measurement, but surface measurement quality is high regardless of the spectrum of light used.
Fig. 6 Comparison of different spectrum for fixed heart measurements. (a) One fringe pattern using blue light; (b) One fringe pattern using green light; (c) One fringe pattern using red light; (d) 3D result using blue light; (e) 3D result using green light; (f) 3D result using red light.
3.3.2. Beating rabbit heart measurement: spectrum of light influence
As mentioned previously, we measured a beating rabbit heart with different projection wavelengths to investigate the combination of motion artifacts and light-tissue interaction on surface measurement quality. Figure 7 and the associated videos show the comparing results. Figures 7(a)– 7(c) respectively show the fringe patterns when the blue, green and red light was used. In comparison with fringe patterns captured for a fixed heart shown in Figs. 6(a)–6(c), the fringe contrast varies drastically for different wavelengths. Figures 7(d)– 7(f) show the corresponding reconstructed 3D results. Here, we used the projection speed of 2,000 Hz to ensure no motion induced artifacts. To fairly compare data recorded from red, green, and blue wavelengths, frames were selected from the same specific phase in the cardiac cycle. Unlike the fixed heart, the beating rabbit heart measurement results vary drastically the projector light wavelength, with blue light giving the best results and red light giving the worst results. Specifically, surface reconstructions from blue light appear to have smoother surface geometry and fewer fringe artifacts. We believe one possible cause for this is the optical properties of rabbit cardiac tissue for blue light relative to green or red. Blue light with a wavelength of 464 nm will have low penetration into cardiac tissue and high scattering compared to green or red light. As a result, measurement error from red and green light could be a result of light contributions from mid-myocardial light scattering and not simply light reflecting from the epicardial surface [17]. Fixation of cardiac tissues increasing scattering and may explain the wavelength independence we observed in our formalin fixed heart measurements. Based on these observations, only blue light was utilized for our other beating heart experiments.
Fig. 7 Comparison of different spectrum for beating heart measurements ( Media 1 and Media 2). (a) One fringe pattern using blue light; (b) One fringe pattern using green light; (c) One fringe pattern using red light; (d) 3D result using blue light; (e) 3D result using green light; (f) 3D result using red light.
3.3.3. Live rabbit heart measurement: measurement speed influence
Typically, an explanted rabbit heart beats at approximately 180 beats/min. The 3D shape measurement technique discussed requires the capture of a sequence of fringe patterns to reconstruct one 3D frame. The fundamental assumption during acquisition is that the heart surface is static during the projection and acquisition of the fringe patterns. Yet, for a beating heart, the surface is constantly deforming. Therefore, in order to properly capture such a dynamically deforming object, the sampling rate must be high enough such that the motion-induced artifacts can be negligible. To determine the minimum frame rate to properly capture the heart surface without obvious motion artifacts, we measured the heart at frame rates of 200, 300, 400, 500, 800 and 1000 Hz. Figure 8 and the associated video ( Media 3) show the 3D reconstructed results with these speeds when the heart surface displays greatest deformation. Figures 8(a) and 8(b) show the results with large bumpy striping artifacts, which were induced by motion. As a comparison, when the capturing speed increases to above 800 Hz, the results, such as those shown in Fig. 8(e)– 8(f), become rather smooth. Based on these results, we determine that in order to properly capture a beating rabbit heart with an ex vivo intrinsic rate of 180 beats/min, the minimum capturing speed required is approximately 800 Hz when a two-frequency phase-shifting technique is adopted.
Fig. 8 Comparison of different speed for beating heart measurements ( Media 3). (a)–(f) respectively shows the 3D results with 200, 300, 400, 500, 800 and 1000 Hz.
It important to note that, to our knowledge, there was no existing “better” technology that could measure the dynamic geometric surface motion of the beating rabbit heart, and thus there was no quantitative measure for these comparisons besides qualitative measure.
4. Conclusions
This paper presents a two-wavelength phase-shifting-based system to measure rabbit hearts. Our system requires binary structured patterns and can achieve up to 4,000 Hz 3D shape measurement speed. Due to the low contrast of cardiac tissue, the projector and the camera must remain focused. An OPWM technique was used to generate high-frequency fringe patterns, and a dithering technique was employed to generate low frequency fringe patterns. Two-frequency phase-shifting was utilized in this system to achieve highest possible absolute 3D shape measurement speed. Our experiments found that for a formalin-fixed rabbit heart, the projection wavelength does not substantially influence the measurement quality; but for a beating rabbit heart, the projection wavelength drastically alters the measurement quality with blue light generating the best results. Furthermore, we observed the motion induced artifacts for a beating rabbit heart are minimized at a minimum capture speed of 800 Hz when a two-frequency phase-shifting technique is adopted.
In summary, the major contributions of this paper are: (1) presented the first 3D shape measurement system that combines the OPWM method with the error-diffusion dithering method using the binary defocusing technique to achieve superfast absolute shape measurement; (2) introduced the error-diffusion based dithering technique, for the first time, to the field of 3D shape measurement; (3) successfully conquered the challenges of measuring live rabbit hearts; and (4) presented the guidelines for measuring live rabbit heart in terms of selecting spectrum of light and minimum capturing speed.
Acknowledgments
We would like to thank the National Science Foundation (projector number: CMMI-1150711) and the National Institute of Health (projector number: R21 HL112278) for the financial support.
References and links
1. Q. Lou, C. M. Ripplinger, P. V. Bayly, and I. R. Efimov, “Quantitative panoramic imaging of epicardial electrical activity,” Annals Biomed. Eng. 36, 1649–1658 (2008) [CrossRef] .
2. S. Zhang, “Recent progresses on real-time 3-D shape measurement using digital fringe projection techniques,” Opt. Laser Eng. 48(2), 149–158 (2010) [CrossRef] .
3. G. Zhang, J. Sun, D. Chen, and Y. Wang, “Flapping motion measurement of honeybee bilateral wings using four virtual structured-light sensors,” Sensors and Actuators A: Physical 148, 19–27 (2008) [CrossRef] .
4. S. Zhang, D. van der Weide, and J. Olvier, “Superfast phase-shifting method for 3-D shape measurement,” Opt. Express 18(9), 9684–9689 (2010) [CrossRef] [PubMed] .
5. Y. Xu, L. Ekstrand, J. Dai, and S. Zhang, “Phase error compensation for three-dimensional shape measurement with projector defocusing,” Appl. Opt. 50(17), 2572–2581 (2011) [CrossRef] [PubMed] .
6. Y. Wang and S. Zhang, “Optimum pulse width modulation for sinusoidal fringe generation with projector defocusing,” Opt. Lett. 35(24), 4121–4123 (2010) [CrossRef] [PubMed] .
7. G. A. Ajubi, J. A. Ayubi, J. M. D. Martino, and J. A. Ferrari, “Pulse-width modulation in defocused 3-D fringe projection,” Opt. Lett. 35, 3682–3684 (2010) [CrossRef] .
8. J. Laughner, S. Zhang, H. Li, and I. R. Efimov, “Mapping cardiac surface mechanics with structured light imaging,” Am. J. Physiol. 303(6), H712–H720 (2002).
9. T. L. Schuchman, “Dither Signals and Their Effect on Quantization Noise,” IEEE Trans. Commun. Technol. 12(4), 162–165 (1964) [CrossRef] .
10. Y. Wang and S. Zhang, “Three-dimensional shape measurement with binary dithered patterns,” Appl. Opt. 51(27), 6631–6636 (2002) [CrossRef] .
11. D. Malacara, ed., Optical Shop Testing, 3rd ed. (John Wiley and Sons, New York, 2007) [CrossRef] .
12. Y. Wang and S. Zhang, “Comparison among square binary, sinusoidal pulse width modulation, and optimal pulse width modulation methods for three-dimensional shape measurement,” Appl. Opt. 51(7), 861–872 (2012) [CrossRef] [PubMed] .
13. W. Purgathofer, R. Tobler, and M. Geiler, “Forced random dithering: improved threshold matrices for ordered dithering,” IEEE International Conference on Image Processing 2, 1032–1035 (1994).
14. B. Bayer, “An optimum method for two-level rendition of continuous-tone pictures,” IEEE International Conference on Communications 1, 11–15 (1973).
15. T. D. Kite, B. L. Evans, and A. C. Bovik, “Modeling and quality assessment of Halftoning by error diffusion,” IEEE International Conference on Image Processing 9(5), 909–922 (2000) [CrossRef] .
16. P. Stucki, “MECCAa multiple-error correcting computation algorithm for bilevel hardcopy reproduction,” Tech. Rep., IBM Res. Lab., Zurich, Switzerland (1981).
17. W. J. Bowen, “The absorption spectra and extinction coefficients of myoglobin,” J. Biol. Chem. 179, 235–245 (1949) [PubMed] .
