Open Access
Add to BibSonomy
Abstract
Human-induced pluripotent stem cell-derived cardiomyocytes (hiPS-CMs) beating can be efficiently characterized by time-lapse quantitative phase imaging (QPIs) obtained by digital holographic microscopy. Particularly, the CM’s nucleus section can precisely reflect the associated rhythmic beating pattern of the CM suitable for subsequent beating pattern characterization. In this paper, we describe an automated method to characterize single CMs by nucleus extraction from QPIs and subsequent beating pattern reconstruction and quantification. However, accurate CM’s nucleus extraction from the QPIs is a challenging task due to the variations in shape, size, orientation, and lack of special geometry. To this end, we propose a novel fully convolutional neural network (FCN)-based network architecture for accurate CM’s nucleus extraction using pixel classification technique and subsequent beating pattern characterization. Our experimental results show that the beating profile of multiple extracted single CMs is less noisy and more informative compared to the whole image slide. Applying this method allows CM characterization at the single-cell level. Consequently, several single CMs are extracted from the whole slide QPIs and multiple parameters regarding their beating profile of each isolated CM are efficiently measured.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Digital holographic microscopy (DHM) providing quantitative phase images (QPIs), is a promising tool in the field of label-free and computer-aided assessments [1–3]. The benefits of numerical reconstruction of digitally recorded holograms made it possible to study micrometer-sized living cells [4–6]. DHM demonstrating the capability of monitoring cell’s dry mass is particularly well suited to achieve label-free screening of living cells [7]. Cardiomyocytes (CMs) are the main contractile elements of the heart leading to pump the blood to the entire body through the vessels by an orchestrated contraction–relaxation cycle coordinated by electrical stimuli generated by pacemaker cells [8]. CMs change their shape rapidly within a beat cycle with meaningful intermediate events named contraction and relaxation. Human-induced pluripotent stem cell-derived cardiomyocytes (hiPSC-CMs) are elastic elements for modeling human disease, cardiotoxicity, and therapy in vitro to regenerate viable cardiac tissue due to the limited capability of the heart to regenerate functional cardiac tissue [9]. There are several critical issues regarding stem cell therapy including the transfer and survival of the implanted stem cells in the myocardium [10]. During the CM beating activity, the cell tissue fiber consecutively shortens and lengthens. The fiber flexibility is essential for the cell’s proper functionality. While cardiac cell’s beating activity, dry mass redistribution can be monitored with DHM. The authors in [11,12] used label-free holographic microscopy for quantitative evaluation of cell’s volume variation after nanographene oxide internalization to the cell memberane [11] and cell’s biophysical parameters measurments including cell’s volume, projected area, thickness and dry mass [12]. In our previous study, we also used quantitative phase digital holographic microscopy (QP-DHM) for non-invasive label-free studying of drug-treated cardiomyocytes and subsequent beating activity quantification [13].
Many efforts have been made to develop advanced complementary methods for CM characterization in vitro for reducing drug development costs and cardiotoxicity-related drug attrition. The CM characterization methods are including patch clamping [14,15], calcium imaging [16,17], and image processing-based contraction-relaxation studies [18,19]. The mechanical probe is another widely applied method for CM characterization [20]. There are limitations associated with each method that requires more expertise and costly equipment or require the CMs to be plated onto specialized material which renders the process challenging. In addition, the above-mentioned methods have been conducted for the whole slide image of multiple CMs while single CM characterization methods are lacking. Therefore, it is crucial to conduct high throughput and reliable methods to facilitate CM characterization in vitro at the single-cell level. The label-free imaging technique is a robust method to characterize hiPS-CMs using the non-invasiveness property [21]. The hiPS-CMs nucleus section from time-lapse QPI imaging can efficiently reflect the rhythmic beating pattern, which is less noisy and more informative for subsequent characterization. Thus, the CM beating activity can be efficiently characterized in multiple single-cell levels solely by nucleus dry mass redistribution monitoring. To be able to efficiently characterize single CM’s beating activity, each QPI image is segmented into ROI (nucleus) and non-ROI which includes surrounding cytoplasm and membrane. The non-ROI negatively influence the beating profile as unwanted noisy peaks which pose difficulties for further dynamic activity characterization. By their very nature, CMs can appear in any kind of shape, size, and orientation [22]. Thus, the ROI extraction form QPIs is a challenging task. Deep learning methods can extract hidden features of images that do not implicitly require any type of features design by human experts. Deep learning methods have been applied to a variety of medical image analysis to overcome drawbacks of employing hand-craft features to describe data characteristics. Discriminative features can be extracted automatically using deep learning techniques considering sufficient training samples [23,24]. Deep fully convolutional neural network (FCN)-based architectures have shown great potential in the medical image segmentation field. However, the features learned by sequences of standard convolution layers are not distinctive when the target object and the other objects in the image are similar in terms of intensity, location, shape, and size which causes poor performance of the U-Net architecture as a popular deep learning model in CM’s ROI extraction.
In the current study, we present a method for single CM characterization by nucleus extraction from QPIs using a fully convolutional neural network. Our proposed FCN-based architecture configuration is similar to semantic segmentation networks. First, we use the off-axis DHM technique to acquire cardiomyocytes phase images. Afterward, we propose an FCN-based network architecture that is made up of parallel multi-pathways features concatenation by taking advantage of features extracted various convolutional kernel sizes, dense connections with residual connections for accurate cardiac cell’s nucleus extraction. The evaluation of the proposed method for CM’s nucleus extraction against the U-Net network model is carried out. Our experimental results indicate that our network model outperforms the U-Net network model. Applying the proposed method allowed CMs characterization at the single-cell level. Finally, several single CMs from QPI and multiple parameters related to the beating profile (contraction period, relaxation period, resting period, beating interval, and beat rate) of every CM are measured.
2. Digital holographic microscopy
The off-axis DHM is used to obtain the phase image of CMs in this study. In the off-axis configuration, the laser source is divided into an object and reference beams (see Fig. 1). The microscope objective (MO) magnifies the object wave and CCD camera records the hologram generated by the interference of the magnified object wave (O) and the reference wave (R) incident at a small angle “θ” to provide the off-axis property. Figure 2(a) shows the recorded hologram of CMs, which can be expressed as the sum of four terms:
where ${|R |^2}$ is the intensity of the reference wave and ${|O |^2}$ is that of the object wave. R* and O* denote the complex conjugates of the two waves. Two separate real image from twin image and zero-order noise a spatial filter in the frequency domain is designed. This filter can only preserve the bandwidth of the real image and eliminates the unwanted bandwidth. Finally, the phase images of cardiomyocytes are numerically reconstructed using numerical reconstruction algorithms on the computer [25–27]. Figure 2 shows one off-axis digital hologram of cardiac cells and its corresponding reconstructed optical path difference (OPD) images of cardiac cells.
Fig. 1. Schematic representation of off-axis digital holographic microscopy used in this experiment to acquire cardiomyocyte phase image.
Fig. 2. (a) A recorded hologram of cardiomyocytes. Inset shows in the 3D portion of the hologram. (b) Phase image after numerical reconstruction. The phase image provides high contrast data for the quantitative analysis. (c) A single cardiac cell with a nucleus section marked with a red line defined as ROI to be extracted for dynamic beating profile quantification.
3. Cardiomyocytes preparation
Human-induced pluripotent stem (iPS) cell-derived CM obtained from Cellular Dynamics Int. (Madison, WI) were cultured and grown according to the manufacturer’s instructions for 14 days before recording the hologram. Measurements were acquired in a Chamlide WP incubator system with a 96-well plate (LCI, South Korea) set at 37°/5% CO2 with high humidity. For the staining with Hoechst dye, the following protocol is used. Hoechst dyes at 1 ug/mL is added to the complete culture medium. Culture medium is removed from the cells and replaced with medium containing dye. Cells are incubated at 37°C for 10 minutes and then imaged. Images were recorded by a commercially available DHM T-1001 from LynceeTec SA (Lausanne, Switzerland) equipped with a motorized stage (Märzhäuser Wetzlar GmbH & Co. KG, Wetzlar, Germany, ref. S429). Images were obtained using a Leica 20×/0.4NA objective (Leica Microsystems GmbH, Wetzlar, Germany, ref. 11566049). The CCD resolution is 1920 ×1200 pixel (the hologram size is 1024×1024 efficient for FFT computation). The phase stability of imaging system is around Δφ=0.05°, which in studying the biological samples with limited reflective index is equivalent to a thickness of several nanometers. The 666nm laser source delivered an intensity of ∼200W/cm2 to the specimen plane, which is nearly six orders of magnitude less light, and the required exposure time was only 0.4ms. The reconstruction process of the cardiac cells phase image was conducted afterward using a standard PC, at a rate of several images per second. 540 were recorded at a sampling frequency of 10Hz.
4. ROI identification with dynamic beating activity analysis
To precise ROI identification, the dynamic beating activity comparison is carried out between the ROI and non-ROI as shown in Fig. 2(c) and the results of beating is shown in Fig. 3. The QPIs in this study are related to the optical path difference (OPD). The dynamic of beating profile is obtained from the spatial variance between successive time-lapse QPI images as explained in [28]. As it has been mentioned, OPD variance reflects the time course of the cell dry mass redistribution occurring during the cardiomyocytes contraction-relaxation cycle. Since the OPD value redistribution of the ROI is dominant comparing to non-ROI (the signal amplitude and significant contraction-relaxation peaks; see Fig. 3), the OPD variance in the ROI can significantly reflect beating activity. In contrast, the OPD variance in the non-ROI demonstrate no beating activity.
5. Convolutional neural network
Deep learning methods have been applied to a wide variety of problems more especially for medical image analysis [29]. In many cases, deep learning-based methods have surpassed state-of-the-art methods performance by capturing hierarchical representations of data. These models are based on the sequential application of the convolutional layer, where the output of one convolution layer is the input to the next one. Each convolutional layer provides one feature hierarchies at a specific level. Layer's weights are initialized by a set of random weights and a set of biases. Convolutional neural networks (CNN) is a type of deep learning model for capturing hierarchical features of data using locally shared weights. A convolution filter is formed by shared weights to the same output. CNN captures features of input data via multiple consecutive convolution kernels followed by max-pooling layers for data dimension reduction. CNN mainly consists of the following elements. 1) A set of learnable filters to extract local features. 2) A nonlinear function as an activation function and 3) a max-pooling layer which aggregates the local feature specification to reduce the data dimensions. The max-pooling operation used for the down-sampling purpose which obtains the maximum value of each filter in the convolution layer. The summation of each convolution layer is applied to a nonlinear function named as a rectified linear unit (ReLU) as an activation function. The ReLU function is a nonlinear function applied to increase the nonlinearity of the CNN feature maps. However, depending on the different tasks, different network architectures are proposed which might be more efficient than simple CNN-based networks. A fully convolutional neural network (FCN) is a type of CNN in which the fully connected layer is replaced with another convolution layer [30]. The current study focuses on single hiPS-CMs characterization by CM’s nucleus extraction using the FCN-based deep learning model.
5.1 U-Net
U-Net is one of the most popular widely used end-to-end network models for semantic segmentation which consists of encoder and decoder pathway, with skip connections between the corresponding layers that yield good segmentation performance. The designed architecture is based on symmetric pathways for accurate localization. The first section of the U-Net extracts deep features and the second part is responsible for the segmentation based on previously extracted features. However, there are some limitations with U-Net architecture that can be denoted as less flexibility and lack of scalability. While deeper networks yield better segmentation, at the same time increases parameter space and cause gradient vanishing [31].
5.2 Proposed network model
This section introduces the concepts employed for efficient CM’s ROI extraction. A novel FCN-based network architecture is proposed for accurate CM’s nucleus extraction which takes advantage of parallel multi-pathways features concatenation with dense connection blocks and residual connections [32]. The overall structure and different building blocks of the proposed network architecture are shown in Fig. 4. The overall network structure leads to a better training performance when compared to the U-Net model meanwhile it improves the pixel classification accuracy. The proposed FCN-based network model’s building blocks are briefly explained below:
Fig. 4. The proposed FCN-based network architecture for cardiac cells ROI extraction has been made up of several modular blocks which are explained as follows: Parallel multi-pathway features concatenation (blue box) which takes the advantage of different kernel sizes: 1×1, 3×3, and 5×5. Each pathway is a composition of a convolutional layer + batch normalization layer + rectified linear unit. The features are concatenated at the end. 2×2 max-pooling layer with a stride of two is used for down-sampling data. The dense connection technique is used for efficient gradient propagation to prevent vanishing gradient. The residual connections are denoted with dotted horizontal red arrows representing residual skip connections.
5.2.1 Parallel multi-pathways features’ concatenation
In multi-pathways features concatenation technique, different feature maps extracted by various kernel sizes are concatenated. Regarding the convolution layers’ kernel size, it is hard to decide which kernel size might be more efficient for the task in hand and different kernel sizes will result in different features. The commonly used convolutional kernel size is 3×3. Therefore, we proposed to use different kernel sizes in different pathways in parallel consisting of namely 1×1, 3×3, and 5×5 followed by batch normalization (BN) and rectified linear unit (ReLU). Features from different pathways are concatenated at the end. Max-pooling layer with the size of (2×2) with a stride of 2 is used for down-sampling feature maps from different pathways to reduce the data dimension.
5.2.2 Dense connection
We propose to use convolutional dense connection blocks explained in [33]. Within each dense block, layers are directly connected with their preceding layers, which is implemented via the concatenation of feature maps in subsequent layers. The major advantages of dense connection blocks can be mentioned as efficient gradient propagation to prevent vanishing gradient which normally happens in deep networks and reuse of the feature maps from previous layers instead of only the last layer which results in better network performance.
5.2.3 Residual connection
Residual connection technique is to utilize skip connections, or short-cuts to jump over some layers that facilitate the training of deep networks [34,35]. In addition, the shorter connection between layers close to the output and input yields better performance, reduces the number of parameters, and easier to train. Besides, pooling operations cause to lose some spatial information. These skip connections allow the network to retrieve the lost spatial information. In our proposed network model, the output of the standard 3×3 convolutional layer prior to the pooling operation is transferred to the corresponding output of the up-sampling layer.
5.3 Patch extraction
Generally, deep learning methods require plenty of samples for the training. There are several different methods to increase the number of training samples, for instance, data augmentation method. Still, these methods cannot increase the data sample significantly. To tackle this problem, we utilized a patch extraction method, which can significantly increase the number of samples to a sufficient level for training of FCN-based deep learning models. We extract patches from the QPI containing multiple CMs using a sliding window and capturing patches along with corresponding ground truth (manually extracted) with a size of 32 ×32 pixels. The captured patches using a sliding window are not overlapped (see Fig. 5).
Fig. 5. Sliding window patch extraction method for training data preparation. (a) Original QPI of cardiomyocytes containing multiple cells (yellow bar represents 20 µm). (b) Magnified portion of the original phase image with indicated patches. (c) Corresponding ground truth patches with ROI (yellow color) and non-ROI (dark blue color) portions.
6. Experimental results
After the manually annotated images are reconstructed, we trained the proposed network by using the training dataset images (generated by patch extraction method) and their corresponding ground truth labels. The whole dataset contains 2500 extracted patches along with corresponding ground truth. From the whole dataset 80% (n = 2000) is used for training and 20% (n = 500) is used as a test dataset. We evaluated our trained network model for ROI extraction using some test images containing multiple cardiac cells (see Fig. 6(a)). To ease the visual evaluation on network performance, predicted masks from the proposed network model and predicted mask by the U-Net network model are shown in Fig. 6(b) and (c), respectively. The ground truth mask is shown in Fig. 6(d). From the experimental results, we can see the predicted mask by the U-Net model contains extra seeds and shows over and under segmentation. The experimental results show that despite the difficulties mentioned above for ROI extraction including ROI size, shape, and orientations, the proposed network model effectively handles different edge directions and specific ambiguous attributes of the ROI and non-ROI.
Fig. 6. Results of ROI extraction using the proposed FCN-based method in comparison with the U-Net network model. (a) Original phase image of multiple cardiac cells obtained by DHM. (b) Predicted mask using our trained FCN-based model. (c) Predicted mask using the U-Net network model. (d) Ground truth mask extracted manually.
Figure 7 demonstrates an example of the beating activity comparison of the whole slide of QPI of multiple CMs before and after ROI extraction (see Figs. 7(a), (c)) and their corresponding beating activity (see Figs. 7(b), (d)). As mentioned previously, the beating activity before ROI extraction showed to be noisy with extra wrong peaks due to the OPD variance in the non-ROI section that can cause difficulties for beating profile quantification. It has been demonstrated that the ROI section can reflect a clean and less noisy beating activity with removed wrong peaks and more informative for CM dynamic characterization.
Fig. 7. Example of CM beating profile reconstruction before and after ROI extraction. (a) and (b) QPI containing multiple CMs before ROI extraction and corresponding beating activity profile respectively. (c) and (d) QPI after ROI extraction using the proposed method (green outline) and corresponding beating activity profile.
6.1 Single cardiomyocyte beating profile quantification
During the first step of our approach, the cardiac cells ROI is extracted and the resulting mask image is multiplied by each QPI image in the sequence. Finally, the spatial variance between successive images is calculated to reconstruct the beating profile as explained in [28]. This parameter is sensitive to the redistribution of dry mass within cardiomyocytes to monitor the characteristics of the cardio beating over time. The corresponding equation is as follows:
where opdi and opdi-1 are the ith and i–1th images. Specifically, opdvar (OPD variance) reflects the time course of the cell dry mass redistribution while CM beating activity. The opdvar signal contains information about the redistribution of dry mass within cardiomyocytes namely contraction and relaxation durations. After ROI extraction, we can monitor single CM’s beating activity and measure different parameters. Descriptions of quantification parameters are explained in Table 1.
Table 1. Description of cardiomyocyte dynamic parameters quantification.
To further examine the use of the proposed method for single-cell level characterization, six individual cells are extracted from the whole slide QPI of cardiomyocyte shown in Fig. 8(a). The corresponding beating profile is reconstructed from Eq. (2) (see Fig. 8(b)). We measured several parameters related to the beating activity profile for each individual cell. In order to characterize the single cells dynamic beating profile activity using ROI extraction, first, we detected two main peaks of contraction and relaxation using the Otsu thresholding method. The first peaks which in most cases have a larger amplitude value stands for contraction and the corresponding relaxation peak is the second peaks with lower amplitude value. Afterward, three auxiliary points 1) Start-of-Contraction, 2) End-of-Contraction and 3) End-of-Relaxation showed in Fig. 8(c) are defined using contraction and relaxation peaks. Locating the auxiliary points are found as follows: The End-of-Contraction point (Start-of-Relaxation) (see Fig. 8(c) blue color points) is obtained by finding the smallest amplitude value between contraction and corresponding relaxation peaks. The auxiliary Start-of-Contraction point and the End-of-Relaxation are detected using a search strategy around the contraction and relaxation peaks. Then the time difference between End-of-Contraction (Start-of-Relaxation) and End-of-Relaxation is considered as relaxation period. The time difference between Start-of-Contraction and corresponding End-of-Contraction point specifies the contraction period. The time difference between two consecutive contraction peaks determines the beating intervals. The resting period is the time difference between End-of-Relaxation and the next Start-of-Contraction point. The extracted features of the beating profile and the corresponding descriptions are presented in Table 1. The beating activity profile reconstructed from cells #1, cell #2, cell #3, cells #4, cell #5 and cell #6 with detected contraction and relaxation peaks and auxiliary point for precise single CM characterization are shown in Fig. 9.
Fig. 8. (a) Original phase image of multiple cardiomyocytes in which single cells are denoted for further quantification. (b) The beating activity profile derived from cells #1 (30 seconds is shown) and (c) details on quantification parameters explained in Table 1.
Fig. 9. Beating profiles of single cells extracted from sample number five (Cell #1, Cell #2, Cell #3, Cell #4, Cell #5, and Cell #6). The sample is recorded with a 10 Hz sampling frequency for 54 seconds.
The dynamic beating profile quantification of single CMs show that the contraction period is shorter than the relaxation period due to the presence of different ion channels and transporters expressed in cardiomyocytes and the mechanisms by which their activities are sequentially orchestrated while cell contract and relax (see Fig. 10). The average beating rate, average beating intervals, and resting time are similar for all cells as shown in Fig. 10.
7. Network performance accuracy measurement
The proposed method was developed using MATLAB 2018a on a personal computer equipped with Intel Core i7-7700, 3.60 GHz, 16 GB RAM, and NVIDIA GeForce GTX 1050. A set of evaluations is carried out to compare the proposed method’s performance against the U-Net model in terms of training process performance, pixel classification accuracy, and Dice coefficient. The following network configuration was found to be optimal. Learning rate: 1.000e-03, momentum: 0.9000, mini-batch size: 5, and max epoch: 55. The learning curve evaluation shows the proposed method convergence and corresponding loss is faster when compared with the U-Net model (see Figs. 11(a), (b)). The confusion matrix of pixel classification demonstrates the accuracy percentage of correctly classified pixels versus the misclassification rate. As shown in Fig. 11(c), the classification accuracy rate using the proposed FCN-based method 99.76% and 99.28%, respectively while the same evaluation for the U-Net model shows 93.69% and 91.25% accuracy (see Fig. 11(d)). The experimental results indicate the robustness of the proposed method by accurate pixel classification. In order to evaluate the proposed method’s segmentation performance of the proposed method against the U-Net model from the statistics point of view, for each cardiomyocyte sample, we used the well-known Dice coefficient analysis (see Table 2). The Dice coefficient is calculated using the following equation:
where TP, FP, and FN are the numbers of true positive, false positive, and false negative detections, respectively.Fig. 11. Comparison of the learning curves and confusion matrix for pixel classification of the proposed FCN-based model versus the U-Net model. (a) and (b) Training process accuracy in 55 epochs and corresponding loss curves respectively. (c) Confusion matrix of pixel classification prediction accuracy for the proposed model. (d) Confusion matrix of pixel classification prediction accuracy for the U-Net model.
Table 2. Segmentation performance evaluation using the Dice coefficient analysis of the proposed method against the U-Net model.
8. Discussion
Single CM characterization in vitro is crucially important for drug compound testing and cardiotoxicity screening. We proposed a method for single CM characterization by using nucleus extraction (ROI) and the other CM sections are considered as non-ROI (surrounding cytoplasm and membrane). In our proposed method, the hiPS-CMs beating activity profile obtained by DHM is reconstructed by using the OPD variance calculation between the successive QPI frames. The experimental results show that the CM beating profile reconstructed from the extracted nucleus is less noisy and more informative for further characterization. On the other hand, the OPD variance in the non-ROI section leads to create a noisy beating profile with multiple wrong peaks causing difficulties for further CM characterization. Due to the specific attributes of the nucleus section, we proposed a novel FCN-based method for nucleus extraction combined with further analysis which allows cardiac cell characterization at single-cell levels. The proposed platform can be integrated into other technologies for precise single CM characterization. The proposed FCN-based architecture showed a better segmentation performance when compared to the one of the popular deep learning-based biomedical image segmentation U-Net model. The overall goal of this approach is to facilitate the cardiac cells beating profile characterization both in multiple and single-cell levels. We found that segmentation was necessary for a single CM characterization. To further examine the potential of the proposed method, we extracted multiple isolated CMs from segmentation results and multiple parameters regarding the beating profile of every single cell are measured. All findings from the experimental results demonstrated that single CMs can be effectively characterized by the cell’s nucleus section extracted by the proposed method. The combined measurements showed the robustness of the proposed FCN-based method for CM nucleus extraction and consequent characterization in a single-cell level suitable for screening compounds cytotoxic effects.
To validate the proposed method for single cardiomyocyte characterization using the nucleus section and to examine whether the segmentation area includes nuclei, one sample is stained with Hoechst dye. We applied our segmentation method prior to and after nuclei staining and ROI and non-ROI sections are marked for the visual comparison. As one can see from Fig. 12, the ROI section mainly includes the nuclei section (nuclei is marked in blue color).
Fig. 12. (a) Segmentation result of original QPI of multiple cardiomyocytes before Hoechst nuclei staining, (b) The original QPI with nuclei staining is overlaid on the segmented image. The segmented sections (ROI) mostly include the nuclei part. The inset shows a single cardiomyocyte with a nuclei section marked with a green line defined as ROI to be extracted for dynamic beating profile quantification.
9. Conclusion
In this paper, for the first time, we proposed an automated FCN-based platform for CM’s nucleus extraction and beating pattern characterization at the single-cell level. We demonstrated that the cardiac cell nucleus section (denoted as ROI) from QPI can effectively reflect the beating pattern suitable for CM characterization at the single-cell level. We proposed a novel FCN-based method to discriminate CM’s nucleus section from other sections of cardiac cells using pixel classification techniques. The predicted mask is used for further characterization of dynamic contraction-relaxation of single CM. The proposed FCN architecture outperformed U-Net for segmentation mask prediction by over 99% pixel classification accuracy. We evaluated the proposed model’s segmentation mask prediction using pixel classification accuracy along with network training accuracy and dice similarity coefficient metrics. To validate our FCN-based platform, we used multiple quantitative phase images of cardiomyocytes containing multiple cells under different circumstance including shape, size, and, orientations. In the next step, we extracted several individual cardiac cells for beating profile characterization at the single-cell level and multiple parameters associated with the dynamic beating profile of each cell after removing non-ROI are measured. The Single CM beating profile quantification is precisely performed using contraction-relaxation peak detection and multiple auxiliary points. The experimental results showed that the proposed method is robust for cardiomyocytes ROI extraction and subsequent cell dynamic beating profile characterization at the single-cell level.
Funding
National Research Foundation of Korea (NRF-2015K1A1A2029224); Daegu Gyeongbuk Institute of Science and Technology (20-CoE-BT-02).
Disclosures
The authors declare that there are no conflicts of interest related to this article.
References
1. B. Rappaz, P. Marquet, E. Cuche, Y. Emery, C. Depeursinge, and P. Magistretti, “Measurement of the integral refractive index and dynamic cell morphometry of living cells with digital holographic microscopy,” Opt. Express 13(23), 9361–9373 (2005). [CrossRef]
2. P. Marquet, B. Rappaz, P. J. Magistretti, E. Cuche, Y. Emery, T. Colomb, and C. Depeursinge, “Digital holographic microscopy: a noninvasive contrast imaging technique allowing quantitative visualization of living cells with subwavelength axial accuracy,” Opt. Lett. 30(5), 468–470 (2005). [CrossRef]
3. E. Ahmadzadeh, K. Jaferzadeh, J. Lee, and I. Moon, “Automated three-dimensional morphology-based clustering of human erythrocytes with regular shapes: stomatocytes, discocytes, and echinocytes,” J. Biomed. Opt. 22(7), 076015 (2017). [CrossRef]
4. I. Moon and B. Javidi, “3-D visualization and identification of biological microorganisms using partially temporal incoherent light in-line computational holographic imaging,” IEEE Trans. Med. Imaging 27(12), 1782–1790 (2008). [CrossRef]
5. B. Javidi, I. Moon, S. Yeom, and E. Carapezza, “Three-dimensional imaging and recognition of microorganism using single-exposure on-line (SEOL) digital holography,” Opt. Express 13(12), 4492–4506 (2005). [CrossRef]
6. K. Jaferzadeh, I. Moon, M. Bardyn, M. Prudent, J. Tissot, B. Rappaz, B. Javidi, G. Turcatti, and P. Marquet, “Quantification of stored red blood cell fluctuations by time-lapse holographic cell imaging,” Biomed. Opt. Express 9(10), 4714–4729 (2018). [CrossRef]
7. B. Rappaz, E. Cano, T. Colomb, J. Kühn, C. Depeursinge, V. Simanis, P. J. Magistretti, and P. Marquet, “Noninvasive characterization of the fission yeast cell cycle by monitoring dry mass with digital holographic microscopy,” J. Biomed. Opt. 14(3), 034049 (2009). [CrossRef]
8. C. Wheeler-Jones, “Cell signaling in the cardiovascular system: An overview,” Heart 91(10), 1366–1374 (2005). [CrossRef]
9. N. Shaked, L. Satterwhite, N. Bursac, and A. Wax, “Whole-cell-analysis of live cardiomyocytes using wide-field interferometric phase microscopy,” Biomed. Opt. Express 1(2), 706–719 (2010). [CrossRef]
10. A. Totaro, F. Urciuolo, G. Imparato, and P. Netti, “Engineered cardiac micromodules for the in vitro fabrication of 3D endogenous macro-tissues,” Biofabrication 8(2), 025014 (2016). [CrossRef]
11. M. Mugnano, G. Lama, R. Castaldo, V. Marchesano, F. Merola, D. Giudice, A. Calabuig, G. Gentile, V. Ambrogi, P. Cerruti, P. Memmolo, V. Pagliarulo, P. Ferraro, and S. Grilli, “Cellular Uptake of Mildly Oxidized Nanographene for Drug-Delivery Applications,” ACS Appl. Nano Mater. 3(1), 428–439 (2020). [CrossRef]
12. M. Mugnano, P. Memmolo, L. Miccio, S. Grilli, F. Merola, A. Calabuig, A. Bramanti, E. Mazzon, and P. Ferraro, “In vitro cytotoxicity evaluation of cadmium by label-free holographic microscopy,” J. Biophotonics 11(12), e201800099 (2018). [CrossRef]
13. K. Jaferzadeh, B. Rappaz, F. Kuttler, B. K. Kim, I. Moon, P. Marquet, and G. Turcatti, “Marker-Free Automatic Quantification of Drug-Treated Cardiomyocytes with Digital Holographic Imaging,” ACS Photonics 7(1), 105–113 (2020). [CrossRef]
14. D. Ossola, M. Amarouch, P. Behr, J. Vörös, H. Abriel, and T. Zambelli, “Force-controlled patch clamp of beating cardiac cells,” Nano Lett. 15(3), 1743–1750 (2015). [CrossRef]
15. A. Brüggemann, C. Haarmann, M. Rapedius, T. Goetze, I. Rinke, M. George, and N. Fertig, “Characterization of iPS Derived Cardiomyocytes in Voltage Clamp and Current Clamp by Automated Patch Clamp,” Biophys. J. 112(3), 236a (2017). [CrossRef]
16. N. Huebsch, P. Loskill, M. Mandegar, N. Marks, A. Sheehan, Z. Ma, A. Mathur, T. Nguyen, J. Yoo, L. Judge, C. Spencer, A. Chukka, C. Russell, P. So, B. Conklin, and K. Healy, “Automated video-based analysis of contractility and calcium flux in human-induced pluripotent stem cell-derived cardiomyocytes cultured over different spatial scales,” Tissue Eng., Part C 21(5), 467–479 (2015). [CrossRef]
17. E. Grespan, S. Martewicz, E. Serena, V. Houerou, J. Rühe, and N. Elvassore, “Analysis of calcium transients and uniaxial contraction force in single human embryonic stem cell-derived cardiomyocytes on microstructured elastic substrate with spatially controlled surface chemistries,” Langmuir 32(46), 12190–12201 (2016). [CrossRef]
18. A. Ahola, A. L. Kiviaho, K. Larsson, M. Honkanen, K. Aalto-Setälä, and J. Hyttinen, “Video image-based analysis of single human induced pluripotent stem cell derived cardiomyocyte beating dynamics using digital image correlation,” Biomed. Eng. 13(1), 1–18 (2014). [CrossRef]
19. C. Bazan, D. Barba, P. Blomgren, and P. Paolini, “Image processing techniques for assessing contractility in isolated neonatal cardiac myocytes,” Int. J. Biomed. Imaging 2011, 729732 (2011). [CrossRef]
20. I. Nitsan, S. Drori, Y. Lewis, S. Cohen, and S. Tzlil, “Mechanical communication in cardiac cell synchronized beating,” Nat. Phys. 12(5), 472–477 (2016). [CrossRef]
21. D. Carl, B. Kemper, G. Wernicke, and G. Bally, “Parameter-optimized digital holographic microscope for high-resolution living-cell analysis,” Appl. Opt. 43(36), 6536–6544 (2004). [CrossRef]
22. Y. Abassi, B. Xi, N. Li, W. Ouyang, A. Seiler, M. Watzele, R. Kettenhofen, H. Bohlen, A. Ehlich, E. Kolossov, X. Wang, and X. Xu, “Dynamic monitoring of beating periodicity of stem cell-derived cardiomyocytes as a predictive tool for preclinical safety assessment,” Br. J. Pharmacol. 165(5), 1424–1441 (2012). [CrossRef]
23. K. Jaferzadeh, S. Hwang, I. Moon, and B. Javidi, “No-search focus prediction at the single cell level in digital holographic imaging with deep convolutional neural network,” Biomed. Opt. Express 10(8), 4276–4289 (2019). [CrossRef]
24. Y. Wu, A. Ray, Q. Wei, A. Feizi, X. Tong, E. Chen, Y. Luo, and A. Ozcan, “Deep Learning Enables High-Throughput Analysis of Particle-Aggregation-Based Biosensors Imaged Using Holography,” ACS Photonics 6(2), 294–301 (2019). [CrossRef]
25. U. Schnars and W. Juptner, “Digital recording and numerical reconstruction of holograms,” Meas. Sci. Technol. 13(9), R85–R101 (2002). [CrossRef]
26. E. Cuche, P. Marquet, and C. Depeursinge, “Simultaneous amplitude-contrast and quantitative phase-contrast microscopy by numerical reconstruction of Fresnel off-axis holograms,” Appl. Opt. 38(34), 6994–7001 (1999). [CrossRef]
27. M. Herráez, D. Burton, M. Lalor, and M. Gdeisat, “Fast two-dimensional phase-unwrapping algorithm based on sorting by reliability following a noncontinuous path,” Appl. Opt. 41(35), 7437–7444 (2002). [CrossRef]
28. B. Rappaz, I. Moon, F. Yi, B. Javidi, P. Marquet, and G. Turcatti, “Automated multi-parameter measurement of cardiomyocytes dynamics with digital holographic microscopy,” Opt. Express 23(10), 13333–13347 (2015). [CrossRef]
29. G. Litjens, T. Kooi, B. Bejnordi, A. Setio, F. Ciompi, M. Ghafoorian, J. Laak, B. Ginneken, and C. Sánchez, “A survey on deep learning in medical image analysis,” Med. Image Anal. 42, 60–88 (2017). [CrossRef]
30. A. Garcia, S. Escolano, S. Oprea, V. Martinez, P. Gonzalez, and J. Rodriguez, “A survey on deep learning techniques for image and video semantic segmentation,” Appl. Soft Comput. J. 70, 41–65 (2018). [CrossRef]
31. O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” Lect. Notes Comput. Sci. 9351, 234–241 (2015). [CrossRef]
32. L. Chen, P. Bentley, K. Mori, K. Misawa, M. Fujiwara, and D. Rueckert, “DRINet for Medical Image Segmentation,” IEEE Trans. Med. Imaging 37(11), 2453–2462 (2018). [CrossRef]
33. G. Huang, Z. Liu, L. Maaten, and K. Weinberger, “Densely connected convolutional networks,” IEEE Conf. Comput. Vis. Pattern Recognition, CVPR 2017, 2261–2269 (2017).
34. M. Khened, V. Kollerathu, and G. Krishnamurthi, “Fully convolutional multi-scale residual DenseNets for cardiac segmentation and automated cardiac diagnosis using ensemble of classifiers,” Med. Image Anal. 51, 21–45 (2019). [CrossRef]
35. S. Jastrzębski, D. Arpit, N. Ballas, V. Verma, T. Che, and Y. Bengio, “Residual Connections Encourage Iterative Inference,” ICLR 2018, 1–14 (2018).
