1. Introduction

Human white blood cells (leucocytes), which originate from haematopoietic stem cells in the bone marrow, provide protection from a wide variety of potential pathogens. Leucocytes can be further divided into granulocytes (neutrophils, eosinophils and basophils), monocytes and lymphocytes, each with individual functions, but which often act in a co-ordinated manner to combat infection. In particular, granulocytes, having prominent cytoplasmic granules, contain reactive enzymes which kill microbes and influence inflammation [1].

The most abundant type of granulocytes, numbering between 50 % and 70 % of circulating leucocytes under normal conditions are neutrophils which specialize in engulfing, capturing and killing microbes and bacteria like Staphylococus aureus [2]. Neutrophils have numerous immunomodulatory functions [3], including those in sepsis [4] and pattern recognition receptor functionality [5]. Defects in neutrophil granulocyte function can result in reduced resistance to infection [6], and chemotherapy-induced neutropenia is a common side-effect in patients undergoing breast cancer therapy [7].

The second major type of granulocyte are eosinophils, representing between 1 % and 3 % of leucocytes, which are frequently associated with defence against helminth and other parasites [8], where they can have dual functionality of pathogenesis and protective immunity. [9]. They also have crucial roles in host defence against respiratory viral pathogens [10], and in allergic lung disease [11–14].

Current identification methods often require destructive fixation, chemical staining or fluorescent labeling of the cells, which can be both time consuming and risk modifying the initial cell status if under subsequent biological study. In order to circumvent these disadvantages, optical techniques have been implemented for label-free hemograms [15], including wavelength modulated Raman spectroscopy (WMRS) [16], or more recently digital holographic microscopy (DHM) [17]. Whilst WMRS extracts information on molecular composition, DHM probes the morphology of the cells. A particular advantage of DHM is the significant potential for higher throughput when compared with Raman analysis due to its speed of acquisition. Recently, DHM has been proven to show typical acquisition rates of 600 cells per minute [17] as opposed to 3 cells per minute for WMRS.

To achieve a high classification accuracy, the experimental data acquired from label-free methods require a post-processing step involving multivariate analysis. These data processing techniques can be used to identify the patterns in the experimental data by systematically reducing their dimensionality. For example, principal component analysis (PCA) is a typical multivariate analysis method to extract the maximum variation present in the data. However, there is no metric by which one could claim PCA to be the optimal solution [18]. Artificial neural networks (ANNs) are the latest advancement in the field of machine learning for the classification based problems. They have the capability of classifying raw input data by automatically discovering feature representations [19]. Their inherent non-linearity improves classification abilities. The automated feature detection also provides a robust and easy solution for any classification process. Therefore, neural networks have been applied in many classification tasks, such as implementing multi-task learning to grade the severity of psoriatic plaques [20], classifying between different stages of metastatic cells [21], detecting stained blood cells between immune and cancer types [22], diagnosing lung cancer [23], prostate cancer [24] and skin cancer for dermatological evaluation [25], classifying mammography mass lesions [26] and lung patterns present during interstitial lung diseases [27], for rapid optical screening of anthrax spores [28] and classification of multiphoton microscopy images [29].

In this paper, we isolate neutrophils and eosinophils from human blood and record the WMRS and DHM data for the two cell types. Subsequently, we explain the method for finding the optimal ANN architectures in order to classify both WMRS and DHM data. We show that DHM combined with a convolutional neural network can be used as a robust, high-throughput and stand-alone hemogram for the classification of these immune cells. In contrast to the previous work [22], our approach implements ANNs for a label-free (i.e. unstained) method of cellular classification. This shows an advancement in technology with real promise for an optically enabled hemogram of future clinical relevance.

2. Methods

2.1. Cell isolation

Cell isolations were performed on whole blood collected from three different healthy donors after obtaining informed written consent, under ethical permission granted by the School of Medicine at the University of St Andrews. 20-30 ml of blood samples were collected in heparinised tubes. For isolation of eosinophils, whole blood was separated over Histopaque (Sigma, Poole UK). Untouched eosinophils were isolated using a MACsxpress eosinophil isolation kit (Mitenyi Biotec, UK, cat:130-092-010) following manufacturer’s instructions. Untouched neutrophils were directly isolated from the whole blood by using MACSxpress neutrophil isolation kit (Miltenyi Biotec, UK, cat: 130-104-434) following manufacturer’s instructions. Small numbers of remaining erythrocytes were removed using an erythrocyte lysis kit (Milteny Biotec, UK cat: 130-094-183) following manufacturer’s instructions. Isolated neutrophils were cultured in the plasma from which they were isolated in the kit, and eosinophils were cultured in RPMI supplemented with 5% Foetal Bovine Serum (FBS, both Invitrogen, UK). For optical analysis, cells were isolated by centrifugation and resuspended in Phosphate Buffer Saline (PBS) with 0.5% FBS solution to avoid aggregation. For the optical measurements, 20 μl of the prepared cell suspension was transferred to the center of a quartz slide (25.4 mm × 25.4 mm × 1 mm) chamber, formed by use of an 80 μl thick vinyl spacer. This chamber was covered from the top using a thin quartz slide (25.4 mm × 25.4 mm, 0.11 mm - 0.15 mm thickness) and the whole assembly was inverted and left for 20-30 minutes to avoid cell motion.

 

Fig. 1 Schematic of the Convolutional Neural Network. The network takes phase images as its input and processes these though four convolution blocks and finally connects to a sigmoid neuron for the binary classification. Here we see the (a) Input phaseimage, (b) the convolution layer containing 30 5×5 filters with 1×1 stride and 1×1 padding, and (c) the convolution layer containing 30 4×4 filters with 1×1 stride and 1×1 padding, (d) the convolution layer containing 30 3×3 filters with 1×1 stride and 1×1 padding,(e) the convolution layer with 30 2×2 filters with 5×5 stride and 1×1 padding. Each convolution layer is followed by a batch normalization and ReLu Layer. The final convolution block has a max-pooling layer with 2×2 filter size and 1×1 padding and (f) shows the binary representation of output classification layer with a single sigmoid neuron which is fully connected to the output of the final convolution block.

Download Full Size | PDF

2.2. Flow cytometry

Both purified populations of cells were washed into PBS with 0.5 % FBS and blocked with 10 % human plasma for 10 mins on ice prior to labelling with antibodies. Anti-CD3-FITC (eBioscience, clone OKT3, cat: 11-0037-41) was used as a negative control for both cell types. Purified neutrophils were stained with anti-CD15-FITC (eBioscience, clone H198, cat: 11-0159-41), and eosinophils with anti-CD66b-FITC (eBioscience, clone G10FS, cat: 11-0666-42). Flow cytometry was performed on a Guava 8HT (Merck-Millipore, UK).

2.3. Digital holographic microscopy acquisition

Holographic images of cells were captured using a Mach-Zehnder interferometer-based off-axis digital holographic microscope [17]. The incident laser (at a wavelength of 532 nm, Spectra-Physics Millennia Vs) beam was passed through a beam splitter to feed into both signal (power = 25.5 μW at sample plane) and reference (power = 30.1 μW) arms of the interferometer. The light beam in the reference arm was passed through a 2X a focal telescopic system of lenses (f = 2.5 cm and 5.0 cm). The signal arm consists of a long working distance objective lens (Mitutoyo UK, M Plan Apo 10x NA = 0.23) coupled with a 60X microscopic objective (NA = 0.85, Nikon). The sample was placed between the two objectives and the image was interfered with the light from the reference arm at the surface of the CCD camera. This was set at 40 fps with an exposure time of 33.3 ms.

 

Fig. 2 Schematic of the Multi-layered perceptron. The network takes a WMR spectrum as the input (layer with 351 neurons) and processes it though a hidden layer (90 neurons) and finally connects to a sigmoid neuron for the binary classification. (a) Representation of mean WMR spectra for the two cell lines, (b) input Layer with nodes equal to the number of data points in each spectrum,(c) hidden layer with 90 neurons and tan hyperbolic as the activation function, (d) representation of classification layer with a single logistic sigmoid neuron for binary classification, here red neuron represents the Eosinophil class whereas the green neuron represents the Neutrophil class.

Download Full Size | PDF

2.4. Spectral acquisition

Single cell WMR spectra with a resolution of 2.4 $c{m}^{-1}$ were collected by placing the prepared sample on a confocal wavelength modulated Raman system (details of which may be found in earlier studies [30, 31]). The central wavelength was set at 785 nm and a modulation step of 0.2 nm was applied for 5 steps. The laser power at the sample plane was set at 200 mW and the spectra were acquired for 5 seconds each at each incidence laser wavelength.

2.5. Phase image calculation and classification

Wide field phase images were calculated from the holographic images by calculating the 2D Fourier transform followed by a 2D phase unwrapping algorithm as explained in our earlier work [17]. Single cell phase images (170 pixel × 170 pixel) of neutrophils and eosinophils were cropped from the wide field image and further downsized to 150 pixel × 150 pixel using bi-cubic interpolation for analysis. The single cell images were downsized to 150 pixel × 150 pixel, whilst retaining the features in the input phase images, permitting augmentation and a reduction in the overall computation time. These phase images capture the refractive index variation across the cytoskeleton of a cell which acts as a morphological fingerprint for the specific cell type.

The classification of neutrophils and eosinophils can be thought of as a binary problem. Here, the input can be considered as the phase image extracted from the DHM fringe patterns such that the output is a binary label $k\in \left\{0,1\right\}$ indicating the two classes of cells. The classification of the single cell phase images was performed (in Matlab2017b, trained using Nvidia Quadro P5000 GPU) using a deep learning based 17 layered convolutional neural network (CNN) as represented in Fig. 1. The CNN inputs phase images and processes them by passing through four fundamental convolution blocks (CBs) containing convolution layer, batch normalization layer [32] and, rectifier unit layer (ReLu). The final block was connected with a max pooling layer. The output of max pooling was fully connected to the classification layer containing one neuron with logistic sigmoid as the activation function. The architecture represented in Fig. 1 was chosen for gaining maximum accuracy over the validation dataset after changing the number of CBs from 2 to 10, changing the filter sizes in each CB from 2×2 to 5×5 and by changing the number of convolution layers from one to three in each CB.

The network was trained on 235 single cell phase images (150 × 150 pixel) for each class. For a single example in the training set, the binary cross entropy loss function, as expressed in equation 1, was optimized.

In the above equation, $p(k|j)$ is the probability that the network assigns to the label j with $A_1=\left|N_1\right|/\left(\left|N_1|+|N_2\right|\right)$ and $A_2=\left|N_2\right|/\left(\left|N_1|+|N_2\right|\right)$ with N₁ and N₂ being the total number of cases for each class. The weights and biases of the network were initialized by seeding the random environment. The network was trained end-to-end using stochastic gradient descent with momentum as $4\times {10}^{-1}$ and L2 regularization as $5\times {10}^{-6}$. Maximum epochs were set to 100 with validation patience of 5 iterations and a mini batch size of 40 images was considered for training the model with a learning rate of $5\times {10}^{-5}$. Before the training process, a zero center normalization was implemented on the dataset.

To benchmark the performance of our CNN, a classification based on the previously suggested technique of applying PCA in combination with LOOCV was also performed over the calculated intensity histograms [17]. Intensity histograms were calculated to represent the information about the size of image, maximum optical path difference (OPD), and total OPD over the entire cellular phase image. These histograms were then used as the one dimensional descriptor vectors for the datasets and PCA was conducted on the new dataset. The number of principal components were optimized to gain maximum classification accuracy over the training and validation datasets.

2.6. Spectral processing and classification

The collected single-cell Raman spectra were normalized with respect to the total spectral intensity which allowed us to cancel any noise generated in the signal due to laser power fluctuations. The Raman shift of the system was linearly calibrated with respect to the characteristic 1001.4 cm${ }^{-1}$ Raman peak of a polystyrene bead. WMRS is based on the physical principle that the fluorescence background essentially remains constant whereas the Raman scattering is shifted by the periodical modulation of the laser wavelength [33]. A modulated Raman spectrum was calculated as PC1 for the PCA on normalized five Raman spectra collected from a single cell by using each excitation wavelength step as a parameter [16]. In this way, fluorescence-background-free spectra can be effectively obtained without using any fitting or numerical correction.

 

Fig. 3 Flow cytometric analysis of purified untouched eosinophils (a) and neutrophils (b) demonstrating forward and side scatter profiles of purified cells and antibody staining with anti-CD3-FITC (negative control) and anti-CD66b-FITC for eosinophils,and anti-CD3-FITC (negative control) and anti-CD15-FITC for neutrophils.

Download Full Size | PDF

The classification of the WMRS dataset can be considered as a pattern recognition problem for a machine. Every cell type exhibits a signature one dimensional WMRS pattern. To classify these 1D patterns, a multi layer perceptron network (MLP) architecture (patternNet, MATLAB 2017b, trained using CPU, Intel Xeon CPU E5-1630 v4 @ 3.70GHz) was implemented. The MLP comprised of an input layer with 351 neurons, a hidden layer having 90 neurons with a tan hyperbolic activation function and a final output layer of one neuron with logistic sigmoid as the activation function. A schematic of the MLP is represented in Fig. 2. The aforementioned MLP model represented in Fig. 2 was chosen for obtaining maximum accuracy over the validation dataset by changing the number of hidden layers ranging from one to three, and changing the number of neurons in each layer from 50 to 200 over the step of 10 neurons. The MLP classification was optimized using stochastic gradient descent with momentum and adaptive learning rate [34] with momentum as $5\times {10}^{-1}$ and L2 regularization as $1\times {10}^{-4}$. Maximum epochs were set to 2000 with validation patience of 5 iterations and a mini-batch size of 20 spectra was considered for training the model. We use an initial learning rate of $1\times {10}^{-3}$ which was decayed by a factor of 10 each time the validation loss plateaus.

To compare and benchmark the results from the MLP, a non-parametric nearest neighbor algorithm based on PCA was also implemented [16]. The model based on PCA was optimized by observing the effect of number of principal components on the classification accuracy of LOOCV over the training and validation dataset. First 8 PCs corresponding to 87.1 % of the total variance were considered and a classification was performed using LOOCV based on the nearest neighbor algorithm. Increasing the number of PCs does not improve the classification accuracy since higher PCs only contribute to the noise present in the spectra.

 

Fig. 4 (a) A subsection of the accumulated bright field image using our microscope, (b) fringe image for the same section, (c) phase image extracted from (b). Scale bars: 10 μm (image), 5 μm (inset).

Download Full Size | PDF

The above mentioned neural network architectures and training processes can be further remodeled and improved to train over multiple classes of immune cells.

3. Results

3.1. Cell isolation

Untouched human blood neutrophils and eosinophils were obtained by negative depletion, i.e. all other cells not of interest were removed using cell-lineage specific antibodies. Flow cytometry confirmed the isolation of monodisperse cell populations by forward and side scatter profiles, which were negative for the T lymphocyte marker CD3, but which stained 100 % positive for CD15 in the case of neutrophils and 100 % for CD66b for eosinophils (Fig. 3). Significantly, as in our previous studies [16, 17], the isolation of untouched cell populations by means of a negative depletion approach means that the cells under study are completely free of any labeling with antibodies or beads typically used in other isolation approaches, and therefore represent as far as possible cells in situ.

3.2. Digital holographic microscopy

Bright field images and holographic fringe images of a 50 μm × 40 μm area were collected at a rate of 40 frames per second and corresponding phase images were calculated for each frame. As depicted in the Fig. 4, depending on the cell count per ml, each wide field image can have a maximum of 9 cells and a minimum of 1 cell, which enables the high-throughput capabilities of the system with an acquisition rate of more than 100 cells per second. Single cell phase images (170 × 170 pixel) for the neutrophils and eosinophils were cropped manually as shown in Fig. 4(c) which is a limitation in our current study. However there exist low computation cost methods for attention based automatic image cropping [35] which can be used to overcome this limitation and shall be a part of future studies. As a preprocessing step, these images were further downsized to 150 × 150 pixel using bicubic interpolation method forthe analysis. Figure 5 represents the single cell phase images of eosinophils and neutrophils. The collected images clearly signify the morphological similarities present between the cells. Both the cellular structures show a round morphology of similar size (≈10 μm in diameter) with differences across the central cellular structure representing the granularity of these cells.

To consider the inter-donor variability, we accumulated the complete dataset from three different donors and divided the phase images into a training set from donor 1, validation set from donor 2 and a testing set from donor 3 (the number of wide field images considered for each cell type from each donor are explained in Table 1 whereas the number of single cell phase images are expressed in Table 2). After training the convolutional neural network (CNN) model, a classification was performed on the test dataset for each cell type. The performance of CNN over the training and validation sets are represented in the form of confusion matrices in Tables 3 and 4. The results are represented in Table 5 in the form of a confusion matrix which shows a sensitivity of 100 % and a specificity of 82.7 %. The trained network classified the input images at a rate of 435 frames per second which was more than 3 times the single-cell image acquisition rate. This indicates that CNN complements the high-throughput capabilities of the DHM system and can be used in real time for the classification purpose.

The classification of the training and validation dataset using the CNN based model was performed in approximately 109 seconds. We note, that the model can rapidly classify the test dataset at a frame rate of 435 frames per second.

 

Fig. 5 Normalized phase images of the Granulocytes. (a), (b), (c) Neutrophils; (d),(e),(f) Eosinophils. These images represent the inter-cellular structural variation in the form of refractive index map which expresses the granularity of the two cell types. Color bar represents the normalized phase difference between the signal and reference arm.

Download Full Size | PDF

Table 1. Summary of DHM wide field phase images considered for cropping the individual phase images.

View Table | View all tables in this article

Table 2. Summary of the single cell DHM phase image dataset collected from the three donors

View Table | View all tables in this article

Table 3. Confusion matrix summarizing the classification performance of the CNN over the training set of the phase images

View Table | View all tables in this article

Table 4. Confusion matrix summarizing the classification performance of the CNN over the validation set of phase images

View Table | View all tables in this article

Table 5. Confusion matrix representing the prediction accuracy of the trained CNN and PCA/LOOCV for classification of the neutrophils and eosinophils on the test dataset. Each row of the matrix expresses the total number of phase images of the cells available for classification, whereas each column represents the predicted cell lines. Thus, the diagonal elements of the confusion matrix represent the correct predictions made by the network whereas off-diagonal terms represent the inaccurate predictions.

View Table | View all tables in this article

The single cell test phase images were also classified by applying PCA in combination with leave one out cross validation (LOOCV) over the calculated intensity histogram for each image [17]. This method classified the images at a rate of 5 frames per second and gave a sensitivity of 87.9 % and a specificity of 72.4 %, which can be calculated from Table 5. The performance of the PCA based technique for classification of training and validation data is expressed in terms of PC scatter plots (Fig. 6) and confusion matrices (Table 6). The PC scatter plots clearly show an absence of variability between the two classes of the cell types which confirms the similarity between the morphology of the two cell types.

Interestingly, the PCA based model can classify the training and validation dataset in the same amount of time (109 seconds) while this very same model can classify the test dataset at a much lower frame rate of 5 frames per second compared to the CNN based model.

 

Fig. 6 (a) PC1-PC2 scatter plot for the two cell types; (b) PC2-PC3 scatter plot for the two cell types; (c) PC3-PC1 scatter plot for the two cell types; Eosinophils are represented by the data points in red whereas the neutrophils are represented by the data points in blue

Download Full Size | PDF

Table 6. Confusion matrix summarizing the classification performance of the PCA over the training and validation set of phase images

View Table | View all tables in this article

The above represented results confirm that digital holographic microscopy when combined with deep learning can be used to create a highly-accurate hemogram device with high-throughput for the label-free classification of the cells of the immune system.

3.3. Raman spectroscopy

Standard and WMR spectra were collected from the neutrophils and eosinophils. Figure 7 depicts the WMR spectra for the two cell subsets.

 

Fig. 7 Wavelength modulated Raman spectra illustrating pairwise comparison between eosinophils and neutrophils. Solid lines show the mean spectrum for each cell subset and shadowed regions represent the standard deviation.

Download Full Size | PDF

The collected WMRS dataset was classified using the multi layered perceptron (MLP) model explained in the methods section. To consider inter-donor variability, we trained the MLP on the data considered from the two donors and tested its performance over the data accumulated from an independent donor, this approach of splitting the data resulted in low classification ability of the MLP based model (validation accuracy of 97.1 % and test accuracy of 67.0 % with 120 hidden neurons). The reason for the same can be attributed to the presence of experimental variations while accumulating the data for individual donors which in turn contribute to the biasing for the training of MLP based model. In addition, the absence of inter-donor variability can be verified by considering the PCA and t-SNE (Fig. 8) analysis of the WMRS data. Each method gave a clear clustering of the WMR spectra of the two cell types over all the data collected from all the three donors. Hence to overcome the effects of experimental bias [36] in the dataset, the complete dataset was randomly divided into a training set (70 % of the total spectral dataset), a validation set (15 % of the total spectral dataset) and a test dataset (15 % of the total spectral dataset) [37–39]. The performance of MLP over training and validation datasets are expressed in the form of confusion matrices in Tables 7 and 8. The MLP showed a sensitivity of 96.7 % and a specificity of 100 % on the test dataset as depicted by the confusion matrix presented in Table 10.

The time taken for the training and validation with MLP based model was less than 1 second (0.93 seconds) whereas the classifying of the test spectral dataset was more rapid, taking a mere 70 ms.

 

Fig. 8 Principal component analysis and t-SNE scatter plots. (a) 3D PC scatter plot showing the clustering of the neutrophils and eosinophils. The red points on the scatter plot correspond to the Eosinophil WMR spectra whereas the green points on the scatter plot correspond to the neutrophil WMR spectra. (b) t-SNE scatter plot in 2D showing a clear clustering of the complete dataset. The red points in the scatter plot correspond to Eosinophils WMR spectra whereas blue points correspond to Neutrophil WMR spectra.

Download Full Size | PDF

Table 7. Confusion matrix summarizing the performance of MLP on the training dataset

View Table | View all tables in this article

Table 8. Confusion matrix summarizing the performance of MLP on the validation dataset

View Table | View all tables in this article

PCA in combination with LOOCV was also performed on the collected Raman dataset and first eight PCs accounting for 87.1 % of the total variance, were considered (the performance of PCA/LOOCV based classification model over the training and validation sets is expressed in the form of confusion matrix in Table 9). The PC scatter plot, shown in Fig. 8(a), of the analyzed data points displayed their clustering corresponding to individual classes in the PC space. The classification accuracy of the analysis was calculated using LOOCV. The PCA based analysis resulted in a sensitivity of 93.5 % and specificity of 98.3 % as calculated from the confusion matrix depicted in Table 10.

The time (10.57 seconds) taken for the training and validation process using the PCA based model more than an order of magnitude longer than using MLP based model. The PCA based model could more rapidly classify the test spectral dataset in 600 ms, though this is still approximately eight-fold longer in time than the time required by the MLP based model.

Table 9. Confusion matrix summarizing the performance of PCA on the Training and validation datasets for WMRS

View Table | View all tables in this article

The results described above show that the classification of cells using either WMRS and DHM can be improved by implementing the artificial neural networks, provided the environmental variances are resolved.

Table 10. Confusion matrix representing the prediction accuracy of the trained MLP and PCA/LOOCV for classification of the neutrophils and eosinophils on the test dataset. Each row of the matrix expresses the total number of WMRS of the cells available for classification, whereas each column represents the predicted cell lines. Thus, the diagonal elements of the confusion matrix represent the correct predictions made by the network whereas off-diagonal terms represent the inaccurate predictions.

View Table | View all tables in this article

4. Discussion

In this study, we set out to determine if the ANNs may improve the classification sensitivity of the label-free optical methods of WMRS and DHM. The comparison of ANN analysis with the previously implemented PCA/LOOCV based method, for the WMRS and DHM data representing the neutrophils and eosinophils, shows that the classification efficiency of the systems is improved. Hence, we have determined that the application of artificial neural networks represents a significant step towards a label-free haemogram of immune cells, especially to enhance the performance of imaging modality.

During the training process, the neural networks automatically learn to classify the training dataset. This classification ability is in-fact gained by learning the generalized transformations of the input dataset. These transformations are learned by calculating the cost function at the output layer and updating the layer-wise parameters with respect to the variation in cost function.

In a CNN, these parameters are the shared filters which convolve with the input layer to form the output representations. These representations generally contain the collective features present in the input dataset. Each layer, extracts different level of representation suggesting common features present in the input image dataset, hence suppressing irrelevant or noisy variations. The hierarchical feature extraction for the applied CNN is shown in the Fig 9. As expressed in the figure, the input phase image is repeatedly transformed at each layer using the learned parameters. After the final feature extraction, the network predicts the class of input image based on the most relevant features present in the dataset.

In a MLP, all the layers are densely connected which means that each node in the input layer is connected to each node in the output layer with an associated parameter. These parameters are updated in the same way as explained above.

 

Fig. 9 Demonstration of the automated feature detection and classification using a CNN. (a) Input phase image of an Eosinophil from test dataset. Different column images represent five of the total layer activations, here (b), (c), (d) and (e) represent the network activations at the four convolution layers.

Download Full Size | PDF

Regarding the WMRS data, a modest improvement of 2.5% in the classification ability was observed by using an MLP approach when compared to PCA. A fundamental difference between WMRS and DHM is that WMRS provides highly specific molecular information. Therefore, for morphologically similar but biochemically distinct cells, one would expect that WMRS would give a very high specificity (evident in Fig. 8) when compared to DHM (see in Fig. 6). Thus, in this case it is not surprising that this specificity is translated to a very comparable high performance for both the MLP and PCA methods.

The morphological similarity in the cells leads to a lower specificity in DHM data, as might be expected. Convolutional neural networks add significant value to the classification ability of DHM (an improvement of 11.1% when compared to PCA). This improvement can be attributed to the presence of non-linear modules in the neural network architectures. Each non-linear module transforms the representation at one level into a representation at a higher, slightly more abstract level [19]. Specifically, for a two dimensional data-set such as the phase images, PCA when applied with a histogram approach fails to identify the cells of the immune system. This can be attributed to the loss of variation between pixels (in 2D) when generating the one dimensional histograms for the classification process. This spatial pixel variation is, by design, observed and parameterized by the CNNs in the form of filters.

When compared to PCA, there are significant improvements in processing speed when using ANNs. For the WMRS dataset, MLP shows a reduction in processing time for training/validation datasets and test datasets respectively as mentioned earlier. In case of the DHM dataset, though both CNN and PCA take the same amount of time for the training and validation phase, CNN shows a significant reduction in processing time by nearly two orders of magnitude (i.e. eighty-seven fold), when compared to PCA. Therefore, such high throughput capabilities with a combination of CNNs and DHM promise a powerful route for the classification of morphologically similar cells of the immune system.

In addition to improving the classification ability of the two optical systems, we also show a 10 fold improvement in the throughput of the DHM system as compared to the previous work. This was achieved by optimizing the camera and expanding the field of view from 10 μm × 10 μm to 50 μm × 40 μm [17].

In this study, the MLP based analysis shows a low classification accuracy when following a donor-wise data splitting method. This limitation can be attributed to the presence of high experimental variance across the data-set which in turn contributes to the biases [36] or prejudices for the training of MLP based model. However, as demonstrated above, this limitation can be overcome by randomly dividing the complete dataset, which in turn resolves the presence of experimental variance and hence the MLP based model performs with high efficiency.

Another beneficial advantage of deep learning based algorithms is that they provide flexibility to remodel and improve the neural network architectures to train over multiple classes and different problems.

In the current study, both the modalities of characterization were measured individually. In the future, we propose to implement a multi-modal approach for both measurement [17] and analysis [40] which can provide an online multi-modal solution to the cellular classification problem. Additionally, the DHM system may be further enhanced by an increase in the field of view to increase throughput. This would lead to an interesting future study to see if CNNs may classify lower resolution phase images of morphologically similar cells of the immune system in this case.

5. Conclusion

In conclusion, this study has used multivariate analysis based approaches for label free hemograms using either WMR spectra or phase images obtained using digital holographic microscopy from two cell lines. For morphologically similar cells, DHM with CNNs can provide a throughput rate of greater than 100 cells per second at a classification accuracy of 91.3%. In particular, our approach indicates that digital holographic microscopy when combined with CNNs can be used as an automated, highly efficient and stand-alone system to classify clinically important cells of the immune system.

This study also indicates that WMRS by itself proves to be highly efficient when the samples of study are biochemically very different. Hence, including multiple modalities in the optical system can be very beneficial for cell classification. The efficiency of the above system shows promise for rapid cell classification and application in the field of hematology. This approach would avoid the use of antibody labeling techniques as it confirms that label-free optical techniques yield equivalent accuracy.