Open Access
Add to BibSonomy
Abstract
Fiber specklegram sensors (FSSs) traditionally use statistical methods to analyze specklegrams obtained from fibers for sensing purposes, but can suffer from limitations such as vulnerability to noise and lack of dynamic range. In this paper we demonstrate that deep learning improves the analysis of specklegrams for sensing, which we show here for both air temperature and water immersion length measurements. Two deep neural networks (DNNs); a convolutional neural network and a multi-layer perceptron network, are used and compared to a traditional correlation technique on data obtained from a multimode fiber exposed-core fiber. The ability for the DNNs to be trained against a random noise source such as specklegram translations is also demonstrated.
© 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Optical fibers as sensors have been extensively researched, and notable commercial applications include gyroscopes and structural health monitoring [1]. Interest in this technology is due to the properties of optical fiber sensors (OFSs), including their light weight, small size, absence of moving parts, and resistance to electromagnetic fields [1–4]. Generally, an OFS functions by correlating changes in an external measurand of choice (such as temperature, strain or pressure) [5], with measurable changes in the phase, polarization or intensity of the light propagating through it. A particularly sensitive approach is to measure the phase properties of the propagating light through interference methods. After propagating through the fiber, the phase of the light at the output depends on the optical path length (OPL), which depends on the effective refractive index of the propagating mode, as well as any perturbations on the fiber affecting its refractive index and length. Directly tracking changes in the phase of light is not feasible due to its very high frequency. Therefore, phase changes are often measured through interferometric techniques such as Fabry-Perot, Mach-Zehnder or, in the case of multimode fiber (MMF), interference between propagation modes.
A specklegram is the total intensity pattern at the end facet of the fiber, and consists of bright and dark features of constructive and destructive interference caused by the relative phases between the modes [6]. Fiber specklegram sensors (FSSs) are OFSs that use an MMF’s specklegram to quantify changes in these relative phases. Given the OPL of each mode will respond differently in the presence of environmental changes, the relative phases between these modes and hence the specklegram becomes highly sensitive to the environment surrounding the fiber [7]. FSSs also have the advantage of only needing low cost, lightweight interrogation equipment. A lens, coherent light source and webcam is enough to capture a specklegram [8,9] so they have found many unique applications, including motion sensing and surveillance [10,11], strain sensing [12], vibration sensing [7], temperature sensing [13] and wavelength detection [14]. The analysis and comparison of specklegrams from an FSS generally follows one of two statistical methods. The first, correlation, relies on multiplying the pixels of two specklegrams together in order to correlate them and hence uses their similarity as images [7,15]. The second, differencing, relies on subtracting the pixels of the two images in order to use their differences as images [11,16].
Despite the numerous applications FSSs have found solutions for, the current methods of analyzing and comparing specklegram images are at heart purely statistical. The cumulative effect of external perturbations is averaged along the fiber, furthermore, all pixels in the specklegram contribute to a single value quantifying a correlation/difference [8]. In this way, they do not offer any physical insight into how the multimode transmission is occurring throughout the length of the fiber, nor do they offer distributed sensing capabilities, and suffer dynamic range limits. The MMF transmission process is so sensitive to small perturbations that building a mathematical model to describe the transmission can be complex and often limited to special cases. Research has been done in describing the physical transmission of light through an MMF such as building transmission matrices (TMs) [17,18] and phase conjugation [19,20], with applications in imaging and biomedical sensing, for example microscopy [21] and endoscopy [22]. Despite such successes, these analytical methods are highly vulnerable to changes in the fiber’s environment, such as bends or defects. Furthermore, they are not suitable for sensing purposes, as the nonlinear effects of environmental changes on MMF transmission go beyond the capabilities of a linear transmission matrix.
The use of deep learning has been explored as a solution to describe MMF transmission by inferring the relationship between an input and a speckle pattern without the need for prior knowledge of the physics and fiber properties governing the transmission [23,24]. A deep neural network (DNN) is a tool to map between potentially very high dimensional data using a non-linear transformation with many parameters, through statistical learning [25]. Deep learning can be considered state of the art in terms of artificial intelligence, displaying capabilities to learn complex patterns in high-dimensional spaces, and solve problems in many fields, most notably image and speech recognition [26,27]. The power of DNNs has also been explored in the context of mapping the complex MMF transmission. Two maps of interest that DNNs are able to learn in regards to MMF transmission include imaging, where the fiber’s environment is held constant and a map is learnt between the input and output of the fiber [23,28–32], and sensing, where the input of the fiber is held constant and a map is learnt between the fiber’s environment and its output [33,34]. To date, the sensing application of DNNs has been far less explored compared to imaging.
We have previously demonstrated the use of a statistical correlation method [13]; the zero-normalized cross correlation (ZNCC) function, to make an FSS out of a multimode exposed core fiber (ECF). An ECF is a microstructured optical fiber with one side open to the external environment, allowing sensing of external materials through the evanescent field [35]. We have previously demonstrated the use of such a fiber as a suitable tool for biological sensing [36–38]. In this work we utilize two DNN architectures to analyze the change of specklegrams from such an ECF with respect to temperature and refractive index in the form of variable water immersion, the latter of which is only made possilengthble by use of the ECF which allows media of different refractive indices to be in contact with the core of the fiber. We emphasize that the fiber environment is designed to have minimal external perturbations and noise other than the measurand of interest so as to provide a controlled proof of concept. For a demonstration of a DNN’s ability to extract a measurand amongst strong noise, see our previous work [33]. We demonstrate that two different DNNs improve on the ZNCC method by extending the dynamic range of the sensor, as well as demonstrating robustness against specklegram image translations.
2. Concept and experiment design
2.1 Experimental Principle
The light guided along an optical fiber can be expressed as a superposition of orthogonal modes, each being a unique solution to Maxwell’s equations given the boundary conditions imposed by the fiber geometry. At a distance L from the entrance to the fiber, the field transverse to the direction of propagation can be expressed as a superposition of guided modes, as in Eq. (1),
The specklegram’s characteristic speckle effect comes from interference between the modes, with bright regions corresponding to constructive interference (modes in phase) and dark regions corresponding to destructive interference (modes with a π phase difference). Figure 1 shows the specklegram from an ECF being imaged using a lens, which can then be captured using a camera or webcam.
The OPL that each mode experiences is given by the product of its effective refractive index and the geometric length travelled (OPL = neffL). All non-degenerate modes will have phase differences along the fiber due to differing neff. The sensitivity of these phase differences are highlighted by the fact that a full cycle of phase (2π) for modes in a fiber has a length of λ / neff, which is on the order of microns. The output specklegram is also highly sensitive to deviations from an ideal fiber such as noise or fiber defects. In the ideal case of no noise, a specklegram will change continuously and deterministically with a continuous change in the fiber’s environment, but this breaks down in the presence of strong noise. This leads to the idea that specklegrams can be used to sense changes in a fiber’s environment, provided noise is not too strong.
Fig. 1. Representation of multimode transmission of light through an exposed core fiber. The electric field in the fiber is a sum of all its modes. These modes and their relative phases respond differently to external perturbations. A lens is used to image the intensity pattern (specklegram) at the end facet. Also displayed is a cross-sectional image of the fiber used. Reprinted from [13].
Figure 2 shows three specklegrams, each from air temperatures 3°C apart. Although to the human eye it is apparent the specklegram is changing, and the movement/evolution of some features are able to be tracked, the overall trend governing this change cannot be easily determined, and certainly not extended beyond this temperature range. Statistical methods compare how specklegrams differ or correlate pixel to pixel in order to quantify this trend. They can, however, suffer from limited dynamic range as the images become increasingly different from the reference image and further changes in the specklegram reveal less information about how the specklegram has changed. Hence, in this work, we demonstrate that deep learning is a more powerful tool for mapping the seemingly arbitrary relationship between a fiber’s specklegram and its environment over a larger dynamic range, improving both accuracy and robustness against unwanted noise. We have previously demonstrated this approach using deep learning on the wavelength dependent transmission spectrum of a multimode fiber coupled to a single mode fiber, rather than specklegram, under strong noise conditions for temperature sensing [33].
Fig. 2. Specklegrams at the end facet of the ECF corresponding to temperatures of (a) 23°C, (b) 26°C, and (c) 29°C.
2.2 Experimental setup and data collection
The ECF used in this experiment was fabricated by drilling six holes into a pure silica preform using an ultrasonic CNC machine [13]. A diamond endmill was used to open a slot, exposing one of the holes to the outside environment. It was drawn with an optical fiber draw tower at 2005°C. An example cross-sectional image of the fiber is shown in Fig. 1. The length of ECF used for temperature and sensing was 13 cm and 11 cm respectively. This length finds a balance between the sensitivity required to resolve temperature changes, and the dynamic range. water immersion length
The experimental setup, shown in Fig. 3, consisted of a He-Ne laser beam (633 nm, 5 mW) passing through a neutral density filter before being launched into the ECF via a 20× microscope objective lens, and from the end facet of the ECF imaged onto a webcam via a 40× microscope objective lens. The ECF was placed in a fiber bathing apparatus manufactured out of an aluminium block with a Peltier device underneath, to control temperature and water immersion length. The webcam captured footage at an 800×600 resolution and at 15 frames per second. For each discrete temperature/immersion length, approximately 10 seconds of footage was taken, with each frame producing a single specklegram for use in analysis. This resulted in approximately 150 specklegrams per measurand label. The specklegrams within each video differ in that they are affected by inherent noise in the fiber’s environment and electrical noise in the webcam. Temperature data was taken in increments of 0.5°C over a range of 22.5-30°C, while immersion length data was taken in increments of 5 mm over a range of 0-50 mm. These ranges were chosen as the easiest to achieve while capturing the ZNCC’s full dynamic range. For more information about the experimental setup and fiber fabrication, see our previous work [13].
Fig. 3. Schematic diagram of the experimental setup used to capture specklegrams from an ECF in the presence of temperature and refractive index changes. NDF, neutral density filter; M1, M2, mirrors; L1, 20× microscope objective; L2, 40× microscope objective; ECF, exposed core fiber. Reprinted from [13].
2.3 Zero-normalized cross-correlation (ZNCC)
The correlative statistical technique used in our previous work [13], and again included in this paper for comparison is the zero-normalized cross correlation (ZNCC) function, defined in Eq. (3):
In this work we use a similar image processing method as in [13]. The 800×600 images were cropped to a 200×200 region of interest in order to eliminate the black region surrounding the specklegram. These were then downsampled to 60×60 images in order to save computational power when training the DNNs.
Once the ZNCC scores for each specklegram in a dataset were calculated against the reference frame, a sixth order polynomial calibration curve for the measurand versus ZNCC score curve was fitted. This function was used to make predictions on specklegrams unseen during the calibration process and quantify the performance of the method.
2.4 Deep neural networks (DNNs)
Two broad classes of DNNs, both used in this work, include multi-layer perceptrons (MLPs) [39] and convolutional neural networks (CNNs) [40]. MLPs consist of a series of fully-connected layers, ending with a layer the size of the number of categories to classify or regression variables to calculate. Each node in each layer gets its value from a weighted sum of all nodes in the previous layer. Images and higher dimensional data must be flattened to a single vector before passing through an MLP. On the other hand, a CNN uses convolutional layers, each consisting of a set number of convolutional filters that scan the image, calculating a correlation with the corresponding image pixels at each possible position, generating feature maps that hold spatial information as to the image’s features. MLPs value each connection between each node equally, whereas CNNs will prioritize connections between adjacent datapoints/pixels, which can be useful for learning recurring features in images.
Firstly, we used an MLP, schematically represented in Fig. 4. When flattened, the 60×60 input image meant our input layer had 3600 nodes. Our output layer, given it was predicting a single value (either temperature or immersion length), had just one node. In between were five hidden dense layers, each decreasing in size from the input size to a single node, a common tactic in MLP architecture design. These hidden layers are there to learn the latent space of the input data in order to facilitate the regression output layers. The rectified linear unit (ReLU) activation [41] was applied after each layer. The model had in total 9,752,929 trainable parameters.
Fig. 4. (a) Schematic of the multi-layer perceptron DNN (MLP) used. The pixel values of the input image are flattened to a single vector and make up the input layer, followed by a series of fully-connected layers. Each node’s value is determined by a sum of the contributions from all nodes in the previous layer. The output layer has one node which represents our desired measurand label. (b) Schematic representation of a convolutional neural network (CNN). Each convolutional layer consists of a set number of convolutional filters, each one creating a “feature map”, which is a map of correlations as the filter scans the image. These feature maps are then down sampled using a pooling method, comprising one convolutional layer.
Secondly, we used a CNN architecture based on the VGG-16 architecture, a winner at the ImageNet Challenge 2014 image recognition competition [42]. Despite many follow-up architectures being developed since, the VGG architecture continues to work well for many image recognition applications [43,44], as well as for MMF transmission [23]. The architecture features three VGG ‘blocks’, each consisting of two or three convolutional layers followed by a maxpooling layer [45], which downsamples the result by pooling pixels together and taking the maximum value of each ‘pool’. Our architecture’s three blocks featured two, two and three convolutional layers with 64, 128 and 256 convolutional filters each respectively. Following the third block, the result was flattened and passed through four fully-connected layers, finishing with a single node. All convolutional and dense layers also used the ReLU activation. The model had in total 29,787,329 trainable parameters. This appears as a very large number of parameters, but we believe models of this size are justified given the highly complex nature of the specklegram, which depends on interference between every possible pair of supported modes in the fiber. The ECF used in this research is very highly multimodal, supporting thousands modes, which indicates just how many degrees of freedom such a specklegram depends on. We also believe the use of DNNs over simpler methods of high-dimensional data mapping, such as a supported vector machine [46], is justified. Such methods often require prior knowledge about the domain of the data in order to find and visualize high-dimensional trends, and, while computationally lighter, can be more time consuming overall.
The entire image dataset consisted of approximately 150 specklegrams per measurand label. For the temperature and water immersion length datasets, this was a total dataset of 2,414 and 1,684 specklegrams respectively. These datasets were shuffled and split into three sets; the training set (67.5%), validation set (22.5%), and testing set (10%). The training set was used to train the model while the validation set was used for evaluating the results of this training with images the model had not seen during training. The testing dataset was kept aside during this process and was only seen by the models whilst producing the final results which appear in this paper. This procedure was also used for the ZNCC method, utilizing the same training set for the calibration curve and the same test set for evaluation (the validation set was not used for the ZNCC method as this method has no hyperparameters to tune).
The training process for both models followed a standard procedure. Mini-batches of 64 training samples were randomly sampled from the training dataset and passed through the model. The standard mean squared error (MSE) loss was then calculated. Backpropagation was then performed, whereby the gradient of the loss function with respect to the model’s weights was calculated, and these weights adjusted in the negative direction of this gradient, in order to head towards a model with a lower loss value [47]. After this had been completed for all training samples in the training set, one epoch was completed, and the process repeated. Both models were trained for 100 epochs. The Adam optimizer [48] was used with a learning rate of 10−3. After 100 epochs, the model with the lowest MSE loss was taken as the best model and used for making predictions. This length of training was chosen to coincide roughly with the loss value beginning to converge to its apparent minimum. Given the relatively small training dataset size, it was important to not overtrain the model, as this could lead to overfitting and poor generalisation capabilities.
The training was done using a Lenovo Legion 7i laptop with a 10th Gen Intel Core i7-10875H processor, 16 Gb of memory and an NVIDIA GeForce RTX 2070 graphics card. The CNN took generally just under an hour to train 100 epochs with a training dataset of ∼1500 specklegrams, while the MLP could do this in minutes.
To quantify the performance of both the DNN models and the ZNCC method, an MSE metric was used in the training. A root mean square error (RMSE) metric is used to display results in this paper in order to preserve the original predictions’ units.
3. Results
3.1 ZNCC calibration curve
To make predictions for the measurand associated with a specklegram using the ZNCC method, a sixth order polynomial calibration curve was fitted to the plot of the measurand label against the ZNCC score for each specklegram in the training set against the reference specklegram. This curve then returned a predicted measurand when given the ZNCC score of a specklegram in the test set.
From the plots in Fig. 5, it is seen that an ideal curve passing through each of the points would not be a 1-to-1 function, and hence cannot be used for sensing over the full range of temperature and immersion parameters. This is because the calibration curve needs to return a unique measurand label when given a ZNCC score, whereas it is seen in Fig. 5 that the sensor response reaches a minimum ZNCC score of around 0 and then starts increasing again with continually increasing temperature/immersion. This defines a range limit on the ZNCC method, as once the calibration curve is no longer 1-to-1, it can no longer be used for making unique predictions. It is this fundamental limit of the ZNCC method that we will improve upon by employing DNNs
Fig. 5. Calibration curves for predicting (a) air temperature and (b) water immersion length from the ZNCC score of a specklegram against the reference. The fitted curve is a 6th order polynomial.
3.2 Predictions
The results come in three parts. The first two relate to datasets for temperature and water immersion length sensing, while the third looks at the performance of the analysis methods against randomly translated specklegrams. For the first two parts, each of the two datasets (temperature and immersion length) were treated in the same way. The DNNs were trained and a measurand vs ZNCC score function fitted using the training dataset, then the quality of the models tested by making predictions using the testing dataset. For both the temperature and immersion datasets, two methods were performed. The first is as described, where all data was shuffled and split into training/validation/test sets. For the second method, all data relating to three of the labels (23°C, 26°C and 29.5°C for temperature and 5 mm, 25 mm and 45 mm for immersion) were removed from the dataset prior to the split. These labels were chosen so as to have two labels close to either end and one from the middle of the datasets removed. These models were then trained and tested with the given training/testing sets as normal, but then additionally made to make predictions using the data from temperatures/immersions they had not yet seen. This is to simulate how these methods would be expected to function as real world sensors; sensing any measurand value within the trained range despite only seeing a set of discrete labels during training. Results of these tests are shown in Fig. 6 and Fig. 7.
Fig. 6. Ability of the three models to predict temperature labels from specklegrams. (a), (b), (c). All data shuffled and split into training/validation/testing sets, predictions shown made on the test set. (d), (e), (f). Data pertaining to three temperatures removed before the split of data and training. Crosses - predictions on test set, circles - predictions on unseen temperature data. For both: black line - line of parity, i.e. where correct predictions would lie. The ΔT errors shown use a root-mean-square metric. ΔTall refers to the crosses, i.e. temperature labels which the models were trained on. ΔTint refers to the circles, i.e. interpolated predictions on the temperature labels not seen during training.
Fig. 7. Ability of the three models to predict water immersion labels from specklegrams. (a), (b), (c). All data shuffled and split into training/validation/testing sets, predictions shown made on test set. (d), (e), (f). Data pertaining to two temperatures removed before the split of data and training. Crosses - predictions on test set, circles - predictions on unseen water immersion data. For all: black line - line of parity, i.e. where correct predictions would lie. The ΔT errors shown use a root-mean-square metric. Δxall refers to the crosses, i.e. water immersion labels which the models were trained on. Δxint refers to the circles, i.e. interpolated predictions on the water immersion labels not seen during training.
For the method where all data was combined and shuffled (with no datapoints omitted), the predictions made by the DNNs all lie almost exactly on the line of correct predictions. The ZNCC method’s predictions lie close to the line within the useful part of the calibration curve, but become inaccurate as the edge of the correlation range is reached. This demonstrates the inherent range limitation that the ZNCC method possesses, and it can be seen that the DNNs solve this.
For the second part of the analysis, where datapoints were removed so as to interpolate with, the DNNs appear to still make accurate predictions on the data they were trained on, but mostly worse than when they were trained on all data. This can be attributed to the reduced training set size, as the overall distribution of data is smaller and easier to overfit to. The discrepancy is more noticeable in the immersion length set, which we can be attributed to two potential causes. Firstly, the immersion length data contains less labels, and hence removing three points resulted in a greater relative reduction in the datapoints shown in training. Secondly, given the experimental setup, the immersion length reference measurement (data label) has larger relative error compared to temperature, and as such there may be minor inconsistencies in the labelling of the specklegrams that are not present in the temperature dataset
The interpolated points were predicted best by the more complex VGG network, with the MLP showing a more noticeable deviation from the reference labels with these predictions. The ZNCC results for this second part of the analysis are similar to the first part.
3.3 Translated specklegrams
After the performance of the DNNs and the ZNCC method had been tested on the two sets of data, the final part of the experiment was to test their robustness against random horizontal and vertical translations of the specklegrams. The capturing of the specklegram via a lens and a webcam requires precise alignment. Therefore this process has the potential to introduce transverse translation due to alignment drift. We have chosen to demonstrate the ability of the CNN to automatically compensate for this type of drift, which is not a feature of the correlation function approach.
To test this, the same procedure as in the first two parts of the experiment was carried out with the temperature data, i.e. all the data was shuffled, split, the models trained/ZNCC curve fitted and predictions were made with varying degrees of random specklegram translations in the vertical and horizontal directions. The physical translation size when the specklegram reaches the webcam, after cropping and downsampling, is equivalent to approximately 0.1 mm per pixel. For each translation amount, the models trained on this data made predictions on the testing set. The results are presented as a plot of model accuracy (average error) against the maximum size of the translations, displayed in Fig. 8.
Fig. 8. Performance of the three tested models when trained using artificially translated specklegrams. The x-axis shows the maximum number of pixels that each specklegram in the set could have been translated by in either direction – the actual amount was chosen randomly with a uniform distribution between the positive and negative of the value displayed on the x-axis.
When the DNNs were shown translated images during the training phase, they were able to effectively learn the measurand/specklegram relation for that size of translation, and achieve an error only a factor of around three away from the original, untranslated specklegram analysis at the largest translation size that was tested. The ZNCC method on the other hand begins to diverge in error at even the smallest translation size. It appears to plateau at an RMSE of ∼2.2°C, which turns out to be the maximum error possible as it begins predicting a singular average value for all measurand labels. This is to be expected, as the ZNCC method is simply directly comparing images pixel for pixel, so as images for the same label become more different and their ZNCC scores spread over a larger range, it becomes difficult to fit an accurate prediction curve to the data. At the larger translation sizes, spreads of ZNCC scores were so great that a straight line predicting the same temperature for every specklegram minimized the error.
4. Conclusion
We have demonstrated the ability of deep neural networks to learn the complex relationship between changes in a multimode fiber’s environment (namely temperature and refractive index), and the intensity specklegram it outputs at its end facet. We have achieved higher accuracy and consistency over a larger measurand range than with a traditional statistical correlation (ZNCC) method. Furthermore, we have demonstrated that deep neural networks (DNNs) are able to learn this multimode fiber transmission relationship even when the training and testing specklegrams have been translated, something the correlation method is fatally vulnerable to. Finally, the feature recognition ability of convolutional neural networks (over both the MLP and ZNCC method) was exploited to predict measurands from translated specklegrams, even when none were included in the training dataset.
This work provides the first direct comparison of a standard statistical method to the use of DNNs in fiber specklegram sensing. We have shown that limitations for the ZNCC method are easily handled by the DNNs, prompting the possibility of further work in the use of DNNs with FSSs and testing what their true limitations are.
This approach of using deep learning for sensing with an exposed core fiber has potential to be useful in biosensing, where the exposed core fiber excels. The use of DNNs could improve the versatility of sensors immensely, providing robustness against imaging alignment difficulties and dynamic range issues. Finally, the field of machine learning is vast, and the limits of its application to sensing and imaging are far from being reached. meaning further improvements from the deep learning aspect of this paper are to be expected.
Funding
ARC Future Fellowship (FT200100154); Optofab node of the Australian National Fabrication Facility utilizing Commonwealth and South Australian State Government funding; ARC Centre for Nanoscale BioPhotonics (CE14010003); Fundação de Amparo à Pesquisa do Estado de São Paulo (2018/10409-7).
Disclosures
S.C.W.S. is a director of HT Sensing Pty. Ltd., a company that manufactures optical fiber sensors. HT Sensing Pty. Ltd. did not contribute to or participate in this research in any way.
Data availability
Data underlying the results presented in this paper may be obtained from the authors upon reasonable request.
References
1. B. Lee, “Review of the present status of optical fiber sensors,” Optic. Fiber Technol. 9(2), 57–79 (2003). [CrossRef]
2. D. A. M. Krohn, T. W. MacDougall, and A. Mendez, Fiber Optic Sensors: Fundamentals and Applications, Fourth Edition ed., (SPIE, 2015), (Chap. 2).
3. X. Wang and O. S. Wolfbeis, “Fiber-Optic Chemical Sensors and Biosensors (2015–2019),” Anal. Chem. 92(1), 397–430 (2020). [CrossRef]
4. K. T. V. Grattan and T. Sun, “Fiber optic sensor technology: An overview,” Sens. Actuators, A 82(1-3), 40–61 (2000). [CrossRef]
5. R. A. Potyrailo, S. E. Hobbs, and G. M. Hieftje, “Optical waveguide sensors in analytical chemistry: today's instrumentation, applications and trends for future development,” Fresenius' J. Anal. Chem. 362(4), 349–373 (1998). [CrossRef]
6. S. Wu, S. Yin, and F. T. S. Yu, “Sensing with fiber specklegrams,” Appl. Opt. 30(31), 4468–4470 (1991). [CrossRef]
7. W. Spillman, B. Kline, L. Maurice, and P. Fuhr, “Statistical-mode sensor for fiber optic vibration sensing uses,” Appl. Opt. 28(15), 3166–3176 (1989). [CrossRef]
8. H. S. Efendioglu, “A review of fiber-optic modal modulated sensors: Specklegram and modal power distribution sensing,” IEEE Sens. J. 17(7), 2055–2064 (2017). [CrossRef]
9. A. G. F. Leal-Junior, A. C. Marques, and M. J. Pontes, “Optical Fiber Specklegram Sensors for Mechanical Measurements: A Review,” IEEE Sens. J. 20(2), 569–576 (2020). [CrossRef]
10. 1M. Englund, A. Ipatti, and P. Karioja, “Fiber optic security monitoring sensor,” Proc. SPIE 3099, 281–296 (1997). [CrossRef]
11. A. Dhall, J. K. Chhabra, and N. S. Aulakh, “Intrusion detection system based on speckle pattern analysis,” Experimental Techn. 29(1), 25–31 (2005). [CrossRef]
12. S. C. Schneider, Y. Gautam, and B. G. Zagar, “Application of a locally operating laser-speckle strain sensor,” IEEE Trans. Instrum. Meas. 52(4), 1025–1029 (2003). [CrossRef]
13. T. D. Cabral, E. Fujiwara, S. C. Warren-Smith, H. Ebendorff-Heidepriem, and C. M. B. Cordeiro, “Multimode exposed core fiber specklegram sensor,” Opt. Lett. 45(12), 3212–3215 (2020). [CrossRef]
14. B. Yang, I.-M. Lee, and B. Lee, “Wavelength detection using optical fiber speckle patterns,” in Proc. IEEE 13th Annu. Meeting Lasers Electro-Opt. Soc., (2000), pp. 470–471.
15. B. Gupta, H. N. Bhargaw, and H. K. Sardana, “Qualifying fiber optic temperature sensor using speckle metrology,” Int. Inf. Technol. Knowl. Manage. 1, 337–350 (2008).
16. P. Klokoc, I. Lujo, M. Bosiljevac, and N. Burum, “Optical sensor system for vibration measuring,” in Proc. IEEE 50th Int. Symp. ELMAR, (2008), pp. 625–628.
17. M. Plöschner, T. Tyc, and T. Čižmár, “Seeing through chaos in multimode fibers,” Nat. Photonics 9(8), 529–535 (2015). [CrossRef]
18. M. N’Gom, T. B. Norris, E. Michielssen, and R. R. Nadakuditi, “Mode control in a multimode fiber through acquiring its transmission matrix from a reference-less optical system,” Opt. Lett. 43(3), 419–422 (2018). [CrossRef]
19. A. Yariv, “Phase conjugate optics and real-time holography,” IEEE J. Quantum Electron. 14(9), 650–660 (1978). [CrossRef]
20. I. N. Papadopoulos, S. Farahi, C. Moser, and D. Psaltis, “Focusing and scanning light through a multimode optical fiber using digital phase conjugation,” Opt. Express 20(10), 10583–10590 (2012). [CrossRef]
21. B. Silvio and R. Di Leonardo, “A multi-mode fiber probe for holographic micromanipulation and microscopy,” Lab. Chip 12(3), 635–639 (2012). [CrossRef]
22. Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109(20), 203901 (2012). [CrossRef]
23. N. Borhani, E. Kakkava, C. Moser, and D. Psaltis, “Learning to see through multimode fibers,” Optica 5(8), 960–966 (2018). [CrossRef]
24. P. Caramazza, O. Moran, R. Murray-Smith, and D. Faccio, “Transmission of natural scene images through a multimode fiber,” Nat. Commun. 10(1), 2029 (2019). [CrossRef]
25. Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521(7553), 436–444 (2015). [CrossRef]
26. A. Krizhevsky, I. Sutskever, and G. Hinton, “ImageNet classification with deep convolutional neural networks,” Proc. Advances in Neural Information Processing Systems 25, 1090–1098 (2012).
27. G. Hinton, “Deep neural networks for acoustic modeling in speech recognition,” IEEE Signal Process. Mag. 29(6), 82–97 (2012). [CrossRef]
28. S. Aisawa, K. Noguchi, and T. Matsumoto, “Remote image classification through multimode optical fiber using a neural network,” Opt. Lett. 16(9), 645–647 (1991). [CrossRef]
29. R. K. Marusarz and M. R. Sayeh, “Neural network-based multimode fiber-optic information transmission,” Appl. Opt. 40(2), 219–227 (2001). [CrossRef]
30. U. Kürüm, P. R. Wiecha, R. French, and O. L. Muskens, “Deep learning enabled real time speckle recognition and hyperspectral imaging using a multimode fiber array,” Opt. Express 27(15), 20965–20979 (2019). [CrossRef]
31. W. Xiong, B. Redding, S. Gertler, Y. Bromberg, H. D. Tagare, and H. Cao, “Deep learning of ultrafast pulses with a multimode fiber,” APL Photonics 5(9), 096106 (2020). [CrossRef]
32. E. Kakkava, N. Borhani, B. Rahmani, U. Teğin, C. Moser, and D. Psaltis, “Deep Learning-Based Image Classification through a Multimode Fiber in the Presence of Wavelength Drift,” Appl. Sci. 10(11), 3816 (2020). [CrossRef]
33. L. V. Nguyen, C. C. Nguyen, G. Carneiro, H. Ebendorff-Heidepriem, and S. C. Warren-Smith, “Sensing in the presence of strong noise by deep learning of dynamic multimode fiber interference,” Photonics Res. 9(4), B109–B118 (2021). [CrossRef]
34. A. R. Cuevas, M. Fontana, L. Rodriguez-Cobo, M. Lomer, and J. M. Lopez-Higuera, “Machine Learning for Turning Optical Fiber Specklegram Sensor into a Spatially-Resolved Sensing System. Proof of Concept,” J. Lightwave Technol. 36(17), 3733–3738 (2018). [CrossRef]
35. F. M. Cox, R. Lwin, M. C. J. Large, and C. M. B. Cordeiro, “Opening up optical fibers,” Opt. Express 15(19), 11843–11848 (2007). [CrossRef]
36. L. V. Nguyen, K. Hill, S. Warren-Smith, and T. Monro, “Interferometric type optical biosensor based on exposed-core microstructured optical fiber,” Sens. Actuators, B 221, 320–327 (2015). [CrossRef]
37. S. C. Warren-Smith, H. Ebendorff-Heidepriem, T. C. Foo, R. Moore, C. Davis, and T. M. Monro, “Exposed-core microstructured optical fibers for real-time fluorescence sensing,” Opt. Express 17(21), 18533–18542 (2009). [CrossRef]
38. S. C. Warren-Smith, E. Sinchenko, P. Stoddart, and T. M. Monro, “Distributed fluorescence sensing using exposed-core microstructured optical fiber,” IEEE Photonics Technol. Lett. 22(18), 1385–1387 (2010). [CrossRef]
39. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction (Springer, New York, 2009).
40. S. Albawi, T. A. Mohammed, and S. Al-Azawi, “Understanding of a convolutional neural network,” in 2017 International Conference on Engineering and Technology (ICET), (IEEE, 2017).
41. R. H. R. Hahnloser, R. Sarpeshkar, M. A. Mahowald, R. J. Douglas, and H. S. Seung, “Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit,” Nature 405(6789), 947–951 (2000). [CrossRef]
42. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in ICLR, (2015).
43. M. Rezaee, Y. Zhang, R. Mishra, F. Tong, and H. Tong, “Using a VGG-16 Network for Individual Tree Species Detection with an Object-Based Approach,” in 2018 10th IAPR Workshop on Pattern Recognition in Remote Sensing (PRRS), (IEEE, 2018), pp. 1–7.
44. Q. Guan, Y. Wang, B. Ping, D. Li, J. Du, Y. Qin, H. Lu, X. Wan, and J. Xiang, “Deep convolutional neural network VGG-16 model for differential diagnosing of papillary thyroid carcinomas in cytological images: a pilot study,” J. Cancer 10(20), 4876–4882 (2019). [CrossRef]
45. D. Scherer, A. Muller, and S. Behnke, “Evaluation of pooling operations in convolutional architectures for object recognition,” in Intl. Conf. on Artificial Neural Networks, (2010), pp. 92–101.
46. B. E. Boser, I. M. Guyon, and V. N. Vapnik, “A training algorithm for optimal classifiers,” in Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, 144–152.
47. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” Nature 323(6088), 533–536 (1986). [CrossRef]
48. P. K. Diederik and J. Ba, “Adam: A Method for Stochastic Optimization,” in 3rd International Conference for Learning Representations, (2015).
