1. Introduction

Complex optical interactions with media have captured the imagination of many researchers, becoming a rapidly growing field of cross disciplinary research [1]. When light interacts with complex media, the optical field is distorted by cascaded interactions with spatially distributed scatters or variations in refractive index [2,3]. Specifically within multi-mode fibers, manufacturing errors, bends in the fiber, and material impurities can result in distortions that lead to mode-mixing. Mode-mixing will commonly produce intricate interference between the optical modes supported by the fiber that result in output intensity profiles that resemble random behaviour rather than ordered, predictable interactions [4]. These interactions are commonly referred to as speckles, due to their complex patterning. Monitoring changes in the speckle structure has been demonstrated as a method for sensing applications [5,6]. In addition, changes in temperature and strain in the fiber lead to the intensity profile continually evolving over time, therefore it is difficult to determine explicit external environmental factors that create a particular output intensity or phase profile. This creates challenges for the use of multi-mode fibers in communications, imaging or entangled quantum systems [7].

Overcoming disorder in complex media, including but not exclusive to fiber, has led to many interesting research discoveries in communication [4,8–11], imaging [12–15], quantum optics [16–18], and optical sensors [19–24]. However, the continuous changes in relative phase between the optical modes create phase-noise that limits the accuracy of sensors based on few-mode or multi-mode fibers [24]. Intermodal coupling can be a reliable fiber property for sensing, however a technical challenge is the development of measurement schemes that are less sensitive to changes in phase associated with thermal and mechanical effects on optical modes guided by a fiber. Low complexity, high accuracy shape sensing techniques hold significant potential for the development of fiber sensors for a wide range of novel sensing applications and would allow these sensors to be more widely deployed.

Rayleigh scattering and Bragg-grating techniques are currently the leading approach to shape sensing in fiber [25–28], where optical time domain reflectometry (OTDR) is commonly used for bend or damage location. Some bends in a fiber result in back reflections, therefore accurate measurement of the time a back reflected signal is received can reveal the location of a bend. Improvements in laser sources, interferometric techniques, and timing electronics have provided a potential road map for cm-scale bend location to be achieved. Recent demonstrations of technologies such as phase sensitive or chirped pulse OTDR, and optical frequency domain reflectometry (OFDR) have received notable interest from the community. The use of multi-mode fibers has become an emerging research avenue, with promise to increase the resolution of the bend location to around 20 m over 2 km of multi-mode optical fiber [29]. Short-distance implementations have become commercially available, with 20 cm resolution, based on multi-mode optical fiber deployed in aeroplanes and submarines [30–33]. Bragg-grating technology are use periodic grating structures are introduced into the fiber, that, when perturbed, change their period and subsequently reflect light of a specific wavelength depending on the strain or temperature change near these gratings [34–36]. Bespoke fibers have been shown to provide high accuracy shape detection, where the sensor accuracy is primarily limited by the frequency resolution of the optical spectrum analyser used for measuring frequency shifts in the back reflected signal. Although accurate, both OTDR and Bragg-grating technology are non-trivial to implement for high accuracy shape sensing, especially at mm scale resolution and shallow bend radii. Recent investigations have indicated that coherent detectors can be used in deployed single-mode fiber to retrieve environmental information [37]. Additionally, combinations of traditional OTDR or Bragg-grating technology with machine learning methods can improve the resolution of sensors based on few-mode fibers, but intermodal coupling and phase noise are both major limitations to accuracy of these systems [24]. Optical elements have been successfully used as all optical pre-processors for machine or deep learning to assist computational efficiency and accuracy [38–41]. Therefore, a combination of bespoke optical elements and machine learning could allow for the development of low complexity fiber shape sensors that could be transformative for consumer electronics, such as wearable technology or vehicle sensors.

In this paper, an approach which we refer to as “structured light enhanced machine learning” is demonstrated for millimeter scale bend sensing in up to a 1 km of few-mode optical fiber; an approach that specifically uses intermodal coupling as a measurement technique. Modal decomposition, commonly referred to as mode sorting or mode de-multiplexing [42–44], are optical methods for determining the linear superposition of orthogonal modes that can be used to represent any structured optical field, such as the output from a bent few- or multi-mode optical fiber, see conceptual drawing in Fig. 1. These approaches, in effect, perform an all-optical feature extraction to sort complex spatial information into a discrete set of classes, similar in concept to the use of principal component analysis of large data-sets [45]. When bends occur in fiber, light is coupled into neighbouring spatial modes. This intermodal coupling will yield a distinctive power distribution at the output of the de-multiplexer that can be used to train a machine learning algorithm. Based on statistical analysis of the power distribution, we revealed a continuously varying physical trend that can be used for accurate bend location detection. This structured light enhanced machine learning approach could also be applied to other complex systems to identify physical trends that could be used for novel sensor technologies.

 

Fig. 1. When high order optical beams, such as those that carry OAM, are coupled in optical fibers, intermodal coupling occurs that distorts the beam. This can be measured using a mode de-multiplexer to quantify the distribution of power between OAM modes that results from intermodal coupling. When used as trainable features for machine learning, their power distributions can provide accurate information identifying the positions or degree of bending in a few- or multi-mode optical fiber.

Download Full Size | PDF

2. Mode mixing in fiber

Modes in step index optical fiber are commonly considered as linear polarised (LP) modes; however, one is free to measure the light emitted from the fiber in any alternative complete modal basis. As noted above, small changes in temperature or position of the fiber usually result in chaotic changes to the relative phase noise between the modes in optical fiber, and more importantly, in a different power distribution within the supported modal bandwidth. In our experimental implementation, an orbital angular momentum (OAM) mode de-multiplexer performs an all-optical linear transformation between OAM modes and spatially separated spots on the back focal plane of a camera, [43]. This transformation is largely unaffected by relative phase, where each of the separated modes maintain their relative phase, but have only minimal levels of interference at the detector plane. Therefore, decomposing the light emitted from the fiber using an OAM de-multiplexer provides a method to monitor intermodal coupling with reduced sensitivity to undesired phase noise.

The specific mode mixing that occurs in a fiber can be represented as a transfer matrix ($\mathbf {T}$). Manufactured fibers have small variations in diameter, refractive index, mechanical twisting, and continual micro-bends along the length of the fiber. Therefore, the optical field is continually being aberrated over the full length of the fiber. This means that a transfer matrix is not valid at all length scales or positions and can be separated into distinct sections along the fiber, see Fig. 2, where each section has a ransfer matrix $\mathbf {T}_{p \pm i}$, where subscript variable $i$ indicates the sections either before or after the bend position, indicated by subscript $p$. When the position of a bend is moved, the $\mathbf {T}_{p}$ before the bend, $\mathbf {T}_{p-1}$, and after the bend, $\mathbf {T}_{p+1}$, will change depending on any volumetric changes in the fiber’s internal structure, or the mechanical support for the fiber, and the effective $\mathbf {T}$ for the fiber as a whole will not be the same. This is mathematically represented as

Therefore, as a bend position is moved, a unique distortion to the output beam profile is induced that can be used for sensing bend position. Similarly, the degree of bending will change the $\mathbf {T}_{p}$ for the bent region of the fiber allowing for the determination of the degree of bending. It should be noted that a straight fiber, free of micro-bend or strain would not exhibit this behaviour as $\mathbf {T}$ would not spatially change over the length of the fiber. As this behaviour requires natural variation in the fiber, a universal $\mathbf {T}$ for every fiber does not exist. Therefore, fitted, calibrated, or trained systems are required to build a sensor based on this principle.

 

Fig. 2. (A) The transmission matrix, T, can be separated into several sections depending on the local fiber parameters. For the case of a single bend in the fiber, with two straight sections located before and after the bend, each zone will have its own well-defined $\mathbf {T}_{p\pm i}$. In real world fiber, the straight sections will perturb the propagating optical mode. (B) Therefore, if the bend is moved this will lead to a change in $\mathbf {T}_{p-1}$ and $\mathbf {T}_{p+1}$, becoming $\mathbf {T}_{p+1}^{*}$ and $\mathbf {T}_{p-1}^{*}$, respectively. This will subsequently change the shape of the output beam due to different mode mixing during propagation, which can be used to determine the position of the bend.

Download Full Size | PDF

A simplified fiber model is the matrix product of a series of $\mathbf {T}_{i}$ that represent the bent, $\mathbf {B}$, and straight, $\mathbf {S}_{z}$, regions of the fiber, where $z$ is each arbitrary longitudinal partition. To compute the expected modal distribution from the model, the straight section of the fiber is considered to have low modal mixing. This matrix is written as

With each change in position, the effective transmission matrix of the fiber is different, yielding a change in the output modal distribution, $\mathbf {V}_{n}$, when $\mathbf {S}_{z}$ is not an identity matrix. To numerically compute the expected result, eleven effective bend positions are defined with the same real valued $\mathbf {S}_{z}$ and $\mathbf {B}$ transmission matrices. As each of these $\textbf {T}_{i}$ will have some level of random variation, 1000 random matrices are computed for each bend, where the mean center of mass, COM, and variance is calculated. Presented in Fig. 3, is a plot of the change in average COM, $\Delta$COM, where a clear change is observed for a change in the bend position. This result indicates that for an ideal fiber, which is well supported with little or no micro-bends, no change in $\Delta$COM is calculated. This result is consistent with the work by Loterie et al. [46]. The results indicate that increased intermodal coupling from non-ideal fiber effects result in a clear change in $|\Delta \text {COM}|$ is expected.

 

Fig. 3. Calculated magnitude change in center of mass, $|\Delta \text {COM}|$ for ideal fiber with no intermodal coupling and therefore no power is transferred to any $s_{n,m}^{\Delta }$ terms in Eqn. (3) and yields $\sum {s_{n,m}^{\Delta }}=0$. Intermodal coupling occurs in straight sections of fiber with small defects, therefore $\sum {{j_{n,m}}} \neq 0$. We consider a ratio, $C=(\sum _1^m \sum _1^n {{s_{n,m}^{\Delta }}})/(\sum _{1}^{n}\sum _{1}^{m} s_{n,m}$), between the total power in all $s_{n,m}^{\Delta }$ terms with relation to the total power in all $i_n$ terms, to calculate the expected $|\Delta \text {COM}|$ in the cases where $C= \left \{0.001, 0.1\,\text {and}\,0.2 \right \}$, respectively.

Download Full Size | PDF

3. Simulation

The spatial profile of any beam can be considered as a linear superposition of any complete basis set of spatial modes. For a coherent paraxial beam, an orthogonal set can be determined based on the particular constraints, such as Cartesian coordinates that define Hermite-Gaussian (HG), and polar coordinates that define Laguerre-Gaussian (LG) beams. Both HG and LG modes have been used to decompose optical fields from fiber [44,47], however selection of the wrong basis set could lead to additional errors due to inter-channel crosstalk [44]. LG beams are commonly used for expressing spatial modes bound by circular apertures. Linear-polarised (LP) modes, commonly considered in step index fiber, can be readily expressed as linear super-positions of $\mathrm {LG}_{p,\ell }$ modes, where spatial modes are described by two integers, $p$ and $\ell$, that correspond to radial and azimuthal components, respectively. a linear superposition can be expressed as:

In our experimental implementation we utilise a step index fiber with a core size of ${8.2}\,\mathrm{\mu}\textrm{m}$ (SMF28) and couple in light from a laser of wavelength ${660}\,\textrm{nm}$. At this wavelength, the fiber will support multiple modes and can be considered a few-mode optical fiber. LP modes are commonly considered the orthogonal mode set of step index bounded waveguides. We estimate our fiber can support 6 spatial LP modes, where the intermodal coupling results in the output mode after bending being a linear superposition of these modes. Therefore, to replicate this intermodal coupling, 11 unique, but non-orthogonal, superpositions of LP modes were generated with random relative phases between each spatial mode.

A common approach for spatial modal analysis using machine learning is based on trained neural networks and has shown promise for communication and sensing systems [48]. Therefore, we utilise a neural network as a bench mark to compare mode structured light enhanced machine learning, see Fig. 4B, where we image the output of the fiber directly onto a CCD camera. Our simulated modal decomposition is achieved by computing the overlap integral of the output with HG and OAM modes, as all-optical mode sorting techniques exist for these mode types [44,49], Fig. 4C and D, respectively. These results indicate that OAM decomposition performs substantially better than either direct intensity measurement ${14}{\% }$ or using HG modes ${58.2}{\% }$, yielding ${98.9}{\% }$ accuracy for the identification of intermodal coupling. We note training can also be completed using both phase and intensity information of the output optical beam from the fiber, however our simulations indicate an 11%$\pm$7 accuracy with a loss of 2.76$\pm$0.15. This lower accuracy is related to the large potential variations in phase that occur from random phase-noise, leading to additional error during training for a given training set.

 

Fig. 4. The measurement of bend location from intermodal coupling was simulated to compare different machine learning approaches for the detection of intermodal coupling when relative phase will naturally evolve. (A) a series of sample intensity profiles with a change in COM, with random relative phase noise between modes used for training. (B) a neural network trained using direct intensity measurement of intensity is considered as the bench mark for traditional approaches using machine learning. The use of modal decomposition in both the (C) HG and (D) LG basis sets are considered, indicating OAM is the preferred basis set for accurate identification of intermodal coupling.

Download Full Size | PDF

4. Experimental implementation

Experimentally, a loosely supported fiber was bent at a series of positions and with bends of different radius of curvature, where the spatial mode decomposition into the OAM basis was recorded. Two types of fiber configurations were considered in these experiments, a laser with a wavelength of 660 nm was used in conjunction with ${8.2}\,\mathrm{\mu}\textrm{m}$ core few-mode fiber (SMF28, which is commonly used as single-mode fiber for ${1550}\,\textrm{nm}$) and ${50}\,\mathrm{\mu}\textrm{m}$ core step index fiber (M14L, Thorlabs). Through structured illumination and controlled coupling to the optical fiber, both fundamental and higher order LP modes could be excited. A spatial light modulator (SLM) (Cambridge Correlator SDE1024n) screen is imaged onto the back focal plane of a 10$\times$ microscope objective (Olympus RMS10x) such that the fiber facet is in the farfield of the SLM. Free-space optical to fiber coupling is controlled with a three-axis stage (Thorlabs MBT616D/M). The output of the optical fiber is then either directly imaged onto a CCD camera (Imaging Source Skyris) for the intensity retrieval results, see Fig. 5A, or de-multiplexed into OAM modes for further increase of the accuracy, as shown in Fig. 5B and C. We utilise a passive optical device designed for the sorting of beams that carry OAM to perform a modal decomposition of the light emanating from the fiber. This device performs a log-polar transformation, which subsequently allows a spherical lens to convert beams that carry angular momentum into a series of separated spots at the back focal plane of a lens [49]. The sorter utilised in our experiment comprised two free-form optical surfaces to perform the required transformation, where the optical profiles for each of the surfaces are fully outlined in [50]. The surfaces used had an aperture diameter of 12.5 mm, which were directly machined on each side of a 10 cm long bar of polymethyl methacrylate (PMMA), utilising the same machining method as outlined in [50].

 

Fig. 5. To determine the usefulness of our modal decomposition approach we investigated three setup configurations to measure the output from the fiber. (A) First, the directly measured intensity of the output mode to train the CNN, for the results shown in Fig. 7A. (B) a log-polar optical transformer was used to perform OAM de-multiplexing of the output from the both few- and multi-mode optical fiber, for the results shown in Fig. 7 B and C. (C) To explore the use of the technology for remote sensing, a ${1}\,\textrm{km}$ delay line of few-mode fiber was introduced. For each realisation, the fiber was bent at 11 fixed locations using a computer controlled translation stage. (D) a picture of the experimental setup, showing the loose support for the fiber, the mode-sorter and moving bend position.

Download Full Size | PDF

The bends are induced by pressing down on the fiber with a ${5}\,\textrm{mm}$ diameter bar that is mounted with a low drag ball bearing to prevent torsion on the fiber. The fiber is supported by memory foam with a channel to prevent bending in locations other than those specifically induced, see inset of Fig. 5D. Although supported, the fiber will naturally change shape slightly in other positions; however, resembles real-world scenarios that would be experienced in applications such as wearable fiber sensors. This bar is mounted to a linear translation stage (Thorlabs LTs300/M) to allow for repeatable bending. The radius of curvature was altered by varying the height of the bar inducing the bend in the fiber. For the experimental simulation of remote sensing applications, we introduce ${1}\,\textrm{km}$ of SMF-28 fiber delay line and repeat these same measurements.

5. Results

Each bend position and radius was recorded 100 times, to record changes in laser power or optical coupling. The process was repeated 10 times so that experimental variation in the support, temperature, and strain on the fiber were fully recorded, see Fig. 6. Selected distributions from one data set are presented in Fig. 6A, where a small variation in the weighting of the normalised power distribution can be observed with a change in position of the bend. A common method of analyzing power distribution of OAM is calculating the center of mass (COM), [51]. As this distribution will vary slightly over time or repeated relocation of the bend conditions, the average and standard deviation of the COM was calculated for all 1000 distributions measured for both few- and multi-mode fiber, see Fig. 6B.

 

Fig. 6. A modal decomposition of the output of a few-mode optical fiber was measured for a range of position and radius of curvatures, utilising an all-optical log-polar transformer. The unique distribution was recorded at each position, which depended on the input mode and accumulative transfer matrix of the fiber. (A) When a $\mathrm {LG}_{0,\ell =2}$ mode is coupled into the fiber and a change in bend position is made, it can be seen that there is a subtle change in power distribution. (B) When analyzed, on average, a continuous change in center of mass (COM) was observed in both few- (yellow curve) and multi-mode fiber (blue curve).

Download Full Size | PDF

As the bend position is moved, the propagation length within the fiber after the bend is increased, this creates further intermodal coupling due to the change in effective transmission matrix of the straight region after the bend. As a result, the COM varies linearly when the bend is positioned further away from the receiver, as predicted in the simulations from Fig. 3. We further observe a difference in the gradient when there is a change in radius of curvature of the bend. The increased bend radius will result in increased intermodal coupling in the fiber and will accelerate coupling to other modes, see Fig. 6B.

These physical features confirm that machine learning can provide a useful tool for classifying the bend positions within the data. The experimental data was processed using a convolutional neural network (CNN) for images of the directly recorded optical field intensity, showing low accuracy in the recognition of bend locations in the fiber, see Fig. 7A. Given the non-standard nature of the images we wish to process from the optical fiber, we developed a simple CNN comprising five layers; a convolution layer ($20$ outputs and $4\times 4$ kernel size), pooling layer, flatten layer, linear layer and softmax layer. We note, the purpose of this investigation is not to obtain the best possible accuracy CNN model, but simply to give a benchmark that we can directly compare our modal decomposition approach with the same physical parameters. Our training set comprised of data from three independent measurement data sets from the experimental system and our verification was from a fourth independently measured data-set. This approach was to ensure we do not train for random fluctuations in the fiber that occur from temperature, small mechanical changes, or fiber mounting. The memory requirement for images used in this training was approximately 5Gb, comprising 50$\times$50 8-bit images. The computer used was an Apple M1 MacMini, with 8 CPU cores, 7 core neural engine, and 16GB of physical memory. The neural network was implemented in Mathematica, utilizing the NetTrain tools to develop our network. Training took approximately 4 hours. The 100 images connected with each position were labeled with the position and input mode only, the network was not aware of the specific measurement data set that each image was recorded within.

 

Fig. 7. (A) The output from a few-mode optical fiber was directly recorded for a range of different bend locations, indicating low accuracy in training based on a CNN. (B) Using the same few-mode fiber, training was completed on model de-multiplexed into the OAM modal basis showing high accuracy in bend location. (C). The experiment was repeated using ${50}\,\mathrm{\mu}\textrm{m}$ core step index multi-mode fiber, again showing high accuracy in bend location. For all plots, verification was completed using dedicated measurement completed later in the day.

Download Full Size | PDF

We implemented support vector machine (SVM) training on the OAM spectrum for both few- and multi-mode fibers, showing greater accuracy in classification, up to ${94.1}{\% }$, as shown in Fig. 7B and C. A further advantage of this form of machine learning training is the reduction in training time required; where over 4 hours is required for image based CNN and only 3 minutes for OAM spectrum based implementations. Our verification in both cases utilises data collected from a different experimental realisation approximately 3 hours after the initial training data is recorded, therefore no data in the training set is used for verification. This confirms that the data is robust to the natural changes throughout the day. Due to the reduced processing time we could increase the number of samples to 200 samples at each position used for training to capture more variation from environmental effects. This is increase in samples lowered our accuracy by approximately 2% due to increased thermal drift. Experimental studies were conducted in an environment with a temperature variation of 2-3 degrees Celsius over the length of the day. To explore the use of the technology for remote sensing, a 1 km delay line of few-mode fiber was introduced and the accuracy of bend location measured. Table 1 shows the combined results for few-mode fiber measurements for different distances and steps between the locations of the bends.

Table 1. Accuracy results for different fiber lengths of few-mode for a measurement based on OAM sorting and trained network using SVM, similar to the results shown in Fig. 7B and C.

View Table

6. Discussion

Reducing the sensitivity to relative phase noise through the use of modal decomposition reveals features within the intermodal coupling that can provide accurate information about the shape or position of bends in the fiber. For convenience, our initial investigation used an SLM to excite specific modes in the fiber. However, a simpler approach to induce these same higher order modes is to illuminate the fiber using an off axis Gaussian laser [4]. Our current experimental approach, is a proof of concept bench top experiment, however physical size and complexity could be reduced through the implementation of compact mode sorters that directly attached to fiber such as those demonstrated by Lightman et al [53]. Our current demonstration is limited to bends in a single direction and over known fiber lengths, where future work will investigate the viability to detect multiple bends with modifications to the system training or illumination modes. We note in our current implementation, that bends in multiple directions, or over longer or shorter fiber could result in errors in classification. Retraining will be required for substantive changes in system, such as the laser source, optical coupling components or fiber type; however, once fully trained, retraining is not generally required for weeks to months, under normal heat cycling. Further investigation for extreme changes in temperature, such as those for aeroplanes, would be required for the development of prototype systems. For sensitivity to multiple directions, we suggest multi-core fibers could be used to measure the differential in bend state at each fiber, similar to that done with bespoke Bragg grating fibers. In addition to the reduced training complexity, an implementation advantage to using mode de-multiplexing based sensing, is it will be directly compatible with space division multiplexing systems, where sensing could be implemented along side communications without the requirement for the addition of high resolution cameras. We propose one could develop systems for fiber shape sensing in a range of applications, including movement sensors for wearable technologies, flexure hinge monitoring, or aeroplane wing shape sensing. Further investigation for extreme changes in temperature, such as those for aeroplanes, would be required for the development of prototype systems.

Previously demonstrated speckle-based sensors could potentially address the challenges of phase noise by increased complexity in machine learning approaches and using larger data sets to sample the potential variation in the speckle from non-mechanical environmental factors. We expect this would dramatically increase the required training for speckle-based approaches compared to mode-demultiplexing approaches we have presented. Our all-optical processing, using modal decomposition, optimizes the amount of information that is required to be collected when analyzing spatial information due to geometrical similarities between particular modes. For example, in order to perform a phase and intensity reconstruction of an OAM mode, we will require an array of complex values to record the profile in both phase and intensity across a system aperture. The Nyquist sampling imposes a strict limit on the minimum number of spatial samples that are required to be taken to fully resolve an optical field, which requires 2 sample points per phase change of $2\mathrm{\pi}$. Therefore, for an OAM mode with $\ell =10$ any measurement system would require at least 20 sample points around the annulus of the beam, plus one additional pixel to identify the direction of the twisted phase front. Usually, optical information is recorded on a camera, therefore these 21 sample points mapped onto a square grid will require each side of the grid to have $d=21\mathrm{\pi} ^{-1}\approx 7$ therefore requiring a grid of at least 49 pixels. However, when partnered with a mode-sorter, the same OAM mode can be recorded as a single complex number. When complex data is recorded as two 8-bit values for intensity and phase, 16 bits per pixel are required. Our experimental implementation utilises 21 OAM modes, and after de-multiplexing we only record intensity information, which corresponds to 8-bits per mode. Therefore, a reduction factor, $\beta$ fundamentally required information of

In conclusion, we have developed a robust, novel fiber bend sensor that can reveal the location of bends using the intermodal coupling induced by this bend. A mode de-multiplexing approach is used to perform efficient all optical pre-processing of the observed intermodal coupling, simultaneously increase measurement accuracy, from 13% accuracy to over 80%, and reduce the training information required for the machine learning algorithms used for bend position recognition by a factor of 120. More generally, mode de-multiplexing combined with machine learning is a potentially powerful technique that could be deployed for a wide range of applications, where the constant evolution in relative phase between modes can be considered as noise within the system. These applications could include monitoring atmospheric conditions or retrieving information from dense scattering environments. Similar mode-demultiplexing approaches could be used to analyze statistical variations in relative phase to further increase system sensitivity.