Open Access
Add to BibSonomy
Abstract
This paper aims to present initial proof of concept of a non-invasive early diagnostic tool for Alzheimer disease (AD). The approach is based on the identification using Raman spectroscopy and machine learning algorithms of two proteins that are linked with AD and exist in the cerebrospinal fluid (CSF). As demonstrated in previous studies, the concentration of the proteins amyloid-β and tau may indicate the existence of AD. The proteins’ concentration in the CSF signifies the condition of AD. The current study can contribute to the existing body of knowledge by enabling the development of a non-invasive diagnostic tool that may help with early diagnosis of AD.
© 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Alzheimer’s disease (AD) is the most common type of dementia in the world today [1]. In 2019, AD was the 6th leading cause of death in the United States and the 5th leading cause of death among Americans above 65 years old [2]. Researchers predict that by mid-century the amount of people that suffering from AD will increase significantly [2]. AD is characterized by a gradual loss of short-term memory, progressive difficulty performing familiar motor tasks, language problems, temporal and spatial disorientation, impairments in abstract thinking, and disturbances in behavior and personality, including sleep disturbances, depression, anxiety, psychosis and struggles with daily routines [3–5]. Alzheimer’s disease is thought to begin decades before symptoms appear [6–13] which make it harder to diagnose AD in its early stages.
As there is currently no cure for AD, finding early identification methods that would allow for interventions to delay the disease’s progression would be most effective. Early treatment has the possibility of interfering with the pathological process before irreversible damage occurs and could keep patients independent and functional longer [14]. Diagnosis based on physical exam is relevant only after symptoms appear which means AD isn’t in the early stages anymore. Similarly, methods that are based on tests such as structural neuroimaging techniques (magnetic resonance imaging (MRI) for example) for diagnosing AD are effective only at the mid or late stages of the disease and not in the early stages [15]. Moreover, methods that are based on functional neuroimaging techniques can track AD related biomarkers in the cerebrospinal fluid (CSF) in the early stages of the cognitive impairment of the disease but are considered to be invasive and expensive [15]. The possibility of developing ways to easily identify, differentiate and access AD biomarkers in the early stages of the disease would be an invaluable diagnostic tool.
Beyond the ability to detect AD before clinical symptoms appear, identifying specific biomarkers has an added value for it can also make differential diagnoses. There are many ways of distinguishing patients with various health conditions form healthy individuals, but it would be clinically important for researchers to differentiate biological markers between psychiatric diseases with shared symptoms. Mild cognitive impairment (MCI) includes a cluster of symptoms that are displayed in the early stages of several diseases, including AD and Parkinson’s disease (PD) [16]. Yet, there is not a cognitive test that differentiates between a person with PD dementia or AD in the early stages. Although cognitive tests can not distinguish between AD and PD, the presence of certain biomarkers can. The literature recognizes two AD-related biomarkers - amyloid-β and tau (proteins), in the CSF [17–21]. Patients who were diagnosed with AD and with MCI observed with 500$\frac{{\eta g}}{L}$ concentration (in mean terms) for each biomarker [22,23]. This could prove to be valuable in the differential diagnosis of AD from other neuro-psychiatric diseases, as well as in assessing prognosis in the early stages of AD [24]. Identifying these two proteins will not only help to indicate the presence of AD but also, depending on the concentration and aggregation of these proteins, it can also give important information about the stage of the disease [23].
Raman spectroscopy measures the scattered energy from the interaction between the resonance laser to the sample. This method has been proven to produce high accuracy identifying chemicals and their concentrations in a controlled environment [25]. The same measurements technique has already been tested successfully for other purposes such as finding Amyloid-β in blood serum, salt in water, bacterial discovery and detection of different kinds of cancer [26–31].
The task of classify measurements with specific proteins among other measurements called classification task. One of the ways dealing with classification task is using machine learning. Machine learning methods can improve the efficiency and accuracy of classified Raman spectroscopy measurements [31–33]. Because our goal is to help identify AD in early stages the ability to classified measurements with low proteins’ concentration correctly is needed. As lower the proteins’ concentration is the lower the changes in the Raman spectroscopy measurements are [34]. That’s why the use of automatic and intelligent tool as machine learning is necessary.
An interesting finding in previous studies leads to improvement of the Raman spectroscopy resolution by using a novel concept of measuring [35–37]. Since the accuracy of the detecting chemicals in Raman measurements depends on its resolution, the result of better resolution will be higher accuracy detecting Amyloid-β and Tau inside the CSF.
This paper can be the theoretical foundation to use Raman spectroscopy measurements of CSF in non-invasive ways to develop innovative, effective and a relatively simple method of identifying Alzheimer’s biomarkers. In this current work we show the capability of detecting several concentrations of the biomarkers in the CSF with Raman spectroscopy. However, our intention is to move towards a non-invasive technique of focusing light through the body to collect light from the CSF and to perform Raman measurements on the same [38]. Our work is unique as to the best of our knowledge, this is the first time Raman measurements are performed on the biomarkers of AD without extracting it from the body. The existing laboratory techniques alters the natural environment of these biomarkers hence the measurements are also altered. Whereas we measure CSF in its natural form. We don't alter with the concentration nor its natural environment hence providing more accurate and remote sensing of these AD biomarkers. The non-invasive Raman measurements can be obtain using optics techniques to detect AD biomarkers in the CSF through the body. It may allow to detect patients who suffer from AD and monitor their disease progress on weekly basis.
2. Theoretical explanation
2.1 Analysis of Raman spectroscopy plots
Raman spectroscopy is described as the study of the interaction between light and matter where light is inelastically scattered [34]. The basis for Raman spectroscopy is that when electromagnetic energy interacts with a material, it can either be reflected, absorbed, transmitted, or scattered [27]. One type of scattering is Raman scattering. Raman spectroscopy is useful as the Raman scattered light produces data of the vibrational modes of the sample molecules. As such, a Raman spectrum enables the identification of molecules and their functional groups, similar to IR spectroscopy, but with visible range light. During an experiment using Raman spectroscopy, light of single known wavelength is focused onto a sample. Most commonly a laser is used as it is a powerful monochromatic source. The photons (energy) from the laser interact with the samples’ molecules and are scattered inelastically. The Raman collects the scattered photons and generate spectrum plot.
Raman spectroscopy is a well-known technique capable of producing very accurate identification of molecules as well as estimating their concentration in a controlled environment [25]. Raman spectrum holds information about the sampled materials that can help to understand the internal structure of the material. following are some of the things that can be derived from the Raman spectroscopy spectrum; The position of the peak – lead to the identification of the chemical species which means that different materials will be represented differently on the Raman spectrum. In addition, the peak intensity – related to the corresponding chemical species concentration. These two features enable the identification of different kind of chemicals within CSF liquid and their concentration
2.2 Supervised learning and logistic regression
Machine learning models can be separated into two groups: supervised and unsupervised learning models. Supervised learning models use annotated training data (as examples) to learn how to solve the relevant task (classification or regression) while unsupervised learning models trying to learn patterns in the training data without any guidance or at least not explanation for each sample in the training data. While learning the training data the model changes its weights in order to minimize the loss function for the relevant task.
Among supervised learning models we can find logistic regression model. Logistic regression uses logistic function to predict the occurrence’s probability of an event by using the training data to fit the function. The logistic regression hypothesis is defined as:
where x is the input, $\theta $ is the function’s parameters that need to be optimized and the function f is sigmoid function that defined as:The values of the sigmoid are in range $[{0,1} ]$, as presented on Fig. 1:
The cost function is defined as:
3. Materials and methods
Raman spectroscopy measurements were recoded and analyzed for various concentrations of amyloid $-\beta $ and tau in CSF for this paper. The Raman spectrometer was set to use excitation laser with 50$mW$ and 532$nm$ wavelength. The CSF was donated by healthy humans, the proteins were inserted with a method called Spike (according to the factory instructions); each sample’s volume was between 1$ml$–200$\mu l$ and the protein concentration was between 100$\frac{{\mu g}}{{ml}}$ – 1$\frac{{\eta g}}{{ml}}$ (only when the sample contain protein, when it’s pure CSF there are no proteins inside the sample). The exposure was set to 150 seconds for each sample. The resolution obtained with this setup was 1.7cm−1.
In order to identify the existence of tau or amyloid $-\beta $ in the CSF, 3 different types of samples were used: Pure CSF, CSF with amyloid $-\beta $, CSF with tau. The output of the Raman spectroscopy measurements for each sample was a vector with the shape of (2,800). The first row is wavelength $({c{m^{ - 1\; }}} )$ and the second row is Intensity $\left( {\frac{W}{{c{m^2}}}} \right)$. In Fig. 2 we can see a demonstration of the Raman spectroscope and one of the samples with AD’s biomarker. The input is CSF sample with or without biomarker and the output is Raman spectroscopy spectrum. All the measurements share the same wavelength row, therefor intensity was used to classify between the different types. To classify between the different groups, supervised learning models were used. A label was attached to each measurement to identify the sample’s group. In supervised learning models, labeled samples pass “through” the model, the model learns to detect the different sample’s group. As part of the model training, the data was separated into 2 sets, train set and test set. The train set was used to pass “through” the model, and the model fit its hidden variable so it could be classified as the different sample’s group. The test set is used to evaluate the accuracy of the model by predicting the samples’ labels in the test set. Choosing well-represented train and test sets has a significant effect on the accuracy of the model. Therefore, the train and test sets must not be skewed to one of the samples’ groups. The artificial intelligence (AI) and machine learning (ML) models were applied using Python tools and packages.
4. Preliminary measurements and preliminary results
In order to present the proof of concept 4 different experiments were conducted. Each experiment used a different concentration of proteins’ (1$\boldsymbol{mg}$, 10$\boldsymbol{\mu g}$, 100$\boldsymbol{\eta g}$, 1$\boldsymbol{\eta g}$ for 1$\boldsymbol{ml}$). In Fig. 3 we can see examples of different tau's concentrations and Fig. 4 present examples of different amyloid $-\boldsymbol{ \beta }$'s concentrations.
Fig. 4. Raman signature for samples of pure CSF and CSF with different volumes of amyloid $-\beta $ scheme.
Each experiment was simulated 4 times, each time with a different separation of the data into train and test sets (20%-80%, 30%-70%, 40%-60%, 50%-50%). Here, there was an attempt to differentiate between 3 groups, pure CSF, CSF with tau, CSF with amyloid $-\beta $.
The confident chart (for 95%) and the confusion matrices are presented for the classification task between the 3 different samples’ type. The results for proteins’ concentration equal to ${10^6}\frac{{\eta g}}{{ml}}$ is shown in Fig. 5, Fig. 6 present the results for proteins’ concentration equal to ${10^4}\frac{{\eta g}}{{ml}}$. The results for proteins’ concentration equal to ${10^2}\frac{{\eta g}}{{ml}}$ and $1\frac{{\eta g}}{{ml}}$ are shown in Fig. 7 and Fig. 8 respectively.
Fig. 5. 95% confident chart and confusion matrices for classification between pure CSF, CSF with tau and CSF with amyloid $-\beta $, both proteins concentrations are ${10^6}\frac{{\eta g}}{{ml}}$.
Fig. 6. 95% confident chart and confusion matrices for classification between pure CSF, CSF with tau and CSF with amyloid $-\beta $, both proteins concentrations are ${10^4}\frac{{\eta g}}{{ml}}$.
Fig. 7. 95% confident chart and confusion matrices for classification between pure CSF, CSF with tau and CSF with amyloid $-\beta $, both proteins concentrations are ${10^2}\frac{{\eta g}}{{ml}}$.
Fig. 8. 95% confident chart and confusion matrices for classification between pure CSF, CSF with tau and CSF with amyloid $-\beta $, both proteins concentrations are $1\frac{{\eta g}}{{ml}}$.
Each graph holds the results for 4 separations of the data. Table 1 gathers all the experiments’ data and results that took place in this article. From Table 1 it is noticeable that the best results (in accuracy means) obtained with test set size set as 40% from the whole data but, the results for test set as 20% of the data are also acceptable. As mentioned above these initial results present a trend that the Raman signature of amyloid $-\boldsymbol{ \beta }\; $ and tau differs from the Raman signature of pure CSF.
Table 1. Data and results of all the experiments that took place as part of this article
5. Conclusion
In summary, this article presented an innovative method for detecting specific proteins in CSF with high accuracy. As shown in the results it is possible to detect AD’s biomarkers (tau and amyloid $-\beta $) in CSF using machine learning based Raman spectroscopy. This paper is the proof of concept for in-vivo detection using advanced optical techniques. Hopefully this research would motivate the scientific community towards this direction. The implications might be relevant for early diagnosis of AD with non-invasive tests. In the future, this technique will be used in medical health care devices for monitoring Alzheimer disease (AD) in its early stages and it may assist in developing drags to cope with AD since the proposed concept can allow to continuously monitor the AD progress without the need of performing invasive medical procedure and thus it can be done frequently and better prove the medical drag’s efficiency and specificity. AD has serious emotional, social and financial effects for the affected and their families and the proportion of the population that is touched is continually on the rise. Non-invasive and reliable methods to diagnose and monitor AD's progress during the early stages might lead to a better understanding of appropriate treatment and even help to discover efficient therapy protocols or the ability to actually curtail the disease or even cure it.
Disclosures
Author declares no conflict of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. M. Prince, R. Bryce, E. Albanese, A. Wimo, W. Ribeiro, and C. P. Ferri, “The global prevalence of dementia: a systematic review and meta analysis,” Alzheimer’s Dementia 9(1), 63–75.e2 (2013). [CrossRef]
2. Alzheimer's Association, “2019 Alzheimer’s disease facts and figures,” Alzheimer’s Dementia 15(3), 321–387 (2019). [CrossRef]
3. E. Joe and J. M. Ringman, “Cognitive symptoms of Alzheimer’s disease: clinical management and prevention,” BMJ 6, 367 (2019). [CrossRef]
4. M. R. Farlow, “Etiology and pathogenesis of Alzheimer’s disease,” Am. J. Health-Syst. Pharm. 55(suppl_2), S5–S10 (1998). [CrossRef]
5. B. H. Monien, L. G. Apostolova, and G. Bitan, “Early diagnostics and therapeutics for Alzheimer’s disease – how early can we get there?” Expert Rev. Neurother. 6(9), 1293–1306 (2006). [CrossRef]
6. Y. T. Quiroz, H. Zetterberg, E. M. Reiman, Y. Chen, Y. Su, and J. T. Fox-Fuller, “Plasma neurofilament light chain in the presenilin 1 E280A autosomal dominant Alzheimer’s disease kindred: a crosssectional and longitudinal cohort study,” Lancet Neurol. 19(6), 513–521 (2020). [CrossRef]
7. N. R. Barthélemy, Y. Li, N. Joseph-Mathurin, et al., “A soluble phosphorylated tau signature links tau, amyloid and the evolution of stages of dominantly inherited Alzheimer’s disease,” Nat. Med. 26(3), 398–407 (2020). [CrossRef]
8. V. L. Villemagne, S. Burnham, P. Bourgeat, B. Brown, K. A. Ellis, O. Salvado, C. Szoeke, S. L. Macaulay, R. Martins, P. Maruff, D. Ames, C. C. Rowe, and C. L. Masters, & Australian Imaging Biomarkers and Lifestyle (AIBL) Research Group, “Amyloid β deposition, neurodegeneration, and cognitive decline in sporadic Alzheimer’s disease: a prospective cohort study,” Lancet Neurol. 12(4), 357–367 (2013). [CrossRef]
9. E. M. Reiman, Y. T. Quiroz, A. S. Fleisher, et al., “Brain imaging and fluid biomarker analysis in young adults at genetic risk for autosomal dominant Alzheimer’s disease in the presenilin 1 E280A kindred: a case-control study,” Lancet Neurol. 11(12), 1048–1056 (2012). [CrossRef]
10. C. R. Jack Jr, V. J. Lowe, S. D. Weigand, H. J. Wiste, M. L. Senjem, D. S. Knopman, M. M. Shiung, J. L. Gunter, B. F. Boeve, B. J. Kemp, M. Weiner, and R. C. Petersen, the Alzheimer's Disease Neuroimaging Initiative, “Serial PIB and MRI in normal, mild cognitive impairment and Alzheimer’s disease: implications for sequence of pathological events in Alzheimer’s disease,” Brain 132(5), 1355–1365 (2009). [CrossRef]
11. R. J. Bateman, C. Xiong, T. L. Benzinger, et al., “Clinical and Biomarker Changes in Dominantly Inherited Alzheimer’s Disease,” N. Engl. J. Med. 367(9), 795–804 (2012). [CrossRef]
12. B. A. Gordon, T. M. Blazey, Y. Su, et al., “Spatial patterns of neuroimaging biomarker change in individuals from families with autosomal dominant Alzheimer’s disease: a longitudinal study,” Lancet Neurol. 17(3), 241–250 (2018). [CrossRef]
13. H. Braak, D. R. Thal, E. Ghebremedhin, and K. Del Tredici, “Stages of the Pathologic Process in Alzheimer Disease: Age Categories From 1 to 100 Years,” J. Neuropathol. Exp. Neurol. 70(11), 960–969 (2011). [CrossRef]
14. M.J. Prince, A. Comas-Herrera, M. Knapp, M. Guerchet, and M. Karagiannidou, “World Alzheimer report 2016: improving healthcare for people living with dementia: coverage, quality and costs now and in the future,” (2016).
15. P. Carmona, M. Molina, M. Calero, F. Bermejo-Pareja, P. Martínez-Martín, and A. Toledano, “Discrimination Analysis of Blood Plasma Associated with Alzheimer’s Disease Using Vibrational Spectroscopy,” J. Alzheimer’s Dis. 34(4), 911–920 (2013). [CrossRef]
16. H. Hildebrandt, F. Fink, A. Kastrup, M. Haupts, and P. Eling, “Cognitive Profiles of Patients with Mild Cognitive Impairment or Dementia in Alzheimer’s or Parkinson’s Disease,” Dement. Geriatr. Cogn. Disord. Extra 3(1), 102–112 (2013). [CrossRef]
17. M. Pais, L. Martinez, O. Ribeiro, J. Loureiro, R. Fernandez, L. Valiengo, P. Canineu, F. Stella, L. Talib, M. Radanovic, and O. V. Forlenza, “Early diagnosis and treatment of Alzheimer’s disease: new definitions and challenges,” Braz. J. Psychiatry 42(4), 431–441 (2020). [CrossRef]
18. H. Braak, E. Braak, D. Yilmazer, R. A. de Vos, E. N. Jansen, and J. Bohl, “Pattern of brain destruction in Parkinson’s and Alzheimer’s diseases,” J. Neural Transm. 103(4), 455–490 (1996). [CrossRef]
19. L. K. Clinton, M. Blurton-Jones, K. Myczek, J. Q. Trojanowski, and F. M. LaFerla, “Synergistic Interactions between Aβ, Tau, and α-Synuclein: Acceleration of Neuropathology and Cognitive Decline,” J. Neurosci. 30(21), 7281–7289 (2010). [CrossRef]
20. A. Haldenwanger, P. Eling, A. Kastrup, and H. Hildebrandt, “Correlation between Cognitive Impairment and CSF Biomarkers in Amnesic MCI, non-Amnesic MCI, and Alzheimer’s Disease,” J. Alzheimer’s Dis. 22(3), 971–980 (2010). [CrossRef]
21. H. Hildebrandt, A. Haldenwanger, and P. Eling, “False Recognition Correlates with Amyloid-β1–42 but not with Total Tau in Cerebrospinal Fluid of Patients with Dementia and Mild Cognitive Impairment,” J. Alzheimer’s Dis. 16(1), 157–165 (2009). [CrossRef]
22. M. Riemenschneider, N. Lautenschlager, S. Wagenpfeil, J. Diehl, A. Drzezga, and A. Kurz, “Cerebrospinal Fluid Tau and β-Amyloid 42 Proteins Identify Alzheimer Disease in Subjects With Mild Cognitive Impairment,” Archives of Neurology 59(11), 1729 (2002). [CrossRef]
23. T. Grimmer, M. Riemenschneider, H. Förstl, G. Henriksen, W. E. Klunk, C. A. Mathis, T. Shiga, H. J. Wester, A. Kurz, and A. Drzezga, “Beta Amyloid in Alzheimer’s Disease: Increased Deposition in Brain Is Reflected in Reduced Concentration in Cerebrospinal Fluid,” Biol. Psychiatry 65(11), 927–934 (2009). [CrossRef]
24. H. Hampel, K. Bürger, S. J. Teipel, A. L. W. Bokde, H. Zetterberg, and K. Blennow, “Core candidate neurochemical and imaging biomarkers of Alzheimer’s disease,” Alzheimer’s Dementia 4(1), 38–48 (2008). [CrossRef]
25. L. A. Lyon, C. D. Keating, A. P. Fox, B. E. Baker, L. He, S. R. Nicewarner, S. P. Mulvaney, and M. J. Natan, “Raman spectroscopy,” Anal. Chem. 70(12), 341–362 (1998). [CrossRef]
26. I. Durickovic, “Using Raman spectroscopy for characterization of aqueous media and quantification of species in aqueous solution,” Applications of Molecular Spectroscopy to Current Research in the Chemical and Biological Sciences. 405–427, (2016).
27. S. Pahlow, S. Meisel, D. Cialla-May, K. Weber, P. Rösch, and J. Popp, “Isolation and identification of bacteria by means of Raman spectroscopy,” Adv. Drug Delivery Rev. 89, 105–120 (2015). [CrossRef]
28. W. Lee, A. Nanou, L. Rikkert, F. A. W. Coumans, C. Otto, L. W. M. M. Terstappen, and H. L. Offerhaus, “Label-Free Prostate Cancer Detection by Characterization of Extracellular Vesicles Using Raman Spectroscopy,” Anal. Chem. 90(19), 11290–11296 (2018). [CrossRef]
29. A. S. Haka, K. E. Shafer-Peltier, M. Fitzmaurice, J. Crowe, R. R. Dasari, and M. S. Feld, “Diagnosing breast cancer by using Raman spectroscopy,” Proc. Natl. Acad. Sci. 102(35), 12371–12376 (2005). [CrossRef]
30. M. Jermyn, K. Mok, J. Mercier, J. Desroches, J. Pichette, K. Saint-Arnaud, L. Bernstein, M. C. Guiot, K. Petrecca, and F. Leblond, “Intraoperative brain cancer detection with Raman spectroscopy in humans,” Sci. Transl. Med. 7(274), 274ra19 (2015). [CrossRef]
31. E. Ryzhikova, O. Kazakov, L. Halamkova, D. Celmins, P. Malone, E. Molho, E. A. Zimmerman, and I. K. Lednev, “Raman spectroscopy of blood serum for Alzheimer’s disease diagnostics: specificity relative to other types of dementia,” J. Biophotonics 8(7), 584–596 (2015). [CrossRef]
32. F. Lussier, V. Thibault, B. Charron, G. Q. Wallace, and J.-F. Masson, “Deep learning and artificial intelligence methods for Raman and surface-enhanced Raman scattering,” TrAC, Trends Anal. Chem. 124, 115796 (2020). [CrossRef]
33. S. Weng, H. Yuan, X. Zhang, P. Li, L. Zheng, J. Zhao, and L. Huang, “Deep learning networks for the recognition and quantitation of surface-enhanced Raman spectroscopy,” Analyst 145(14), 4827–4835 (2020). [CrossRef]
34. B. Dietzek, D. Cialla, M. Schmitt, and J. Popp, “Introduction to the fundamentals of Raman spectroscopy,” In Confocal Raman Microscopy, Springer Berlin Heidelberg, Berlin, Heidelberg, 21–42, (2010).
35. D. Malka, G. Berkovic, Y. Hammer, and Z. Zalevsky, “Super-Resolved Raman Spectroscopy,” Spectrosc. Lett. 46(4), 307–313 (2013). [CrossRef]
36. D. Malka, B. Adler Berke, Y. Tischler, and Z. Zalevsky, “Improving Raman spectra of pure silicon using super-resolved method,” J. Opt. 21(7), 075801 (2019). [CrossRef]
37. D. Malka, G. Berkovic, Y. Tischler, and Z. Zalevsky, “Super-Resolved Raman Spectra of Toluene and Toluene–Chlorobenzene Mixture,” Spectrosc. Lett. 48(6), 431–435 (2015). [CrossRef]
38. A. Sanjeev, V. Trivedi, and Z. Zalevsky, “Optical reciprocity induced wavefront shaping for axial and lateral shifting of focus through a scattering medium,” Sci. Rep. 12(1), 6387 (2022). [CrossRef]
39. E. Christodoulou, J. Ma, G. S. Collins, E. W. Steyerberg, J. Y. Verbakel, and B. Van Calster, “A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models,” J. Clin. Epidemiol. 110, 12–22 (2019). [CrossRef]
40. T. van der Ploeg, P. C. Austin, and E. W. Steyerberg, “Modern modelling techniques are data hungry: a simulation study for predicting dichotomous endpoints,” BMC Med. Res. Methodol. 14(1), 137 (2014). [CrossRef]
