Open Access
Add to BibSonomy
Abstract
This work combines two existing technologies to demonstrate the possibility for automated, targeted, phototherapy of psoriasis and other skin conditions: 1) Image-to-image translation via a neural network (NN) as a method of image segmentation. 2) Light control using a digital micromirror device (DMD). With a small dataset of just 104 patient photographs (labeled by expert dermatologists) our NN model was trained to identify regions of psoriasis that required treatment, achieving an average accuracy of 96.6%. The image output of the NN model was applied to a DMD and precise control over the shape of the illuminated region was demonstrated. In the proposed automated phototherapy device this would target treatment to the affected regions, minimizing exposure of healthy skin and the associated risks of patient harm.
Published by Optica Publishing Group under the terms of the Creative Commons Attribution 4.0 License. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.
1. Introduction
Despite considerable effort over many decades, image segmentation tasks can still pose a significant challenge for classic image processing code [1,2]. To achieve high performance, applications often require carefully controlled imaging conditions and hand-crafted algorithms that incorporate a priori knowledge about expected image features – resulting in solutions that are fragile (i.e., working parameter ranges for basic operations such as thresholding, morphological erosion/dilation and edge detection can shift significantly with changes in illumination level or image content). Increasing code complexity to handle edge-cases and improve robustness can significantly increase computing time per image which could be problematic for applications where live video must be segmented with low latency. In recent years there has been considerable interest in applying machine learning (ML) to object detection and image segmentation tasks [3,4] with part of the appeal deriving from the following factors: The NN is assumed to act as a universal function approximator [5]. During training, optimal values are determined for the numerical parameters that make up the NN model. In contrast to hand-crafting a conventional code algorithm, which often relies on human intuition and trial-and-error, the ML approach has the advantage of being almost entirely based on the provided training data. Several large datasets and challenges have been devised to test the segmentation performance of different methods, these include the currently popular MS COCO (instance segmentation, 80 object classes [6]) and the longer running PASCAL VOC (semantic segmentation, 20 object classes [7]). Figure 1 shows officially submitted results for the PASCAL VOC segmentation task; interestingly they show a clear step-change in segmentation performance at around 2014-2015. This correlates with the development of NN based segmentation methods such as [8] which claims a 20% improvement on the preceding state-of-the-art result for PASCAL VOC segmentation.
Fig. 1. Scatter plot showing PASCAL VOC mean segmentation accuracy (over 20 object classes) as a function of submission date. Note that between 2007 and 2012 the size of the training dataset was increased each year, results are therefore grouped based on the dataset used. A further distinction is made between methods that train only on the PASCAL VOC dataset (competition 5) and those that use external data (competition 6).
Many of the most important applications of image segmentation are in the field of medicine [9,10] where, for example, it can be used to identify different tissue types [11] or to highlight tumors [12,13] which can help to improve accuracy and maximize throughput in a wide range of medical imaging techniques. Segmentation can even be applied to sets of images, e.g., representing cross-sectional slices of a patient, allowing production of three-dimensional models with individual organs identified and labeled [14]. When used to assist in making a diagnosis, image segmentation tools are typically applied to static images, obtained previously using a medical imaging device. Although segmentation speed is desirable, the difference between a few milliseconds and a few minutes often wouldn’t be critical for diagnosis. In contrast, if image segmentation is used to target a treatment, it can be critical that there is low latency between the imaging system and the targeted output, for example to track patient movements accurately. In some applications it can therefore be attractive to use modern graphics processing units (GPUs) to accelerate image segmentation calculations. Whilst GPUs can be applied to classic image processing operations [15] they are particularly well-suited to the tensor mathematics required for both training, and subsequent inference from NN models. The ready availability of this hardware (and of software APIs for machine learning) means that even very complex NN models can now be used as part of a pipeline for low-latency, real-time, video processing applications.
Our work aims to demonstrate that NN-based image segmentation is suitable for use in real-time control of a medical treatment device. The test application chosen is UV phototherapy, which can be an effective treatment for a variety of skin conditions, including psoriasis [16–18] and vitiligo [19,20]. Although developed independently, our work has similarities with an existing medical device [21–23], in that it uses a digital micromirror device (DMD) to control the treatment beam. (The use of a DMD also facilitates structured light illumination [24] allowing this device to measure the 3d topography of the region to be treated). The aim of image segmentation within the control system is to automatically target treatment only to regions of the patient’s skin that are affected by the condition, and to minimize UV exposure to surrounding healthy skin. (Psoriasis manifests as scaly patches and plaques of inflamed skin that differ in color and texture to the surrounding healthy skin, making them suitable for identification and targeting via image segmentation.) The control system of the existing medical device is proprietary and therefore not know, however, we believe that in comparison to classic image processing methods [25] an NN based system could offer significant advantages in terms of the robustness, precision, and speed of segmentation, and in terms of the flexibility to train multiple NN models based on datasets for different skin conditions. Indeed, various NN-based image segmentation methods have been applied, for example, to psoriasis skin biopsies [26], and to photographs of psoriasis for identification [27,28], classification [29–32], and severity grading [33–35]. The key novelty that we claim for this work is in bringing together the existing concepts of NN-based image segmentation and DMD targeted phototherapy in order to demonstrate (for the specific case of psoriasis) the feasibility for this combination to achieve high performance in a new medical treatment device.
2. Methods
2.1 Experimental apparatus
In this proof-of-concept experiment, a computer monitor was used in place of the patient (see Fig. 2(a)). The monitor displays patient skin images, allowing us to rapidly present many examples of psoriasis to the NN within the treatment device (via its camera). To avoid risks associated with unnecessary UV exposure, the UV phototherapy beam was simulated using visible spectrum light from a data projector, which contains a digital micromirror device (DMD), a large 2D array of computer-controlled mirrors which allow the projected light to be switched on or off on a pixel-by-pixel basis [36]. The data projector (via a process termed spatial light modulation) is therefore able to shape its light output, to illuminate only regions that the NN has labeled as requiring treatment. A calibration procedure was required to determine the relationship between pixel coordinates on the monitor, camera images and the projector output; part of this process is shown in Fig. 2(b). Once the necessary coordinates are determined it is possible to calculate the image-transforms that must be applied to the projector and monitor such that they display perfectly overlapping images as viewed by the camera. The proposed automated phototherapy device will use a DMD similar to the one inside the data projector, in precisely the same manner, to achieve localized UV treatment as illustrated in Fig. 2(c).
Fig. 2. a) Experimental setup, with a computer monitor to display patient skin images, and a data projector to simulate the phototherapy device. b) Photograph showing part of the calibration process. c) Schematic flow diagram of the automated UV phototherapy device.
2.2 Data collection
For this initial work, medical staff took photographs of psoriasis (with patient consent) from patients attending the Psoriasis Clinic and UV treatment center at the Dermatology department at the University Hospital Southampton NHS Foundation Trust. Ethical approval (reference number 22/WS/0015) for the study was granted by the West of Scotland REC4 Research Ethics Committee and by the NHS Health Research Authority (HRA) and Health and Care Research Wales (HCRW). Photographs were taken using an Apple iPad with no special consideration given to consistency of, for example, lighting, orientation, physical scale, etc. The photographs varied in pixel dimensions (due to cropping and orientation). The width varied between 1708 and 4032 pixels whilst the height varied between 1393 and 4032 pixels.
Psoriasis photographs were accumulated over time as patients attended for treatment and/or review of their skin disease. This gave us the opportunity to investigate the effects of increasing the amount of data available to the NN. An initial data group that consisted of 45 psoriasis images (obtained from 10 patients) and the associated labeled images was later expanded with the addition of 59 more psoriasis images (from 4 patients, one of whom was also part of the initial data group). The dataset was further expanded by adding 65 images showing the background of the UV treatment and patient consultation rooms, with no psoriasis or patient skin in view (see Table 1).
Table 1. Summary of data augmentation (* example of partitioning scheme)
2.3 Labeling
Each photograph was viewed by two dermatologists who used image processing software to manually label regions that required treatment by painting those areas white (RGB value 255,255,255) and saving the labeled image under a new filename, see Fig. 3(a) and (b). Filenames included an anonymized patient ID number to aid data partitioning (e.g., to ensure that no patient appeared in both the training and testing datasets); this metadata was not accessible to the NN. Between the healthy skin and regions that are clearly psoriasis-affected there is a transition zone (at least a few pixels wide) where there is some uncertainty as to whether treatment should be applied. In these areas the dermatologists used their judgement and placed the labels accordingly, which was an important mechanism by which human expert knowledge and experience were transferred to the NN.
Fig. 3. a) Original patient photograph. b) Expert labeled image. c-h) Examples of augmented image-label pairs.
2.4 Data augmentation
When working with small datasets, data augmentation can be a valuable tool [37]. In this technique, for each item in a dataset, several variations (based on the original data) can be synthesized. Here, augmented data was generated by randomly cropping parts of each image (differing in both size and position). Each randomly cropped image was then rotated to an arbitrary angle, subjected to a random perspective distortion, and had its color intensity randomly altered, before finally scaling it to a size of 256 256 pixels. Identical transforms were applied to the corresponding label image, and the patient photograph and label image were then concatenated side-by-side to produce a single image-label pair (see examples in Fig. 3(c)-(h)). Image manipulation was carried out using standard Python libraries (Numpy and OpenCV) prior to NN training. Data augmentation greatly increased the number of images available for training and incorporated additional variability into the dataset.
During data augmentation, source images were selected one-by-one, with uniform probability. For each selection a new augmented image was generated. Our target number of augmented images for the first group of patient photographs was 10000. For subsequent data groups the same average augmentation factor was maintained (222.2 augmented images per source image as shown in Table 1). To achieve this, more augmented images than needed were generated. Some augmented images were discarded on the basis that: 1) They contained no psoriasis. 2) Their file size was too small (indicating the image contained little information). 3) They were randomly selected for discard with a uniform probability across all remaining augmented images, until the target number for the total of all augmented images was reached. This meant that whilst the average number of augmented images per source image was 222.2, the number of augmented images per source image varied between 133 and 269.
2.5 Neural network implementation
In 2017 researchers at UC Berkeley released their pix2pix software with the stated aim of achieving a “general-purpose solution to image-to-image translation problems” [38]. The code utilizes conditional adversarial networks to convert input images from one visual style to another, for example, converting line drawings to full color renderings. Pix2pix has proved to be extremely successful and has been applied widely in many fields of research [14,39–47]. The great appeal of pix2pix is that it is a general-purpose image-to-image processing tool, allowing the same algorithm to be applied to a wide variety of tasks without significant modification, additionally its computational requirements are relatively modest. In this work we use pix2pix for image segmentation by training a model that transforms digital photographs into labeled binary images. An alternative approach would be to implement a purpose-built image segmentation NN [4,48–50]; this could potentially yield higher performance but would require considerably more powerful computing resources to train and is therefore outside of the scope of this initial study.
Pix2pix is an example of a conditional generative adversarial neural network; it is termed adversarial because its two main parts, the generator and discriminator, compete against each other to improve their individual performance. During training, example data is provided to the NN in the form of side-by-side input-output pairs. In this work the input is an image of the patient’s skin, the output is the same size but is a binary image (the label) where each pixel is either black (indicating no treatment is needed) or white in the region that requires treatment (see Fig. 3). The generator (having a ‘U-net’ structure [51,52]) is only provided with the input images, and from these it must produce its own estimates of the corresponding label images. (Once the NN is fully trained, the generator estimates of the label should accurately fulfill the psoriasis segmentation task). Throughout training the discriminator (a convolutional ‘PatchGAN’ [53]) improves its capability to distinguish between real image-label pairs (where the label was applied by a medical professional), and pairs that include an NN-generated label. The feedback that allows the NN to learn is provided by a combination of three loss terms: The generator L1 loss (error between the predicted and actual labels), the discriminator loss, and the combined adversarial loss [38].
3. Results
During this work, various studies were undertaken to investigate the effect of different hyperparameters and different data partitioning schemes. For example, we determined that a batch size of 1 produced the highest performance and this was therefore used in all subsequent experiments (i.e. during training, NN internal parameters were updated each time an image-label pair was processed).
To assess NN performance it was necessary to devise a set of key performance metrics. The two that are presented in the results below are the ‘mean’ and the ‘above 90%’ metrics. The ‘mean’ metric is evaluated as the mean (across all images in the testing dataset) of the percentage of pixels that are correct (i.e., where the NN prediction of the label agrees with the one applied by a human expert). The ‘above 90%’ metric is the percentage of images in the testing dataset where greater than 90% of the NN predicted pixels are correct; (90% was an arbitrarily chosen threshold).
3.1 Training with different sized datasets
In this section we investigate the effect of increasing the dataset size. We consider separately the effect of adding two additional data groups to the initial one (data group #1, ‘Init’). Firstly, additional photographs of the clinic background (data group #2, ‘Bg’) were added. These do not contain any psoriasis (or even any patient skin). Secondly, an extended set of patient photographs was added (data group #3, ‘Ext’). See Table 1 for clarification of the data groups. We considered four combinations of these data groups, each of which was used to train a separate NN model. Key performance metrics were evaluated at each stage of training as presented in Fig. 4 and the highest values achieved for each metric are summarized in Table 2.
Fig. 4. Performance metrics during training, plotted for various datasets. a) shows the mean percentage of pixels that are correct. b) shows the percentage of images that exceed 90% of pixels being correct. Markers show individual sampling results whilst the solid lines show the moving average, with a window size of 2. Note that the horizontal axes are logarithmic, and the data extends to 1.5 M training steps.
Table 2. Summary of highest performance metrics achieved for each combination of data groups. *See Table 1 for more information about data groups
The uncertainty in the percentages shown in Table 2 can be assessed in several ways: Firstly, the standard deviation, across all augmented images in the test dataset, for the ‘mean’ metric was approximately 3.48%. Secondly, variation in the highest scores achieved was assessed over multiple training runs – this yielded a standard deviation of 1.6% for the ‘mean’ metric and 5.3% for the ‘above 90%’ metric. Finally, variation in the highest achieved scores was assessed for 10 different data partitioning schemes (i.e. with different patients chosen as the testing data). After two epochs of training the standard deviation of the ‘mean’ metric was 2.3% whilst it was 11.0% for the ‘above 90%’ metric. The considerably greater variability in the ‘above 90%’ metric was driven by an outlying case where patient 1 was used as the test data (this had just 62.6% of test images with more than 90% of pixels correct – the next lowest score was for patient 14 where 84.0% of test images had more than 90% of pixels correct).
In this experiment the addition of the clinic background photographs seems to have a largely negative effect on final NN performance. The patient photographs already contain many pixels that do not show psoriasis, so the overall effect of the background photographs seems to be to reduce the signal-to-noise ratio for psoriasis recognition. (Note that in the ‘Init + Bg’ training dataset there are 23,346 augmented images but only 9386 of them contain psoriasis). Despite this, the highest percentage of images that exceed the arbitrary 90% threshold was achieved by the NN model trained with the ‘Init + Ext + Bg’ dataset.
During the early stages of training, the ‘Init + Bg’ model suffered a plateau region where it appears to have been unable to improve its psoriasis recognition. This lasted until approximately 1500 training steps. The NN output during this plateau region was an approximate duplicate of the NN input (patient image) but with greatly reduced color saturation; the NN therefore constantly predicted that no psoriasis was present. This strategy nevertheless yielded higher performance (at this early stage of training) than the NN model trained using the ‘Init + Ext’ dataset, although this situation is reversed before completion of the first epoch (22,497 steps for the ‘Init + Ext’ dataset).
The different vertical scales on Fig. 4(a) and (b) highlight that the ‘above 90%’ metric is far more sensitive than the ‘mean’ metric. According to the more sensitive ‘above 90%’ metric, there is a 4.7% increase in performance with the addition of the extended patient image dataset, demonstrating that there is scope for considerable improvement in NN performance as more patient data is collected.
3.2 Best and worst case NN performance
One of the best performing neural network models was produced after 1.2 M training steps combining training data from all the data groups listed in Table 1 (‘Init + Ext + Bg’). In Fig. 5 we look in more detail at the performance of this NN model on individual test images.
Fig. 5. a) b) best c) median and d) worst-case performance, for each source image in the testing dataset. Top row shows the test images, the lower row shows analysis of the NN output. Numeric values are expressed as percentages and indicate: Mean, the percentage of pixels that are correct. True positive (Tp, green). True negative (Tn, black). False positive (Fp, red), False negative (Fn, yellow).
Figure 5(a) shows the best performing augmented image (in terms of the mean percentage of pixels that were correct), for each of the three source images in the testing dataset (patients 1 and 6). Note: trivial cases such as those that do not feature any psoriasis were disregarded. For all source images, the best performing case achieved 99% accuracy, with fewer than 1% of false negative or false positive pixels. Below each augmented patient image, a color-coded plot indicates regions that were correctly identified by the NN as psoriasis (in green), false positive (in red) and false negative (in yellow). Figure 5(b) provides additional examples of good psoriasis recognition, with the next highest scoring test images.
The worst-case performance for this NN model is presented in Fig. 5(d). For patient 1 image 3 the worst-case performance was 83.9% of pixels correct, with over 10% of false positive pixels occurring. The unsatisfactory results for some augmented images derived from this particular source image (patient 1, image 3) are likely because this image has the largest number of distinct regions of psoriasis in the whole dataset, and because there were relatively few images in the training dataset where the psoriasis presented in this sort of clustered arrangement – making this source image something of an outlier and a particularly challenging test case. Image analysis has shown that the poor worst-case performance for patient 6 image 1 and patient 6 image 2 are chiefly caused by a large scaling factor applied during augmentation, which increases the median size of psoriasis regions above the norm for the dataset. In addition, both of these source images have low image contrast relative to the average for the training dataset. The worst-case performance for patient 6 image 2 still yields >95% of pixels being correct.
The augmented images in the testing dataset produced a distribution of scores for the mean percentage of pixels correct metric. Figure 5(c) shows the augmented images within the testing dataset that scored closest to the median of this distribution and as such, they represent the typical performance of this NN model.
3.3 Demonstration of DMD beam shaping
Figure 6 demonstrates the use of the digital micromirror device (DMD), within the data projector, for beam shaping to achieve localized irradiation of psoriasis. With the experimental arrangement shown in Fig. 2(a) it was possible to display the patient image on the computer monitor, have this captured by the device camera and processed by the trained NN. The data projector then received the NN estimate of the region labeled as psoriasis and illuminated only this region with blue light (simulating the localized UV phototherapy treatment). The patient skin images in Fig. 6 are different augmented versions of the same source image (taken from the testing dataset, and therefore not seen during training). A video is included in the supplementary material (Visualization 1) where multiple augmented images are arranged into a sequence – simulating patient motion within live video. The sequence was recorded with approximately 1 second delay between each image so that the patient skin displayed on the computer monitor and the projected treatment beam could be clearly distinguished. Without any code optimization inference took on average 60 ms per input image (using an Nvidia Quadro RTX 5000 GPU) which is equivalent to approximately 16 frames per second (it is anticipated that, with code optimization, considerably higher frame rates will be feasible).
Fig. 6. From left to right: Column 1, augmented image as displayed on the monitor. Column 2, Analysis of the NN output (colors same as Fig. 5). Column 3, projected treatment beam, as viewed by the device camera. Column 4, Patient skin displayed on the monitor, overlayed with the projected treatment beam, as viewed by the device camera.
4. Conclusion
This work demonstrates that recognition of psoriasis, and differentiation of psoriasis from non-involved skin, are possible via an image-to-image transform neural network. The NN model used to produce the results shown in Fig. 5 and Fig. 6 achieved a mean of 96.6% of pixels correct per image, across the entire testing dataset. The same model exceeded 90% of pixels being correct for 93.8% of the augmented images in that testing dataset (576/614 images). Furthermore, our simulated patient and treatment device apparatus demonstrated that targeted phototherapy can be automated via control signals from a trained neural network. A production medical device could apply image analysis to estimate NN segmentation accuracy and issue a warning, or halt treatment, when input images deviate too far towards the edges of the parameter space sampled by the training data (e.g., If the input image contrast is too low, or if the segmented image predicts too many small distinct regions of psoriasis).
Since segmentation of still images has been successful (with unoptimized calculation times of approximately 60 ms) the trained NN should, in principle, be able to perform the same task on a live-video feed in the proposed automated phototherapy device. The NN-based approach to image segmentation is particularly suited to this as it makes efficient use of GPU acceleration. (Note that, during inference, NN calculation time is essentially independent of image complexity; meaning that input images with many small psoriasis plaques require the same processing time as images with a single plaque or with no psoriasis.)
The NN’s treatment accuracy likely already rivals that which can be achieved with a hand-held localized treatment device, where the beam width would typically be at least a few mm. Nevertheless, it is anticipated that it will be possible to further improve NN performance (reducing false positives in particular) if a state-of-the-art neural network is specifically designed for this image segmentation task. We recognize that, due to its limited size, our dataset cannot contain all possible distributions of psoriasis and skin types but expect that additional performance gains (and wider applicability) will be attained by significantly expanding the training dataset size (as demonstrated in section 3.1).
Funding
National Institute for Health and Care Research (NHSX AI_AWARD02168); Engineering and Physical Sciences Research Council (EP/N03368X/1, EP/T026197/1).
Acknowledgments
Beam-shaping for Laser-based Additive and Subtractive-manufacturing Techniques (BLAST). Engineering and Physical Sciences Research Council (EP/N03368X/1).
Lasers that Learn: AI-enabled intelligent materials processing. Engineering and Physical Sciences Research Council (EP/T026197/1).
This project is funded by the National Institute for Health and Care Research and NHSX (Artificial Intelligence, PhototheRapy Enhanced Via Artificially Intelligent Lasers [PREVAIL], AI_AWARD 02168). The views expressed are those of the author(s) and not necessarily those of the National Institute of Health and Care Research, NHSX or the Department of Health and Social Care.
We gratefully acknowledge the support of NVIDIA Corporation for donation of GPU hardware used for this research.
Disclosures
The authors declare no conflicts of interest.
Data availability
Data associated with this publication is held in the University of Southampton repository [54]. Access to the dataset is restricted as it contains patient images. Applications to access the data, made by bona fide researchers, will be considered by the data access committee.
References
1. A. Distante and C. Distante, “Image Segmentation,” in Handbook of Image Processing and Computer Vision: Volume 2: From Image to Pattern, A. Distante and C. Distante, eds. (Springer International Publishing, Cham, 2020), pp. 271–332.
2. Y. Yu, C. Wang, Q. Fu, et al., “Techniques and challenges of image segmentation: a review,” in Electronics, (2023).
3. S. Minaee, Y. Boykov, F. Porikli, et al., “Image segmentation using deep learning: a survey,” arXiv, arXiv:2001.05566 (2020). [CrossRef]
4. Y. Mo, Y. Wu, X. Yang, et al., “Review the state-of-the-art technologies of semantic segmentation based on deep learning,” Neurocomputing 493, 626–646 (2022). [CrossRef]
5. K. Hornik, M. Stinchcombe, and H. White, “Multilayer feedforward networks are universal approximators,” Neural Netw. 2(5), 359–366 (1989). [CrossRef]
6. T.-Y. Lin, M. Maire, S. Belongie, et al., “Microsoft coco: Common objects in context,” in Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, (Springer, 2014), 740–755.
7. M. Everingham, L. Van Gool, C. K. I. Williams, et al., “The pascal visual object classes (VOC) challenge,” Int J Comput Vis 88(2), 303–338 (2010). [CrossRef]
8. J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015), 3431–3440.
9. A. Norouzi, M. S. M. Rahim, A. Altameem, et al., “Medical image segmentation methods, algorithms, and applications,” IETE Technical Review 31(3), 199–213 (2014). [CrossRef]
10. M. Antonelli, A. Reinke, S. Bakas, et al., “The medical segmentation decathlon,” Nat. Commun. 13(1), 4128 (2022). [CrossRef]
11. D. S. W. Ting, L. R. Pasquale, L. Peng, et al., “Artificial intelligence and deep learning in ophthalmology,” Br. J. Ophthalmol. 103(2), 167–175 (2019). [CrossRef]
12. T. Kooi, G. Litjens, B. van Ginneken, et al., “Large scale deep learning for computer aided detection of mammographic lesions,” Med. Image Anal. 35, 303–312 (2017). [CrossRef]
13. R. Ranjbarzadeh, A. Bagherian Kasgari, S. Jafarzadeh Ghoushchi, et al., “Brain tumor segmentation based on deep learning and an attention mechanism using MRI multi-modalities brain images,” Sci. Rep. 11(1), 10930 (2021). [CrossRef]
14. B. S. Mackay, S. Blundell, O. Etter, et al., “Automated 3D labelling of fibroblasts and endothelial cells in SEM-imaged placenta using deep learning,” in 13th International Joint Conference on Biomedical Engineering Systems and Technologies, 2020), 46–53.
15. E. Smistad, T. L. Falch, M. Bozorgi, et al., “Medical image segmentation on GPUs – A comprehensive review,” Med. Image Anal. 20(1), 1–18 (2015). [CrossRef]
16. “Psoriasis: assessment and management,” in Clinical Guideline [CG153], (National Institute for Health and Care Excellence, 2017).
17. K. Boswell, H. Cameron, J. West, et al., “Narrowband ultraviolet B treatment for psoriasis is highly economical and causes significant savings in cost for topical treatments,” Br. J. Dermatol. 179(5), 1148–1156 (2018). [CrossRef]
18. J. Foerster and R. Dawe, “Phototherapy achieves significant cost savings by the delay of drug-based treatment in psoriasis,” Photodermatol., Photoimmunol. Photomed. 36(2), 90–96 (2020). [CrossRef]
19. K. Ezzedine and N. Silverberg, “A Practical Approach to the Diagnosis and Treatment of Vitiligo in Children,” Pediatrics 138(1), e20154126 (2016). [CrossRef]
20. J. M. Bae, H. M. Jung, B. Y. Hong, et al., “Phototherapy for vitiligo: a systematic review and meta-analysis,” JAMA Dermatol. 153(7), 666–674 (2017). [CrossRef]
21. T. Werfel, F. Holiangu, K. H. Niemann, et al., “Digital ultraviolet therapy: a novel therapeutic approach for the targeted treatment of psoriasis vulgaris,” Br J Dermatol 172(3), 746–753 (2015). [CrossRef]
22. U. Reidel, S. Bechstein, B. Lange-Asschenfeldt, et al., “Treatment of localized mycosis fungoides with digital UV photochemotherapy,” Photodermatol., Photoimmunol. Photomed. 31(6), 333–340 (2015). [CrossRef]
23. T. Graier, A. Hofer, and P. Wolf, “Digital ultraviolet B phototherapy in vitiligo: proof of concept,” Br. J. Dermatol. 182(5), 1293–1294 (2020). [CrossRef]
24. S. Zhang, “High-speed 3D shape measurement with structured light methods: A review,” Optics and Lasers in Engineering 106, 119–131 (2018). [CrossRef]
25. N. van Geel, I. Saeys, J. Van Causenbroeck, et al., “Image analysis systems to calculate the surface area of vitiligo lesions: A systematic review of measurement properties,” Pigm. Cell Melanoma Res. 35(5), 480–494 (2022). [CrossRef]
26. A. Pal, U. Garain, A. Chandra, et al., “Psoriasis skin biopsy image segmentation using Deep Convolutional Neural Network,” Computer Methods and Programs in Biomedicine 159, 59–69 (2018). [CrossRef]
27. M. Dash, N. D. Londhe, S. Ghosh, et al., “PsLSNet: Automated psoriasis skin lesion segmentation using modified U-Net-based fully convolutional network,” Biomedical Signal Processing and Control 52, 226–237 (2019). [CrossRef]
28. S. Zhao, B. Xie, Y. Li, et al., “Smart identification of psoriasis by images using convolutional neural networks: a case study in China,” Journal of the European Academy of Dermatology and Venereology 34(3), 518–524 (2020). [CrossRef]
29. R. Roslan, I. Razly, B. Sabri, et al., “Evaluation of psoriasis skin disease classification using convolutional neural network,” IAES International Journal of Artificial Intelligence (IJ-AI) 9, 349 (2020). [CrossRef]
30. Y. Yang, J. Wang, F. Xie, et al., “A convolutional neural network trained with dermoscopic images of psoriasis performed on par with 230 dermatologists,” Comput. Biol. Med. 139, 104924 (2021). [CrossRef]
31. S. F. Aijaz, S. J. Khan, F. Azim, et al., “Deep learning application for effective classification of different types of psoriasis,” Journal of Healthcare Engineering 2022, 1–12 (2022). [CrossRef]
32. R. Raj, N. D. Londhe, and R. Sonawane, “PsLSNetV2: End to end deep learning system for measurement of area score of psoriasis regions in color images,” Biomedical Signal Processing and Control 79, 104138 (2023). [CrossRef]
33. Y. Li, Z. Wu, S. Zhao, et al., “PSENet: Psoriasis Severity Evaluation Network,” Proceedings of the AAAI Conference on Artificial Intelligence 34(01), 800–807 (2020). [CrossRef]
34. M. J. Schaap, N. J. Cardozo, A. Patel, et al., “Image-based automated psoriasis area severity index scoring by convolutional neural networks,” Journal of the European Academy of Dermatology and Venereology 36(1), 68–75 (2022). [CrossRef]
35. C.-I. Moon, J. Lee, Y. S. Baek, et al., “Psoriasis severity classification based on adaptive multi-scale features for multi-severity disease,” Sci. Rep. 13(1), 17331 (2023). [CrossRef]
36. L. J. Hornbeck, “Digital light processing for high-brightness high-resolution applications,” in Projection Displays III, (SPIE, 1997), 27–40.
37. R. Hao, K. Namdar, L. Liu, et al., “A comprehensive study of data augmentation strategies for prostate cancer detection in diffusion-weighted MRI using convolutional neural networks,” Journal of Digital Imaging 34(4), 862–876 (2021). [CrossRef]
38. P. Isola, J. Y. Zhu, T. H. Zhou, et al., and IEEE “Image-to-image translation with conditional adversarial networks,” in 30th IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Conference on Computer Vision and Pattern Recognition2017), 5967–5976.
39. D. J. Heath, J. A. Grant-Jacob, Y. H. Xie, et al., “Machine learning for 3D simulated visualization of laser machining,” Opt. Express 26(17), 21574–21584 (2018). [CrossRef]
40. S. Kaji and S. Kida, “Overview of image-to-image translation by use of deep neural networks: denoising, super-resolution, modality conversion, and reconstruction in medical imaging,” Radiological Physics and Technology 12(3), 235–248 (2019). [CrossRef]
41. Y. Y. Qu, Y. Z. Chen, J. Y. Huang, et al., “Enhanced Pix2pix Dehazing Network,” in 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Conference on Computer Vision and Pattern Recognition2019), 8152–8160.
42. M. Eslami, S. Tabarestani, S. Albarqouni, et al., “Image-to-images translation for multi-task organ segmentation and bone suppression in chest x-ray radiography,” IEEE Trans. Med. Imaging 39(7), 2553–2565 (2020). [CrossRef]
43. Y. H. Feng, Z. Y. Chen, D. L. Wang, et al., “DeepWelding: a deep learning enhanced approach to GTAW using multisource sensing images,” IEEE Trans. Ind. Inf. 16(1), 465–474 (2020). [CrossRef]
44. J. A. Grant-Jacob, M. Praeger, M. Loxham, et al., “Lensless imaging of pollen grains at three-wavelengths using deep learning,” Environ. Res. Commun. 2(7), 075005 (2020). [CrossRef]
45. P. Klages, I. Benslimane, S. Riyahi, et al., “Patch-based generative adversarial neural network models for head and neck MR-only planning,” Med. Phys. 47(2), 626–642 (2020). [CrossRef]
46. H. Abdelmotaal, A. A. Abdou, A. F. Omar, et al., “Pix2pix conditional generative adversarial networks for scheimpflug camera color-coded corneal tomography image generation,” Trans. Vis. Sci. Tech. 10(7), 21 (2021). [CrossRef]
47. J. A. Grant-Jacob, M. Praeger, R. W. Eason, et al., “Semantic segmentation of pollen grain images generated from scattering patterns via deep learning,” J. Phys. Commun. 5(5), 055017 (2021). [CrossRef]
48. K. He, G. Gkioxari, P. Dollár, et al., “Mask R-CNN,” in arXiv, arXiv e-prints, (2017). [CrossRef]
49. S. Liu, L. Qi, H. Qin, et al., “Path aggregation network for instance segmentation,” arXiv, arXiv:1803.01534. (2018). [CrossRef]
50. A. Kirillov, E. Mintun, N. Ravi, et al., “Segment anything,” arXiv, arXiv:2304.02643 (2023). [CrossRef]
51. O. Ronneberger, P. Fischer, and T. Brox, “U-Net: convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015 (Springer International Publishing, 2015), 234–241.
52. N. Siddique, S. Paheding, C. P. Elkin, et al., “U-Net and its variants for medical image segmentation: a review of theory and applications,” IEEE Access 9, 82031–82057 (2021). [CrossRef]
53. C. Li and M. Wand, “Precomputed real-time texture synthesis with Markovian generative adversarial networks,” arXiv, arXiv:1604.04382. (2016). [CrossRef]
54. M. Praeger, J. Scott, J. A. Grant-Jacob, et al., “Dataset for: automated, localized, phototherapy of psoriasis via machine learning,” University of Southhampton1.0, 2022https://doi.org/10.5258/SOTON/D2485
