Abstract

We present a new class of wavefront sensors by extending their design space based on machine learning. This approach simplifies both the optical hardware and image processing in wavefront sensing. We experimentally demonstrated a variety of image-based wavefront sensing architectures that can directly estimate Zernike coefficients of aberrated wavefronts from a single intensity image by using a convolutional neural network. We also demonstrated that the proposed deep learning wavefront sensor can be trained to estimate wavefront aberrations stimulated by a point source and even extended sources.

© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

Full Article  |  PDF Article
OSA Recommended Articles
Object-independent image-based wavefront sensing approach using phase diversity images and deep learning

Qi Xin, Guohao Ju, Chunyue Zhang, and Shuyan Xu
Opt. Express 27(18) 26102-26119 (2019)

Feature-based phase retrieval wavefront sensing approach using machine learning

Guohao Ju, Xin Qi, Hongcai Ma, and Changxiang Yan
Opt. Express 26(24) 31767-31783 (2018)

Machine learning for improved image-based wavefront sensing

Scott W. Paine and James R. Fienup
Opt. Lett. 43(6) 1235-1238 (2018)

References

  • View by:
  • |
  • |
  • |

  1. N. Ji, “Adaptive optical fluorescence microscopy,” Nat. Methods 14, 374–380 (2017).
    [Crossref] [PubMed]
  2. B. C. Platt and R. Shack, “History and principles of Shack-Hartmann wavefront sensing,” J. refractive surgery 17, 573–577 (1995).
    [Crossref]
  3. P. Schipani, L. Noethe, D. Magrin, K. Kuijken, C. Arcidiacono, J. Argomedo, M. Capaccioli, M. Dall’Ora, S. D’Orsi, J. Farinato, D. Fierro, R. Holzlöhner, L. Marty, C. Molfese, F. Perrotta, R. Ragazzoni, S. Savarese, A. Rakich, and G. Umbriaco, “Active optics system of the VLT Survey Telescope,” Appl. Opt. 55, 1573–1583 (2016).
    [Crossref] [PubMed]
  4. M. Shaw, K. O’Holleran, and C. Paterson, “Investigation of the confocal wavefront sensor and its application to biological microscopy,” Opt. Express 21, 19353–19362 (2013).
    [Crossref] [PubMed]
  5. C. Wu, J. Ko, and C. C. Davis, “Determining the phase and amplitude distortion of a wavefront using a plenoptic sensor,” J. Opt. Soc. Am. A 32, 964–978 (2015).
    [Crossref]
  6. R. Ragazzoni, “Pupil plane wavefront sensing with an oscillating prism,” J. Mod. Opt. 43, 289–293 (1996).
    [Crossref]
  7. F. Roddier, “Curvature sensing and compensation: a new concept in adaptive optics,” Appl. Opt. 27, 1223–1225 (1988).
    [Crossref] [PubMed]
  8. R. A. Gonsalves, “Phase retrieval and diversity in adaptive optics,” Opt. Eng. 21, 829–832 (1982).
    [Crossref]
  9. H. Campbell and A. Greenaway, “Wavefront sensing: From historical roots to the state-of-the-art,” EAS Publ. Ser. 22, 165–185 (2006).
    [Crossref]
  10. R. Horisaki, Y. Ogura, M. Aino, and J. Tanida, “Single-shot phase imaging with a coded aperture,” Opt. Lett. 39, 6466–6469 (2014).
    [Crossref] [PubMed]
  11. R. Horisaki, R. Egami, and J. Tanida, “Single-shot phase imaging with randomized light (SPIRaL),” Opt. Express 24, 3765–3773 (2016).
    [Crossref]
  12. C. Wang, X. Dun, Q. Fu, and W. Heidrich, “Ultra-high resolution coded wavefront sensor,” Opt. Express 25, 13736–13746 (2017).
    [Crossref]
  13. P. Berto, H. Rigneault, and M. Guillon, “Wavefront sensing with a thin diffuser,” Opt. Lett. 42, 5117–5120 (2017).
    [Crossref]
  14. C. M. Bishop, Pattern Recognition and Machine Learning (Springer-Verlag New York, Inc., NJ, USA, 2006).
  15. Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521, 436–444 (2015).
    [Crossref] [PubMed]
  16. T. Ando, R. Horisaki, and J. Tanida, “Speckle-learning-based object recognition through scattering media,” Opt. Express 23, 33902–33910 (2015).
    [Crossref]
  17. R. Horisaki, R. Takagi, and J. Tanida, “Learning-based imaging through scattering media,” Opt. Express 24, 13738–13743 (2016).
    [Crossref]
  18. R. Horisaki, R. Takagi, and J. Tanida, “Learning-based focusing through scattering media,” Appl. Opt. 56, 4358–4362 (2017).
    [Crossref] [PubMed]
  19. A. Sinha, J. Lee, S. Li, and G. Barbastathis, “Lensless computational imaging through deep learning,” Optica 4, 1117–1125 (2017).
    [Crossref]
  20. R. Horisaki, R. Takagi, and J. Tanida, “Deep-learning-generated holography,” Appl. Opt. 57, 3859–3863 (2018).
    [Crossref]
  21. R. L. Kendrick, D. S. Acton, and A. L. Duncan, “Phase-diversity wave-front sensor for imaging systems,” Appl. Opt. 33, 6533–6546 (1994).
    [Crossref]
  22. H. Guo, N. Korablinova, Q. Ren, and J. Bille, “Wavefront reconstruction with artificial neural networks,” Opt. Express 14, 6456–6462 (2006).
    [Crossref]
  23. S. W. Paine and J. R. Fienup, “Machine learning for improved image-based wavefront sensing,” Opt. Lett. 43, 1235–1238 (2018).
    [Crossref] [PubMed]
  24. J. W. Goodman, Introduction to Fourier Optics (McGraw-Hill, 1996).
  25. J. D. Schmidt, Numerical Simulation of Optical Wave Propagation with Examples in MATLAB (SPIE, WA, USA, 2010).
    [Crossref]
  26. A. Stern, Y. Zeltzer, and Y. Rivenson, “Quantization error and dynamic range considerations for compressive imaging systems design,” J. Opt. Soc. Am. A 30, 1069–1077 (2013).
    [Crossref]
  27. F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (IEEE, 2017), pp. 1800–1807.
    [Crossref]
  28. L. Gilles and B. Ellerbroek, “Shack-Hartmann wavefront sensing with elongated sodium laser beacons: centroiding versus matched filtering,” Appl. Opt. 45, 6568–6576 (2006).
    [Crossref]
  29. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” Int. Conf. on Learn. Represent. (ICLR) (2015).
  30. G. Cohen, S. Afshar, J. Tapson, and A. van Schaik, “EMNIST: an extension of MNIST to handwritten letters,” arXiv preprint p. 1702.05373 (2017).
  31. R. J. Noll, “Zernike polynomials and atmospheric turbulence,” J. Opt. Soc. Am. 66, 207–211 (1976).
    [Crossref]

2018 (2)

2017 (5)

2016 (3)

2015 (3)

2014 (1)

2013 (2)

2006 (3)

1996 (1)

R. Ragazzoni, “Pupil plane wavefront sensing with an oscillating prism,” J. Mod. Opt. 43, 289–293 (1996).
[Crossref]

1995 (1)

B. C. Platt and R. Shack, “History and principles of Shack-Hartmann wavefront sensing,” J. refractive surgery 17, 573–577 (1995).
[Crossref]

1994 (1)

1988 (1)

1982 (1)

R. A. Gonsalves, “Phase retrieval and diversity in adaptive optics,” Opt. Eng. 21, 829–832 (1982).
[Crossref]

1976 (1)

Acton, D. S.

Afshar, S.

G. Cohen, S. Afshar, J. Tapson, and A. van Schaik, “EMNIST: an extension of MNIST to handwritten letters,” arXiv preprint p. 1702.05373 (2017).

Aino, M.

Ando, T.

Arcidiacono, C.

Argomedo, J.

Ba, J.

D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” Int. Conf. on Learn. Represent. (ICLR) (2015).

Barbastathis, G.

Bengio, Y.

Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521, 436–444 (2015).
[Crossref] [PubMed]

Berto, P.

Bille, J.

Bishop, C. M.

C. M. Bishop, Pattern Recognition and Machine Learning (Springer-Verlag New York, Inc., NJ, USA, 2006).

Campbell, H.

H. Campbell and A. Greenaway, “Wavefront sensing: From historical roots to the state-of-the-art,” EAS Publ. Ser. 22, 165–185 (2006).
[Crossref]

Capaccioli, M.

Chollet, F.

F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (IEEE, 2017), pp. 1800–1807.
[Crossref]

Cohen, G.

G. Cohen, S. Afshar, J. Tapson, and A. van Schaik, “EMNIST: an extension of MNIST to handwritten letters,” arXiv preprint p. 1702.05373 (2017).

D’Orsi, S.

Dall’Ora, M.

Davis, C. C.

Dun, X.

Duncan, A. L.

Egami, R.

Ellerbroek, B.

Farinato, J.

Fienup, J. R.

Fierro, D.

Fu, Q.

Gilles, L.

Gonsalves, R. A.

R. A. Gonsalves, “Phase retrieval and diversity in adaptive optics,” Opt. Eng. 21, 829–832 (1982).
[Crossref]

Goodman, J. W.

J. W. Goodman, Introduction to Fourier Optics (McGraw-Hill, 1996).

Greenaway, A.

H. Campbell and A. Greenaway, “Wavefront sensing: From historical roots to the state-of-the-art,” EAS Publ. Ser. 22, 165–185 (2006).
[Crossref]

Guillon, M.

Guo, H.

Heidrich, W.

Hinton, G.

Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521, 436–444 (2015).
[Crossref] [PubMed]

Holzlöhner, R.

Horisaki, R.

Ji, N.

N. Ji, “Adaptive optical fluorescence microscopy,” Nat. Methods 14, 374–380 (2017).
[Crossref] [PubMed]

Kendrick, R. L.

Kingma, D. P.

D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” Int. Conf. on Learn. Represent. (ICLR) (2015).

Ko, J.

Korablinova, N.

Kuijken, K.

LeCun, Y.

Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521, 436–444 (2015).
[Crossref] [PubMed]

Lee, J.

Li, S.

Magrin, D.

Marty, L.

Molfese, C.

Noethe, L.

Noll, R. J.

O’Holleran, K.

Ogura, Y.

Paine, S. W.

Paterson, C.

Perrotta, F.

Platt, B. C.

B. C. Platt and R. Shack, “History and principles of Shack-Hartmann wavefront sensing,” J. refractive surgery 17, 573–577 (1995).
[Crossref]

Ragazzoni, R.

Rakich, A.

Ren, Q.

Rigneault, H.

Rivenson, Y.

Roddier, F.

Savarese, S.

Schipani, P.

Schmidt, J. D.

J. D. Schmidt, Numerical Simulation of Optical Wave Propagation with Examples in MATLAB (SPIE, WA, USA, 2010).
[Crossref]

Shack, R.

B. C. Platt and R. Shack, “History and principles of Shack-Hartmann wavefront sensing,” J. refractive surgery 17, 573–577 (1995).
[Crossref]

Shaw, M.

Sinha, A.

Stern, A.

Takagi, R.

Tanida, J.

Tapson, J.

G. Cohen, S. Afshar, J. Tapson, and A. van Schaik, “EMNIST: an extension of MNIST to handwritten letters,” arXiv preprint p. 1702.05373 (2017).

Umbriaco, G.

van Schaik, A.

G. Cohen, S. Afshar, J. Tapson, and A. van Schaik, “EMNIST: an extension of MNIST to handwritten letters,” arXiv preprint p. 1702.05373 (2017).

Wang, C.

Wu, C.

Zeltzer, Y.

Appl. Opt. (6)

EAS Publ. Ser. (1)

H. Campbell and A. Greenaway, “Wavefront sensing: From historical roots to the state-of-the-art,” EAS Publ. Ser. 22, 165–185 (2006).
[Crossref]

J. Mod. Opt. (1)

R. Ragazzoni, “Pupil plane wavefront sensing with an oscillating prism,” J. Mod. Opt. 43, 289–293 (1996).
[Crossref]

J. Opt. Soc. Am. (1)

J. Opt. Soc. Am. A (2)

J. refractive surgery (1)

B. C. Platt and R. Shack, “History and principles of Shack-Hartmann wavefront sensing,” J. refractive surgery 17, 573–577 (1995).
[Crossref]

Nat. Methods (1)

N. Ji, “Adaptive optical fluorescence microscopy,” Nat. Methods 14, 374–380 (2017).
[Crossref] [PubMed]

Nature (1)

Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521, 436–444 (2015).
[Crossref] [PubMed]

Opt. Eng. (1)

R. A. Gonsalves, “Phase retrieval and diversity in adaptive optics,” Opt. Eng. 21, 829–832 (1982).
[Crossref]

Opt. Express (6)

Opt. Lett. (3)

Optica (1)

Other (6)

F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (IEEE, 2017), pp. 1800–1807.
[Crossref]

J. W. Goodman, Introduction to Fourier Optics (McGraw-Hill, 1996).

J. D. Schmidt, Numerical Simulation of Optical Wave Propagation with Examples in MATLAB (SPIE, WA, USA, 2010).
[Crossref]

D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” Int. Conf. on Learn. Represent. (ICLR) (2015).

G. Cohen, S. Afshar, J. Tapson, and A. van Schaik, “EMNIST: an extension of MNIST to handwritten letters,” arXiv preprint p. 1702.05373 (2017).

C. M. Bishop, Pattern Recognition and Machine Learning (Springer-Verlag New York, Inc., NJ, USA, 2006).

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (12)

Fig. 1
Fig. 1 Schematic and experimental diagram of the deep learning wavefront sensor. LED: light emitting diode. P: Polarizer. SLM: Spatial light modulator. Xception: A convolutional neural network. DO: Dropout layer. FC: Fully connected layer.
Fig. 2
Fig. 2 An example pair from the test datasets. (a) The Zernike coefficients and (b) the corresponding phase map, which is normalized in the interval [−π, π].
Fig. 3
Fig. 3 Experimental results in the in-focus setup with a point source. (a) The captured image with the Zernike coefficients in Fig. 2(a) and (b) the estimated Zernike coefficients.
Fig. 4
Fig. 4 Experimental results in the overexposure setup with a point source. (a) The captured image with the Zernike coefficients in Fig. 2(a) and (b) the estimated Zernike coefficients.
Fig. 5
Fig. 5 Experimental results in the defocus setup with a point source. (a) The captured image with the Zernike coefficients in Fig. 2(a) and (b) the estimated Zernike coefficients.
Fig. 6
Fig. 6 Experimental results in the scatter setup with a point source. (a) The captured image with the Zernike coefficients in Fig. 2(a) and (b) the estimated Zernike coefficients.
Fig. 7
Fig. 7 Examples of object images from the EMNIST database.
Fig. 8
Fig. 8 Experimental results in the in-focus setup with extended sources. (a) The captured image with the Zernike coefficients in Fig. 2(a) and (b) the estimated Zernike coefficients.
Fig. 9
Fig. 9 Experimental results in the overexposure setup with extended sources. (a) The captured image with the Zernike coefficients in Fig. 2(a) and (b) the estimated Zernike coefficients.
Fig. 10
Fig. 10 Experimental results in the defocus setup with extended sources. (a) The captured image with the Zernike coefficients in Fig. 2(a) and (b) the estimated Zernike coefficients.
Fig. 11
Fig. 11 Experimental results in the scatter setup with extended sources. (a) The captured image with the Zernike coefficients in Fig. 2(a) and (b) the estimated Zernike coefficients.
Fig. 12
Fig. 12 Relationship between the number of training pairs and the accuracy obtained for the estimated coefficients when training for 15, 21 and 32 coefficients, while testing on wavefronts with 32 coefficients using extended sources and the defocus preconditioner.

Tables (2)

Tables Icon

Table 1 Summary of the accuracies (RMSEs) of the estimated Zernike coefficients in the experiments.

Tables Icon

Table 2 Summary of accuracies (RMSEs: rad) of the equivalent wavefronts from the Zernike coefficients estimated in the experiments.

Equations (4)

Equations on this page are rendered with MathJax. Learn more.

g = h f ,
h = | [ w ] | 2 ,
w ( r , θ ) = i = 1 a i z i ( r , θ ) ,
g p = 𝒫 [ h f ] ,

Metrics