Abstract

Deep learning using convolutional neural networks (CNNs) has been shown to significantly outperform many conventional vision algorithms. Despite efforts to increase the CNN efficiency both algorithmically and with specialized hardware, deep learning remains difficult to deploy in resource-constrained environments. In this paper, we propose an end-to-end framework to explore how to optically compute the CNNs in free-space, much like a computational camera. Compared to existing free-space optics-based approaches that are limited to processing single-channel (i.e., gray scale) inputs, we propose the first general approach, based on nanoscale metasurface optics, that can process RGB input data. Our system achieves up to an order of magnitude energy savings and simplifies the sensor design, all the while sacrificing little network accuracy.

© 2021 Optical Society of America

Full Article  |  PDF Article
More Like This
Optical frontend for a convolutional neural network

Shane Colburn, Yi Chu, Eli Shilzerman, and Arka Majumdar
Appl. Opt. 58(12) 3179-3186 (2019)

Convolutional neural network-based approach to estimate bulk optical properties in diffuse optical tomography

Sohail Sabir, Sanghoon Cho, Yejin Kim, Rizza Pua, Duchang Heo, Kee Hyun Kim, Youngwook Choi, and Seungryong Cho
Appl. Opt. 59(5) 1461-1470 (2020)

Neural network enabled metasurface design for phase manipulation

Li Jiang, Xiaozhong Li, Qingxin Wu, Lianhui Wang, and Li Gao
Opt. Express 29(2) 2521-2528 (2021)

References

You do not have subscription access to this journal. Citation lists with outbound citation links are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Supplementary Material (1)

NameDescription
Supplement 1       A design framework for metasurface optics-based convolutional neural networks: supplemental document

Data Availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

You do not have subscription access to this journal. Cited by links are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Figures (8)

You do not have subscription access to this journal. Figure files are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Equations (13)

You do not have subscription access to this journal. Equations are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Metrics