Simple 'smart' glass reveals the future of artificial vision

Smart glass can recognize handwritten digits, even if the writing style is different. The scattered light passes through the glass and then focuses to different locations, which correspond to different digit categories.

The sophisticated technology that powers face recognition in many modern smartphones someday could receive a high-tech upgrade that sounds-and looks-surprisingly low-tech.

Embedding artificial intelligence inside utterly inert objects is a concept that, at first glance, seems like something out of science fiction.

However, it's an advance that could open new frontiers for low-power electronics. Now, artificial intelligence gobbles up substantial computational resources (and battery life) every time you glance at your phone to unlock it with face ID. In the future, one piece of glass could recognize your face without using any power at all.

"This is completely different from the typical route to machine vision." says Yu.

He envisions pieces of glass that look like translucent squares.

But embedded within them, tiny strategically placed bubbles and impurities would bend light in specific ways to differentiate among different images. That's the artificial intelligence in action.

For their proof of concept, the engineers devised a method to make glass pieces that identified handwritten numbers. Light emanating from an image of a number enters at one end of the glass, and then focuses to one of nine specific spots on the other side, each corresponding to individual digits.

The glass was even dynamic enough to detect, in real-time, when a handwritten 3 was altered to become an 8.

"The fact that we were able to get this complex behavior with such a simple structure was really something." says Erfan Khoram, a graduate student in Yu's lab.

Designing the glass to recognize numbers was similar to a machine-learning training process-except that the engineers "trained" an analog material instead of digital codes. Specifically, the engineers placed air bubbles of different sizes and shapes as well as small pieces of light-absorbing materials like graphene at specific locations inside the glass.

"We're accustomed to digital computing, but this has broadened our view." says Yu. "The wave dynamics of light propagation provide a new way to perform analog artificial neural computing."

One such advantage is that the computation is completely passive and intrinsic to the material-and that means one piece of image-recognition glass could be used hundreds of thousands of times.

"We could potentially use the glass as a biometric lock, tuned to recognize only one person's face." says Yu. "Once built, it would last forever without needing power or internet, meaning it could keep something safe for you even after thousands of years."

Additionally, it works at literally the speed of light, because the glass distinguishes among different images by distorting light waves.

Although the up-front training process could be time consuming and computationally demanding, the glass itself is easy and inexpensive to fabricate.

In the future, the researchers plan to determine if their approach works for more complex tasks, such as facial recognition.

"The true power of this technology lies its ability to handle much more complex classification tasks instantly without any energy consumption," says Ming Yuan, a professor of statistics at Columbia University. "These tasks are the key to create artificial intelligence: to teach driverless cars to recognize a traffic signal, to enable voice control in consumer devices, among numerous other examples."

Unlike human vision, which is mind-bogglingly general in its capabilities to discern untold thousands of different objects, the smart glass could excel in specific applications-for example, one piece for number recognition, a different piece for identifying letters, another for faces, and so on.

"We're always thinking about how we provide vision for machines in the future, and imagining application specific, mission-driven technologies." says Yu. "This changes almost everything about how we design machine vision."

Author brief introduction

Zongfu Yu is the Dugald C. Jackson Associate Professor and Vilas Associate at UW-Madison. Graduate students Erfan Khoram, Ang Chen and Dianjing Liu contributed to the research. Qiqi Wang at Massachussetts Institute of Technology and Ming Yuan at Columbia University were collaborators. A DARPA Young Faculty Award program supported the research.

Sam Million-Weaver,, (608) 263-5988

Yu Research Group:





“我们正在利用光学方法把照相机、探测器和深度神经网络等集成在单块薄玻璃上。”来自麦迪逊分校电子与计算机工程系的助理教授Zongfu Yu如此说道。相关研究成果作为封面文章发表在Photonics Research第7卷第8期上。(Erfan Khoram, et al., Nanophotonic media for artificial neural inference).




“利用一种简单的结构来实现如此复杂的效果,这个结果令人振奋。”Yu实验室的研究生Erfan Khoram表示。

设计玻璃来识别数字的过程与机器学习的训练过程非常类似,所不同的是,工程师们“训练”的是一块模拟材料而非数字代码。具体来说,他们在一层仅为20 μm厚的薄玻璃中,将大小不同、形状迥异的空气气泡和小片吸光材料(如石墨烯)放置在设计好的位置上。








“这项技术的真正威力在于它能够在不消耗任何电源的情况下即时处理复杂的分类任务。”哥伦比亚大学统计学教授Ming Yuan表示,“这些任务是创造人工智能的关键,例如让无人驾驶汽车识别交通信号,在消费类设备中实现语音控制等等。”




Zongfu Yu是麦迪逊分校的Dugald C. Jackson副教授和Vilas Associate。研究生Erfan Khoram, Ang Chen和Dianjing Liu对该研究有贡献。麻省理工学院的Qiqi Wang和哥伦比亚大学的Ming Yuan是该研究的合作者。DARPA Young Faculty Award program 支持了该项研究。

Yu 研究小组主页: