Open Access
Add to BibSonomy
Abstract
An optical fingerprinting solution with a switchable backlight is demonstrated for liquid crystal displays (LCDs). In fingerprinting mode, the collimated infrared light is applied to improve the signal-noise ratio (SNR) between the valley and ridge of the fingerprint. Compared with the conventional backlight, the proposed structure can effectively improve the SNR. Furthermore, the fingerprinting mode and the display mode may work at the same time without interference because of the different wavelengths and light paths. It shows the great potential application of LCD-based optical fingerprints.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Fingerprint identification technologies are considered to be one of the most reliable biometric technology, which has been applied in many fields such as mobile payment and mobile device security to protect personal privacy [1–3]. However, as consumers pursue the mobile phone appearance and full-screen, fingerprinting in display has become one of the most competitive solutions. Currently, owing to the simple structure and active luminescence [4,5], there have been several fingerprinting solutions for organic light-emitting diode (OLED) displays [6,7]. But the liquid crystal display (LCD) technology, one of the most popular display technologies in the display market, does not have a good fingerprinting solution due to the complexity of the structure of LCDs and the weak light intensity after passing the polarizers and the liquid crystal layer. Several techniques have been developed to solve these issues. Jeong et al. proposed a dual-gate photosensitive thin-film transistor model for fingerprint sensor integration into active matrix display [8]. Xu et al. reported a dual-gate photosensitive TFT π-shape channel for fingerprint sensor integration in display pixels [9]. These techniques provided a feasible solution of fingerprinting in LCD. But neither of them can provide a high signal-noise ratio (SNR) which is used to characterize the performance of fingerprint identification. The high SNR means good fingerprinting performance. Ma et al. have demonstrated an optical touch screen integrated with fingerprint recognition [10] which improved the SNR. But the fingerprinting structure is too complex. Therefore, a fingerprinting solution with a high SNR and a simple structure is needed for LCDs.
In this paper, we proposed an optical fingerprinting solution with a collimated backlight for LCDs. In fingerprinting mode, the infrared collimated light is applied to improve the SNR between the valley and ridge of fingerprint and eliminate the interference between fingerprinting mode and display mode. The high SNR and non-interference between the fingerprint recognition and display show the great potential application in LCDs
2. Design principle
In fingerprinting mode, the ridge area of the fingerprint directly contacts the upper-substrate of LCD and the valley area of the fingerprint is filled with air, which makes the refractive indices of the fingerprint valley area and ridge area are different. Therefore, according to Fresnel's equation, the effective reflectivity of the contact interface between ridge areas and valley areas is different. The sensors integrated into the screen can recognize the difference of the reflection light intensity between the ridge and valley. And that is the principle of the optical fingerprint imaging. But the small difference of reflected light intensity between ridge areas and valley areas and the high noise light intensity are the main causes that limit the optical fingerprinting application in LCDs.
Where RAF is the ridge area of a fingerprint, VAF is the valley area of the fingerprint. The red lines denote signal light (SL), the green lines denote total internal reflected light (TRL) and the blue lines denote the scattered light from the skin (SLS).
Figure 1(a) shows the light propagation path in LCD with a conventional backlight unit (BLU). In fingerprinting mode, the reflected light received by photodiodes (PDs) can be divided into two parts: the signal light (SL) and the noise light (NL). The SL that the red lines showed in Fig. 1(a) is reflected from the ridge or valley areas directly above the PDs. To get a high SNR fingerprint image, the light intensity difference of the valley area and the ridge area should be as large as possible. Where the SNR is used to judge the difference of the reflected light intensity of the valley and ridge areas. The SL intensity of valley areas (LIV) is always bigger than the SL intensity of the ridge area (LIR). The SNR can be written as Eq. (1):
The NL contains two parts: total internal reflected light in LCD (TRL) and scattered light from the skin (SLS). TRL that the green lines showed in Fig. 1(a) is the noise of fingerprinting. And the TRL is closely related to the divergent angle of BLU. Therefore, to decrease the noise of the fingerprint image, the intensity of the TRL should be as small as possible. SLS that the blue lines showed in Fig. 1(a) is also the noise of the fingerprinting. SLS is closely related to the thickness of the upper-substrate and will increase as the thickness of the upper-substrate increases. Figure 1(b) shows the light propagation path in LCD with a collimated BLU. With a collimated BLU, the reflection angle θ of the TRL will be smaller than that with a conventional BLU. The interference from TRL will be partly suppressed. Therefore, the intensity of NL will be reduced and the SNR of the fingerprint image will be improved.Although the LCD with a collimated BLU could improve the quality of the fingerprint image, the viewing angle of the panel could be reduced greatly. So, a switchable BLU with a large divergent angle and a collimated light is needed. Our previous work demonstrated a switchable BLU. The switchable BLU can provide a divergent backlight with ±55° of full width at half maximum (FWHM) and a collimated backlight with ±10° of FWHM [11–12]. The collimated visible light can improve the SNR in fingerprinting mode, but interference between fingerprinting mode and display mode will appear. Also, the surrounding light will affect the fingerprinting. To eliminate the interference between fingerprinting mode and display mode, the fingerprinting light is set as infrared light. In the display mode, only the divergent visible light source (DVLS) works. When the fingerprint recognition is required, the infrared collimated light source (ICLS) will be turned on. Because the DVLS and the ICLS are at different wavelengths, there is no interference between the display mode and fingerprinting mode. And the ICLS can be turned off after fingerprint recognition to save the power. As the directional light may much improve the SNR, a collimated backlight system is very important for the device. However, the design of the backlight is complicated, the fabrication process is complex, and the yield is relative low. These factors maybe limit the commercial application of the device.
3. Simulation
The fingerprinting performance of LCD with an ICLS was simulated by the commercial optical system modeling software, LightTools. The fingerprinting simulation model is illustrated in Fig. 2. As shown in Fig. 2, the refractive indices of upper-substrate, LC layer, ITO-SiNx equivalent layer and lower-substrate are 1.5, 1.55, 2.0 and 1.5, respectively. The refractive index of the finger is 1.42. The reflection character of the finger could be regarded as the Lambertian body with a scattering of 35%, and the absorption was 65% [11]. The finger model is an elliptical cylinder with a long axis of 12 mm and a short axis of 8 mm. The width of the ridge and valley parts are 0.5 mm and 0.5 mm, respectively. The receiver was set to a 20 mm*20 mm plane. The BM is set to an absorber with an aperture ratio of 41.6%. And a DVLS and ICLS switchable BLU is adopted. The wavelength of DVLS is 550 nm and the wavelength of ICLS is 1550 nm. The entire model is immersed in the air whose refractive index is 1.0.
The divergent angle of conventional BLU is about ±60° of FWHM, the divergent angle of BLU is set from 0° to ±60° of FWHM. And the thickness of the upper-substrate is set from 0.1 mm to 0.8 mm. As revealed in Fig. 3, the SNR almost unchanged about 3.05 with the divergent angle of 0° under the different thicknesses of upper-substrate. Similarly, when the divergence angle is ±15° of FWHM, the SNR changes under different thicknesses of upper-substrate are not large, which is about 2.25. Compared to the 0° of the divergent angle of BLU, it is 26.2% lower. This is because when the divergent angle of BLU is small, the TRL is much bigger than SLS. Therefore, the change of the divergent angle plays a leading role in the change of SNR. And as the divergent angle of BLU increases, SLR and SLS gradually approach until SLS is greater than SLR. When the divergent angle of BLU increases to ±30° of FWHM, the maximum SNR is about 1.95. When compared with 0° and ±15° of FWHM of the divergent angle of BLU, the SNR decreased by 36.1% and 13.3% respectively. And along with the increasing of the thickness of upper-substrate from 0.1 mm to 0.8 mm, the SNR decreased by 13.3%. When the divergent angle increases to ±60° of FWHM, the maximum SNR decreases to 1.71. And with the thickness of upper-substrate increasing from 0.1 mm to 0.8 mm, the SNR decreases by 41.1%. Compared with the divergent angle of BLU of ±30° of FWHM, the maximum and the minimum SNR are reduced by 12.3% and 40.5% respectively.
Figure 4 shows the fingerprint brightness maps on the receiver with the different divergent angles of BLU and different thicknesses of the upper-substrate. Figures 4(a) and (b) show the fingerprint brightness maps with the 0° and ±15° of FWHM of the divergent angle of BLU under different thicknesses of upper-substrate, it can be seen that the fingerprint brightness maps almost remains unchanged. But when the divergent angle increase, as shown in Figs. 3(c) and (d), the quality of fingerprint brightness maps will get worse along with the thickness of the upper-substrate becomes thick. Especially in Fig. 4(d), the boundary of the valley and ridge areas get fuzzy obviously along with the thickness of upper-substrate increased.
Fig. 4. The fingerprint brightness maps on the receiver with the divergent angle of BLU of (a) 0°, (b) 15°, (c) 30°, (d) 60° and the thickness of upper-substrate of 0.1 mm to 0.7 mm
4. Measurement and analysis
To simply verify the feasibility of fingerprinting with a collimated light source, a simplified LCD model was adopted, which contains upper-substrate, PDs, lower-substrate and a switchable collimated BLU with the wavelength equaling to 532 nm. The diagram of the experimental system for measuring SNR is shown in Fig. 5. The long axis and the short axis of the patterned elliptical cylindrical model made of glass cement are 120 mm and 80 mm, respectively. And the width of the ridge and valley parts are 8 mm and 8 mm, respectively. According to the Fresnel's equation, when the light from BLU normal incident to the upper-substrate, the reflectance R can be written as:
Where n1 is the refractive index of upper-substrate, and n2 is the refractive index of the skin in the ridge parts or the refractive index of air in the valley parts. The refractive index of human skin is about 1.42, and the refractive index of the finger model which was made of glass cement is about 1.404. The reflectance deviation between the glass cement finger model and the real finger is about 0.03%. Therefore, the reflectance deviation caused by the refractive index can be ignored. In the SNR measuring system, the surface roughening was made on the patterned elliptical cylinder model to make the scattering rate close to the human skin. So, compared with the real finger, the finger model made of glass cement will not make a much change on SNR. The PD matrix (spectral range of sensitivity of PDs is 400 nm∼900 nm. And the size of PDs are 2 mm*2 mm) is under the upper substrate. In fingerprinting mode, the collimated light source was turned on. The beam emitted from the BLU and reflected on the interface between the finger model and upper-substrate, then hits the surface of PDs. Owing to the different light intensity between the valley and ridge areas of the model, the PDs will generate a different current signal. We can measure the current signal to figure out the SNR under the different divergent angles of BLU.The experimental diagram for capturing the fingerprint was shown in Fig. 6. The finger model is molded from the finger using glass cement. As the size of the CCD camera is too large, it has to be set at the same side of the BLU. Therefore, to imitate the experiment system for measuring SNR, the angle between the incident light and reflect light, Θ, should be as small as possible. The measured SNRs between the valley and ridge of the finger model are about 2.65 and 1.17 with the collimated light whose FWHM is about ±10° and conventional backlight whose FWHM is about ±70°. The measurement SNR shows the positive effect of the collimated light, but the value is lower than the simulation result because the experimental system is lack of BM layer. As shown in Fig. 2, the BM layer is between the finger and the PDs. In the fingerprinting mode, three parts of light, TRL, SLS and SL, reflected from the interface. TRL and SLS are the noise of fingerprinting, as their reflection angles are all larger than that of SL, BM may block part of TRL and SLS. Therefore, the intensity of TRL and SLS will increase without BM, resulting in the decrease of SNR.
Figure 7(a) shows the fingerprint image captured by a CCD camera with collimated light (the divergent angle is about ±10° of FWHM). The ridge and valley areas of the fingerprint can be clearly distinguished. The fingerprint image with a divergent light (the divergent angle is about ±70° of FWHM) is showed in Fig. 7(b). The boundary between the ridge and valley area of the fingerprint image is very fuzzy. The images of collimated light and divergent light well matched with the measurement result and simulation result which confirmed that a LCD with a collimated fingerprinting light can greatly improve the SNR.
Fig. 7. The fingerprint images with a CCD camera under (a) the collimated light and (b) the divergent light
5. Conclusion
An optical fingerprinting solution with a switchable collimated backlight for LCD is proposed. In fingerprinting mode, the collimated infrared light source is applied to improve the SNR between the valley and ridge of the fingerprint. Compared with conventional backlight, the proposed structure can improve the SNR effectively under different upper-substrate. Furthermore, the interference between fingerprinting and display can be absolutely eliminated because of the different wavelengths and light paths. The high SNR and non-interference between fingerprinting and display show the great potential application of LCD based optical fingerprinting.
Funding
National Natural Science Foundation of China (61727808, 61775135).
Acknowledgments
The authors would like to thank our colleagues for supporting our experiments, as well as the reviewers from OE for their thoughtful comments.
Disclosures
The authors declare no conflicts of interest.
References
1. M. G. Galterio, S. A. Shavit, and T. Hayajneh, “A Review of Facial Biometrics Security for Smart Devices,” Computers 7(3), 37–48 (2018). [CrossRef]
2. A. K. Jain, R. Arun, and S. Pankanti, “Biometrics: a tool for information security,” IEEE Trans. Inform. Forensic Secur. 1(2), 125–143 (2006). [CrossRef]
3. B. Miller, “Vital signs of identity [biometrics],” IEEE Spectrum 31(2), 22–30 (1994). [CrossRef]
4. P. E. Burrows, G. Gu, V. Bulovic, Z. Shen, S. R. Forrest, and M. E. Thompson, “Achieving full-color organic light-emitting devices for lightweight, flat-panel displays,” IEEE Trans. Electron Devices 44(8), 1188–1203 (1997). [CrossRef]
5. B. Geffroy, L. R. Philippe, and P. Christophe, “Organic light-emitting diode (OLED) technology: materials, devices and display technologies,” Polym. Int. 55(6), 572–582 (2006). [CrossRef]
6. T. Kamada, R. Hatsumi, K. Watanabe, S. Kawashima, M. Katayama, H. Adachi, T. Ishitani, K. Kusunoki, D. Kubota, and S. Yamazaki, “OLED Display Incorporating Organic Image Sensor,” J. Soc. Inf. Disp. 50(1), 1011–1014 (2019). [CrossRef]
7. P. E. Malinowski, T. H. Ke, H. Akkerman, A. Nakamura, S. Shanmugam, A. V. Breemen, D. V. Velpen, E. Vandenplas, L. M. Hagelsib, L. Verscheuren, J. Genoe, W. Dehaene, A. J. Kronemeier, S. Steudel, G. Gelinck, and P. Heremans, “Organic photo-lithography for displays with integrated fingerprint scanner,” J. Soc. Inf. Disp. 50(1), 1007–1010 (2019). [CrossRef]
8. H. Jeong, M. Won, W. J. Shi, J. A. Weldon, X. Li, and K. Wang, “Feasibility Study of a Dual-gate Photosensitive Thin-Film Transistor for Fingerprint Sensor Integrated Active-Matrix Display,” Dig. Tech. Pap. - Soc. Inf. Disp. Int. Symp. 46(1), 1131–1134 (2015). [CrossRef]
9. M. Xu, H. Li, H. Ou, J. Chen, S. Z. Deng, N. S. Xu, and K. Wang, “Dual-gate photosensitive a-Si: H TFT array enabling fingerprint-sensor-integrated display application,” J. Disp. Technol. 12(8), 835–839 (2016). [CrossRef]
10. P. Ma, W. S. Cao, S. Y. Zheng, X. C. Cui, L. D. Guo, J. Zheng, and Z. C. Ye, “Optical Touch Screen Integrated With Fingerprint Recognition,” Dig. Tech. Pap. - Soc. Inf. Disp. Int. Symp. 48(1), 835–837 (2017). [CrossRef]
11. Y. J. Wang, J. G. Lu, W. C. Chao, and H. P. D. Shieh, “Switchable viewing angle display with a compact directional backlight and striped diffuser,” Opt. Express 23(16), 21443–21454 (2015). [CrossRef]
12. Y. J. Wang, S. H. Ouyang, W. C. Chao, J. G. Lu, and H. P. D. Shieh, “High directional backlight using an integrated light guide plate,” Opt. Express 23(2), 1567–1575 (2015). [CrossRef]
