A thermal deformation measurement system based on calibrated phase-shifting digital holography is proposed. Two synchronized ordinary CMOS cameras are used in the calibrated phase-shifting digital holography system. One is to record the holograms including the object information, and the other is to record the interference fringes to evaluate phase-shifting errors. The calibrated phase-shifting digital holography can provide the high quality reconstructed images which are applied to calculate the thermal deformation of the object. Meanwhile, the thermal images of the object at different temperatures are recorded by a thermal camera. Nanometer-order thermal deformation measurement of an electronic device is achieved in a real experiment. Our measurement system could be useful for electric packaging materials development or the system design.
© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
Electronic devices such as Central Processing Unit (CPU), Graphics Processing Unit (GPU), flash memory continue to grow smaller and more powerful with the passing years. As the heat increases, the three-dimensional (3-D) thermal deformation measurement is becoming increasingly important for the engineers. Although the atomic force microscope (AFM) and confocal laser scanning microscopy (CLSM) are competent for a nanometer- or micrometer-order 3-D surface measurement, they are powerless for measuring a dynamic object because of long scanning time. Besides, a 3-D shape measurement method based on fringe projection techniques has been wildly investigated [1,2]. Whereas, it is difficult to achieve under micrometer measurement due to the lower resolution of the projector. In the previous studies, a method using double exposure holography to measure thermal deformation has been proposed . However, the object information is recorded on the optical sensitive materials which are difficult to provide a quantitative evaluation, and useless for measuring the moving object.
The digital holography  can obtain the 3-D information of the object from the reconstructed phase images with single-shot [5–7] or only several exposures [8–10]. This technique is capable of achieving 3-D measurement of a dynamic object, and has been extensively investigated in many fields, such as particle measurement , microscope , etc. In recent years, a thermal expansion measurement of thermoelectric materials by using in-line digital holography has been reported . Unfortunately, the quality of the reconstructed images was poor in the report because no phase-shifting method was used to reduce the conjugate image and zeroth-order wave. Practically the digital holography falls into two broad categories: in-line and off-axis digital holography, on the basis of the incident angle between the object wave and reference wave to the image sensor. The in-line digital holography is much more efficacious in measuring a large object than the off-axis method due to the limitation on the pixel size of the image sensor. Nevertheless, there is an inevitable problem that the conjugate image and zeroth-order wave are superimposed on the object image in in-line digital holography. To solve this problem the phase-shifting digital holography [5–10,14–16] was proposed. In this technique, a phase-shifting device like a mirror mounted on a piezoelectric transducer (PZT) is introduced in the reference arm to shift the phase of the reference beam to record multiple phase-shifted holograms. Then only the object image can be reconstructed by the phase-shifting calculation. However, phase-shifting errors are always introduced in the phase-shifted holograms, because of non-linearity of the PZT, wavelength fluctuation of the laser, frame loss in the camera and environmental disturbances.
Subsequently, numerous methods were proposed to overcome this problem, such as a closed loop phase control system , random phase-shifting method , self-calibrating algorithms based on statistical method . But some disadvantages exist in these techniques: just a single photodiode is applied to detect the phase-shifting errors in closed loop phase control system which is insufficient, output power fluctuation of the laser dramatically influences the quality of the reconstructed image in the random phase-shifting method, and the object in diffraction field must be assumed as sufficiently random or zero mean in the self-calibrating algorithms. Hence, we proposed a calibrated phase-shifting digital holography (CPSDH) system  capable of detecting the phase-shifting errors. In the previous study, a Japanese one-yen coin was set as the object to demonstrate that the quality of reconstructed images was improved by the calibrated phase-shifting digital holography. However, it was a preliminary experiment without any practical application.
In this paper, we develop a thermal deformation measurement system based on the calibrated phase-shifting digital holography, which is the first challenge for a practical application of the technique. To compare with the conventional calibrated phase-shifting digital holography , a thermal camera is introduced in the thermal deformation measurement system. In the developed system, two synchronized ordinary complementary metal–oxide semiconductor (CMOS) cameras are used in the calibrated phase-shifting digital holography. One is to record the holograms including the object information, and the other is to record the interference fringes to evaluate phase-shifting errors. Meanwhile, the thermal images of the object at different temperatures are recorded by the thermal camera. Nano-order thermal deformation measurement of an electronic device is achieved in the experiment.
2. Calibrated phase-shifting digital holography system (CPSDH)
The schematic of the calibrated phase-shifting digital holography system  is shown in Fig. 1. The collimated laser beam is divided into two beams by beam splitter (BS) 1. One beam illuminates the object, and the reflected beam from the object becomes object beam. The other passes through BS 3 and is reflected in a mirror mounted on a PZT, called the reference beam. The object beam and reference beam are interfered at the image sensor plane of camera 1, and the hologram is recorded. On the other hand, camera 2 records an interferogram with a periodic repetitive fringe pattern generated by two plane waves, which are extracted from BS 2 and 3 respectively. These two cameras work synchronously so that the hologram recorded by camera 1 and the interferogram recorded by camera 2 will synchronously change when the phase of the reference beam is shifted by the PZT. The sampling Moiré technique can accurately measure a minute displacement from a single image with a repetitive fringe pattern. The accuracy of sampling Moiré method can theoretically reach 1/500 of interference fringe pitch . Thus, the sampling Moiré method is used to analyze the interferograms recorded by camera 2, to evaluate the phase-shifting errors. Finally, the object images are reconstructed by the phase-shifting error compensation algorithm. Note that the position of these two cameras is not necessary to adjust strictly because the phase-shifting amount of reference wave at camera 1 and 2 only depends on the change of the optical length of each reference wave arm. In general, the shifting amount of the PZT is not equal to the theoretical value, since the surface of the mirror mounted on the PZT is not strictly perpendicular to the incoming beam. In the conventional phase-shifting digital holography, a mirror instead of the object before the measurement is used to adjust a interferogram with a periodic repetitive fringe pattern in the camera. Then adjusting the shifting amount of the PZT to make the phase-shifting amount of the interferogram in the camera to be the setting value. In the proposed system, the shifting amount of the PZT can be adjusted directly from the interference fringes displayed on the camera 2.
3. Sampling Moiré technique and principle of the phase-shifting error compensation algorithm
3.1 Sampling Moiré technique
The sampling Moiré technique can measure a minute displacement from a single image with a repetitive fringe pattern. Figure 2 shows the principle of the sampling Moiré technique . First the down-sampling processing is applied to the recorded grating pattern. Then multiple phase-shifted Moiré fringes are obtained and used to calculate the phase distribution of the Moiré fringes based on discrete Fourier transform algorithm. The detail processing is described as follows.
The pitch of the recorded grating in the CCD plane is supposed to P, and then the recorded intensity of the grating can be represented as22,23] is described as,
The principle of the sampling Moiré technique to determine the phase-shifting errors is shown in Fig. 3. In the calibrated phase-shifting digital holography system, multiple phase-shifted holograms with object information and interferograms with periodic repetitive fringe pattern are recorded by two synchronized cameras. The phase-shifting amount of reference wave at camera 1 and 2 just depends on the change of the optical length of each reference wave. The optical length of these two reference waves is synchronously shifted by the PZT. Therefore, the phase-shifting amount in camera 1 is equal to that in camera 2, and the phase-shifting amount in camera 2 is equal to the phase difference of the Moiré fringe patterns obtained from two adjacent interference fringe patterns. Thus, the actual phase-shifting amount of two adjacent holograms can be calculated from the phase difference of the Moiré fringe patterns. If the phase-shifting errors exist in the recorded holograms, they will be accurately detected by the sampling Moiré technique using following equation,
3.2 Phase-shifting error compensation algorithm
So far numerous phase-shifting methods with different phase-shifting steps have been proposed [5–10,24]. Supposing that the complex distribution of object wave at the image sensor plane is , the intensity distribution of phase-shifted hologram can be expressed as
Finally, the amplitude and phase distributions of the object wave at the object plane are obtained by applying the diffraction integral calculation.
We developed a thermal deformation measurement system based on the calibrated phase-shifting digital holography. Nano-order thermal deformation measurement of an electronic device is achieved in the experiment.
4.1 Experimental setup
The experimental setup of the thermal deformation measurement system based on the calibrated phase-shifting digital holography is illustrated in Fig. 4. A laser working at 532 nm was used as the light source. Two CMOS cameras (VCXU-50, Baumer, Inc.) with a resolution of 2448 x 2048 pixels and 3.45μm pixel pitch, and an infrared thermal camera (PI160, Optris, Inc.) with a resolution of 160 x 120 pixels were applied in the system. One thermal image was recorded at the end of the completion of each four phase-shifted holograms recording. The phase-shifting amount was set to π/2 by driving a PZT (PAZ005, Thorlabs, Inc.). A USB2.0 memory storage whose metal casing was removed shown in Fig. 5(a) was set as the object at the position 301 mm away from the image sensor of camera 1. The reflectance of the chip area of the object is low so that the intensity of the object wave is weak. Therefore, the object was illuminated by an inclined beam in order to obtain an adequate intensity of the object wave. If the object was illuminated by an in-lined beam using a beam splitter, the reflected wave from the object will pass through the beam splitter again. Then, the intensity of the object wave is too weak to be recorded. The thermal camera was located at the position 150 mm away from the object with about 35 inclined angle as shown in Fig. 5(b). The recording speed of the CMOS cameras were set to 20 frames per second (fps) based on the performance of the hardware such as the hard disk drive (HDD), recording speed of the cameras.
4.2 Experimental results
The temperature of the USB memory storage will increase when the data are read from or written in. The thermal deformation of the USB memory storage was measured continuously by the developed system. One hologram including the object information and one interferogram used for evaluating the phase-shifting errors are shown in Figs. 6(a) and 6(b), respectively. One reconstructed amplitude image by the CPSDH is represented in Fig. 6(c), and the magnified image of the area indicated by the rectangle with a red dotted line in Fig. 6(c) is shown in Fig. 6(d). We can see some shadows in the reconstructed amplitude image because the object was illuminated by an inclined beam. The shadows, chip and substrate areas are indicated in Fig. 6(c). The phase difference images of the area indicated with the red dashed line in Fig. 6(d) are calculated when the object is subjected to different temperatures.
Figure 7 shows the experimental result under several different temperatures. Figures 7(a)-7(d) are the thermal images of the object recorded by the thermal camera. Figure 7(e) is the phase image in case of the object at the state in Fig. 7(a), which is set to the original phase distribution. Here, we used a windowed phase-shifting method  that is a powerful tool for reducing the speckle noise to calculate the phase difference between the phase images reconstructed at the state in Figs. 7(b)-7(d) with Fig. 7(e), and the results are shown in Figs. 7(f)-7(h). The calculation area is indicated with the dashed line in Fig. 6(d) and also indicated in Figs. 7(a)-7(d). We can see that some fringes appear in the chip and substrate areas, and there is no fringe in the shadow area. Comparing Fig. 7(b) with Fig. 7(f), both the temperature of the chip and substrate areas are around 28. But the fringes in the substrate area is narrower than that in the chip area because the chip and substrate are made of different materials. As the data were read from or written in the USB memory storage, the temperature of the chip area increased faster than the substrate area. The fringes in the substrate area became narrower and the deformation kept a gradient with the temperature increased. On the other hand, the surface of the chip became a spherical cap shape when the temperature increased to 45 as shown in Figs. 7(d) and 7(h). The CPSDH can detect whether the phase-shifting errors exist in the phase-shifted holograms and improve the quality of the reconstructed image using the phase-shifting error compensation algorithm. Phase-shifting errors can be caused by a variety of reasons such the non-linearity of the PZT, frame loss in the camera, wavelength fluctuation of the laser and environmental disturbances. In the experiment, 125 groups of four phase-shifted holograms were continuously recorded when the temperature of the object changed from 22 to 45.3. Twice large phase-shifting errors occurred in the recorded holograms because of the frame loss in the camera. To demonstrate the effectiveness of the CPSDH, one example of the images reconstructed from the holograms with large phase-shifting errors is revealed in Fig. 8. The amplitude images reconstructed by the CPSDH and the conventional method using the same phase-shifted holograms are represented in Figs. 8(a) and 8(c), respectively. The phase difference images corresponding with the areas indicated by the red dotted line in Figs. 8(a) and 8(c), are shown in Figs. 8(b) and 8(d). The phase difference in Fig. 8(b) is the same as that in Fig. 7(f), and the fringes are clear. Whereas, a lot of noise exists in the phase difference image calculated by the conventional method as presented in the Fig. 8(d). Obviously, the CPSDH provides high quality image reconstruction that is to say the measurement precision of the CPSDH is higher than the conventional method. The in-plane resolution of the conventional method is decreased owing to the noise, which will influence the partial analysis of the object.
The amount of the thermal deformation in out-of-plane direction can be calculated as followsFig. 8(b) is plotted in Fig. 8(e), and the unwrapped phase distribution is described in Fig. 8(f). The out-of-plane deformation between the point a (phase value: 0.5171) and the point a’ (phase value: −5.9413) is approximately 270 nm. The nanometer-order thermal deformation measurement of the surface shape of the electronic device is achieved by the developed system. Besides, the totally thermal expansion of the object is extremely fast, so that the deformation in a single point of the object as a function of the temperature should be measured by a high-speed camera.
A thermal deformation measurement system based on the calibrated phase-shifting digital holography was developed. Two ordinary CMOS cameras and one thermal camera were used in the system. In the experiment, a series of four phase-shifted holograms were continuously recorded when the temperature of the object changed from 22 to 45.3. The amplitude and phase images were successfully reconstructed by the phase-shifting error compensation algorithm. Then the nanometer-order thermal deformation measurement of an electronic device was achieved. To compare with the conventional method, the proposed system provides more stable performance, which will contribute to various industrial fields.
This study was partially supported by the funding provided by the Mitutoyo Association for Science and Technology.
References and links
1. S. Zhang, “Recent progresses on real-time 3D shape measurement using digital fringe projection techniques,” Opt. Lasers Eng. 48(2), 149–158 (2010). [CrossRef]
2. M. Chang, W. Tsai, J. Lin, and K. Jiang, “In-line monitoring of thermal deformation and surface topography of flip chip substrates,” Proc. SPIE 8321, 83211Q (2016). [CrossRef]
4. J. W. Goodman and R. W. Lawrence, “Digital image formation from electronically detected holograms,” Appl. Phys. Lett. 11(3), 77–79 (1967). [CrossRef]
5. Y. Awatsuji, M. Sasada, and T. Kubota, “Parallel quasi-phase-shifting digital holography,” Appl. Phys. Lett. 85(6), 1069–1071 (2004). [CrossRef]
6. W. Jeong, K. Son, and H. Yang, “Image reconstruction algorithm for speckle noise reduction of 2-step parallel phase-shift digital holography,” in Mathematics in Imaging 2017, OSA Technical Digest (Optical Society of America, 2017), paper MTu2C.3.
7. S. Jiao and W. Zou, “High-resolution parallel phase-shifting digital holography using a low-resolution phase-shifting array device based on image inpainting,” Opt. Lett. 42(3), 482–485 (2017). [CrossRef] [PubMed]
9. X. F. Meng, L. Z. Cai, X. F. Xu, X. L. Yang, X. X. Shen, G. Y. Dong, and Y. R. Wang, “Two-step phase-shifting interferometry and its application in image encryption,” Opt. Lett. 31(10), 1414–1416 (2006). [CrossRef] [PubMed]
10. W. Zhou, H. Zhang, Y. Yu, and T. C. Poon, “Experiments on a simple setup for two-step quadrature phase-shifting holography, IEEE Transactions on Industrial Informatics,” IEEE Trans. Ind. Electron. 12(4), 1564–1570 (2016).
12. X. He, C. V. Nguyen, M. Pratap, Y. Zheng, Y. Wang, D. R. Nisbet, R. J. Williams, M. Rug, A. G. Maier, and W. M. Lee, “Automated Fourier space region-recognition filtering for off-axis digital holographic microscopy,” Biomed. Opt. Express 7(8), 3111–3123 (2016). [CrossRef] [PubMed]
13. T. Thanyarat and B. Prathan, “Measuring a thermal expansion of thermoelectric materials by using in-line digital holography,” Proc. SPIE 10022, 100220C (2016).
14. Y. Morimoto, T. Nomura, M. Fujigaki, S. Yoneyama, and I. Takahashi, “Deformation measurement by phase-shifting digital holography,” Exp. Mech. 45(1), 65–70 (2005). [CrossRef]
21. S. Ri, T. Muramatsu, M. Saka, K. Nanbara, and D. Kobayashi, “Accuracy of the sampling moiré method and its application to deflection measurements of large-scale structures,” Exp. Mech. 52(4), 331–340 (2012). [CrossRef]
22. S. Ri, M. Fujigaki, and Y. Morimoto, “Sampling moiré method for accurate small deformation distribution measurement,” Exp. Mech. 50(4), 501–508 (2010). [CrossRef]
23. S. Ri and T. Muramatsu, “Theoretical error analysis of the sampling moiré method and phase compensation methodology for single-shot phase analysis,” Appl. Opt. 51(16), 3214–3223 (2012). [CrossRef] [PubMed]
24. J. A. N. Buytaert and J. J. J. Dirckx, “Study of the performance of 84 phase- shifting algorithms for interferometry,” J. Opt. 40(3), 114–131 (2011). [CrossRef]
25. Y. Morimoto, M. Toru, M. Fujigaki, and N. Kawagishi, “Subnanometer displacement measurement by averaging of phase difference in windowed digital holographic interferometry,” Opt. Eng. 46(2), 025603 (2007). [CrossRef]