Ptychography is a lensless phase imaging technique that obtains an image by scanning a specimen at several points with respect to a localized illumination beam. For larger specimens, it takes a longer time to complete scanning and hence higher stability is required in the setup which is often not guaranteed. An alternative technique is proposed here that reduces the sequential scanning time for such applications. A pinhole array is used to generate multiple tiny spatially separated beams to scan an object simultaneously at various points. The resulting diffraction patterns are recorded and processed in the Fresnel regime to obtain the images. Unlike other ptychographic methods using multiple beams, the proposed method does not require the use of a multimode ptychography algorithm or autocorrelation filtering of the diffraction patterns. The effectiveness of the method is studied through simulations and experiments. In contrast to conventional single-beam ptychography, the proposed method has the ability to achieve a larger field of view while leaving the number of scanned positions unchanged.
© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
Coherent diffraction imaging (CDI) [1–8] has become a well-established lensless imaging technology for reconstructing the amplitude and phase of a target specimen simultaneously using iterative algorithms. CDI was first proposed by Sayre  in the early 1950s, and experimentally demonstrated by Miao  using X-rays. Without using focusing optics, CDI is now capable of achieving a diffraction-limited spatial resolution, and has become a very useful tool in the imaging field, particularly using X-rays [2–4] and electrons [5–8]. However, CDI also has certain limitations: it requires an isolated specimen, which limits the field of view (FOV) and results in poor reconstructions in the presence of noise or the absence of a tight support. Various ptychography techniques [9–23] that have been introduced in recent years are able to overcome the drawbacks of CDI, and can achieve a large FOV by shifting a probe relative to the target specimen. Ptychography has thus attracted significant attention in recent years, and has been successfully realized using X-ray , optical , electron , extreme ultraviolet , and terahertz (THz)  waves. In ptychography, hundreds of diffraction patterns recorded from different areas of the specimen are utilized to reconstruct a high-resolution specimen. The redundancy in the ptychographic data set provides sufficient information, not only for specimen and probe retrieval [24–27] but also for translation error correction [28–30], mixed state decomposition [31,32] and spectral weight recovery  in wavelength multiplexing imaging; however, this comes at the cost of a substantially increased scanning time, particularly when investigating larger objects. Though it has been shown that a larger illuminating probe can be used in such ptychographic scanning process, it does not efficiently meet the requirement of larger imaging FOV as a sufficiently smaller scanning step size is required in such methods to guarantee convergence [33,34].
Single-shot ptychography [35,36] was recently proposed to reduce the acquisition time, in which a single diffraction pattern is used to recover a complex specimen. However, the reconstructed spatial resolution is reduced compared to the resolution of conventional ptychography  owing to interference occurring between the sub-diffraction patterns corresponding to different beams. Incoherent multiple-beam ptychography  uses two mutually incoherent and spatially separated beams to simultaneously scan a specimen and increase the scanned area, as well as a multimode algorithm [31,32] for image reconstruction. Coherent multiple-beam ptychography  uses mutually coherent illuminating beams to scan a specimen, and with this method, coherent illuminating beams are generated using multiple pinholes, which were originally utilized by Hessing et al.  to establish the ptychographic scanning positions. Coherent multiple-beam ptychography removes the interference in the recorded patterns using an autocorrelation filtering technique before adopting a multimode algorithm for image reconstruction. In , an experimental setup used to naturally remove the interference from the measured diffraction data by carefully choosing the experimental parameters is described. In their method, owing to the use of a Fourier transform lens after the sample, the diffraction from all probes is fully mixed together in the spatial domain at the detector plane, and a multimode algorithm is always necessary for image reconstruction. Because multimode algorithm requires high data redundancy to decompose all illumination modes and obtain a clear reconstructed image, both coherent and incoherent multiple-beam ptychographic methods need more scanning points than common ptychography for the same sample size.
In this paper, another multiple-beam ptychographic imaging method (mb-PIE) is demonstrated, where multiple coherent beams generated using a pinhole array are adopted to scan the sample to be observed. Accordingly, the speed of the data acquisition can be remarkably increased. Using priori knowledge of the distribution of the pinhole array forming the multiple-beam probe, the complex amplitude of the specimen can be directly reconstructed using a conventional ePIE  algorithm. The feasibility of this proposed method was verified through both numerical simulations and experiments.
Conventional ptychography uses a pinhole or slit to form coherent illumination on a specimen fixed on a 2D motorized translation stage, where the distance between the specimen and the pinhole varies from several centimeters to tens of centimeters depending on the wavelength used. A CCD camera in the Fresnel regime of the specimen records the diffracted patterns that are formed. The recording of the diffraction pattern is repeated every time the specimen is shifted a small distance relative to the pinhole until the required FOV is reached. Many algorithms such as ePIE , difference map [25,27], conjugate gradient , RAAR , and maximum likelihood  can be used to reconstruct the specimen from the data set recorded. All these algorithms are developed based on the optimization of inverse problems and mainly rely on two constraints: an overlap constraint and a modulus constraint. In our proposed method, an ePIE routine is adopted for the reconstruction because it is very memory-conserving and has been proven to be robust to noise and other disturbances . With Or and Pr representing the complex amplitude of the specimen and probe, respectively, the ePIE algorithm routine can be summarized as follows:
The exit wavefront immediately after the specimen is defined as
This formula works as overlap constraint, where r represents the spatial coordinate perpendicular to the propagation direction, and j indicates the scanning position. In the absence of noise, the wavefront at the detector plane is obtained byEquations (1)–(4) are iterated until the convergence condition is reached.
The difference between mb-PIE and single-beam ptychography is that a pinhole array is used to generate a multiple-beam probe. The setup of mb-PIE is shown in Fig. 1. The pinhole array is illuminated using a plane wave, and the target specimen is placed immediately behind the pinhole array (the pinhole array can also be projected onto the specimen through a 4f system). A CCD camera is located in the Fresnel regime to record the diffraction patterns.
As shown in Fig. 1, each recorded diffraction intensity includes multiple diffraction patterns formed by different illuminated specimen regions. Owing to the existence of interference between adjacent diffraction patterns, the recorded diffraction intensity cannot be divided into clearly separated diffraction patterns corresponding to individual probes, and therefore, we can only conduct a phychographic reconstruction using a single-beam ptychography algorithm by treating multiple beams as a large illuminating probe. In general, a large illuminating probe always meets two problems related to the resolving capability of the detector and the reconstruction accuracy (or convergence speed). To ensure that the detector can faithfully record the fine details of the diffraction patterns, the maximum illuminating size permitted is D = λz/2p [38,42], where λ is the wavelength of the light, z is the distance from the specimen plane to the detector plane, and p is the pixel pitch of the detector. (Edo et al. [33,34] proposed a sampling criterion that frees the ptychography from the oversampling criterion for traditional CDI, allowing a larger probe to be used to illuminate the sample during ptychography. However, ptychographic systems using this sampling criterion require an increase in the number of scanning points and an increased overlap of the adjacent scanning positions [38,43], which does not meet the requirement for the high-speed imaging of large samples.)
In the proposed mb-PIE method, interference can only take place between neighboring diffraction patterns if the detector is located in the Fresnel region with a proper distance from the sample. For a given pinhole size w, the distance between the detector and the sample z should be greater than 2pw/λ. The distance z cannot be too long because a long distance produces a large diffraction pattern, which will cause an overlap between non-neighboring diffraction patterns on the detector plane. When designing an illumination array, we first choose a small sample-detector distance z, and the size of the pinhole should thus be smaller than λz/2p. Before an actual experiment, we use each generated probe by keeping the individual pinholes open to illuminate the sample separately, observe the size and location of the diffraction pattern on the detector, and adjust the interval between pinholes to avoid overlap between non-neighboring diffraction patterns on the detector. The gap between adjacent pinholes should be less than λz/2p to meet the sampling requirement. In single-beam ptychography, the entire detector chip is used to record the diffraction pattern. In multiple-beam illumination system, multiple diffraction patterns are formed on the detector simultaneously. If each sub-diffraction pattern is divided and used independently, the numerical aperture (NA) of the system will decrease compared to that of conventional single-beam ptychography. In the proposed mb-PIE method, with proper pinhole array and sample-detector distance, the multiple-beam probe is treated as a single probe that meets the sampling requirement, despite that the extend size of the multiple-beam illumination exceeds the maximum illumination size required for sampling criterion. In this case, the multiple diffraction patterns are not divided, and therefore the NA of the mb-PIE method is not reduced compared to that of conventional single-beam ptychography. Thus, for the proposed method, with a proper pinhole size and interval, the largest illumination array permitted is in theory only limited by the size of the detector chip, and thus it is easy to realize a large FOV and high-speed ptychographic imaging.
3. Simulation results
In this section, we demonstrate the mb-PIE numerically. As shown in Fig. 2, a “fruit” image [1,024 × 1,024 pixels, shown Fig. 2(a)] with values in the range of [0–1] is used to simulate the amplitude of the specimen, and a “pepper” image [1,024 × 1,024 pixels, shown in Fig. 2(b)] with values in the range of [0–π] is used to simulate the phase of the specimen. A 5 × 5 pinhole array [Fig. 3(a)] serves as the multiple-beam probe to illuminate a complex specimen. Each pinhole is 1 mm in diameter, and the distance between the centers of two adjacent pinholes is set to 2 mm. The wavelength of the plane wave is 632.8 nm. The diffraction patterns are calculated by Fresnel propagating a distance of 5 cm to the detector plane and adding white Gaussian noise with a signal-to-noise ratio (SNR) of 35 dB. The pixel pitch of the detector is set to 10 μm, resulting in a maximum probe size of 1.58 mm in conventional single-beam ptychography according to the sampling criteria , which is much smaller than the size of a multiple-beam probe (9 mm in width, as shown in Fig. 3(a)).
The specimen translates to 9 × 9 positions with a step size of 0.25 mm, providing an approximate overlap of 75% between adjacent scanning positions for each individual probe. Figure 3(b) shows one of the diffraction patterns at the log scale, and the inset in Fig. 3(b) shows that some interference fringes exist between adjacent diffraction blocks. Unlike the single-shot method [35,36], the mb-PIE method does not divide the diffraction pattern into separate diffraction patterns, and the ePIE algorithm is used to convert the diffraction pattern into a complex image of the specimen. The ePIE algorithm starts with a randomly guessed object function. The probe function is obtained from the distribution of the pinhole array. Figures 3(c) and 3(d) show the reconstructed amplitude and phase of the specimen after 200 iterations of the ePIE algorithm. To compare the results using a multiple-beam probe with the results using a large probe (as large as the multiple-beam probe), we generate a probe with a width of 9 mm to illuminate the specimen, and shift the specimen to the same positions as in mb-PIE (9 × 9 positions with a step size of 0.25 mm). Figure 3(e) shows the large probe, and Fig. 3(f) shows one of the diffraction patterns. The reconstructed amplitude and phase of the specimen after 200 iterations of the ePIE algorithm are shown in Figs. 3(g) and 3(h), respectively. Compared to the image quality in Fig. 3(c), we can see that the image quality in Fig. 3(g) is degraded.
In addition, fPIE  and sPIE  are also used to process the diffraction patterns generated by the large probe. By up-sampling the diffraction pattern by a factor of 4 × 4 with these methods, the reconstructions using fPIE and sPIE after 200 iterations are shown in Figs. 4(a)–4(b), and Figs. 4(c)–4(d), respectively.
We also compared the results of the mb-PIE method with those of conventional ptychography using a small probe (a single pinhole with the same diameter). The width of the small probe beam used is 1 mm, which meets the sampling requirement (the maximum allowed width is 1.58 mm). The specimen translates to the same positions as in mb-PIE (9 × 9 positions with a step size of 0.25 mm). Ptychographic results using mb-PIE and conventional ptychography are compared, and the reconstructed complex specimens after 200 iterations of the ePIE algorithm are shown in Fig. 5.
The root mean squared error metric is used to evaluate the convergence when applying these two methods:Fig. 6 indicate the reconstruction error for mb-PIE, whereas the blue squares represent the results of conventional ptychography. The errors of the two different methods converge to the same value. We can conclude from Figs. 5 and 6 that the mb-PIE method can obtain a similar reconstruction when compared with conventional ptychography.
4. Experimental results
We built a ptychography microscope operating with a 3 × 3 beam array to demonstrate the effectiveness of the mb-PIE method. A He-Ne laser beam (λ = 632.8 nm) is passed through a spatial filter with a 40 × objective lens, a 15 μm pinhole, and a focusing lens of f = 15 cm, forming a plane wave illuminating the pinhole array (3 × 3 pinhole array, where each pinhole is 0.75 mm in diameter and the distance between the centers of two adjacent pinholes is 1.5 mm). The pinhole array is imaged on a target specimen through a 4f system. Each lens of the 4f system has a focal distance of 10 cm. A CCD camera (AVT PIKE F-421B, 7.4 μm, 2,048 × 2,048 pixels) is placed 4 cm behind the specimen to record the diffraction patterns. According to the sampling criteria, the maximum probe size allowed is 1.7 mm in single-beam ptychography, which is smaller than the size of the multiple-beam probe (3.7 mm width) used.
The specimen is shifted to 10 × 10 positions with a step size of 0.15 mm, and a ~20.3 mm2 FOV is reached. The overlap ratio between adjacent scan positions is about 80% for each individual probe. The ePIE algorithm is applied to the specimen reconstruction. The algorithm starts with a randomly guessed object function and an initial probe function obtained from the distribution of the pinhole array. Figures 7(a1) and 7(b) show the reconstructed amplitude and phase of the biological specimen (crosscut of a pumpkin stem) using the mb-PIE method after 200 iterations of the ePIE algorithm, respectively. Figure 7(a2) shows a magnified image of the yellow box in Fig. 7(a1). The multiple-beam probe and one of the recorded diffraction patterns (at the log scale) used with the mb-PIE method are shown in Figs. 7(c) and 7(d), respectively. To compare the results with single-beam ptychography, we conducted an experiment using only a single-beam generated by keeping only the central pinhole in the pinhole array open to illuminate the specimen. In the single-beam experiment, we moved the specimen to the same positions as in the mb-PIE experiment. The single-beam experiment resulted in an FOV of ~2.25 mm2, which is about one-ninth that obtained by the mb-PIE method. This means that the acquisition speed of mb-PIE is about 9 times that of conventional ptychography. The reconstructed complex distribution of the specimen using single-beam ptychography after 200 iterations of the ePIE algorithm is shown in Figs. 7(e1) and 7(f). Figure 7(e2) shows a magnified image of the yellow box in Fig. 7(e1). The probe and one of the recorded diffraction patterns of the single-beam ptychography are shown in Figs. 7(g) and 7(h), respectively. Comparing Fig. 7(a2) with Fig. 7(e2), we can see that the resolution is basically the same.
Figures 8(a) and 8(b) show the reconstruction results from the cropped central blocks [220 pixels, shown in the dotted red box in Fig. 7(d)] of the multiple-beam diffraction pattern at each scanning position. Figure 8(c) shows a magnified image of the red box in Fig. 8(a), and Fig. 8(d) is a magnified image obtained from the mb-PIE (without separating the diffraction pattern into blocks) reconstructions. Compared with the image in Fig. 8(c), the resolution of the image in Fig. 8(d) is increased, and more details can be observed. This can be explained by the fact that the interference signals in the cropped central blocks are treated as noise  during the recovery process.
Experiments using an USAF 1951 resolution chart were conducted to demonstrate that the mb-PIE method obtains images without a loss in resolution when compared with the images obtained with conventional ptychography. The resolution chart was shifted to 10 × 10 positions in both the mb-PIE and single-beam ptyhcography experimental setups. The central part of the reconstructions is shown in Figs. 9(b) and 9(f) for the mb-PIE method and conventional ptychography, respectively. Figures 9(c) and 9(g) show magnified images of the red square region in Figs. 9(b) and 9(f), respectively. From the line plot images (Figs. 9(d) and 9(h)), the mb-PIE method reaches the same resolution (Element 3 in Group 6) as that with single-beam ptychography.
In this paper, we both numerically and experimentally demonstrated a pinhole-array based multiple-beam ptychography method that does not require division or autocorrelation filtering of the diffraction patterns, or use of a multimode ptychography algorithm. The method achieves a higher imaging speed than conventional ptychography, and the FOV can be further magnified if a pinhole array with more pinholes is employed. The number of pinholes is only limited by the size of the detector chip. The setup of the mb-PIE method is very compact, and does not require separated beams incoherent to each other. The only modification required is to replacing the pinhole in conventional ptychography with a pinhole array.
In mb-PIE, if the size of the individual pinhole is large, the detector should be placed farther away to meet the sampling requirement. In this case, the non-neighboring diffraction patterns might overlap with each other, and the resolution of the mb-PIE method will decrease. Thus, in the proposed mb-PIE method, the better choice is a small pinhole size, multiple pinholes, and a small recording distance.
The reported method may find applications in the fields of microscopy, label-free phase imaging for biomedical specimens, or other areas that require rapid imaging.
National Natural Science Foundation of China (NSFC) (61675215); Shanghai Sailing Program (18YF1426600).
1. D. Sayre, “Some implications of a theorem due to Shannon,” Acta Crystallogr. 5(6), 843 (1952). [CrossRef]
2. J. Miao, P. Charalambous, J. Kirz, and D. Sayre, “Extending the methodology of X-ray crystallography to allow imaging of micrometre-sized non-crystalline specimens,” Nature 400(6742), 342–344 (1999). [CrossRef]
3. W. Hua, G. Zhou, Y. Wang, P. Zhou, S. Yang, C. Peng, F. Bian, X. Li, and J. Wang, “Measurement of the spatial coherence of hard synchrotron radiation using a pencil beam,” Chin. Opt. Lett. 15(3), 033401 (2017). [CrossRef]
4. J. M. Zuo, I. Vartanyants, M. Gao, R. Zhang, and L. A. Nagahara, “Atomic Resolution Imaging of a Carbon Nanotube from Diffraction Intensities,” Science 300(5624), 1419–1421 (2003). [CrossRef] [PubMed]
5. J. Miao, T. Ohsuna, O. Terasaki, K. O. Hodgson, and M. A. O’Keefe, “Atomic resolution three-dimensional electron diffraction microscopy,” Phys. Rev. Lett. 89(15), 155502 (2002). [CrossRef] [PubMed]
6. K. S. Raines, S. Salha, R. L. Sandberg, H. Jiang, J. A. Rodríguez, B. P. Fahimian, H. C. Kapteyn, J. Du, and J. Miao, “Three-dimensional structure determination from a single view,” Nature 463(7278), 214–217 (2010). [CrossRef] [PubMed]
7. R. Dronyak, K. S. Liang, Y. P. Stetsko, T. K. Lee, C. Feng, J. Tsai, and F. Chen, “Electron diffractive imaging of nano-objects using a guided method with a dynamic support,” Appl. Phys. Lett. 95(11), 111908 (2009). [CrossRef]
8. O. Kamimura, K. Kawahara, T. Doi, T. Dobashi, T. Abe, and K. Gohara, “Diffraction microscopy using 20 kV electron beam for multiwall carbon nanotubes,” Appl. Phys. Lett. 92(2), 024106 (2008). [CrossRef]
9. J. M. Rodenburg and H. M. Faulkner, “A phase retrieval algorithm for shifting illumination,” Appl. Phys. Lett. 85(20), 4795–4797 (2004). [CrossRef]
11. J. M. Rodenburg, A. C. Hurst, A. G. Cullis, B. R. Dobson, F. Pfeiffer, O. Bunk, C. David, K. Jefimovs, and I. Johnson, “Hard-x-ray lensless imaging of extended objects,” Phys. Rev. Lett. 98(3), 034801 (2007). [CrossRef] [PubMed]
14. X. He, C. Liu, and J. Zhu, “On-line beam diagnostics based on single-shot beam splitting phase retrieval,” Chin. Opt. Lett. 16(9), 091001 (2018). [CrossRef]
15. A. Sun, X. He, Y. Kong, H. Cui, X. Song, L. Xue, S. Wang, and C. Liu, “Ultra-high speed digital micro-mirror device based ptychographic iterative engine method,” Biomed. Opt. Express 8(7), 3155–3162 (2017). [CrossRef] [PubMed]
16. M. J. Humphry, B. Kraus, A. C. Hurst, A. M. Maiden, and J. M. Rodenburg, “Ptychographic electron microscopy using high-angle dark-field scattering for sub-nanometre resolution imaging,” Nat. Commun. 3(1), 730–737 (2012). [CrossRef] [PubMed]
17. J. Zhang, T. Xu, X. Wang, S. Chen, and G. Ni, “Fast gradational reconstruction for Fourier ptychographic microscopy,” Chin. Opt. Lett. 15(11), 111702 (2017). [CrossRef]
18. W. Yu, S. Wang, S. P. Veetil, S. Gao, C. Liu, and J. Zhu, “High-quality image reconstruction method for ptychography with partially coherent illumination,” Phys. Rev. B 93(24), 241105 (2016). [CrossRef]
19. P. Hessing, B. Pfau, E. Guehrs, M. Schneider, L. Shemilt, J. Geilhufe, and S. Eisebitt, “Holography-guided ptychography with soft X-rays,” Opt. Express 24(2), 1840–1851 (2016). [CrossRef] [PubMed]
21. M. D. Seaberg, B. Zhang, D. F. Gardner, E. R. Shanblatt, M. M. Murnane, H. C. Kapteyn, and D. E. Adams, “Tabletop nanometer extreme ultraviolet imaging in an extended reflection mode using coherent Fresnel ptychography,” Optica 1(1), 39–44 (2014). [CrossRef]
23. L. Valzania, E. Hack, P. Zolliker, R. Brönnimann, and T. Feurer, “Resolution limits of terahertz ptychography,” Proc. SPIE 10677, 1067720 (2018).
28. A. M. Maiden, M. J. Humphry, M. C. Sarahan, B. Kraus, and J. M. Rodenburg, “An annealing algorithm to correct positioning errors in ptychography,” Ultramicroscopy 120(5), 64–72 (2012). [CrossRef] [PubMed]
29. F. Zhang, I. Peterson, J. Vila-Comamala, A. Diaz, F. Berenguer, R. Bean, B. Chen, A. Menzel, I. K. Robinson, and J. M. Rodenburg, “Translation position determination in ptychographic coherent diffraction imaging,” Opt. Express 21(11), 13592–13606 (2013). [CrossRef] [PubMed]
30. A. Tripathi, I. McNulty, and O. G. Shpyrko, “Ptychographic overlap constraint errors and the limits of their numerical recovery using conjugate gradient descent methods,” Opt. Express 22(2), 1452–1466 (2014). [CrossRef] [PubMed]
33. T. B. Edo, D. J. Batey, A. Maiden, C. D. Rau, U. Wagner, Z. Pesic, T. A. Waigh, and J. M. Rodenburg, “Sampling in x-ray ptychography,” Phys. Rev. A 87(5), 053850 (2013). [CrossRef]
34. D. J. Batey, T. B. Edo, C. D. Rau, U. Wagner, Z. Pesic, T. A. Waigh, and J. M. Rodenburg, “Reciprocal-space up-sampling from real-space oversampling in x-ray ptychography,” Phys. Rev. A 89(4), 043812 (2014). [CrossRef]
35. X. Pan, C. Liu, and J. Zhu, “Single shot ptychographical iterative engine based on multi-beam illumination,” Appl. Phys. Lett. 103(17), 171105 (2013). [CrossRef]
36. P. Sidorenko and O. Cohen, “Single-shot ptychography,” Optica 3(1), 9–14 (2016). [CrossRef]
37. R. Karl, C. Bevis, R. Lopez-Rios, J. Reichanadter, D. Gardner, C. Porter, E. Shanblatt, M. Tanksalvala, G. F. Mancini, M. Murnane, H. Kapteyn, and D. Adams, “Spatial, spectral, and polarization multiplexed ptychography,” Opt. Express 23(23), 30250–30258 (2015). [CrossRef] [PubMed]
38. C. Bevis, R. Karl Jr, J. Reichanadter, D. F. Gardner, C. Porter, E. Shanblatt, M. Tanksalvala, G. F. Mancini, H. Kapteyn, M. Murnane, and D. Adams, “Multiple beam ptychography for large field-of-view, high throughput, quantitative phase contrast imaging,” Ultramicroscopy 184(Pt A), 164–171 (2018). [CrossRef] [PubMed]
39. D. R. Luke, “Relaxed averaged alternating reflections for diffraction imaging,” Inverse Probl. 21(1), 37–50 (2005). [CrossRef]
40. P. Thibault and M. Guizar-Sicairos, “Maximum-likelihood refinement for coherent diffractive imaging,” New J. Phys. 14(6), 063004 (2012). [CrossRef]