1. Introduction

Over the past a few decades, many imaging techniques have been developed to surpass Rayleigh’s criterion [1] that imposes a resolution limit on optical, X-ray, and electron microscopy. Ptychography is one of those to exploit coherence to decode sample information at a spatial resolution beyond the limitation of the focusing optics [2–4]. Though it was first proposed in electron microscopy [5], ptychography has been actively developed at synchrotron radiation facilities in the past decade for various applications: with the reconstructed complex-valued sample function, it has been used for high-resolution 2D/3D structural imaging [6,7], quantitative chemical analysis [8–10], materials strain and magnetization studies [11,12], sample dynamics visualization [13,14]; with the retrieved illumination function, it can be applied for wave-field and nano-focusing optics characterization [15,16], image deconvolution [17,18].

The final achieved spatial resolution in ptychography is theoretically limited by the cumulative dose on the sample [19,20] provided that no obvious radiation damage occurs. For a source with a given coherent flux, increasing the exposure time in the data acquisition is a typical way to impart more dose on the sample to get high statistics on a series of recorded diffraction patterns. As a coherent diffraction imaging method, ptychography also relies on a high degree of coherence in acquired diffraction patterns to reconstruct high-resolution sample features. However, various sources of spatial and temporal variations, especially coming from the illumination and the sample, inevitably exist in the real experiments. Therefore, longer exposure time would result in unwanted decoherence effects on the acquired diffraction patterns by integrating these variations, thus bringing difficulties in high-resolution image reconstruction. Recently developed advanced mixed-state reconstruction approaches [21] are able to reconstruct stationary mixed states of any origin from the illumination source, sample and detector, thus allowing one to deal with some variation sources that contribute to individual diffraction patterns equally (for example, sample or illumination motion with a constant linear speed [22–24] or a periodic vibration with a frequency higher than the acquisition frequency [25]). However, mixed-state approaches still have limitations dealing with variations which affect individual diffraction patterns nonuniformly and thus can’t be modeled by stationary mixed states, such as a gradually or non-periodically varying illumination or sample at different scan positions. Such slow-varying effects can be taken into account when using orthogonal probe relaxation (OPR) [26].

In this work, we first demonstrate high-speed ptychography with a fast detector frame rate to reduce the impact of unwanted variations on diffraction patterns. The results provide evidence that superior image quality can be obtained with a low-dose scan than with a high-dose scan with 100 times longer exposure. Thanks to less decoherence and sufficient constraints provided by diffraction patterns acquired in short exposure time, a varying illumination can even be recovered when a physical model used in the reconstruction can account for such variations, such as the above-mentioned OPR method to account for probe variation during the scan.

While fast data acquisition can reduce the effects from various experimental imperfections, with dose-limited resolution one would still want to collect enough cumulative photons per area in order to have sufficient photon statistics to resolve features at a given resolution. Here we further demonstrate continuous multi-pass scan via low-dose high-speed ptychography to gradually accumulate the required dose on the sample to achieve high-resolution imaging. Measuring a series of low-dose images was proposed in cryo electron microscopy to get superior image quality using the image correlation method [26]. Data collection protocols have also been optimized in X-ray crystallography by taking low-dose diffraction patterns using fast-framing detectors until radiation effects begin to alter the specimen [27,28]. Such a low-dose data acquisition method can be extended to tomography to collect projected views when one considers dose fractionation, as described in [29,30]: “A three-dimensional reconstruction requires the same integral dose as a conventional two-dimensional micrograph provided that the level of significance and the resolution are identical. The necessary dose $D$ for one of the $K$ projections in a reconstruction series is, therefore, the integral dose divided by $K$.” This argument applied to multi-pass scanning suggests that the necessary dose at each pass is the required integral dose divided by the number of passes. As a result, this multi-pass scan strategy may generate many low-signal diffraction patterns, which, however, can reliably be reconstructed in ptychography provided that proper physical and noise models are used in the reconstruction [31]. As many synchrotron facilities around the world, including the Advanced Photon Source (APS), are undergoing major upgrades to provide higher X-ray flux by two or three orders of magnitude, radiation damage may become a main issue for a variety of specimens to do high-resolution imaging. In this case, low-dose strategies [32], such as those demonstrated in this work, need to be well studied and employed to overcome radiation damage effects. The fast scan strategy with continuous multi-pass scan shown here also has potential for time-resolved ptychography with a high X-ray flux provided by upgraded sources.

2. Experimental results

2.1 High-speed ptychography with experimental variations

Ptychographic scans with different data acquisition speeds were performed at the Velociprobe beamline [33] at the APS. A monochromatic X-ray beam at 8.8 keV was focused by a Fresnel zone plate with 50 nm outermost zone width and 180 $\mu$m diameter. A gold Siemens Star test pattern with 100 nm finest spokes was placed about 600 $\mu$m downstream of the focus position. A pixelated area detector (Dectris Eiger X 500K) with 75 $\mu$m-size pixels was located 1.92 m downstream from the sample to collect diffraction patterns. Two types of ptychographic datasets were collected at a detector frame rate of 2000 Hz and 20 Hz, respectively, just after we moved a double crystal monochromator to change the X-ray energy. Both scans used the fly-scan scheme following a “snake” trajectory [33] with a 100 nm average step size in the horizontal direction and 500 nm in the vertical direction, the relative scan positions between the sample and the zone plate were accurately recorded by laser interferometers (Attocube IDS3010). Attempts to reconstruct the object images on these scans were carried out by using a mixed-state approach [21] with 10 probe modes used to solve the speckle-contrast reduction caused by the continuous scan (100 nm effective step size per exposure) and partially coherent illumination. Reconstructed phase images are shown in Figs. 1(a) and 1(b). Both images contain reconstruction artifacts which are expected due to the illumination variation caused by the unstable monochromator. The spatial resolution given by Fourier ring correlation (FRC) with 1/2 bit threshold [34] in Fig. 2(a) shows that the reconstruction quality for 20 Hz scans is worse than 2000 Hz scans even though it has 100 times more X-ray dose on the sample. To account for the illumination change in the scan, three additional OPR modes were added in ptychography reconstruction together with the 10 mixed-state modes used above. Figures 1(c) and 1(d) show reconstructed phase images for 2000 Hz and 20 Hz scans, respectively, with corresponding reconstructed OPR modes shown in Figs. 1(e) and 1(f). The reconstruction quality is greatly improved after using OPR modes, especially for scans acquired at 2000 Hz with the spatial resolution increasing from 45.3 nm to 16.3 nm as shown in Fig. 2(a). The reconstruction quality improvement using OPR for 20 Hz scans is not as good as that of 2000 Hz as shown by Fig. 1(d) and the FRC result in Fig. 2(a).

 

Fig. 1. The comparison of two ptychographic reconstructions of a Au test pattern acquired at different exposure times with a varying probe. Phase images reconstructed with 10 mixed-state probe modes on scans acquired at a frame rate of 2000 Hz and 20 Hz are shown in (a) and (b), respectively. Additional three orthogonal relaxation modes were added into the first primary mixed-state probe mode to account for the probe variation during the scan. (c) and (d) are the reconstructed phase images using combined 10 mixed-state modes and 3 OPR modes for scans acquired at 2000 Hz and 20 Hz, respectively. Four reconstructed principal probe components used to account for probe evolution via OPR [26] are shown for scans acquired at 2000 Hz (e) and 20 Hz (f). Visualization 1 demonstrates the time evolution of reconstructed probes in (e).

Download Full Size | PDF

 

Fig. 2. (a) Reconstruction quality comparison for 2000 Hz and 20 Hz scans with mixed-state approach (MS) and combined OPR plus mixed-state approach (OPR+MS). 1/2 bit threshold is used to define the FRC resolution. (b) Fourier transform of the time evolution of the first three dominating OPR modes as shown in Fig. 1(e), with the inset showing the 0 to 400 Hz on a logarithmic scale.

Download Full Size | PDF

The above comparison shows the advantages of fast data acquisition in ptychography: with a short exposure time (0.4 ms for scan acquired at 2000 Hz), the diffraction pattern suffers little from both spatial and temporal variations in the experiment and still preserves a high degree of coherence; the high continuous frame rate results in a almost constant or slightly different illumination for every diffraction pattern, depending on the time scale of the variation. Therefore, the dynamic variation of the illumination can be reconstructed by OPR approach that can account for slow-varying effects [26]. The time evolution of the varying probe reconstructed from the 2000 Hz dataset is shown in Visualization 1. Figure 2(b) shows the Fourier transform of the time evolution of the first three dominating OPR modes. The frequency of oscillations dominates at < 400 Hz range, which suggests that a detector frame of at least 400 Hz should be used to detect the vibrations. For the scans acquired at 20 Hz, effects of higher frequency vibration are averaged into acquired diffraction patterns. Thus, the varying illumination in the 20 to 400 Hz range is not captured in the data acquisition and cannot be accounted for in the 20 Hz scans with the OPR approach. The capability of reconstructing the probe wavefront variation also shows the promising application of high-speed ptychography for dynamic wavefront characterization which can be applied not only to a synchrotron radiation source but also to other illumination sources, such as X-ray free-electron lasers (XFELs) [35,36] and electrons.

2.2 High-resolution ptychography achieved by multi-pass fast scan

As the achieved spatial resolution in ptychography is dose-limited, a certain amount of photons need to be deposited on the sample to meet the dose requirement for a specific resolution. Here we demonstrate two scan schemes via high-speed ptychography to accumulate the required dose by continuous multiple passes for high-resolution imaging.

2.2.1 Continuous repeating scan

The continuous repeating scan was performed at the Bionanoprobe [37] where the original scan controller was replaced by a high-bandwidth FPGA-based softGlueZynq [38] controller. A Ni test pattern with 24 nm finest spokes was scanned back and forth by continuously driving the sample piezo X, Y stages with triangle voltage signals from a waveform generator (Agilent 33600A). The waveform amplitude for both X and Y stages was set to cover a 4 $\mu$m travel range, and a frequency of 5 Hz and 0.125 Hz was set, resulting in a motion speed of 40 $\mu$m/s and 1 $\mu$m/s for X and Y stages, respectively. A Fresnel zone plate with 70 nm outermost width was used to focus an X-ray beam at 10.7 keV with an illumination spot size on the sample of about 300 nm. The relative positions between sample and the zone plate in a two-dimension (2D) scan were measured by four laser interferometers which were mounted on a common reference base. A Dectris Pilatus 300K detector located 2.4 meters downstream of the sample was running in internal continuous mode at 400 frames/s (1.5 ms exposure time and 1 ms data readout time for each frame). The four interferometer positions were recorded at 1000 Hz, and triggers from the detector were recorded along with interferometer positions so that focal-spot locations could later be associated with individual diffraction patterns. The data acquisition here employed time-based triggers from the detector, which has advantages over pixel-based triggering in the high-speed scan [39] because the controller doesn’t need to frequently check whether the stages arrive at the requested position prior to triggering the detector. In addition, interferometer positions can be logged with a frequency higher than the detector frame rate, therefore, the sample locations during an exposure can be accurately measured.

Figure 3 shows the scan trajectories and ptychographic images of the test pattern reconstructed at different time stamps. The scan trajectories represent the relative motion between sample and zone plate as measured by laser interferometers. The stages had slight fluctuations during the motion, which, however, were well measured by the laser interferometers. As shown in Fig. 3(a), a single pass covering the whole imaging area took about 4 seconds, the corresponding phase image on the top shows spokes with even 24 nm smallest features visible. However, the reconstructed image is a little noisy with some obvious artifacts due to the insufficient photon statistics. As the scan continued, the reconstruction quality gradually improved as more X-ray photons were deposited on the sample. FRC analysis was conducted between the first $n$ passes and the following $n$ passes. As a total of 30 passes were acquired, the values of $n$ are $1,2,\ldots, 15$. Figure 4 shows the FRC result as a function of the number of passes (or the imaging dose). The achieved spatial resolution increased as more X-ray dose deposited on the sample, with $\sim$10 nm resolution for scans with 15 passes which is corresponding to about $3.8\times 10^4$ photons per reconstruction pixel ($6.3\times 6.3$ nm$^2$). A fit of resolution versus log scale of the imaging dose gives a slope of 1/3.2. Based on the Rose criterion with a signal-to-noise ratio (SNR) of 5, the required imaging dose has an inverse-fourth-power scaling with the resolution [40,41]. The slope value obtained here is slightly away from inverse-fourth-power scaling, but falls within a reasonable range between 1/4 to 1/3 as discussed in Ref. [40,42].

 

Fig. 3. A continuous repeating-scan ptychography enabled by a high-bandwidth FPGA-based softGlueZynq controller. Sample X, Y piezo stages were driven back and forth by triangle waveforms generated from a waveform generator at a frequency of 5 Hz and 0.125 Hz, respectively, covering an area of 4 $\mu$m $\times$ 4 $\mu$m. Four laser interferometers for sample X, Y stages and zone plate X, Y stages were recorded by the softGlueZynq controller with trigger signals sent from the operating Pilatus detector. The relative positions between sample and optics can be obtained from such recorded position information. (a) Ptychography scan of the first pass covering the image area, which took about 4 seconds. The top shows half of the ptychographic image of the Ni test pattern, while the bottom shows the scan trajectory for the bottom-half scan. Other figures here follow the same display method. As the scan continued, the phase image and scan trajectory were updated at 8 seconds (b), 20 seconds (c), 40 seconds (d), 80 seconds (e), and 120 seconds (f).

Download Full Size | PDF

 

Fig. 4. Reconstruction quality as a function of the number of passes used in the reconstruction. 1/2 bit threshold is used to define the achieved spatial resolution.

Download Full Size | PDF

2.2.2 High-speed sparse sampling with gradual increase in fill-factor

One of requirements in the ptychographic scan is that adjacent illuminated spots must overlap with each other in order to provide sufficient constraints for the reconstruction. The influence of the overlap parameter on ptychography reconstruction has been well studied [43]. In general, ptychography benefits from large overlaps which can provide more information by dense sampling. In practice, excessively dense sampling should also be avoided in conventional ptychographic scans as it may result in a long experiment time with the need to acquire many scan points, especially for the step-scan scheme where the stage overhead contributes a significant percentage of the experimental time [23]. Here we demonstrated a “smarter” continuous scan scheme, similar to the Lissajous-scan [44,45], by acquiring the data points from sparse sampling to dense sampling gradually until a sufficient overlap is reached for high-quality ptychography.

The experiment was carried out on the Velociprobe with the same experimental setup as described in Sec. 2.1 to image a CMOS integrated circuit (IC) on a $\sim$ 240 $\mu$m Si substrate [46]. The scan adopted a snake pattern as shown in Fig. 5(a), with 10 $\mu$m linear travel distance for the horizontal fast axis and 3 $\mu$m-diameter semi-circle curve to turn the scan direction of the fast axis at the end of each scan line. The velocity of stage motion was set to about 100 $\mu$m/s, and the Eiger X 500K detector was operated in internal continuous mode at 2000 frames/s, yielding an average step size of 50 nm in the fly-scan direction and 3 $\mu$m step size in the vertical direction. A single pass of this snake scan covered an area of 13 $\mu$m $\times$ 12 $\mu$m, which took about 0.75 seconds. As the first-pass scan that moved the sample from the top to the bottom completed, the scan controller shifted the vertical center of the scan region down by half the vertical step size (1.5 $\mu$m), and scanned the sample from the bottom to the top (see Fig. 5(a)). All of the following passes used the same approach by shifting the vertical center to fill the gap between the horizontal lines gradually, so that the overlapping ratio in the vertical direction increased. A total of 50 pass scans shown in Fig. 5(c) were taken within about 38 seconds to acquire $\sim$75400 diffraction patterns.

 

Fig. 5. High-speed continuous ptychographic scan with gradually increasing fill-factor of scan points on an integrated circuit sample. The scan trajectory that covered the scan area by the first 2 passes (a), 25 passes (b), and 50 passes (c). (d) Reconstructed phase images of the sub-region marked in (c) at different time stamps when the beam scanned across the sample at 1st, 3rd, 5th, 10th, 25th, 50th pass. (e) Phase image of a conventional single-pass scan with longer exposure time, but with the same accumulated X-ray dose.

Download Full Size | PDF

An example diffraction pattern acquired from the IC at 2000 Hz frame rate is shown in Fig. 6(a), which has low scattering signals outside the primary beam (donut-shape area). For such a dataset with low-signal diffraction pattern, a maximum-likelihood (ML) algorithm with Poisson noise model [47] was used for ptychography reconstruction. Due to the memory limit of the Nvidia RTX 2080 Ti GPU card (11 GB), only a portion of diffraction patterns acquired from a 4 $\mu$m $\times$ 15 $\mu$m region (marked in a cyan rectangular box in Fig. 5(c), containing about 400 diffraction patterns per pass) were selected for the reconstruction. In the reconstruction, diffraction patterns with 256 $\times$ 256 pixels were cropped, yielding a real-space pixel size of 14.1 nm. Figure 5(d) shows the reconstructed phase images for different numbers of scan passes. Because the vertical step size in a single-pass scan was larger than the probe size ($\sim$ 1.5 $\mu$m), there was not overlap between scan lines in the vertical direction so that the areas that were not covered by the illumination cannot be reconstructed; while a fly-scan step size of 50 nm along the horizontal direction provided sufficient overlap, enabling successful reconstruction of those scan lines as shown in the first column in Fig. 5(d). The quality of those reconstructed scan lines was limited by the low X-ray dose. As the scan continued with more passes across the sample, the overlap condition, especially in the vertical direction, was gradually improved, and the gap areas were recovered with more coming data points. The scan with 10 passes already provided a good coverage on the sample, which is almost equivalent to a conventional raster scan with 50 nm horizontal step size and 375 nm vertical step size; with a good overlapping condition, all pixels in this region were all well reconstructed at this point. The spatial resolution was continuously improved with more scan passes. The reconstructed quality as a function of the number of passes was evaluated by FRC in the same way used above, which is plotted in Fig. 7. The plot shows that the trend of improving reconstruction quality as the number of scan passes increased. The reconstruction from scans with 25 passes achieved a spatial resolution of 23.5 nm. The linear fitting of resolution versus log-scaled imaging dose was performed by excluding the first two data points due to the low reconstruction quality on unscanned regions, showing a slope of 1/4.3.

 

Fig. 6. View of the ptychographic data produced by high-speed multi-pass scan and single-pass scan at the same dose level. (a) An example of a single frame acquired at 2000 Hz detector frame rate. (b) The sum of all diffraction patterns that acquired in the 50-pass scan. (c) An example of a single frame acquired at 100 Hz detector frame rate at the same sample position as (a). (d) The sum of all diffraction patterns that acquired in the single-pass scan. (e) Power spectral density calculated from (b) and (d).

Download Full Size | PDF

 

Fig. 7. Reconstruction quality as a function of the number of passes used in the reconstruction. 1/2 bit threshold is used to define the achieved spatial resolution. The first two data points (1 and 2 passes) are excluded in the plot fitting.

Download Full Size | PDF

The final reconstructed image of the total 50 passes is shown in the last panel in Fig. 5(d). The extension of the curve fit to the data in Fig. 7 gives a spatial resolution of about 20 nm for the scan with 50 passes. Even though data acquired in such a short exposure time has low scattering signals on individual diffraction patterns (see Fig. 6(a)), sufficient photon statistics can still be obtained by accumulating X-ray dose through collecting a large number of low-dose diffraction patterns. Figure 6(b) is the summation of all diffraction patterns from 50 passes, showing an integral of scattered signals extended to the edge of detector which is corresponding to 7 nm. For comparison, conventional single-pass ptychography scans were carried out, with 200 nm and 300 nm scan step size for the horizontal and vertical direction, respectively. This resulted in 20 times larger step sizes in two dimensions compared with the effective step sizes (horizontal 50 nm and vertical 60 nm) in the multi-pass scan. The detector was operated 20 times slower with a continuous frame rate of 100 Hz, finally resulting in a similar cumulative X-ray dose on the same scan area. Figure 6(c) shows an example diffraction pattern which was acquired with 20 times more exposure time compared with the one acquired at 2000 Hz (Fig. 6(a)), with more signals scattered outside the primary beam. The summation of all diffraction patterns ($\sim$3800) from one single-pass scan at 100 Hz is shown in Fig. 6(d) which displays a similar integral intensity distribution as Fig. 6(b). The plotted power spectral densities (PSDs) of both integral diffraction patterns are highly consistent with each other as shown in Fig. 6(e), indicating that these two scan approaches reached similar photon statistics. The reconstructed image of the IC sample acquired at 100 Hz in this single-pass scan was displayed in Fig. 5(e), with a spatial resolution of 24.8 nm estimated by FRC (Fig. 7).The image quality is slightly worse than that reconstructed from the 50-pass scan. Figures 8(a) and 8(b) show a same zoom-in region labeled by the red and blue boxes in Fig. 5 for 50-pass scan at 2000 Hz and single-pass scan at 100 Hz, respectively. The interlayer connection vias with a period of 160 nm are well visualized in both images. Line profiles on a row of vias show that 2000 Hz multi-pass scan gives more uniform dots structure and a slightly smaller full-width at half-maximum (FWHM) dot size (an average size of 76 nm compared with 84 nm for 100 Hz single-pass scan), as shown in Fig. 8(c). The slight degradation of the image quality in single-pass scan is probably due to a larger fly-scan step size (200 nm) which would degrade the reconstruction quality in fly-scan ptychography [22,23]. A new reconstruction approach using a common single mode and the knowledge of sample trajectory has been developed for arbitrary-path fly-scan ptychography [48], which has shown better reconstruction quality in fly-scans with a big step size compared with the mixed-state approach used in this paper. The reconstruction quality of this single-pass scan with 200 nm step size could be further improved by this so-called “A-fly method”. Furthermore, a denser sampling of scan points was finally achieved by the multi-pass scan. The improved overlapping condition presumably also contributes to a better reconstruction.

 

Fig. 8. (a) A zoom-in region of the reconstruction of 50-pass scan acquired at 2000 Hz, denoted by a red box in Fig. 5(d). (b) The same zoom-in region of the single-pass scan (100 Hz) reconstruction, denoted by a blue box in Fig. 5(e). (c) Line profiles indicated by red-dashed line in (a) and blue-dashed line in (b).

Download Full Size | PDF

3. Discussion

3.1 Reducing decoherence effect via fast data acquisition

The comparison of ptychographic scans acquired at different frame rates in Sec. 2.1 indicates that the final achieved resolution or quality of ptychography is not simply dose-limited in practice, it is also affected by the coherence degree in the measurement. Any origin of decoherence would cause a reduction of speckle contrast on diffraction patterns [21], thus jeopardizing the ptychographic reconstruction. Therefore, a long exposure time at each diffraction pattern is not always a good approach for coherent imaging because various spatial and temporal variations exist in the real experiments and the longer exposure time would integrate more decoherence effects from those variations.

The high-speed scan demonstrated here acquires data continuously with a short exposure time to minimize the effect of various experimental imperfections on diffraction patterns, so highly resolved image can be obtained (Fig. 1(c)). In addition, with individual highly coherent diffraction patterns acquired in such a fast mode, the spatial and temporal variation (such as illumination variation and sample dynamics) can be modeled by advanced algorithms [26,49]. In this work, the OPR approach was used to model a slightly different probe for every diffraction pattern, and a varying probe was reconstructed as shown in Visualization 1. This capability has significant applications for dynamic wavefront characterization.

3.2 Advantages with multi-pass fast ptychographic scan

Multi-pass ptychographic scanning with fast data acquisition provides a number of advantages. First, it provides a better imaging strategy so that an overview of the entire sample scan region can be obtained quickly at the early stage of scanning. Compared with a conventional single-pass scan, such information can only be known when the scan completes, which may take a long time. We demonstrated this multi-pass fast scan in two schemes, both of which were able to give an overview of the scanned sample region after the first pass, as can be seen in Fig. 3(a) and Fig. 5(d), though with a comparatively lower quality at the beginning. The capability of quickly getting the overview of the sample during the scan sometimes is very important, especially for the cases that such information is necessary for on-site decision making in the experiment.

Second, multi-pass scan scheme provides better photon utilization. Conventional scans put all dose in a single-pass scan, the exposure time is usually set with empirical knowledge of the incident illumination, sample materials and measurement environment, which may result in underdose or overdose situation. With multi-pass scan, the image quality is gradually improved as the deposited dose increases with more scan passes on the sample, and the scan can be aborted when the image quality is sufficient. Higher photon utilization can be realized with more efficient scan schemes, for example, the second scan scheme that gradually increases the fill-factor of scan points makes full use of all data points from the coarse scan to the fine scan in the reconstruction; in addition, the more complete overlap provided by such a scan scheme improves image reconstruction. Furthermore, the diffraction patterns acquired in such a short exposure time have less blurring effect from spatial and temporal variations, which is believed to more photon efficiency from the reconstruction perspective [48]. Note that short exposure to reduce decoherence effects is not an advantage connected to only multi-pass scan, it can be also implemented in single-pass scan by taking multiple short exposures at each scan pixel.

Third, multi-pass fast scan with a low dose at each pass provides a better solution to minimize the radiation damage effect on the sample during the imaging. The sample in a conventional single-pass scan receives X-ray dose in an all-at-once fashion at each scan point, the achievable image resolution and quality are limited as specimens may be chemically and structurally modified by a high dose imparted on the sample in a short time. Such changes would also affect the measurements at adjacent scan points due to the overlapping requirement in ptychography. Multi-pass scan with a low dose uses a similar concept of dose fractionation theorem [29] in 3D imaging to distribute the dose at multi-pass scan with a low dose at each pass, and highly faithful images can be obtained as dose is gradually delivered on the sample over time until obvious radiation damage occurs. Acquiring a number of low-dose images to avoid structural alteration by radiation damage and then to form a high-resolution image has been used in single-particle cryo-electron microscopy [50]. Both this 2D low-dose image acquisition and 3D dose fractionation have an assumption that 2D images and 3D projections can be well aligned. Ptychography with multi-pass low-dose scan doesn’t have this alignment restriction since all data points from multiple passes can be fed into one reconstruction. However, it does require accurate position information which, for example, was recorded by laser interferometers in this study. Such a low-dose fast-data-acquisition strategy provides an efficient and quantitative tool to evaluate the critical dose at which radiation damage effects begin to appear for the specimens. This has been used in electron microscopy [51] and X-ray crystallography [27,28] to take a large number of very low-dose images which are then analyzed with image correlation methods to study radiation effects on the sample. In addition to the benefit of studying radiation damage, the multi-pass fast scan described in this paper may also provide a solution to protect sample from deleterious radiation effects, such as sample heating which may occur [52,53] with a high-flux X-ray beam (e.g., next generation synchrotron sources with about two orders of magnitude increase in coherent flux) and/or the absence of a good heat transfer on sample. X-ray induced heating could cause temperature rise on the sample above the phase transition temperature and increase the rate of damage. For example, temperature rises of the order of 10-20 K have been reported in cryocooling microcrystals [52] and a simulation with a hard X-ray nanofocused beam [53]. The “stroboscopic” dose deposition by short exposures may be able to prevent the sample from reaching the maximum temperature [50,53], and the time gap between passes shown in this paper will provide time for sample to dissipate the heat before the illumination comes back for next pass scan. This is also helpful for some samples which have mechanisms to repair radiation damage at a low-dose exposure [32], yielding a higher tolerance with radiation damage for high-resolution imaging.

Last, such continuous scan is suitable for time-resolved ptychography to do fast imaging of dynamic processes [13], which is very attractive but still hasn’t been fully optimized in ptychography yet. The diffraction patterns acquired continuously in this scheme can be chopped at different time scale to form images to show sample dynamics.

3.3 Experimental and computational requirements for multi-pass fast scan

This multi-pass fast scan scheme also posts additional requirements during the measurement. First, a fast detector with a high continuous frame rate is required for this high-speed scan. In the second multi-pass scan demonstration in this study, an Eiger detector with a continuous frame rate of 2000 Hz was used. As the scan speed continues to increase with higher flux sources (such as the coming upgraded synchrotron sources), the required frame rate of a detector will increase proportionally (e.g., 100-1000X). Second, the detector is also required to have a low noise background, thus single-photon counting detectors that are free of readout noise and dark current will be very helpful; otherwise, the noise background would accumulate as the number of measurements increases. Nevertheless, a sustained effort in developing such a fast noise-free detector is essential to make full advantages of this technique especially with upgraded high-flux sources. Third, this multi-pass fast scan requires accurate position information. As a large number of data points are acquired in the multi-pass scan, position errors would build up numerous degrees of freedom in the reconstruction. Therefore, highly accurate scan position information, such as those measured by high-bandwidth interferometry here, is a key factor for the success of this multi-pass scan for high-resolution imaging.

From the computational perspective, this multi-pass fast scan scheme brings challenges together with opportunities. The need to acquire more data points in this scan strategy results in a large data volume which requires vast computational resource. For example, compared with the conventional single-pass fly-scan, the two scans demonstrated here produced 30 and 20 times more data, respectively. This factor is expected to be even larger when compared with conventional step-scan in which one can use a larger step size (e.g., 5 times of the fly-scan step size used in Fig. 1) but at a cost of longer experiment time due to stage overheads [22,54]. In addition, fly-scan requires multiple probe modes in the reconstruction to deal with decoherence effects from the continuous motion, which is more computationally demanding than a conventional step scan. Note that the current reconstruction speed for Fig. 5(d) was much slower than the scan speed by about two orders of magnitude (an average reconstruction time of 4.5 mins per pass compared with 0.75 s scan time). Large data volume together with comparatively slow iterative reconstruction approach precludes real-time ptychographic imaging. The current real-time feedback on such a fast scan is only possible through conventional imaging contrasts that can quickly map the transmitted intensity or the center of mass of diffraction patterns at scanned points, but with a limited spatial resolution. The bottleneck of the real-time imaging will be gradually addressed as the reconstruction speed continues to be improved by advanced algorithms and optimization techniques together with the use of high performance computation (for example, multi-CPUs or GPUs cores on supercomputer for parallel data processing) [47,55,56]. The recent development of machine learning (ML) techniques [57] enables quick image inversion from diffraction patterns without going through long-time iterative phase retrieval, which provides a very promising tool for real-time ptychographic imaging. These ML techniques can be further integrated into this multi-pass scan scheme to do smart-scan ptychography to further optimize the dose distribution. In addition, with the same photon budget, the number of photons collected at each diffraction pattern in this multi-pass scan decreases, resulting in low-signal diffraction patterns. Therefore, a reconstruction code with faithful physics and noise models (e.g., maximum likelihood algorithm with Poisson noise model used in this study) is required to correctly deal with the noisy data. Finally, additional optimization and refinement techniques are also very important as the measurements can not be always collected in perfect conditions, and the steady development of those techniques in return can also alleviate stringent experimental requirement.

4. Conclusion

In this work, we first show that high-speed ptychographic scanning has advantages to address the spatial and temporal variations which jeopardize coherent imaging; with faithful data acquired in a fast fashion, we then demonstrate that high-resolution imaging can be obtained by multi-pass fast scan. Such a multi-pass fast scan provides a better imaging strategy and more efficient photon utilization during high-resolution imaging. Furthermore, it presents another avenue to minimize radiation damage imparted on the sample. As this tool and corresponding reconstruction methods continue to develop, we believe that it will have widespread applications for not only X-ray but also electron ptychography experiments, both of which pursue high-resolution imaging with a minimum dose, especially for those studies of radiation sensitive specimens. With the improved time resolution given by multi-pass high-speed scan, this approach will also open up new opportunities to a myriad of studies which high spatial resolution and temporal resolution are both required in dynamic samples.