1. Introduction

Fourier Ptychographic Microscopy (FPM) [1–8], sharing its roots with aperture synthesis and phase retrieval algorithms, is a developed technique in computational microscopy. FPM achieves a large field of view (FOV), high-resolution (HR) imaging, and quantitative phase imaging simultaneously by turning on a sequence of light-emitting diode (LED) patterns that provide angle-varied illuminations. In conventional FPM systems, an off-the-shelf planar LED array is used as the light source to provide illumination from different angles and even with different wavelengths. However, this kind of planar LED array has some limitations, such as a restricted angular illumination range, low light throughput, introducing flickering artifacts [9] and raster grid artifacts [10]. Recently, specialized LED illuminators have been proposed to overcome these issues. For instance, quasi-dome LED arrays have been designed and utilized in high numerical aperture (NA) imaging [11,12]. Non-uniform grid LED arrays have been introduced to mitigate the raster grid artifact problem [10]. While these designs effectively address the aforementioned issues, their calibration difficulty is greatly increased due to the lack of geometric constraints of rectangular arrays.

In FPM reconstruction algorithm, correctly stitching each low-resolution (LR) image in the Fourier domain sub-region relies on the accurate knowledge of each illumination angle. In simpler terms, the quality of the reconstruction hinges on the accurate knowledge of each illumination angle. According to this characteristic, some researches based on searching each correct sub-region in the Fourier domain have been proposed. For instance, Yeh et al. used a global Newton’s method to correct each illumination angle in the spectrum [13]. Eckert et al. first made a fast-direct estimate of bright-field (BF) illumination angles and further refined both BF and dark-field (DF) illumination angles using a robust spectral correlation method [14]. Some methods rooted in deep learning have also been proposed to get each correct illumination angle in the Fourier domain [15]. However, these data-driven methods may face some challenges though they can also be applied to the specialized LED illuminators, including low signal-to-noise ratio (SNR) for DF images, time-consuming, sensitive to the initial search parameters, easy to fall into local solution and sensitive to the number of BF images.

Given that most off-the-shelf planar LED arrays have a rigid structure, which implies the possibility of introducing additional rectangular constraints in the LED pose calibration, several physics-based models have been proposed to correct the LED misalignment. For instance, Sun et al. proposed an LED array model based on the rigid structure of the planar LED array to correct the global LED misalignment using an improved simulated annealing (SA) algorithm [16]. Zheng et al. proposed a physics-based defocusing strategy to calibrating the angle-varied illumination [17]. Yang et al. calibrated the global LED misalignment based on the brightfield-to-darkfield (B-D) images [18]. Zheng et al. corrected the global LED misalignment based on the geometric features of the B-D edges [19]. While the additional rectangular constraints can enhance calibration efficiency, they may encounter accuracy issues due to limitations in welding precision. Moreover, for special structure LED arrays, like domes or rings, it becomes challenging to identify suitable and precise rectangular constraints. Consequently, when dealing with these special structure LED arrays, it is generally necessary to calibrate each LED illumination angle individually.

Recently, several studies have been proposed to calibrate the 3D pose of LED arrays based on geometric imaging relationship by introducing additional special made samples. For instance, Guo et al. calibrated angle-varied illumination using a double-sided mask in multi-angle structured illumination lens-less (MASIL) systems [20]. Song et al. calibrated angle-varied illuminations using a monolayer of blood cells in lens-less imaging [21]. These methods obtain the 3D positions of the illuminator and can be applied to freeform illuminators. However, it necessitates initial calibration of accurate 3D LED positions using a lens-less system before it can be adapted to the FPM system. As a result, accurate misalignment pose-parameters of the illuminator within the microscopy system need to be recalibrated using methods like the SA method in [16] or physics-based method in [18]. Additionally, practically implementing a monolayer of blood cells is challenging because the cover glass of the image sensor needs to be removed. The distance between the blood-coated layer and the pixel array should be determined by digitally propagate method and visual inspection as mentioned by the authors in [21]. Furthermore, Jiang et al. also provide a summary of two different options for calibrating the incident angles of different LED elements in their research [22]. The first option involves utilizing the B-D transition features of the captured image and assumes that the LED array has a well-defined grid pattern. On the other hand, the second option involves conducting cross-correlation analysis to obtain the incident angle information from the captured images corresponding to the defocused sample. Different to the first option, these approach does not necessitate a well-defined grid pattern, making it more versatile. However, since the cross-correlation analysis should be applied to the BF images, the second option requires a high-NA objective lens to capture BF images as much as possible.

In this paper, we propose a hybrid full-pose parameter calibration method for freeform LED illuminators based on the stereo vision and the geometric constraints of the microscopic platform. We employ a stereovision system to calibrate the accurate 3D position of each LED element of the freeform LED illuminator firstly. Then, within the microscopic imaging system, we calibrate the coordinate system of the LED array and that of the optical imaging by using the geometric features of the B-D edges within the microscopic imaging system. Finally, we demonstrate the feasibility of the proposed method through experiments with four different types of freeform LED illuminators.

This paper is organized as follows. The principle of the proposed method including stereo vision and calibrating the precise misalignment pose parameters is presented in Section 2. In Section 3, the pure USAF phase target and biological samples are used to demonstrate the effectiveness of our method. Conclusion and discussion will be presented in Section 4.

2. Principle

The flow chart of the proposed method is shown in Fig. 1. In step 1, as shown in Fig. 1 (a), we use the in-line LED element to construct a freeform illuminator using a 3D printer. Such LED can be easily controlled by the micro-controller (STM32 or MCS 51). In step 2, we change the LED patterns one by one and capture the corresponding images with two cameras shown in Fig. 1 (b). Then, in step 3, we use the stereo vision method to calibrate the accurate 3D position of each LED element in Fig. 1 (c). In step 4, the calibrated illuminator is installed into the microscopic system, and we capture a sequence of B-D boundary images without placing samples to obtain the accurate misalignment pose-parameters between the LED illuminator and the microscopic system, as shown in Fig. 1 (d2). Finally, in step 5, we recover the results of biological sample with the FPM algorithm as shown in Fig. 1 (e). We note that, for a fixed illuminator, step 1 to step 3 should be performed just once, then, the illuminator can be used in the FPM system. However, the step 4 should be repeated if the illuminator is installed into a different imaging system.

 

Fig. 1. Flow chart of the proposed method. (a) Constructing freeform LED illuminator with in-line LED element. (b) 3D position calibration process of the freeform illuminator. (c) 3D position overview of (b). (d1) FPM system without samples. (d2) B-D boundary images. (e1) FPM system with samples. (e2) recovered HR amplitude of biological sample. (e3) LR image captured by the camera. (e4) recovered HR phase of biological sample. (e5) recovered spectrum of biological sample.

Download Full Size | PDF

2.1 3D position calibration with stereo vision

Stereo photogrammetry [23] is a widely used technique in 3D measurement, which consists of two cameras and a projector that provides some feature points if the object is no texture. This technique operates based on the principles of triangulation rangefinder. As shown in Fig. 2, in a typical stereo photogrammetry setup without the projector, the axes of the two cameras are parallel and point in the same direction, perpendicular to the stereo base.

 

Fig. 2. Typical case of stereo photogrammetry with two cameras.

Download Full Size | PDF

Suppose the point P in Fig. 2 is located in the left camera’s perspective center, whose 3D position is described as $(X,Y,Z)$, the coordinates of point P can be described as follows,

Eq. (2) can be rewritten as,

The disparity $px^{\prime}$ of the projection position corresponding to the point P between the left and right cameras is described as the product of the number of pixels and the camera’s pixel size. Therefore, by combining Eq. (1) and Eq. (3), we can obtain the 3D position $(X,Y,Z)$ of point P. Following the principles of stereo photogrammetry, we implement the 3D position calibration process as follows.

The flow chart of the 3D position calibration is shown in Fig. 3. First, we use a checkerboard to calibrate the coordinates between two cameras, as shown in Fig. 3 (a). The step size of the checkerboard (PointVision, CC-100-17${\times}$19-5.0-1.0) used in our experiment is 5 mm. The objective lens (CHIOPT, FA1202D) and cameras (IMAVISION, MER-302-56U3M) are used in the calibration process. It’s worth noting that stereo photogrammetry has become widely used across diverse fields, making the calibration of coordinates between two cameras a well-understood process. There are numerous toolkits available for this purpose, including but not limited to OpenCV (available in C++ or Python) [24], MATLAB toolkits [25] and others, which can be utilized easily and quickly.

 

Fig. 3. Flow chart of the 3D position calibration of the freeform illuminator. (a) calibrating the coordinates of two cameras. (b) baseline correction and calculating the disparity of the projection position of each point in two cameras. (c) 3D position results of all points corresponding to the freeform illuminator.

Download Full Size | PDF

Second, the LED elements of the illuminator are turned on sequentially, and the corresponding images are captured for subsequent processing. As shown in Fig. 3 (b), the images captured by the two cameras should be processed by the baseline correction method as shown in step 1 of Fig. 3(b). And then, we apply binarization to the images using Eq. (4) in step 2 and calculate the center of points to get the disparity of the projection position of the point P between the left and right cameras. The center of points is obtained using the center-of-gravity method, as demonstrated in Eq. (4),

Finally, by repeatedly executing step 1 and step 2 to obtain all LEDs’ point centers, we calculate the disparity of the projection position of each point between two cameras and get the 3D position according to Eq. (1) and Eq. (3). The 3D position of a freeform illuminator we designed with a 3D printer is shown in Fig. 3(c).

2.2 Principle of Fourier ptychographic microscopy

A conventional FPM setup consists of an LED array for providing angle-varied illumination, a microscopic system for optical imaging, and a camera for capturing the LR images. In experiments, a sequence of LEDs is turned on to illuminate the sample that is placed at the sample plane. For a subregion of the sample, the illumination wave from $LE{D_i}$ can be treated as a monochrome plane wave with wave vectors of $({u_i},{v_i})$. Thus, the LR intensity image captured by the camera can be described as Eq. (5),

As shown in Fig. 4, FPM shares roots with the synthetic aperture imaging and phase retrieval in coherent diffraction imaging [26,27]. The complex distribution of the sample can be obtained by recovering the phase component and stitching the captured LR images together in the Fourier domain. Usually, the initially complex distribution can be initialized with the interpolated BF LR image, zeros matrix or ones matrix. And then, the alternating projection optimization process will be utilized in the FPM reconstruction as shown in the middle square in Fig. 4.

 

Fig. 4. FPM reconstruction flow chart.

Download Full Size | PDF

Considering that FPM has high requirements for the accuracy of the sub-region position when stitching the captured LR images together in the Fourier domain, the LED misalignment correction is important.

2.3 Pose-parameter calibration

A global LED array model has been proposed to correct the LED misalignment pose-parameters based on SA algorithm [16]. As shown in Fig. 5 (a), this model employs four parameters $(\Delta x,\Delta y,{\theta _z},h)$ to represent the LED misalignment between LED array and the microscopic system. Here, the parameters $(\Delta x,\Delta y)$ indicate the lateral shifts, ${\theta _z}$ is the z-axis rotation angle, and h is the height between LED array and sample plane. Consequently, the corrected LED array model can be described as,

 

Fig. 5. LED misalignment model. (a) global LED misalignment model. (b) deviation between the actual position and designed position.

Download Full Size | PDF

However, such a model cannot obtain the accurate 3D position of each LED. As shown in Fig. 5 (b), limited to the soldering precision, a deviation exists between the actual and designed positions. Most of the conventional LED misalignment correction methods overlook this soldering deviation. These methods are suitable for regularly arranged LED arrays or those with a known distribution, such as planar array. However, for random freeform LED illuminators, determining the distribution can be challenging. In the method we proposed, we calibrate the accurate 3D position of each LED using the method in Section 2.1 firstly. Subsequently, the misalignment pose-parameters between the illuminator and the microscopic system will be corrected with the following method.

We construct a physics-based model to calibrate the misalignment pose-parameters of the LED illuminator, building upon our previous research [19]. As shown in Fig. 6 (a1), when the illumination NA approximates the objective’s NA ($N{A_{obj}} = n \cdot \sin \theta$, where $n = 1$ in the air and $\theta$ is defined as the maximum illumination angle that objective lens can capture), the obtained image is formed by superimposing the field stop and the projection of the circular aperture stop in the image plane, resulting in an arc-shaped B-D image as shown in Fig. 6 (a2). The schematic diagram of the physics-based model is presented in Fig. 6 (b), where ${h_1}$ indicates the height between the entrance pupil and the entrance window. As the position of entrance pupil and entrance window are fixed, ${h_1}$ is a constant value of the imaging system. $GH$ can be calculated by $GH \approx {h_1} \cdot NA$. $C{C_0}$ is the radius of the projection of the entrance pupil on the entrance window plane, which can be described as $C{C_0} = {R_{BD}}/Mag$, where $Mag$ is the magnification of the imaging system and ${R_{BD}}$ indicates the radius of the B-D image shown in Fig. 6 (a2). So, with the similar triangle in Fig. 6 (b), we can get

 

Fig. 6. physics-based model of the proposed method. (a1) forward model of the proposed method. (a2) B-D image captured by the camera. (b) schematic diagram of the physics-based model in Fig. 6 (a1).

Download Full Size | PDF

By substituting $C{C_0} = {R_{BD}}/Mag$ and $GH \approx {h_1} \cdot NA$ into Eq. (7), we can reformulate Eq. (7) as follows:

Building upon the accurate 3D position obtained in Section 2.1, we can designate the 3D position $({x_0},{y_0},{z_0})$ of central LED element as the origin point $(0,0,0)$. This allows us to calculate the center of the ith B-D image circle, corresponding to the ith LED element with a 3D position of $({x_i},{y_i},{z_i})$ as Eq. (9).

In general, Eq. (8) and Eq. (9) constitute the forward model of our method, which means the circle’s center points of the B-D image corresponding to the ith LED element can be calculated by using the ith LED 3D position $({x_i},{y_i},{z_i})$, height between LED array and sample plane, and the NA of the objective lens. Therefore, combining with Eq. (6), Eq. (8) and Eq. (9), we can obtain a set of circle’s center points $({x_{BD,i}},{y_{BD,i}})$ of the B-D image which contains unknown LED misalignment. Furthermore, we can also get another set of circle’s center points $({x_{BD,i,c}},{y_{BD,i,c}})$ of the B-D images captured in experiments with circular fitting method, details can be found in our previous work [19]. Meanwhile, the radius ${R_{BD}}$ of B-D images can be obtained through the circular fitting. Notable that each B-D image can obtain a radius ${R_{BD,i}}$ from the circular fitting process. Due to the vignetting of the objective lens, we found that the series of ${R_{BD,i}}$ follows a normal distribution. Therefore, to obtain a more precise value for the radius ${R_{BD}}$, we utilize the method as Eq. (10), where we assign each ${R_{BD,i}}$ a weight value of ${w_i}$. $\varepsilon$ indicates the standard deviation of the series of ${R_{BD,i}}$. $\overline {{R_{BD}}}$ indicates the mean value of the series of ${R_{BD,i}}$, and ‘| |’ describes the absolute value.

Thus, the optimization process can be described as follows:

In more detail, we can obtain the optimized misalignment pose-parameters $(\Delta x,\Delta y,{\theta _z},h)$ with the function shown in Eq. (12) by using the nonlinear fitting:

In general, in the proposed method, we can calibrate the accurate 3D position of each LED element in a freeform LED illuminator firstly using the stereo vision method. Then, with the optical model we proposed, we can also obtain the misalignment pose-parameters between the illuminator and the microscopic system. Different B-D images may contain varying proportions of BF parts. In other words, not all B-D images are suitable in circular fitting. Specifically, if the BF parts of a B-D image account for less than 10% or more than 90% of the B-D image, they are not suitable because the circular fitting method will fail to achieve correct results. Thus, we first binarize the ith B-D image, and then calculate the proportion of the BF part according to Eq. (13), where ${G_i}(x,y)$ is the binarized image of ith B-D image. X and Y indicate the number of pixels along the x- and y-axes, respectively. If the proportion value ${\eta _i} \in [0.1,0.9]$, we select that specific B-D image.

3. Experiments and results

We build an FPM system to validate the proposed method, as shown in Fig. 7 (a). An objective lens (Olympus, 4x, NA = 0.13) is used for optical imaging. A camera (FLIR, BFS-U3-200S6M-C) is used for capturing LR images. A quasi-dome LED is used as an illuminator whose design parameters is shown in Fig. 7 (b). The quasi-dome LED illuminator incorporates a central radius of 80 mm and features a total of 37 LEDs. The angle maintained between adjacent LED rings is specified as ${5^ \circ }$, ${6^ \circ }$, ${7^ \circ }$ and ${8^ \circ }$. Following the guidance provided in Fig. 1 (a), we sequentially link each LED’s input and output line. In addition to the above, we also conceive some different freeform illuminators, the details will be introduced subsequently. Throughout our experiments, we use three different exposure times (70 ms, 100 ms, and 200 ms) to achieve HDR combination when capturing the LR images. The capturing process can further be simplified with a color camera [28] according to our previous work. The gain value for the camera is set to 0 and the wavelength of the illumination is 470 nm. More experimental details of the FPM system setup can be found in [22]. It’s worth noting that during our experiments, the objective lens with NA = 0.13 cannot able to capture enough B-D images (typically number of B-D images should be four at least in our method to ensure the optimization process can achieve the results). Thus, in the pose-parameters calibration process, we use an objective lens (4x, NA = 0.1) to capture the B-D images.

 

Fig. 7. FPM system setup. (a) FPM system with a quasi-dome illuminator. (b) designed parameters of the quasi-dome illuminator with 37 LEDs.

Download Full Size | PDF

3.1 Ablation experiment

In order to validate the presence of a deviation between the designed position and the actual position, and to find how this deviation impacts the reconstructed quality, we implement an ablation experiment as follows. First, we design a quasi-dome illuminator shown in Fig. 7 (b), and calibrate the accurate 3D position of each LED element according to Section 2.1. Therefore, we can obtain both the actual position and the designed position simultaneously (the designed position is obtained according to the design parameters, such as using SolidWorks in our experiment). Second, we install the quasi-dome illuminator into the FPM system with accurate pose adjustment [29,30], ensuring there is no LED misalignment between the illuminator and the microscopic system. In other words, the pose-parameters $(\Delta x,\Delta y,{\theta _z})$ are set to zeros and h is fixed at 80 mm. Finally, we compare the FPM results of the pure USAF phase target corresponding to both the designed and actual positions, as shown in Fig. 8.

 

Fig. 8. Ablation experiment results. (a) 3D overviews of the actual and designed position. (b1-b2) FPM reconstructed phase with the designed position and the corresponding enlarged sub-region. (c1-c2) FPM reconstructed phase with the actual position and the corresponding enlarged sub-region. (d) LR image captured by the camera. (e) quasi-dome illuminator designed with a 3D printer.

Download Full Size | PDF

Fig. 8 (a) shows the 3D overviews of the actual and designed positions of the quasi-dome illuminator in Fig. 8 (e). We can find that there is a slight deviation between the actual and the designed positions. Then, as shown in Figs. 8 (b) and (c), this deviation will be reflected in the FPM reconstructed results. Fig. 8 (b1) is the FPM reconstructed phase corresponding to the designed position, and Fig. 8 (b2) is the enlarged sub-region. Fig. 8 (c1) is the FPM reconstructed phase corresponding to the actual position, and Fig. 8 (c2) is the enlarged sub-region. Comparing these two results, we found that the reconstructed quality of the designed position is worse than that of the actual position. The lines in group 9, element 2 in Fig. 8 (b2), in particular, appear chaotic. We analyze that it may be caused by the deviations of the LEDs with illumination angles corresponding to this resolution range. However, by utilizing the actual position calibrated with our method, the results are significantly improved, thus validating the feasibility of the proposed method. Furthermore, based on our design parameters, the illumination NA can reach $N{A_{illu}} = 0.44$, and the objective’s NA is 0.13. Thus, the final synthesized NA will be $N{A_{syn}} = 0.57$. Given that the illumination wavelength is 470 nm, we can calculate the frequency limitation $f = N{A_{syn}}/\lambda \approx 1213(lp/mm)$ with our system parameters, aligning with the result shown in Fig. 8 (c1) where we can achieve at least the lines in group 10 and element 2 (whose frequency limitation is 1149 $lp/mm$). Fig. 8 (d) is the BF LR image captured by the camera. This result indicates that we can improve the resolution limitation of this FPM system from 2um to less than 500 nm with just 37 LEDs.

Furthermore, we use a mouse heart slice to demonstrate the feasibility of the quasi-dome illuminator for biological imaging. As shown in Fig. 9, we select a sub-region (256${\times}$256 pixels) for FPM reconstruction. Figs. 9 (a) and (c) are the HR amplitude and phase, respectively. Fig. 9 (b) is the LR image captured with the camera. Fig. 9 (d) is the reconstructed spectrum in the Fourier domain. On comparing Fig. 9 (a) with Fig. 9 (b), we found that the resolution is improved significantly. Moreover, the phase shown in Fig. 9 (c) can be recovered effectively.

 

Fig. 9. Biological results of FPM. (a) HR amplitude of FPM. (b) LR image captured by camera. (c) HR phase of FPM. (d) spectrum of FPM using the quasi-dome illuminator.

Download Full Size | PDF

3.2 Random position experiments

In this experiment, we use four different LED illuminators to demonstrate the feasibility of calculating the misalignment pose-parameters mentioned in Section 2.3. As shown in Fig. 10, the illuminators in Figs. 10 (a1) and (d1) are designed using LED elements referenced in Fig. 1 (a). Fig. 10 (b1) is a planar LED array designed by ourselves, incorporating LED elements (TH-S3528F4C1TC-53) packaged as surface mounted devices (SMD). Comparing with the LED elements in Figs. 10 (a1) and (b1), the SMD-packaged LED element has a smaller size (typical size is 2.5 mm or 3 mm), which allows for a reduced distance between adjacent LEDs. However, the SMD package requires soldering onto a circuit board. Given that circuit boards are typically flat such as the rings in Fig. 10 (c1), the SMD package offers limited degree of freedom. On the contrary, the in-line LED element in Figs. 10 (a1) and (d1) can be installed without soldering and circuit board, which allows us to use a 3D printer to create differently shaped fixed housings. It provides us with more design flexibility, especially for complex freeform surfaces.

 

Fig. 10. Random position experimental results of the proposed method. (a1) 3D position of the quasi-dome LED array. (a2) reconstructed phase of FPM with (a1). (a3) recovered spectrum with (a1). (b1) 3D position of the planar LED array. (b2) reconstructed phase of FPM with (b1). (b3) recovered spectrum with (b1). (c1) 3D position of the quasi-dome LED with the off-the-shelf LEDs. (c2) reconstructed phase of FPM with (c1). (c3) recovered spectrum with (c1). (d1) 3D position of a planar LED array. (d2) reconstructed phase of FPM with (d1). (d3) recovered spectrum with (d1).

Download Full Size | PDF

The LED illuminator in Fig. 10 (d1) is designed with two layers, where the center layer consists of nine LEDs, and the peripheral layer has twelve LEDs. According to our design, there is a 3 mm height difference between these two layers. We also design an off-the-shelf LED array whose LED element’s model is WS2812. As shown in Fig. 10 (c1), we design a quasi-dome housing using a 3D printer to fixed three ring LED arrays with different radii. The three ring LED arrays are randomly affixed within the quasi-dome housing using double-sided tape. Generally, these four different illuminators operate under the same drive mode, regardless of their different packages. Notably, the control code and designed parameters can be made open source for non-commercial use. The driver in our experiments is the STM32F103C8T6 and different types of STM32 may require different compile methods.

Initially, we calibrate the accurate 3D position corresponding to four different LED illuminators using the method detailed in Section 2.1. The 3D overviews are shown in Figs. 10 (a1) -(d1), respectively. Next, four LED illuminators are installed into the microscopic system, each with random misalignment pose-parameters. According to Section 2.3, we capture the B-D images without placing biological sample to calculate the LED misalignment pose-parameters for each illuminator. The correct results are shown in Figs. 10 (a1) -(d1), where the parameters $(\Delta x,\Delta y)$ describe the x-y axis lateral shifts, ${\theta _z}$ is the rotation angle of the z-axis and h indicates the height from center LED to sample plane, respectively. The units for lateral shift and height are in mm and rotation angle’s unit is in degree. According to our results, the maximum lateral shift we can calculated exceeds ${\pm} 1.6mm$ whose shift extent is already visible to the naked eye. Then, we place the sample of pure USAF phase target on the sample plane and capture the corresponding raw data in the FPM to assess the reconstructed quality of FPM. The LED number are 37 LEDs, 121 LEDs, 35 LEDs, and 21 LEDs corresponding to Figs. 10 (a1) -(d1), respectively. According to our calculations, with the objective lens (4x, NA = 0.13), the synthetic NA of the four illuminators can achieve 0.57, 0.5, 0.47, 0.36, and the reconstructed phase results are shown in Figs. 10 (a2) -(d2), respectively. Figs. 10 (a3) -(d3) are the corresponding recovered spectrum.

Comparing the results in Fig. 10, we can find some conclusions. First, quasi-dome LED illuminators can provide a larger illumination angle and require fewer LEDs. On the contrary, the planar LED in Fig. 10 (b1) requires more LEDs to achieve the same resolution. However, even though the planar LED illuminator requires a greater number of LEDs compared to quasi-dome LED illuminators, it can potentially provide more convergent results due to the huge amount of raw data. The reconstructed results will be clearer than that reconstructed with other illuminators. Second, most of the frequency components of pure USAF phase target concentrate in the cross-shaped of their spectrum as shown in their spectrum distribution in Figs. 10 (a3) –(d3). Therefore, once the illuminator in Fig. 10 (d1) has a rotational angle like that in the Fig. 10 (d3), the reconstructed quality will be decrease. However, this phenomenon will not exist with circular spectrums like Figs. 10 (a3) and (c3).

4. Conclusion and discussion

In conclusion, we proposed a hybrid full-pose parameter calibration method for freeform LED illuminators based on stereoscopic 3D imaging and geometric constraints of the microscopic platform. During the design process, we use in-line LEDs to construct freeform illuminators with a 3D printer. Compared with the off-the-shelf SMD-packaged LEDs, in-line LEDs avoid the need to design a circuit board, allowing greater design flexibility. We then employ the stereo vision method to obtain the accurate 3D position of each LED of the freeform illuminator and construct a rigid 3D coordinate LED array system. Next, we utilize a geometric imaging method to correct the LED misalignment between the LED illuminator and the microscopic system based on a set of B-D images. We demonstrate the feasibility of the proposed method by four different freeform LED illuminators with the pure USAF target and biological samples. Generally, the lateral shift can be corrected more than ${\pm} 1.6mm$, and we can increase the resolution from 2um to less than 500 nm using just 37 LEDs.

Although the experimental results have demonstrated the feasibility of the proposed method, there are still some issues that warrant further discussion. First, according to our previous work [19], the LED misalignment model is described by six parameters as $(\Delta x,\Delta y,{\theta _z},{\theta _x},{\theta _y},h)$ rather than four parameters as $(\Delta x,\Delta y,{\theta _z},h)$, where ${\theta _x}$ and ${\theta _y}$ are tilt angles along the x and y axis. However, in the forward model of the proposed method, we found that the combined influence of the tilt misalignment and the latera shift of $(\Delta x,\Delta y)$ significantly affect the lateral shift in the B-D image. The lateral shift of $(\Delta x,\Delta y)$ contributes more significantly in this particular case. It means that a substantial variation in the tilt angle could only induce a relatively minor shift in the B-D image (i.e. ${10^ \circ }$ to 48um). Therefore, when we add $({\theta _x},{\theta _y})$ in the forward model, the optimization process tends to find a solution by exaggerating the tilt misalignment. This not only yields inaccurate tilt results but also affect the accuracy of other parameters, because other parameters need to adjust to offset the influence triggered by the erroneous tilt angle. Such case may be mitigated with additional B-D images. However, considering the minimum LED gap and the objective’s NA in our experiment, we do not add the parameters in the forward model. The tilt misalignment in our experiments is corrected with a spirit level like other researches. Second, we think the proposed method can be also utilized in other computational microscopy to help them correct the LED misalignment, such as 3D tomography [31] and differential phase contrast (DPC) [32].