1. INTRODUCTION

Nowadays, nearly all imaging systems include image processing algorithms to improve image quality. Hence, when designing an imaging system, it seems natural to jointly optimize the lens and the image processing algorithms. This approach is called joint optimization or also co-design. Co-design was pioneered by Dowski et al. in 1999 [1], who proposed phase masks that extend the depth of field (DOF) at the price of a degradation of the raw image quality, which was compensated for by a digital deconvolution algorithm. Later, the co-design problem was formulated in a rigorous signal processing framework. This consisted of defining the optimization criterion as the mean square difference between an ideally sharp image and the final image produced by the global system composed of the image forming lens and the digital image processing implemented by a Wiener deconvolution filter [2–4]. This approach has been used to optimize DOF-extending phase masks placed in the pupil of ideal or already optimized complex camera lenses [5,6]. These masks have been implemented with success in real-world imagers [7,8]. This approach has also been used to design an iris recognition system by jointly optimizing phase masks placed in subapertures with Wiener filter [9]. Zammit et al. proposed systems with optimized antisymmetric phase masks and a deconvolution filter with removed artefacts for 3D reconstruction [10], which was recently applied to microscopy [11]. The design of the optical components or the phase masks by taking post-processing into account has also been made using other criteria different from the above-mentioned image quality criterion [12–16].

Co-design based on final image quality optimization has also been applied to the design of complex lenses made of several optical elements. To design such lenses, it is necessary to use modern optical design software since it can efficiently take into account all the optical and manufacturing constraints related to system-level specifications and the physical and practical requisites for the elements and their opto-mechanical mounts. Robinson and Stork extended the co-design framework to complex lens design using the Zemax optical design software [3]. They jointly optimized the lens, the sensor, and the processing algorithm by implementing the mean-square image quality criterion directly in Zemax using C-extensions. This method has been used to design relatively simple lenses with two or three optical elements, like a Cooke triplet, in order to increase their performance in terms of optical path difference or of DOF. A similar approach has been applied to the joint optimization of infrared optical systems including a phase mask, other optical elements, and a deconvolution filter in order to make the image quality invariant over the field [17].

These first demonstrations of lens co-design have shown the feasibility and the benefits of this approach. They have been performed on simple lenses with few optical components and/or moderate apertures. In order to go further and address the design of lenses with a larger number of optical elements and/or higher specifications in terms of aperture or imaging field, it is necessary to remember that optical design of such lenses is an art as well as a science. In the classical design of complex lenses, the skilled optical designer uses a powerful optical design software with efficient ray-tracing routines and efficient optimization algorithms, which are adapted to manage the numerous optical and mechanical constraints that an optical system must fulfil to be practically manufacturable. The designer also uses many hints from experience in order to steer the lens optimization process, which consists of a succession of local optimizations and adjustments of the weights of the optimization criterion [18]. To make this iterative process convenient, optimization time is a key factor.

We think that in the near and even mid-future, co-design of hybrid optical system with complex lenses cannot be simply an automatic procedure and a skilled optical designer will still be required. In order to benefit from the co-design approach, a key point is thus to introduce this approach into the usual optical design process, while staying as close as possible to the usual practice and environment of the optical designer, including familiar optical design software and figures of merit. The purpose of the present paper is to propose a method to reach this objective.

The rigorous image quality criterion involves intricate calculations on the whole computation of the lens’s 2D optical transfer function. It is therefore computationally intensive to evaluate at each iteration of the optimization algorithm, and its direct optimization is thus highly time consuming. Furthermore, the rigorous image quality criterion is extremely difficult to implement in such optical design software applications, like Code V, as it involves advanced computations on mega-pixel images or mega-element arrays in the Fourier domain.

Therefore, we define a surrogate lens optimization criterion based on a merit function, which uses, in a non-standard way, physical quantities and parameters that are natively and efficiently computed by optical design software. This approach is first evaluated on a simple co-design problem where the surrogate criterion and the image quality criterion can be compared and are shown to be equivalent. The proposed method is then applied to the design of a very fast (f/0.75) complex lens. This co-designed lens is found to be superior to a classically designed lens in terms of lens weight and homogeneity of image quality in the field.

The paper is organized as follows. Section 2 describes the imaging chain model, the mean-square joint design criterion based on final image quality (IQ), and discusses its implementation in optical design software. In Section 3, we propose a surrogate optical design (SOD) criterion, which can be more easily implemented in optical design software as it uses a merit function based on native and efficiently computed physical quantities such as the spot diagram diameter or the modulation transfer function (MTF) at given frequencies, fields, and orientations. In Section 4, the efficiency of this criterion is demonstrated on the co-design of a complex lens with a very large aperture.

2. IMAGE QUALITY CRITERION FOR CO-DESIGN

In this section, we define the mean-square image quality criterion that is generally used for co-design applications. We then discuss the problem of its implementation and its practical use in commercial optical design software.

A. Image Quality Criterion

Let us consider an imaging system that consists of a lens, an array sensor, and a deconvolution algorithm that processes the acquired image. The problem we consider is to make the imaging system invariant with respect to a given parameter, such as the defocus value in order to increase the DOF or the field. In a local area of the imaging field, the lens is assumed to behave as a linear and translation-invariant filter and is characterized by its point spread function (PSF), denoted ${\mathrm{PSF}}_{\psi }^{\mathit{\theta }}$. In this notation, $\mathit{\theta }$ is a vector representing the set of optical parameters on which the system response depends, such as, for example, surface curvature, lens materials, glass thickness, or air spacing. The scalar $\psi$ represents the parameter for which we want the system to be invariant, e.g., the defocus values or positions in the field, and takes value among the set $\{{\psi }_1,\dots ,{\psi }_K\}$. The finite size of the sensor pixels, which leads to some modular transfer function (MTF) drop at high spatial frequencies, is represented by an impulse response $h_p$ depending on the pitch $p$ and a uniform spatial distribution of sensitivity over the pixel, assuming here a 100% fill factor. The total impulse response is

Let us denote $O(\mathbf{r})$ the image of the scene (minus its mean) that would be formed by an ideal imaging system devoid of any limitation. It is considered as a stationary random process with zero mean and power spectral density (PSD) $S_{OO}(\mathit{\nu })$, where $\mathit{\nu }$ denotes the spatial frequency coordinates. The image acquired by the system is modeled as

The purpose is to find the deconvolution filter $d$ and the optical system parameters $\mathit{\theta }$ that minimize the MSE defined in Eq. (3) for the set of $K$ possible values of $\psi$. We choose the deconvolution filter that minimizes the mean of the MSE over the $K$ values of $\psi$,

Knowing the closed-form expression of $d_{\mathrm{opt}}^{\mathit{\theta }}$ given in [19], the optimal value of $\mathit{\theta }$ can be estimated by minimizing the MSE averaged on all values of ${\psi }_k$. In practice, it has been noticed that this minimization can lead to high values of the MSE for some values of ${\psi }_k$. In order to better control the image quality disparity among the ${\psi }_k$ set, we will rather use the following minimax criterion proposed by Diaz et al. [19], which is to minimize

We will also define the obtained image quality IQ for a given set of parameters $\mathit{\theta }$ at a given $\psi$ value as

B. Implementation of the Image Quality Criterion in Optical Design Software

Our goal is to optimize the lens parameters $\mathit{\theta }$ following $J_{\mathrm{IQ}}$ defined in Eq. (5). For that purpose, it is necessary to take into account all the nonlinear constraints that should be fulfilled by these parameters, such as, for example, that the effective focal length must be equal to a specified value, that the lens thickness at the center and edges must be strictly positive, or that the weight and size of the whole lens must not be above specified values. It is also of utmost importance to keep optimization times similar to what optical designers are used to in classical optical design—that is to say, a few seconds or minutes for a local optimization. As the optimization algorithms implemented in optical design software are powerful and natively take into account the constraints specific to optical design, the natural strategy to reach our goal is to implement $J_{\mathrm{IQ}}$ inside the optical design software. For that purpose, we first need to consider software limitations.

Robinson and Stork [3] and Vettenburg et al. [17] made use of the Zemax software. They implemented the computation of a criterion similar to $J_{\mathrm{IQ}}$ thanks to extensions programmed in C language. However, they do not precisely mention the used optimization algorithm, its sensitivity to choice of starting point, or the typical computation time required. This latter point is of high importance regarding optimization practicality. Indeed, the optimization algorithms implemented in optical design software are adapted to classical merit functions, like the root mean square (RMS) diameter of the spot diagram. Employing a different user-defined objective function with different structure like $J_{\mathrm{IQ}}$, which requires computation of all the impulse responses ${\{h_{{\psi }_k}^{\mathit{\theta }}\}}_{k=1\dots K}$ at each merit function evaluation, would lead to much higher computation time and possibly poor convergence to one of the numerous under-optimal local minima. One must also note that the direct implementation of the criterion $J_{\mathrm{IQ}}$, which involves products of Fourier transforms of PSFs, was not technically possible until recently in the optimization procedure of the optical design software we used (Code V).

Taking into account these limitations, we decided to investigate an approach consisting of approximating $J_{\mathrm{IQ}}$ with a surrogate criterion that can be efficiently implemented in standard optical design software. In other words, it should only make use of physical quantities natively and efficiently computed by optical design software. These physical quantities should be used in a way that mimics as best as possible the behavior of $J_{\mathrm{IQ}}$ so as to obtain a lens with performance similar to that obtained by directly optimizing $J_{\mathrm{IQ}}$. In the following section, our purpose will be to infer this surrogate optical design criterion from an analysis of the main imaging characteristic of a system optimized with $J_{\mathrm{IQ}}$.

3. CONSTRUCTION OF THE SURROGATE OPTICAL DESIGN CRITERION

We first analyze in sub-section A the imaging performance of a co-designed system optimized using the IQ criterion $J_{\mathrm{IQ}}$. From this analysis, we propose in sub-section B a surrogate criterion to $J_{\mathrm{IQ}}$ that can be simply and efficiently implemented with native quantities in optical design software. Finally, we validate this surrogate criterion in sub-section C by showing that its optimization leads to a system very similar to that obtained by directly optimizing $J_{\mathrm{IQ}}$.

A. Analysis of the Imaging Performance of a System Co-Designed with the IQ Criterion

The imaging system we consider is the co-designed f/1.2 visible and near-infrared camera described in [8]. It consists of an already optimized complex lens and of a six-ring binary phase mask placed in the stop plane of the lens. This optical system is co-optimized with the deconvolution algorithm to extend the DOF of the camera. The set of variables on which it depends are $\mathit{\theta }=\{r_1,\dots ,r_5,\delta \}$, where ${\{r_n\}}_{⟦1,5⟧}$ are the five radii of the inner rings of the binary phase mask, and $\delta$ is an optical parameter that allows one to adjust the internal focus setting of the camera. It was required that the imaging performance be invariant for object distances $\psi \in \{4.8\mathrm{m},9.6\mathrm{m},10\mathrm{km}\}$. In [8], these parameters were optimized with $J_{\mathrm{IQ}}$. The optimization algorithm was the simplex method [20], implemented with Matlab numerical computation software. Code V software was used only as a ray-tracing subroutine to recompute at each iteration the PSFs of the system for all the considered object distances. Due to the communication time between the two software applications and simplex algorithm convergence speed, optimization took around one day. The optimal parameters obtained in this way are given in Table 1, column 1 $(J_{\mathrm{IQ}})$.

Table 1. Binary Phase Mask Optimal Parameters and Associated Optimal Value of Defocus Setting Parameter for ${\mathsf{J}}_{\mathsf{IQ}}$ (Column 1) and ${\mathsf{J}}_{\mathsf{SOD}}$ (Column 2)

View Table | View all tables in this article

We will now analyze two quantities characterizing the on-axis optical performance of this optimized imaging system, namely its PSF, denoted ${\mathrm{PSF}}_{{\psi }_k}^{\mathit{\theta }}$, and its modulation transfer function (MTF), denoted ${\mathrm{MTF}}_{{\psi }_k}^{\mathit{\theta }}$, which is the modulus of the Fourier transform of the PSF. The PSFs for the three object distances of 4.8 m, 9.6 m, and 10 km are represented in Figs. 1(a), 1(b), and 1(c), respectively. They are normalized so that $\iint {\mathrm{PSF}}_{{\psi }_k}^{\mathit{\theta }}(\mathbf{r}){\mathrm{d}}^2\mathbf{r}=1$. We observe that each of the three PSFs presents ringing induced by the presence of the binary phase mask in the stop plane of the lens. Among these three PSFs, the one at 9.6 m is the most spatially spread, as is shown more precisely by their horizontal cross sections in Fig. 1(d). However, it can be noted that the central peaks of each PSF are of quasi-similar width.

 

Fig. 1. Logarithmic scale (dB) display of hybrid system PSFs before deconvolution when $\psi$ is (a) 10 km, (b) 9.6 m, and (c) 4.8 m. (d) Linear scale PSF cross sections along $x$ and for $y=0$ including the PSFs’ maxima.

Download Full Size | PDF

The MTFs of the same system are represented in Fig. 2(a) for the same three object distances. The MTF curves are plotted up to $\nu =60\mathrm{cy}/\mathrm{mm}$, as the lens had initially been designed to resolve details up to this spatial frequency [8]. We also represented the evolution with spatial frequency of the normalized noise level $\zeta (\mathit{\nu })$, defined as

 

Fig. 2. MTF of the optical system including the binary phase mask that was optimized following (a) $J_{\mathrm{IQ}}$ and (b) $J_{\mathrm{SOD}}$. Global MTF after pixel filtering and deconvolution of the system with phase mask optimized following (c) $J_{\mathrm{IQ}}$ and (d) $J_{\mathrm{SOD}}$.

Download Full Size | PDF

From this analysis we can conclude that an optimization of the system that would only rely on optical response (PSF or MTF) computations, while implicitly taking into account that images will have to be deconvolved, should be steered according to the three following criteria:

In the following section, we will thus build along these lines a surrogate optimization criterion, denoted “SOD” for surrogate optical design, that can be implemented using the native quantities and merit function of optical design software. From the observations made in this section, this SOD criterion will specifically be based on the quasi-invariance of the PSF size and the MTF values as $\psi$ varies. The goal is to obtain MTF curves that are quasi-invariant with respect to $\psi$ while having the largest value possible.

B. SOD Criterion

In optical design software, the merit function to minimize is classically specified as a sum of weighted quadratic terms involving native physical quantities. Following the analysis made in the previous section, we chose to define the SOD objective function as follows:

The third $\sum$ term of Eq. (8) implements the new unconventional constraints (with weights $w_{\psi ,\nu ,\xi }^{\mathrm{\Delta }\mathrm{MTF}}$) we introduced in order to enforce that the diffractive-MTF curves should be as similar as possible between each other among the $\{{\psi }_k\}$ set. Each of their sub-terms is computed at a given spatial frequency $\nu$ and for a chosen angle of azimuth $\xi$ ($\xi =0°$ corresponds to the sagittal plane and $\xi =90°$ to the tangential plane). Several sub-terms with different spatial frequencies can be used in order to set several control points on MTF curves. It is also possible to set inequality constraints on MTF values, especially when the MTF are too low, close to, or below the normalized noise level. Such inequality constraints would ensure that the MTF is high enough to limit noise amplification after deconvolution, as described in sub-section 3.A, assuming that the degrees of freedom of the lens are sufficient to allow such quality to be reached. However, for the design problems considered in this paper, we found that implementing such inequality constraints was not necessary.

The fourth $\sum$ term of Eq. (8) implements new unconventional quasi-invariance constraints among the $\{{\psi }_k\}$ set on the spot diagram RMS diameter with weight denoted $w_{\psi }^{\mathrm{\Delta }\varphi }$. This term, together with the first term, also allows one to control indirectly but efficiently the overall behaviour of the MTF curves at all frequencies. It is thus a good complement to the third term of Eq. (8), which enforces the similarity of the MTF at discrete frequencies. During the optimization process, if $w_{\psi ,\nu ,\xi }^{\mathrm{\Delta }\mathrm{MTF}}$ and $w_{\psi }^{\mathrm{\Delta }\varphi }$ are high enough, the quadratic terms corresponding to MTF and spot size quasi-invariance compete with the usual minimization of the spot diagram RMS diameter.

The final merit function is a nonlinear quadratic function of $\mathit{\theta }$ that can be minimized with nonlinear least-squares algorithms. Code V uses by default for local optimization the Levenberg–Marquardt algorithm [20], also called damped least squares. The computer implementation of this optimization algorithm is highly optimized in commercial optical design software so that computation time is dramatically low. For example, the optimization of a conventional system of 6 optical elements over 10 iterations lasts a few seconds on present desktop computers. Even though global optimization is now available on commercial optical design software, the optical designer, instead of using random blind brute force, still usually leads the optimization process point to a well-behaved solution with her or his intuition, knowledge and skills. She or he uses a number of local optimization steps with slight adjustments of the system and/or the merit function weights to progressively shape the lens and its optical behavior to meet the specifications.

In our case, optimization of the $J_{\mathrm{SOD}}$ criterion is steered as follows. At the beginning of the optimization process, the weights $w_{\psi ,\nu ,\xi }^{\mathrm{\Delta }\mathrm{MTF}}$ are generally set to zero. The RMS spot size is minimized while adding quasi-invariance constraints over the RMS spot sizes for the different configurations, which should give the same optical behavior to later allow the deconvolution of the raw image by a unique kernel. This means that minimizing the spot size is in competition with setting the same spot size for every value of $\psi$. The values of weights $w_{\psi }^{\mathrm{\Delta }\varphi }$ should be quite small to obtain low ${\varphi }_{\psi }^{\mathit{\theta }}/\mathrm{high}$ MTF values, but they are non-zero in order to start with PSFs close to one another. As the design moves forward, the MTF quasi-invariance constraints at one or several spatial frequencies, that is, the weights $w_{\psi ,\nu ,\xi }^{\mathrm{\Delta }\mathrm{MTF}}$, are increased. They strongly influence the optimization path.

The design process is thus conducted in a way similar to classical lens design, where weighting coefficients are adjusted by the optical designer, with these adjustments depending on the obtained performance but also and strongly on the experience of the designer. The advantage of the proposed method is that the optimization environment as well as the optimization algorithm remain the same as in classical optical design. The only change is in the merit function, which mimics the behavior of $J_{\mathrm{IQ}}$ while using the native merit functions of the software.

C. Comparison of Imaging System Optimized with SOD and IQ Criteria

We will now validate the SOD criterion on a design example, that is, verify that optimization of SOD criterion leads to an imaging system similar to that obtained by optimizing $J_{\mathrm{IQ}}$. For that purpose, we consider again the optimization of the f/1.2 visible camera with the DOF-extending binary phase mask introduced in Section 3.A. The advantage of this optimization problem is that it can be solved by both $J_{\mathrm{IQ}}$ and $J_{\mathrm{SOD}}$, and it thus makes it possible to compare the optimal system parameters and image qualities obtained with both criteria.

It is, however, worth noting that diffractive optical elements (DOE) like the binary phase mask have nearly no effect on ray tracing and thus no pertinent effect on the spot diagram. As a consequence, the spot diagram of an optical system with DOE is no more representative of the PSF. This problem is specific to the $J_{\mathrm{SOD}}$ criterion. This is not the case for the $J_{\mathrm{IQ}}$ criterion, which is based on diffractive PSF computations and does clearly take properly into account the effect on the binary phase mask. Consequently, the $J_{\mathrm{SOD}}$ first and fourth $\sum$ terms in Eq. (8) involve ${\varphi }_{{\psi }_k}^{\mathit{\theta }}$ computations relying on pure ray tracing that does not see the DOE-induced phase steps. We, however, choose to keep the first $\mathrm{\Sigma }$ term with ${\varphi }_{{\psi }_k}^{\mathit{\theta }}$, as it serves to stabilize optimization convergence and the $\delta$ parameter value at the first design steps. We thus perform optimization with $w_{{\psi }_k}^{\mathrm{\Delta }\varphi }=0$, considering the fact that terms related to $\mathrm{\Delta }{\varphi }_{{\psi }_k}^{\mathit{\theta }}$ are unuseful in that case. Please note that this problem is specific to DOE-enhanced optical systems and does not at all impact the more common DOE-free optical systems like the example that will be presented in Section 4.

In Code V, we create three configurations of the system. Each of them corresponds to one of the three considered object distances {4.8 m, 9.6 m, 10 km}. All the optical parameters (except the object distances) and variables are common to them, and the three configurations will be optimized simultaneously. The starting point of the optimization is chosen arbitrarily without any a priori information on the final solution. The initial values are $r_1=1.0\mathrm{mm}$, $r_2=2.5\mathrm{mm}$, $r_3=4.0\mathrm{mm}$, $r_4=5.5\mathrm{mm}$, $r_5=7\mathrm{mm}$, and $\delta =7\mathrm{mm}$. In Code V, it is mandatory to constrain the radii values by setting the following inequality constraints: $r_1>0$, $r_5<R$, and $\forall \ell \in \{2,3,4,5\},r_{\ell }-r_{\ell -1}>0$. In this particular case, we used only two quadratic terms related to MTF quasi-invariance following Eq. (8), where ${\psi }_1=10\mathrm{km}$, ${\psi }_2=9.6\mathrm{m}$, and ${\psi }_3=4.8\mathrm{m}$. They are computed at spatial frequencies ${\mathit{\nu }}_1=20\mathrm{cy}/\mathrm{mm}$ and ${\mathit{\nu }}_2=60\mathrm{cy}/\mathrm{mm}$ so that we have a total of four MTF invariance constraints.

The parameters of the imaging system optimized in this way are presented in Table 1, column 2. We notice that the parameters of the mask are quite similar to those of the mask optimized with $J_{\mathrm{IQ}}$. The same observation holds for the optimal values of $\delta$. The MTFs obtained with both imaging systems are represented for comparison in Fig. 2(a) for the system optimized with $J_{\mathrm{IQ}}$ and in Fig. 2(b) for the system optimized with $J_{\mathrm{SOD}}$. We can see that for both systems, MTFs are quasi-invariant with respect to object distance. The MTF curves for 9.6 m or 10 km are particularly similar. The MTF curve for an object positioned at 4.8 m is slightly lower for the system optimized with $J_{\mathrm{SOD}}$ than for the other one. The MTFs after deconvolution for both systems are presented in Figs. 2(c) and 2(d). They are accordingly quasi-similar for distances of 9.6 m and 10 km. For the distance of 4.8 m, the curve of the system optimized with $J_{\mathrm{SOD}}$ is lower, as could be expected from the observed difference in MTF curves before deconvolution, but it remains well above the deconvolved noise level for all spatial frequencies until 60 cy/mm.

The simulated images, together with IQ values, corresponding to both $J_{\mathrm{SOD}}$- and $J_{\mathrm{IQ}}$-optimized systems are compared in Fig. 3, with respect to the object distance. Column 1 corresponds to the images given with the $J_{\mathrm{IQ}}$-optimized system before deconvolution, and column 2 corresponds to the images obtained with the $J_{\mathrm{SOD}}$-optimized system also before deconvolution. It is seen that for both systems, image qualities are quasi-uniform with respect to the object distance, and that the IQ value [defined at Eq. (6)] is also of similar values from one system to the other at a given object distance. The same observations hold after deconvolution, where the images given by the $J_{\mathrm{IQ}}$-optimized system (column 3) and the $J_{\mathrm{SOD}}$-optimized system (column 4) are all enhanced and of similar image qualities. The slight yet acceptable drop in IQ for the image given by the $J_{\mathrm{SOD}}$-optimized system at 4.8 m is directly related to the drop in MTF that was observed in Fig. 2(d).

 

Fig. 3. Simulated images with respect to the object distance provided before deconvolution by the system with the $J_{\mathrm{IQ}}$-optimized phase mask (column 1) and with the $J_{\mathrm{SOD}}$-optimized phase mask (column 2), and after deconvolution by the system with the $J_{\mathrm{IQ}}$-optimized phase mask (column 3) and with the $J_{\mathrm{SOD}}$-optimized phase mask (column 4).

Download Full Size | PDF

We can thus conclude that optimization with the surrogate criterion $J_{\mathrm{SOD}}$ provides imaging performance very close to that given by the rigorous image quality criterion $J_{\mathrm{IQ}}$. This surrogate criterion has several advantages. First, it has been designed to be fully implementable in an optical design software like Code V. It can thus be used like any conventional design method in the same environment familiar to the optical designer. Second, thanks to the use of native physical quantities in the merit function and to the efficiency of its optimization algorithm, the computation time has been dramatically reduced relatively to the direct $J_{\mathrm{IQ}}$ optimization. Indeed, for the considered example, the whole optimization process, including several local optimizations followed by modifications of the merit function weights by the designer, took only a dozen minutes, whereas the optimization of $J_{\mathrm{IQ}}$ took around one day with the Nelder–Mead simplex optimization algorithm and the communication between the numerical computation software steering the optimization process and the optical design software used for ray tracing. This point is very important, since in more complex optical design tasks, where all the lens surfaces have to be optimized with associated constraints, optimization of $J_{\mathrm{IQ}}$ with a numerical computation software is practically unfeasible, whereas the optimization of $J_{\mathrm{SOD}}$ inside the optical design software is possible while taking into account all the optical constraints that are required in optical design. The purpose of the next section is precisely to demonstrate the efficiency of the proposed method for the co-design of a complex lens.

4. CO-DESIGN OF A VERY FAST F/0.75 LENS WITH THE SOD CRITERION

In this section, our objective is to jointly optimize all the parameters of a complex lens, for which optimization of $J_{\mathrm{IQ}}$ would be very difficult to perform. We will consequently use the $J_{\mathrm{SOD}}$ optimization. We consider the design of a very fast f/0.75 lens with the following parameters: focal length of $f=24.7\mathrm{mm}$, maximum half-field of view (HFoV) of 20°, and sensor pitch $p=8\mathrm{\mu m}$ corresponding to Nyquist frequency ${\mathit{\nu }}_{\mathrm{Nyq}}=62.5\mathrm{cy}/\mathrm{mm}$. The design will be performed in the near-infrared spectral range $\lambda \in [600\mathrm{nm};900\mathrm{nm}]$.

Such a very high aperture is required when imaging in dim light conditions, where it is necessary to collect as many photons as possible. The lens considered in this section is aimed at equipping night vision goggles mounted on a soldier’s helmet. It has to be as light as possible, and its center of mass needs to be as close as possible to the helmet. Typical apertures for this type of optical system are currently f/0.95 [21]. Conventional design of such lenses is complex. Very large apertures induce optical aberrations with high-order and large amplitudes that are difficult to correct. Moreover, image quality becomes very sensitive to changes in optical parameters. Obtaining good imaging performance at f/0.75 while keeping low weight is thus a big challenge.

The potential added value of the co-design approach for this type of lens is to allow for a higher yet controlled amount of aberrations than conventional design, since these aberrations can be compensated for by digital post-processing. Allowing and controlling the presence of aberrations can relax some constraints on the shapes of lenses and thus enable simplifying and lightening the optimized lens. In the following, we will compare results of both conventional and joint optical designs.

A. Conventional f/0.75 Lens Design

A first conventional f/0.75 lens was designed by a skilled optical designer from Thales Angénieux. The optimization was performed in the traditional way by minimizing the RMS spot-diagram diameter. The design constraints were the following:

During optimization, performance assessment tools such as MTF or RMS spot diagram diameters were computed on a large spectral range [600 nm; 900 nm], and for the following horizontal field of view (HFoV) values $F_k=(k-1)\times 5°$ with $k\in \{1,\dots ,5\}$.

The obtained optical system is shown in Fig. 4. It consists of six optical elements denoted $\{L_k;k\in ⟦1,6⟧\}$ with one aspherical surface on the first surface of the front lens and a supplementary diaphragm placed in front of the system, which introduces vignetting in order to reduce the amplitude of field aberrations. The diaphragm consequently induces a drop in relative illumination across the field. It is seen in Table 2 that this illumination drops to 30% at maximal HFoV. The barrel distortion amplitudes with respect to the field are given in Table 2, column 2. The maximal value is 2% at the 20° field.

 

Fig. 4. Conventional f/0.75 lens.

Download Full Size | PDF

Table 2. Relative Illumination and Distortion

View Table | View all tables in this article

Images of optical PSFs and the values of the RMS spot diameters are given in Fig. 5 for the following HFoV values: on-axis, intermediate field of 10°, and maximum field of 20°. We observe on the images that the PSF size increases rapidly with the field and the RMS spot size increases accordingly. The PSF is wider in the sagittal plane (i.e., following the horizontal $x$ axis for fields in the $y$ direction) than in the tangential plane, which is a characteristic feature of astigmatism. Correspondingly, it is observed in Fig. 6 that MTF values for low HFoV (0° and 5°) are high and similar to each other, reaching 0.7 at ${\nu }_{Nyq}$. They decrease with the field but stay higher than 0.35 at ${\nu }_{Nyq}$ in the tangential plane, as specified in the design constraints. However, for high HFoV, sagittal MTF values are much lower than tangential MTF values, the sagittal MTF taking zero value at 55 cy/mm. Again, this asymmetry between tangential and sagittal MTF is typical of coma and astigmatism.

 

Fig. 5. Logarithmic scale (dB) display of polychromatic PSF with associated RMS spot size ${\mathrm{\varphi }}_{\mathrm{RMS}}$. Conventional system: (a) on-axis, (c) for intermediate HFoV of 10°, and (e) for maximum HFoV of 20°. Co-designed system: (b) on-axis, (d) for HFoV of 10°, and (f) for HFoV of 20°.

Download Full Size | PDF

 

Fig. 6. MTF in tangential plane (plain curve) and sagittal plane (dots) for (a) the conventional optical system and (b) the co-designed system. MTFs after deconvolution for (c) conventional system and (d) co-designed system.

Download Full Size | PDF

The weights of the optical elements have been estimated using the “WEI” command of Code V, which takes into account the volumes and densities of each lens. The results are given in Table 3, column 1. The heaviest elements of the system are the front lens (L1), the third convergent lens (L3) and the second lens of the doublet (L5). Since the weight of L1 is due to its large diameter required by the system high aperture, it may be difficult to reduce it. In contrast, L3 and L5 weights may be modified by reducing their widths and curvatures, while ensuring that the mass center stays close to the image plane.

Table 3. Estimated Weight of Each Individual Lens Composing the Conventional System (Column 1) and the Co-Designed System (Column 2) Using the Code V WEI Command

View Table | View all tables in this article

B. Co-Designed f/0.75 Lens

Our purpose is now to reduce the weight of the lens while keeping at least equivalent performance by using a co-design approach. For that purpose, we will optimize the SOD criterion defined in Eq. (8). Since we want to obtain a good performance uniformity in the field, the parameter $\psi$ to which we require the performance to be invariant is the field angle, that is, $\psi \in \{F_1,\dots ,F_5\}$. The optimization variables $\mathit{\theta }$ are the curvature radii of all surfaces, the width of all optical elements, air spacing between lenses, and potential aspherical coefficients. The distance between the last optical surface and the image plane is set to 0.1 mm. The number of optical elements is not changed during the optimization, and will remain equal to 6. The lens configuration used as the starting point of the co-design optimization is the conventionally optimized lens represented in Fig. 4.

At the beginning of the optimization of $J_{\mathrm{SOD}}$, MTF weights $w_{\psi ,\nu ,\xi }^{\mathrm{\Delta }\mathrm{MTF}}$ are set to zero. The terms related to quasi-invariance of the RMS spot size [fourth term of Eq. (8)] are computed with respect to the reference field, i.e., ${\psi }_1=F1=0°$. Once the MTF curves start to take off from nearly zero values at medium spatial frequencies due to the standard optimization criterion and are sufficient close to one another due to the ${\varphi }_{{\psi }_k}^{\mathit{\theta }}$ quasi-invariance unconventional constraints, terms related to MTF quasi-invariance are added. They are computed for the two azimut angles $\xi =0°$ and $\xi =90°$, and at a given spatial frequency that evolves during optimization. Associated weights are also evolving once the MTFs are stabilized.

The lens resulting from this optimization keeps a shape similar to the conventional one, with one aspherical surface on the front lens. As shown in Table 2 (second column), relative illumination stays above 34.5%, and barrel distorsion amplitude is low and under 0.43%. The PSF images and corresponding RMS spot sizes are given in Fig. 5 (right column). After optimization of $J_{\mathrm{SOD}}$, the RMS spot sizes are quite uniform in the field and much more symmetric. As expected, the MTFs represented in Fig. 6 (right column) are also very similar to each other, whether on the sagittal or tangential plane. Only the MTF curves at 20° are further apart, with a noticeable difference between tangential and sagittal curves that reaches a maximal value of 0.1 at low spatial frequency (20 cy/mm).

We have represented the global MTF after deconvolution obtained for both conventional and co-designed systems in Figs. 6(c) and 6(d), respectively. To compute the Wiener filter, we have assumed that the input SNR is equal to 34 dB. For the conventional lens, the average Wiener filter has the same closed-form expression as for the co-designed lens, but it is computed using the OTFs of the conventional lens. It is observed in Fig. 6 that global MTFs are globally enhanced, except where the original MTF curves are close to or lower than the noise level. The global MTFs of the co-designed system are again quasi-invariant with respect to HFoV and azimuth angle. As their values before deconvolution were high enough and above the noise level, they are all significantly enhanced and are comparable to an ideal system with the same optical characteristics.

Simulated images corresponding to conventional and co-designed systems, before and after deconvolution, are given in Fig. 7 together with the IQ value, for different values of HFoV. The first column corresponds to images obtained with the conventional lens. It is seen that image quality sharply decreases with the field. For instance, the roots on the pot become blurry for intermediate and high HFoV. On the contrary, it can be seen in the second column that the co-designed lens provides images of very similar quality all over the field. Thanks to this property, the quality of images from the co-designed lens after deconvolution (fourth column) is excellent for all values of the field. It is much better than the deconvolved image from the conventional lens (third column) at a 20° field angle, whereas it is similar for the two other values of the field.

 

Fig. 7. Simulated images with respect to HFoV $Fk$, provided before deconvolution by the conventional system (column 1) and by the co-designed system (column 2), and provided after deconvolution by the conventional system (column 3) and by the co-designed system (column 4).

Download Full Size | PDF

The weights of the different elements of the co-designed lens are given in Table 3, column 2. The total mass of the co-designed system is 61.8 g, that is, 10 g less than that of the conventional system. The third lens weight has been significantly reduced and is the main contributor to the weight reduction of the system. We also note that the center of mass is closer to the sensor plane.

5. CONCLUSION

We have proposed a surrogate criterion (SOD) to replace the rigorous mean-square-error-based image quality (IQ) criterion in order to efficiently perform joint digital-optical design with commercial optical design software. This SOD criterion has been validated on the optimization of a phase mask for depth of field extension, and the obtained performance has been shown to be quite similar to that obtained by optimizing the rigorous IQ criterion. The new criterion has then been used for the design of a very fast (f/0.75) complex lens. This co-designed lens has been found to be superior to a classically designed lens in terms of lens weight and homogeneity of image quality in the field.

This surrogate criterion has several advantages. First, it is easily implementable in optical design software like Code V or Zemax, as it makes use of a merit function based on the default one completed by easy-to-compute terms involving natively implemented physical quantities and parameters that are well known to optical designers. Second, and as a direct consequence of the first point, it can fully exploit the efficiency of optical design software optimization algorithms. The computation time can thus be dramatically reduced and made comparable to that of classical designs.

In this paper, the performance of the SOD criterion has been illustrated on only a few design examples. The main perspective of this work is to validate this approach on many other design problems with more complex lenses and/or other design trade-offs. Of course, depending on the problem, the suitable form of the SOD criterion may be different, involving different combinations of native terms in the merit function. However, the philosophy should remain the same: to introduce as smoothly as possible the co-design approach into the classical optical design process in order to make the most out of optical designers’ skills and art and of digital processing.