Open Access
Add to BibSonomyAbstract
This paper explores practical design considerations for selecting Q for an electro-optical earth imaging system, where Q is defined as (λ FN) / pixel pitch. Analytical methods are used to show that, under imaging conditions with high SNR, increasing Q with fixed aperture cannot lead to degradation of image quality regardless of the angular smear rate of the system. The potential for degradation of image quality under low SNR is bounded by an increase of the detector noise scaling as Q. An imaging test bed is used to collect representative imagery for various Q configurations. The test bed includes real world errors such as image smear and haze. The value of Q is varied by changing the focal length of the imaging system. Imagery is presented over a broad range of parameters.
© 2013 Optical Society of America
1. Introduction
Design considerations for selecting the ratio of focal length to pixel size for electro-optical imaging systems were explored in detail by Fiete [1]. Increasing the value of Q with fixed aperture size leads to a reduced ground-sample-distance (GSD), the dominant term that improves image quality. However, due to optical MTF, signal-to-noise (SNR), and smear considerations, the image quality improvement of increasing Q is not as substantial as that implied by the GSD reduction.
In general, selecting the value of Q for an imaging system is a trade between image quality and image collection rate. In addition, the system architect must ensure that increasing Q does not reduce image quality [1] describes how increasing Q results in increased smear in the imagery. Due to the higher smear, increasing Q may lead to degraded image quality. Ensuring that this does not happen is a trade between the exposure time and the angular smear rate of the system.
Realizing a significant image quality improvement for high Q systems requires excellent line-of-sight control. In addition, a significant image quality improvement usually requires a longer exposure. As a result, line scanning systems may require more time delay integration and/or slower line rates. More TDI may be difficult and costly to manufacture, and slower line rates impact collection performance.
Higher Q systems have more pixels across a target than lower Q systems. To compensate, more pixels must be added to the sensor, or the image area is reduced. The increased data associated with more pixels may impact the bandwidth requirements for the system. There may also be costs associated with manufacturing a sensor with more pixels. Depending on the requirements of the system, the increased costs associated with having more pixels may not be worth the image quality improvement. Alternatively, if the number of pixels does not increase, the system architect must trade image quality and image collection.
This paper explores practical design considerations for selecting Q for an electro-optical imaging system. Attention is restricted to the selection of Q for a fixed aperture size or, equivalently, the ratio of focal length to pixel size. This ratio determines the GSD relative to the optical PSF projected onto the object. The goal is to provide insight into the following two questions: does increasing Q necessarily result in better image quality, and is the resulting image quality improvement worth the loss in image area? The first issue is addressed using analytical means by demonstrating a design rule-of-thumb that ensures that increasing Q does not reduce image quality. It is shown that careful selection of the exposure time prohibits an image quality reduction when increasing Q provided that Q ≤ 2.
The issue regarding the worthiness of increased Q is addressed using image data. An electro-optical imaging test-bed at the Lockheed Martin Advanced Technology Center in Palo Alto, CA is used to explore image quality as a function of Q. The test-bed is designed to collect imagery for Q values between 1 and 2. Q is varied by either changing a variable diameter pupil mask inserted at the system exit pupil or by changing the focal length of the imaging system. There is also a capability to introduce well-controlled smear into the test-bed. The test-bed is used to collect data at variable levels of Q, GSD, SNR, and smear. The imagery can be used as a guide to aid the selection of Q, namely to help determine if the image quality improvement of increasing Q is worth the area collection rate reduction.
2. Bounding the risk of increasing Q
A result in [1] suggests that increasing Q with fixed aperture can result in reduced image quality despite the decrease in GSD. In this example, a theoretical Q = 1 system has 1 pixel of smear for a given exposure time. As Q is increased, the exposure time is increased by a factor of Q2 to produce constant signal-to-noise. Under these assumptions, the smear in units of pixels increases as Q3. For this scenario, the image quality peaks at Q slightly less than 1.3.
The example in [1] provides valuable insight into the trade considerations associated with the selection of Q. However, the result is valid only for the assumption that exposure time is increased to hold SNR constant. In the example, the GSD improved and the SNR was held fixed, but the substantial increase in smear more than offset the image quality improvement from the GSD. An interesting question is whether or not a different exposure time relationship between low and high Q systems can be found that prohibits an image quality loss with increasing Q.
The question can be answered using the concept of signal-to-noise in the frequency domain. Consider the image i(x) of an object o(x), where x denotes pixel location. Only one dimension is shown for simplicity. If the system point spread function is h(x), and the image noise is n(x), then the image is
where ⊗ denotes convolution. The Fourier Transform of the image, denoted by the ~ symbol, is given bywhere υ is a spatial frequency coordinate and OTF(υ) is the optical transfer function of the imaging system. The frequency domain signal-to-noise metric [2] is defined by identifying in Eq. (2) the absolute value of convolution of the object with the OTF as the signal:where MTF(υ) is the modulation transfer function. For an NxN pixel image with detector read noise σ, Eq. (3) can be shown to be [3]where P is the total number of signal photoelectrons in the image and Phaze is the total number of haze photoelectrons. Haze is defined as non-imaged or background photoelectrons. The following is a reasonable statement regarding frequency domain SNR: if at all spatial frequencies in an image the SNR is increased or held equal, then the resulting image quality and interpretability is improved or held equal.The image quality impacts of increasing Q are complex. The improvement to image quality is due to the decreased GSD, but there is also a degradation of image quality due to decreased optical MTF relative to the GSD, increased sensitivity to smear, and decreased spatial SNR. Furthermore, if the exposure time is increased to improve the spatial SNR, then there is an additional increased smear. The complexity of the interactions often makes it difficult to clearly understand and quantify the risks of higher Q systems. A simple way to clarify these issues is to consider an idealized system with a noiseless detector. For this case, it can be shown that image quality and interpretability cannot be degraded if Q is increased, provided that exposure time is held constant relative to the lower Q system.
To prove this claim, consider a baseline Q = Q0 system. Assume that Q is increased by decreasing the pixel size. Decreasing pixel size simplifies the argument, but it can be shown that increasing focal length is equivalent. Also assume that, as Q is increased, the same field-of-view is imaged. As a result, the number of pixels in the higher Q image increases by the factor (Q/Q0)2. Other than the pixel size and the number of pixels, the two systems are the same in every other way including their angular smear rates. Again, the domain is restricted to Q≤2.
The optical cutoff frequency for an imaging system is υcut = 1/λFN, where FN is the f-number. The information content in a Q≤2 image also depends on the spatial sampling rate (i.e., pixel size p). The Nyquist frequency for the image is defined as υNyquist = 1/2p. Inserting the definition of Q into the equation for optical cutoff frequency yields the following:
In the spatial domain, the image quality improvement associated with increasing Q is described by the reduction in GSD. In the frequency domain, the image quality improvement can be described in the following way: increasing Q increases the Nyquist frequency, so more spatial frequencies are not aliased and contribute productively to image quality.To demonstrate that increasing Q cannot degrade image quality for a noiseless detector, consider two spatial frequency domains. The first domain consists of frequencies less than the Nyquist frequency for the Q0 system υ0. It is shown that the SNR at these frequencies does not degrade if Q is increased. As a result, the information present in the Q0 image is in not compromised by increasing Q. The second domain consists of frequencies greater than υ0, i.e. the spatial frequencies passed only by the higher Q system. It is argued that the presence of these higher spatial frequencies in the image cannot corrupt the information in the lower frequencies contained in the original Q0 image. This is true regardless of the SNR of the higher frequencies.
For a particular set of imaging conditions, assume that an ideal exposure time τ is used for the Q0 system. Assume that the same exposure time is also used for the higher Q system. Since the field-of-view is imaged by both systems, the same total numbers of signal and haze photons are collected for each system. Using Eq. (4), the SNR ratio between the higher and lower Q systems for spatial frequencies less than the υ0 is given by
where p0 is the pixel size in the Q0 system. The only change to the SNR is associated with the MTF of the detector footprint. Since the pixel size for the higher Q system is smaller, the ratio of the SNR is > 1. Equation (6) shows that the SNR for all spatial frequencies less than υ0 are slightly improved for higher Q.For the higher Q system, it is reasonable to argue that extremely poor SNR at spatial frequencies higher than υ0 may corrupt the perceptibility of the lower spatial frequency information. If this is the case, then a simple low-pass digital filter can be used to remove the higher spatial frequency content from the image. In the higher Q image, a top-hat filter that passes only spatial frequencies less than υ0, in conjunction with Eq. (6), guarantees an image no worse than the original Q0 image. Such a filter is, of course, not recommended in practice, but the argument proves that increasing Q in the absence of detector read noise cannot possibly make an image worse regardless of the angular smear rate. Of course, no detector is noiseless, but the preceding arguments hold true under high SNR conditions where the detector noise is negligible compared to the photon noise from both the signal and haze.
The detector noise is the only system parameter that allows for the possibility of image quality degradation with higher Q. For spatial frequencies below υ0, the SNR ratio between the higher and lower Q system is given by:
where N02 is the number of pixels in the Q0 image. To remove the dependence on image size, let l be the average number of signal and haze photoelectrons per pixel in the Q0 image. Equation (7) becomesEquation (8) shows that, for frequencies below υ0, the SNR may degrade as Q increases because, effectively, the detector noise increases by the factor Q/Q0.To summarize, the risk to image quality due to increasing Q can be bounded by making the following conservative simplifications. Consider only frequencies below υ0 by assuming that any image can be low-pass filtered to produce an image with effective Q = Q0. Also, ignore the MTF associated with the detector footprint, a conservative assumption because the footprint MTF is better for a higher Q system. Assuming a Q0 system with detector noise σ0, the image quality and interpretability of a higher Q image is no worse than the Q0 image with detector noise equal to the following:
3. Experiment description
A full description of the experimental testbed has been given in [4]. We will give a brief refresher in this paper to facilitate understanding of the results.
We have designed a testbed that will allow variation of exposure times, signal to noise ratio, angular smear rates, focal lengths, haze levels, wavelength band and Q. The testbed consists of two identical telescopes pointing at one another through a common pupil. The pupil consists of a flat steering mirror that sends the beam from one telescope into the other. One telescope is the scene projector and the other acts as the imaging telescope. Various filters and additional light sources allow variation of wavelength band and noise levels. Multiple aperture masks of different sizes allow the aperture of the imaging telescope to be varied as a means of varying Q. A set of lenses acts as a zoom system and is used to vary the focal length of the imaging system.
The zoom system is illustrated in Fig. 1. It consists of a pair of off-the-shelf doublets designed to produce a 1-2x magnification capability on the imaging telescope leg while providing diffraction limited performance across the visible wavelengths and keeping the image plane at a fixed location. The doublets are mounted on a set of stages that translate them along the optical axis producing repeatable movement to the correct positions to give desired zoom settings. The stage positions are calibrated to the desired levels by adjusting the doublet pair to optimize the focus of an image across a range of zoom settings, and then directly measuring the detector sampling at each setting using a pupil mask with small holes of known spacing.
Fig. 1 The zoom lens for varying the system focal length is shown in the 1x and 2x configuration. The positions of the two doublets are varied to adjust the system focal length.
Smear is introduced using a PZT tip/tilt stage driving a small fold mirror that directs the optical path to the zoom lenses and camera. It is driven by a signal generator that produces a saw-tooth waveform of a period synchronized to the camera data collection rate. The amplitude of the smear in the imagery shown here is calibrated to produce 0.8 pixels for the 1x zoom case at the nominal integration time.
The imaging focal plane is a Kodak KAF1401E CCD with 1317 by 1035 pixels and a 6.8 um pixel pitch. The camera is a Roper Scientific camera and the Kodak chip is thermoelectrically cooled to reduce read noise and dark current. This pixel pitch allows a Q range of 1 to 2 to be easily explored with the various pupil masks.
4. Test bed characterization
4.1 Zoom calibration
The aperture masks are highly controllable, but the zoom lens must be calibrated. For example, the magnification of the Q = 1.0 is actually 0.98. Measurements of the number of pixels across fixed targets in the imagery shows that the other Q values for the 1.5 and 2.0 cases are actually 1.51 and 1.95 respectively.
4.2 MTF measurements
An MTF model of the imaging system on standard Fourier Optics theory [6] was developed. A digital sampled pupil map is created, and the Fourier Transform of map is calculated using the ‘fft2’ command in MATLAB. The normalized square of the result is the optical point spread function (PSF) of the system. The sampling rate of pupil map is selected such that the sampling rate of PSF is equal to the pixel size of the science camera, or ½ of the final image pixel size. Wavefront error is added to the simulation using methods described in [6]. In addition, the detector MTF is added to the model using the following simplified equation:
In Eq. (10), α is a constant used to represent detector diffusion and other detector effects, and the sinc function represents the detector footprint. The spatial frequency υ is in units of cycles per pixel. In the system MTF model, the optical MTF and detector MTF are multiplicative.The MTF of the imaging system is measured from imagery of a knife edge target. MTF measurements were taken at three zoom levels corresponding to Q = 0.98, 1.51, and 1.95. The results shown in Fig. 2 are in reasonable agreement with the model and suggest that the zoom lens accurately changes Q without introducing substantial wavefront error.
4.3 Smear calibration and validation
The test bed introduces variable levels of controllable smear into the imagery. The levels of smear are validated by measuring the effect of the smear on the system MTF. Smear is measured for several values of Q. For these measurements, the smear rate of the PZT mirror is set to produce a prescribed amount of smear for the Q = 0.98 imagery. Q is then varied using the zoom lens, and the exposure time scales as Q2. The smear in the imagery is then estimated from the MTF measurements at each Q level. The measurements are summarized in Table 1. The estimated smear differed from the input smear by an additive bias of approximately 0.4-0.5p of smear, but the scaling of the estimated smear across the Q values is comparable to theory, i.e. that smear scales as Q3.
Table 1. Results of smear characterization and validation.
5. Image processing
For this experiment, it is critical to sharpen the imagery in an un-objective and repeatable way. The imagery is sharpened using a Wiener Filter based on the modeled system MTF and the measured SNR of the imagery. An estimate of the object oe(x) is recovered from the image using the following filtering process in the frequency domain:
The Wiener Filter Φ(υ) is given by [7]In Eq. (12), c is a constant and SNR(υ) is given by Eq. (4). The value c = 1.0 is used for this study.Implementation of Eq. (12) requires an estimation of the object spectrum. It has been shown that the amplitude spectrum of objects tends to follow, on average, a power scaling law [8]:
where α typically near 1 for many scenes. The constant α can be estimated from the imagery. Inserting Eq. (13) into Eq. (2), ignoring the noise term, and taking the logarithm givesUsing the model for MTF previously described, an estimate of α is obtained by performing a linear fit to the logarithm of the Fourier Transform of the image with the logarithm of the modeled MTF subtracted from the data. The fit is performed using the radially averaged image amplitude spectrum, and the fit is only performed over spatial frequencies where the signal (object multiplied by the MTF) is substantially larger than the noise.An example of the amplitude spectrum estimation is shown in Fig. 3. The object amplitude of a Q = 1.95 image is estimated by linearly fitting the logarithm of the image minus the logarithm of the MTF. It is possible to identify the spatial frequency where noise begins to dominate because subtracting the logarithm of the MTF produces a spike. In this example, the noise dominates starting at approximately log10(υ) = −0.5, so the fit is done only for spatial frequencies less than that. For this example, the estimate of α is 1.16.
It is worth noting that the Wiener Filter used for this study does not correct for the smear. It is assumed that on a typical electro-optical imaging system, smear is a random process that is not measured and cannot easily be worked into the PSF for image sharpening. An example image sharpening result for Q = 1.95 with no smear is shown in Fig. 4.
5. Results
5.1 Data overview
Imagery is collected with several values of Q, SNR, and smear. Q is varied using the zoom lens. The SNR and smear values are set for Q = 1, and then the illumination levels and angular smear rate are then held constant for all Q. As Q varies, the exposure time is either held fixed for variable SNR cases or increased as Q2 for a fixed SNR cases.
The scene illumination, haze, and exposure time are selected to provide several SNR levels. The SNR is measured from the white region of the MTF targets, where the pixels are 50% transmissive. The approximate ratio of signal to haze for a 100% transmissive pixel and the measured SNR for a 50% transmissive pixel are shown in Table 2. Image examples for high and low SNR levels for Q = 1 are shown in Fig. 5.
Table 2. Summary of SNR levels.
5.2 Image results
Imagery is presented in this section for variable Q and fixed aperture size at high and low SNR. Q is varied using the zoom lens. Imagery is collected with exposure time held fixed for all Q as well as scaling as Q2 (“variable”). Imagery is also collected with no smear and at an angular smear rate corresponding to ~0.8 pixel in the Q = 1 imagery.
All imagery shown here has been sharpened using the Wiener Filter. The image size for the Q = 2.0 imagery is 482x628. For Q<2.0, the images are shown at equal magnification, so the image sizes in pixels are smaller by a factor of Q/2.0.
Figures 6 and 7 show imagery at high SNR with no smear with variable and fixed exposure. At high SNR, leaving the exposure fixed had only a minor impact to the image quality improvement with increasing Q. The same cannot be said at low SNR as shown in Fig. 8-9. The image quality loss for holding integration time fixed is noticeable at high Q, but both the Q = 1.51 and 1.95 images are better than the Q = 0.98 image even with fixed integration time.
Figure 10 is an experimental version of the previously described result in [1]. The imagery is collected with an angular smear rate, horizontal to the image, corresponding to ~1 pixel of smear in the Q = 0.98 image. Depending on the orientation of the features in the scene, the higher Q images are worse than the Q = 0.98 image if the exposure scales with Q, and this is in agreement with [1]. However, Fig. 11 shows the same conditions with exposure held fixed. In this example, both of the higher Q images are better than the Q = 0.98 image.
Figures 12 and 13 repeats the previous experiment at low SNR. For this example, fixing the exposure time across Q again produces better image quality for the higher Q examples. The Q = 1.51 image is better than the Q = 0.98 image, but it is unclear if the Q = 1.95 image is better. The imagery shows a resolution improvement with a spatial SNR degradation. It is difficult to clearly trade the two, so the choice of which image is better is subjective. To conclusively prove that the Q = 1.95 image is not worse than the Q = 0.98 image, a low pass filter is applied prior to the Wiener Filter. The image is then sharpened using the Q = 0.98 Wiener Filter. The resulting image is very similar to the actual Q = 0.98 image. This is in agreement with the assertion in this paper that increasing Q cannot reduce image quality, even with high smear and low spatial SNR.
Fig. 13 Low SNR, 0.8p Smear, Fixed Exposure (CW from upper left: Q = 0.98, 1.51, 1.95, 1.95 low-pass filtered)
6. Summary
The theory and image data presented here demonstrate that image quality cannot degrade when increasing Q with fixed aperture. This assumes that the exposure can be held fixed as Q increases, although no claim is made that this is the optimal exposure relationship for all systems. The results hold true for spatial SNR conditions where the detector noise is not the dominate noise term, an assumption true for most systems under reasonable conditions. In the event that detector noise dominates, there may be a degradation of image quality with higher Q consistent with an increase in the detector noise scaling as Q. The imagery shown here suggests that increasing Q does not present a risk to image quality. The user of a higher Q system can expect a reasonable image quality improvement under good SNR conditions, and little-to-no improvement under poor SNR conditions, depending on the angular smear rate of the system. The most serious risk to higher Q systems is that the image quality improvement is not worth the loss of area collection rate. The imagery shown here can be used to help assess the worthiness of a Q increase for an electro-optical imaging system.
References and links
1. R. D. Fiete, “Image quality and λFN/p for remote sensing systems,” Opt. Eng. 38(7), 1229–1240 (1999). [CrossRef]
2. R. D. Fiete and T. A. Tantalo, “Comparison of SNR image quality metrics for remote sensing systems,” Opt. Eng. 40(4), 574–585 (2001). [CrossRef]
3. J. R. Fienup, “MTF and Integration Time versus Fill Factor for Sparse-Aperture Imaging Systems,” in Imaging Technology and Telescopes, edited by J. W. Bilbro, J. B. Breckinridge, R. A. Carreras, S. R. Czyzak, M. J. Eckart, R. D. Fiete, P. S. Idell, Proc. SPIE 4091 (2000).
4. R. L. Kendrick, A. Cochrane, K. Schulz, R. Bell, and E. Smith, “Image Quality effects due to image plane sampling: Experimental Results,” in Sensors, Systems, and Next-Generation Satellites XV, Proc. SPIE , 8176 (2011).
5. A. T. Cochrane, G. C. Robins, G. J. Baker, R. M. Bell, V. G. Zarifis, and B. J. Herman, “Practical aspects of image system validation using transillumination,” in Modeling, Systems Engineering, and Project Management for Astronomy II, edited by M. J. Cullum, G. Z. Angeli, Proc. SPIE 6271 (2006).
6. J. W. Goodman, Introduction to Fourier Optics, Second Edition (The McGraw-Hill Companies, Inc., 1996), Chap. 6.
7. W. H. Press, B. P. Flannery, S. A. Teukolsky, and W. T. Vetterling, Numerical Recipes, The Art of Scientific Computing, Second Edition (Cambridge University Press, 1997), Chap. 13.
8. D. J. Field, “Relations between the statistics of natural images and the response properties of cortical cells,” J. Opt. Soc. Am. A 4(12), 2379–2394 (1987). [CrossRef] [PubMed]
