Abstract

Focus and depth of field are conventionally addressed by adjusting longitudinal lens position. More recently, combinations of deliberate blur and computational processing have been used to extend depth of field. Here we show that dynamic control of transverse and longitudinal lens position can be used to decode focus and extend depth of field without degrading static resolution. Our results suggest that optical image stabilization systems may be used for autofocus, extended depth of field, and 3D imaging.

© 2015 Optical Society of America

Full Article  |  PDF Article
OSA Recommended Articles
Imaging properties of extended depth of field microscopy through single-shot focus scanning

Sheng-Huei Lu and Hong Hua
Opt. Express 23(8) 10714-10731 (2015)

Coding for compressive focal tomography

David J. Brady and Daniel L. Marks
Appl. Opt. 50(22) 4436-4449 (2011)

Single shot three-dimensional imaging using an engineered point spread function

René Berlich, Andreas Bräuer, and Sjoerd Stallinga
Opt. Express 24(6) 5946-5960 (2016)

References

  • View by:
  • |
  • |
  • |

  1. D. J. Brady, in Optical Imaging and Spectroscopy (Wiley, 2009).
  2. D. Miau, O. Cossairt, and S. K. Nayar, in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2013).
  3. E. R. Dowski and W. T. Cathey, Appl. Opt. 34, 1859 (1995).
    [Crossref]
  4. K. Venkataraman, D. Lelescu, J. Duparré, A. McMahon, G. Molina, P. Chatterjee, R. Mullis, and S. K. Nayar, ACM Trans. Graphics 32, 1 (2013).
    [Crossref]
  5. C. Zhou, S. Lin, and S. K. Nayar, in IEEE 12th International Conference on Computer Vision (IEEE, 2009), pp. 325–332.
  6. A. Levin, R. Fergus, F. Durand, and W. T. Freeman, ACM Trans. Graphics 26, 70 (2007).
    [Crossref]
  7. S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, Proc. Natl. Acad. Sci. USA 106, 2995 (2009).
  8. R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, Comput. Sci. Tech. Rep. 2, 11 (2005).
  9. P. Llull, X. Yuan, X. Liao, J. Yang, L. Carin, G. Sapiro, and D. J. Brady, in Computational Optical Sensing and Imaging (Optical Society of America, 2014), paper CM2D-3.
  10. X. Lin, J. Suo, G. Wetzstein, Q. Dai, and R. Raskar, in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2013).
  11. D. J. Brady and D. L. Marks, Appl. Opt. 50, 4436 (2011).
    [Crossref]
  12. S. Quirin and R. Piestun, Appl. Opt. 52, A367 (2013).
    [Crossref]
  13. A. A. Wagadarikar, N. P. Pitsianis, X. Sun, and D. J. Brady, Opt. Express 17, 6368 (2009).
    [Crossref]
  14. T.-H. Tsai and D. J. Brady, Appl. Opt. 52, 2153 (2013).
    [Crossref]
  15. P. Llull, X. Liao, X. Yuan, J. Yang, D. Kittle, L. Carin, G. Sapiro, and D. J. Brady, Opt. Express 21, 10526 (2013).
    [Crossref]
  16. Y. Bando, B.-Y. Chen, and T. Nishita, Comput. Graphics Forum 30, 1869 (2011).
    [Crossref]
  17. A. Mohan, D. Lanman, S. Hiura, and R. Raskar, in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2009).
  18. D. Krishnan and R. Fergus, in Advances in Neural Information Processing Systems (NIPS, 2009), pp. 1033–1041.

2013 (4)

2011 (2)

Y. Bando, B.-Y. Chen, and T. Nishita, Comput. Graphics Forum 30, 1869 (2011).
[Crossref]

D. J. Brady and D. L. Marks, Appl. Opt. 50, 4436 (2011).
[Crossref]

2009 (2)

A. A. Wagadarikar, N. P. Pitsianis, X. Sun, and D. J. Brady, Opt. Express 17, 6368 (2009).
[Crossref]

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, Proc. Natl. Acad. Sci. USA 106, 2995 (2009).

2007 (1)

A. Levin, R. Fergus, F. Durand, and W. T. Freeman, ACM Trans. Graphics 26, 70 (2007).
[Crossref]

2005 (1)

R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, Comput. Sci. Tech. Rep. 2, 11 (2005).

1995 (1)

Bando, Y.

Y. Bando, B.-Y. Chen, and T. Nishita, Comput. Graphics Forum 30, 1869 (2011).
[Crossref]

Biteen, J. S.

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, Proc. Natl. Acad. Sci. USA 106, 2995 (2009).

Brady, D. J.

Brédif, M.

R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, Comput. Sci. Tech. Rep. 2, 11 (2005).

Carin, L.

P. Llull, X. Liao, X. Yuan, J. Yang, D. Kittle, L. Carin, G. Sapiro, and D. J. Brady, Opt. Express 21, 10526 (2013).
[Crossref]

P. Llull, X. Yuan, X. Liao, J. Yang, L. Carin, G. Sapiro, and D. J. Brady, in Computational Optical Sensing and Imaging (Optical Society of America, 2014), paper CM2D-3.

Cathey, W. T.

Chatterjee, P.

K. Venkataraman, D. Lelescu, J. Duparré, A. McMahon, G. Molina, P. Chatterjee, R. Mullis, and S. K. Nayar, ACM Trans. Graphics 32, 1 (2013).
[Crossref]

Chen, B.-Y.

Y. Bando, B.-Y. Chen, and T. Nishita, Comput. Graphics Forum 30, 1869 (2011).
[Crossref]

Cossairt, O.

D. Miau, O. Cossairt, and S. K. Nayar, in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2013).

Dai, Q.

X. Lin, J. Suo, G. Wetzstein, Q. Dai, and R. Raskar, in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2013).

Dowski, E. R.

Duparré, J.

K. Venkataraman, D. Lelescu, J. Duparré, A. McMahon, G. Molina, P. Chatterjee, R. Mullis, and S. K. Nayar, ACM Trans. Graphics 32, 1 (2013).
[Crossref]

Durand, F.

A. Levin, R. Fergus, F. Durand, and W. T. Freeman, ACM Trans. Graphics 26, 70 (2007).
[Crossref]

Duval, G.

R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, Comput. Sci. Tech. Rep. 2, 11 (2005).

Fergus, R.

A. Levin, R. Fergus, F. Durand, and W. T. Freeman, ACM Trans. Graphics 26, 70 (2007).
[Crossref]

D. Krishnan and R. Fergus, in Advances in Neural Information Processing Systems (NIPS, 2009), pp. 1033–1041.

Freeman, W. T.

A. Levin, R. Fergus, F. Durand, and W. T. Freeman, ACM Trans. Graphics 26, 70 (2007).
[Crossref]

Hanrahan, P.

R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, Comput. Sci. Tech. Rep. 2, 11 (2005).

Hiura, S.

A. Mohan, D. Lanman, S. Hiura, and R. Raskar, in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2009).

Horowitz, M.

R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, Comput. Sci. Tech. Rep. 2, 11 (2005).

Kittle, D.

Krishnan, D.

D. Krishnan and R. Fergus, in Advances in Neural Information Processing Systems (NIPS, 2009), pp. 1033–1041.

Lanman, D.

A. Mohan, D. Lanman, S. Hiura, and R. Raskar, in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2009).

Lelescu, D.

K. Venkataraman, D. Lelescu, J. Duparré, A. McMahon, G. Molina, P. Chatterjee, R. Mullis, and S. K. Nayar, ACM Trans. Graphics 32, 1 (2013).
[Crossref]

Levin, A.

A. Levin, R. Fergus, F. Durand, and W. T. Freeman, ACM Trans. Graphics 26, 70 (2007).
[Crossref]

Levoy, M.

R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, Comput. Sci. Tech. Rep. 2, 11 (2005).

Liao, X.

P. Llull, X. Liao, X. Yuan, J. Yang, D. Kittle, L. Carin, G. Sapiro, and D. J. Brady, Opt. Express 21, 10526 (2013).
[Crossref]

P. Llull, X. Yuan, X. Liao, J. Yang, L. Carin, G. Sapiro, and D. J. Brady, in Computational Optical Sensing and Imaging (Optical Society of America, 2014), paper CM2D-3.

Lin, S.

C. Zhou, S. Lin, and S. K. Nayar, in IEEE 12th International Conference on Computer Vision (IEEE, 2009), pp. 325–332.

Lin, X.

X. Lin, J. Suo, G. Wetzstein, Q. Dai, and R. Raskar, in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2013).

Liu, N.

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, Proc. Natl. Acad. Sci. USA 106, 2995 (2009).

Llull, P.

P. Llull, X. Liao, X. Yuan, J. Yang, D. Kittle, L. Carin, G. Sapiro, and D. J. Brady, Opt. Express 21, 10526 (2013).
[Crossref]

P. Llull, X. Yuan, X. Liao, J. Yang, L. Carin, G. Sapiro, and D. J. Brady, in Computational Optical Sensing and Imaging (Optical Society of America, 2014), paper CM2D-3.

Lord, S. J.

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, Proc. Natl. Acad. Sci. USA 106, 2995 (2009).

Marks, D. L.

McMahon, A.

K. Venkataraman, D. Lelescu, J. Duparré, A. McMahon, G. Molina, P. Chatterjee, R. Mullis, and S. K. Nayar, ACM Trans. Graphics 32, 1 (2013).
[Crossref]

Miau, D.

D. Miau, O. Cossairt, and S. K. Nayar, in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2013).

Moerner, W. E.

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, Proc. Natl. Acad. Sci. USA 106, 2995 (2009).

Mohan, A.

A. Mohan, D. Lanman, S. Hiura, and R. Raskar, in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2009).

Molina, G.

K. Venkataraman, D. Lelescu, J. Duparré, A. McMahon, G. Molina, P. Chatterjee, R. Mullis, and S. K. Nayar, ACM Trans. Graphics 32, 1 (2013).
[Crossref]

Mullis, R.

K. Venkataraman, D. Lelescu, J. Duparré, A. McMahon, G. Molina, P. Chatterjee, R. Mullis, and S. K. Nayar, ACM Trans. Graphics 32, 1 (2013).
[Crossref]

Nayar, S. K.

K. Venkataraman, D. Lelescu, J. Duparré, A. McMahon, G. Molina, P. Chatterjee, R. Mullis, and S. K. Nayar, ACM Trans. Graphics 32, 1 (2013).
[Crossref]

C. Zhou, S. Lin, and S. K. Nayar, in IEEE 12th International Conference on Computer Vision (IEEE, 2009), pp. 325–332.

D. Miau, O. Cossairt, and S. K. Nayar, in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2013).

Ng, R.

R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, Comput. Sci. Tech. Rep. 2, 11 (2005).

Nishita, T.

Y. Bando, B.-Y. Chen, and T. Nishita, Comput. Graphics Forum 30, 1869 (2011).
[Crossref]

Pavani, S. R. P.

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, Proc. Natl. Acad. Sci. USA 106, 2995 (2009).

Piestun, R.

S. Quirin and R. Piestun, Appl. Opt. 52, A367 (2013).
[Crossref]

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, Proc. Natl. Acad. Sci. USA 106, 2995 (2009).

Pitsianis, N. P.

Quirin, S.

Raskar, R.

X. Lin, J. Suo, G. Wetzstein, Q. Dai, and R. Raskar, in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2013).

A. Mohan, D. Lanman, S. Hiura, and R. Raskar, in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2009).

Sapiro, G.

P. Llull, X. Liao, X. Yuan, J. Yang, D. Kittle, L. Carin, G. Sapiro, and D. J. Brady, Opt. Express 21, 10526 (2013).
[Crossref]

P. Llull, X. Yuan, X. Liao, J. Yang, L. Carin, G. Sapiro, and D. J. Brady, in Computational Optical Sensing and Imaging (Optical Society of America, 2014), paper CM2D-3.

Sun, X.

Suo, J.

X. Lin, J. Suo, G. Wetzstein, Q. Dai, and R. Raskar, in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2013).

Thompson, M. A.

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, Proc. Natl. Acad. Sci. USA 106, 2995 (2009).

Tsai, T.-H.

Twieg, R. J.

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, Proc. Natl. Acad. Sci. USA 106, 2995 (2009).

Venkataraman, K.

K. Venkataraman, D. Lelescu, J. Duparré, A. McMahon, G. Molina, P. Chatterjee, R. Mullis, and S. K. Nayar, ACM Trans. Graphics 32, 1 (2013).
[Crossref]

Wagadarikar, A. A.

Wetzstein, G.

X. Lin, J. Suo, G. Wetzstein, Q. Dai, and R. Raskar, in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2013).

Yang, J.

P. Llull, X. Liao, X. Yuan, J. Yang, D. Kittle, L. Carin, G. Sapiro, and D. J. Brady, Opt. Express 21, 10526 (2013).
[Crossref]

P. Llull, X. Yuan, X. Liao, J. Yang, L. Carin, G. Sapiro, and D. J. Brady, in Computational Optical Sensing and Imaging (Optical Society of America, 2014), paper CM2D-3.

Yuan, X.

P. Llull, X. Liao, X. Yuan, J. Yang, D. Kittle, L. Carin, G. Sapiro, and D. J. Brady, Opt. Express 21, 10526 (2013).
[Crossref]

P. Llull, X. Yuan, X. Liao, J. Yang, L. Carin, G. Sapiro, and D. J. Brady, in Computational Optical Sensing and Imaging (Optical Society of America, 2014), paper CM2D-3.

Zhou, C.

C. Zhou, S. Lin, and S. K. Nayar, in IEEE 12th International Conference on Computer Vision (IEEE, 2009), pp. 325–332.

ACM Trans. Graphics (2)

K. Venkataraman, D. Lelescu, J. Duparré, A. McMahon, G. Molina, P. Chatterjee, R. Mullis, and S. K. Nayar, ACM Trans. Graphics 32, 1 (2013).
[Crossref]

A. Levin, R. Fergus, F. Durand, and W. T. Freeman, ACM Trans. Graphics 26, 70 (2007).
[Crossref]

Appl. Opt. (4)

Comput. Graphics Forum (1)

Y. Bando, B.-Y. Chen, and T. Nishita, Comput. Graphics Forum 30, 1869 (2011).
[Crossref]

Comput. Sci. Tech. Rep. (1)

R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, Comput. Sci. Tech. Rep. 2, 11 (2005).

Opt. Express (2)

Proc. Natl. Acad. Sci. USA (1)

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, Proc. Natl. Acad. Sci. USA 106, 2995 (2009).

Other (7)

P. Llull, X. Yuan, X. Liao, J. Yang, L. Carin, G. Sapiro, and D. J. Brady, in Computational Optical Sensing and Imaging (Optical Society of America, 2014), paper CM2D-3.

X. Lin, J. Suo, G. Wetzstein, Q. Dai, and R. Raskar, in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2013).

C. Zhou, S. Lin, and S. K. Nayar, in IEEE 12th International Conference on Computer Vision (IEEE, 2009), pp. 325–332.

D. J. Brady, in Optical Imaging and Spectroscopy (Wiley, 2009).

D. Miau, O. Cossairt, and S. K. Nayar, in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2013).

A. Mohan, D. Lanman, S. Hiura, and R. Raskar, in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2009).

D. Krishnan and R. Fergus, in Advances in Neural Information Processing Systems (NIPS, 2009), pp. 1033–1041.

Supplementary Material (2)

NameDescription
» Supplement 1: PDF (5063 KB)      Supplemental Document
» Visualization 1: AVI (1860 KB)      This video shows the target scene as observed with quarter-circular 3D image translation in slow motion.

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (7)

Fig. 1.
Fig. 1. Image translation sensing process. The lens or sensor may be translated to produce the image, whose spatial frequency content depends on the object range.
Fig. 2.
Fig. 2. Simulated clear aperture PSFs (top row) and image translation PSFs (bottom row) for various levels of defocus for a diffraction-limited imaging system. The peak irradiance point traverses a quarter circle over the operating range. The simulated pixel pitch is 1 μm.
Fig. 3.
Fig. 3. MTF cross sections for several focal coding strategies. The solid and dashed lines denote the MTFs observed for 0 and .03 diopters of defocus, respectively.
Fig. 4.
Fig. 4. Experimental prototype camera and image translation trajectories. The trigger pulses to the camera initiate the integration period; during this time, the focal plane moves with linear motion while the tip/tilt actuators move the image transversely.
Fig. 5.
Fig. 5. Image translation PSFs hs2,zo recorded by the camera for three object distances. These PSFs have an angle (relative to the projected radius of curvature) of peak irradiance that varies linearly with dioptric distance from the camera.
Fig. 6.
Fig. 6. Experimental scene. (a) Target volumetric scene. (b)–(d) Scene as observed by the prototype when focused at z1=174, z2=119, and z3=95, respectively. See Visualization 1 for a video showing this scene as observed by the prototype when the image translation trajectory is slowed down by a factor of 14.
Fig. 7.
Fig. 7. Experimental results. (a) Raw data captured by the prototype camera. (b) All-in-focus reconstructed image. (c) Reconstructed depth map with colors corresponding to the distances z1=174, z2=119, and z3=95. Lighter areas denote distances closer to the camera, as shown by the color bar on the right, which has units of diopters.

Equations (4)

Equations on this page are rendered with MathJax. Learn more.

gzo(x,y,t)=f(x,y,z,t)hzo(xx,yy,zzo)×rect(xxΔ,yyΔ,ttΔt)dxdydzdt,
hs1,zo(x,y,z)=0Δthzo(x,y,zγ(t))dt,
hs2,zo(x,y,z)=0Δthzo(xξ(t),yη(t),zγ(t))dt,
f^zo(l)=argf˜zi(l)ming(l)hs2,zi(l)*f˜zi(l)22+λTV(hs2,zi(l)*f˜zi(l)),

Metrics