Abstract

Fluorescence lifetime imaging (FLI) is a popular method for extracting useful information that is otherwise unavailable from a conventional intensity image. Usually, however, it requires expensive equipment, is often limited to either distinctly frequency- or time-domain modalities, and demands calibration measurements and precise knowledge of the illumination signal. Here, we present a generalized time-based, cost-effective method for estimating lifetimes by repurposing a consumer-grade time-of-flight sensor. By developing mathematical theory that unifies time- and frequency-domain approaches, we can interpret a time-based signal as a combination of multiple frequency measurements. We show that we can estimate lifetimes without knowledge of the illumination signal and without any calibration. We experimentally demonstrate this blind, reference-free method using a quantum dot solution and discuss the method’s implementation in FLI applications.

© 2015 Optical Society of America

1. INTRODUCTION

Fluorescence lifetime imaging (FLI) is a significant research area that spans many engineering applications. Knowledge of a sample’s fluorescence lifetime allows, for example, DNA sequencing [1], tumor detection [2,3], fluorescence tomography [4,5], in vivo imaging [6], and high-resolution microscopy [7]. Typically, FLI is categorized into two complementary modes [4,8]. In time-domain FLI [7,9] (TD-FLI), an impulse-like excitation pulse probes the fluorescent sample, and the time-resolved reflection is used to calculate lifetimes. In frequency-domain FLI [1012] (FD-FLI), the sample is excited with (sinusoidal) intensity modulated light, and the measured phase shift of the reflected signal at the same modulation frequency encodes the lifetime. FD-FLI is theoretically appealing in that phase measurements at one given modulation frequency suffice to resolve the sample lifetime. From a practical standpoint, model mismatch [10] and sample contamination due to multiple lifetimes [13] often limit the accuracy of single-frequency-based systems. Frequency diversity [13,14] may be used for imaging multiple lifetimes. Since this approach requires sweeping frequencies over a given bandwidth, a frequency diversity method is not suitable for wide-field imaging of dynamic samples. In either case (TD-FLI or FD-FLI), because of the costly equipment, system constraints are often strict, so that precise knowledge of the illumination signal and calibration measurements (to compensate for path length delays) are required [15].

Alternatively, time-of-flight (ToF) sensors, which are far more cost-effective and operate in real time, offer a great deal of flexibility in design. ToF sensors, such as the Microsoft Kinect, are essentially depth/range imaging devices and are at the heart of the entertainment industry. Recently, ToF sensing has found prodigious use in computer graphics, computer vision [16], and computational imaging [1618], with applications in multipath imaging [1922], ultrafast imaging [22,23], and imaging through scattering media [17,24].

Here, we demonstrate an extension of ToF sensing to FLI that is both blind and calibration-free. To do so, we generalize (from the distinct time or frequency domains) the active illumination signal, which is common to both modalities, to show that neither calibration measurements nor knowledge of the signal are necessary for estimation of fluorescent lifetimes. This blind, reference-free method is implemented in a consumer-grade ToF sensor to estimate simultaneously the range and lifetime of a CdSe–CdS quantum dot sample. This method suggests a cost-effective alternative to the usual FLI methods.

This paper is organized as follows: starting from first principles, we propose a ToF-sensor-based image formation model and discuss its role in depth/range estimation in Section 2. We then discuss the two complementary modes of operation of ToF sensors: time and frequency domains. Section 2.A deals with TD-ToF-sensor-based depth imaging, while Section 2.B discusses frequency-domain depth imaging. In Section 3, we provide a mathematical model for FLI with ToF sensors. We show that consumer ToF sensors can be used to estimate the lifetime of a fluorescent sample. We develop theoretical models for both time- and frequency-domain modes, and our theory is corroborated with experiments for both. These complementary approaches are discussed in Sections 3.A and Section 3.B. Finally, we conclude this work with possible future directions in Section 5.

2. ToF IMAGE FORMATION MODEL

ToF sensors operate using the lock-in principle [25]: active illumination probes a sample, and the reflected light is cross-correlated electronically to calculate range and amplitude information. In this way, each ToF sensor exposure results in two images: the usual intensity image and a range image in which each pixel relates to the depth of the scene. Usually, the signal is sinusoidal, essentially classifying a ToF sensor as a homodyne detection system, but the illumination signal can be far more general. Both ToF sensors and current FLI technologies acquire scene information in either the time or frequency domains, so the present analysis for ToF sensing is compatible with both.

Regardless of the operating domain or physical implementation, the ToF imaging process can be understood as follows. A scene is illuminated with a time-dependent Δ-periodic probing intensity signal p(t), such that p(t+Δ)=p(t), Δ>0. Similar to FLI, TD-ToF [21,22] systems use a time-localized pulse p(t) (not necessarily an impulse); the FD-ToF counterpart uses a modulated probing function p(t)=1+cos(ω0t), where ω0 (usually in the megahertz range) is the modulation frequency. Below, however, we do not use a specific form for p(t).

This signal interacts with the scene, whose response is characterized by a time-dependent scene response function (SRF) h(t,t), where t is a time-domain variable which models the time-variant SRF. This interaction results in the reflected signal r(t):

r(t)=p(t)h(t,t)dt.

A ToF sensor detects r(t) and electronically cross correlates it with p(t):

m(t)=p¯(t)r(tt)dt,
where p¯(t)=p(t). The K stored measurements are digital samples of this cross correlation, mk=m(kTs), k=0,,K1 (Ts>0 is the sampling rate).

In many cases of interest, the SRF is shift-invariant, that is (see [26], Sec. 2.2.4, p. 50),

h(t,t)=hSI(tt),
so that Eq. (1) represents the convolution operation defined by
r(t)=(p*hSI)(t)=p(t)hSI(tt)h(t,t)dt.

In the case of shift-invariant SRFs, the measurements simplify to

m(t)=(p*p¯*hSI)(t)(ϕ*hSI)(t),
implying linear filtering of cross-correlation function ϕ(t) with the scene response filter hSI.

Conventional ToF sensors [16] are designed for range imaging and depth estimation. For simple reflections from an object (with reflection coefficient ρ) that is at a depth d from the sensor, the SRF becomes

h(t,t)=ρδ(tt2d/c),
where c is the speed of light, and δ is the Dirac distribution. The reflected signal in this case is simply a delayed version of the probing function and the delay is proportional to the depth parameter d. More precisely, in view of the ToF sensor operation, the reflected signal in Eq. (1) reads r(t)=ρp(tt0), where the delay t0=2d/c. In this case, the measurements amount to
m(t)=ρϕ(t)(tt0)ρ(p*p¯)(tt0).

Estimation of the scene parameters {ρ,d} by the ToF sensor results in range and intensity images. The estimation process depends on the choice of probing function p, which may be a time-localized pulse or an amplitude-modulated continuous-wave (AMCW) function leading to time-domain [Fig. 2(a)] and frequency-domain [Fig. 2(b)] modes of operation, respectively.

A. TD-ToF Imaging

For TD-ToF imaging, the probing function is ideally well localized in time, that is, a Dirac delta distribution. In this case, p(t)δ(t), and the measurements in Eq. (4) simplify to m(t)=hSI(t). In practice, maximum length sequences (MLSs) [27] provide an optimal time-localized probing function. Such sequences are a function of the signal length or period Δ.

In view of the depth estimation problem, where h(t,t)=ρδ(tt2d/c), the depth is estimated by the operation

t˜0=maxtm(t)=maxtϕ(tt0),t0=2dc.

More sophisticated methods have been developed for the case of multiple reflections or multipath interference [21,22].

The exact characterization of the true TD-ToF probing function p(t) is challenging due to the electronics of the sensor and the physical process involved. By using a Fourier series expansion [21] to represent the probing function,

p(t)=1Δn=p^nejnω0t,ω0=2π/Δ,
where
p^n=0Δp(t)ejnω0tdt,
we define the cross correlation of the probing function p by
ϕ(t)=(p*p¯)(t)=n=ϕ^nejω0nt.

The Fourier series coefficients of the cross-correlated signal and the probing signal are related by

ϕ^n=p^np^n*=|p^n|2.

Practically, a band-limited approximation, Cp,p¯(t), suffices to represent ϕ(t):

Cp,p¯(t)1Δ|n|N0ϕ^nejnω0t,
where we have retained the first N0 Fourier coefficients of ϕ(t). Figure 1 shows an experimentally obtained ϕ(t) and its Fourier approximation Cp,p¯, with their Fourier spectra with N0=30, for an impulsive probe function.

 

Fig. 1. Experimental cross-correlated probing function [Eq. (8)] (blue line) and its Fourier series approximation (red dashed–dotted line) [Eq. (9)], with N0=30. Since the time-domain signal Cp,p¯ lasts for Δ=309.902ns, the corresponding fundamental Fourier harmonic is ω0/2π=3.2268MHz. Consequently, Fourier coefficient N0=30 corresponds to 96.80MHz. Inset: Fourier series coefficients ϕ^n,n=N0,,N0.

Download Full Size | PPT Slide | PDF

B. FD-ToF Imaging

In FD-ToF imaging, the scene is probed with a continuous wave, sinusoidal probing function with a modulation frequency ω:

p(t)=1+p0cos(ωt),
where p0 is the modulation amplitude. For pure depth estimation, with the SRF specified in Eq. (5), the reflected signal becomes
r(t)=ρp(tt0)=ρ(1+p0cos(ω(tt0))).

The ToF lock-in sensor acts as a homodyne detector array: each pixel cross correlates the reflected signal with the probing signal to produce measurements:

m(t)=limB12BB+Bp¯(t)r(tt)dt
=ρ(1+p022cos(ω(t+t0))),
where, in Eq. (10), we generalize the cross correlation in Eq. (2) for sinusoids.

The two quantities of interest, ρ and d, are estimated with the four digital measurements of the measured signal in Eq. (11), that is, mk=m(πk/2ωTs), k=0,,3. Based on these discrete measurements, we define a complex number zC:

z=(m0m2)+j(m3m1).

The scene parameters are thus estimated by

ρ˜=|z|p02andd˜=c2ωz.

Note that this technique for extracting phase is identical to conventional phase-shifting holography [28].

To summarize, object range is estimated via time delays in TD-ToF [Fig. 2(a)], whereas FD-ToF [Fig. 2(b)] encodes range into the signal phase. A quantitative summary of the ToF-sensor-based depth estimation problem for the time-domain and frequency-domain approaches is presented in Table 1 and Table 2, respectively.

Tables Icon

Table 1. TD- and FD-ToF Depth Imaging

Tables Icon

Table 2. TD- and FD-ToF FLI

In most consumer ToF sensors, m(t) [Eq. (2)] is a set of measurements that is used to estimate the scene parameters {ρ,d} [22,25]. The goal of this paper is to show that by using exactly the same set of measurements but with a different SRF, one can recover fluorescent lifetimes in context of the FLI.

3. FLI WITH TOF SENSORS

The experimental setup for FLI using a ToF sensor is depicted in Fig. 2(e). A fluorescent sample is located in an xy plane a distance z=d from the ToF sensor. Pixels corresponding to a nonfluorescent background object (xb,yb) produce only a time delay proportional to 2d/c, precisely the case of conventional ToF imaging with the SRF specified in Eq. (5).

 

Fig. 2. ToF depth imaging and its link with FLI. For the SRF h(t,t)=ρδ(tt2d/c), (a) and (b) compare TD- and FD-ToF principles. (a) Probing and reflected signal for TD-ToF. The time delay is proportional to distance d. (b) Probing and reflected signal for FD-ToF. The phase is proportional to distance d and modulation frequency ω0. For the SRF h(t,t)=hDepth(t,t)+hSample(t,t) in Eq. (13), (c) and (d) compare the TD-ToF and FD-ToF methods for FLI. (c) Probing and reflected signal for TD-ToF FLI. The time delay is proportional to distance d and the waveform shape is linked with lifetime τ. (d) Probing and reflected signal for FD-ToF FLI. The phase is proportional to distance d, modulation frequency ω0, and lifetime τ. (e) Experimental setup for FLI estimation via ToF.

Download Full Size | PPT Slide | PDF

The probing function that interacts with the pixels corresponding to a fluorescent sample at location (xf,yf) undergoes two transformations. The first transformation is attributed to the same depth contribution, d. The second transformation results from fluorescence: a fraction of the incident light excites the sample, which fluoresces with a characteristic decay time τ. The total SRF at (xf,yf) is then

h(t,t)=hDepth(t,t)+hSample(t,t),
where the contributions due to respective components are
hDepth(t,t)=ρδ(tt2d/c),
hSample(t,t)=μe(tt2d/cτ)Π(tt2d/c).
μ is the emission strength of the fluorescent sample, and Π(t) is the unit step function. (Note that hDepth and hSample represent signals at the excitation and emission optical wavelengths, respectively. The former can be eliminated experimentally via, e.g., a dielectric interference filter.) Because Eq. (13) is linear and shift-invariant, the resulting measurements are
m(t)=(4)(p*p¯*hSI)(t)(9)(Cp,p¯*hSI)(t)Cp,p¯*(δ(t2d/c)*(ρδ(t)+μetτΠ(t))).

We emphasize that our ToF system is completely compatible with FLI. In TD-FLI, p(t)=δ(t), whereas in FD-FLI, p(t)=1+cos(ω0t). Conventionally, separate calibration steps provide explicit knowledge of d, which is used for subsequent measurements. However, we make no such assumption. Instead, we simultaneously compute d and τ from our measurements [Eq. (16)]. We need not perform a separate measurement to obtain d explicitly.

A. TD-ToF FLI: Theory and Experiments

1. Theoretical Modeling

As in TD-ToF, we utilize the same truncated cross-correlation function [Eq. (9)] for FLI. Importantly, explicit knowledge of p(t) is not required. To show this, note that the eigenfunctions of Eq. (16) are precisely the complex exponentials of Eq. (9). By using the convolution theorem, we have

m(t)1Δ|n|N0(ϕ^nh^n)ejω0nt,
where h^n=h^(ω0n) and h^(ω) is the Fourier transform of hSI(t),
h^(ω)=h(t)ejωtdt(Fourier Transform)
=ρejω(2dc)+μτ1+jωτejω(2dc).

From here onward and for all practical purposes, we assume that Eq. (17) is an equality instead of an approximation. Expressing h^(ω) in polar form, we have h^(ω)=|h^(ω)|ejh^(ω), where

|h^(ω)|=(ρ+μτ)2+(ωρτ)21+(ωτ)2,h^(ω)=tan1(ωμτ2ρ+μτ+ρ(ωτ)2)2dcω.

Note that the phase of the spectrum encodes both the depth and the lifetime parameters. We may write

h^(ω)=θτ(ω)θd(ω),
where
θτ(ω)=tan1(ωτ2τ+ρ(1+(ωτ)2))andθd(ω)=2dω/c.
(Because we consider a single lifetime, we set μ=1.)

In vector-matrix notation, the discretized system of equations in Eq. (17) can be written as

m=VDϕ^h^,
where
  • m is a K×1 vector of discretized ToF sensor measurements mk=m(kTs), k[0,K1];
  • V is a Vandermonde matrix of size K×(2N0+1) with matrix element [V]k,n=ej(2π/Δ)Tsnk, n[N0,+N0];
  • Dϕ^ is a (2N0+1)×(2N0+1) diagonal matrix with diagonal entries [D]n,n=ϕ^n; and
  • h^ is (2N0+1)×1 vector of the discretized spectrum [Eq. (19)] with entries h^(2πn/Δ), n=N0,,N0.

The estimation problem is thus: given K measurements m, estimate parameters d and τ.

Because ϕ(t) is a real, time-symmetric function by construction, the matrix Dϕ^ does not contribute to the phase of vector h^ in Eq. (22). Hence, we rely on only the measurements m. Thus, p (or ϕ) need not be known, so that we eliminate the calibration requirements that are central to both FLI and ToF imaging [22].

To see this, note that TD-FLI uses [8] p=δ(t)=ϕ, so the corresponding measurements are

mTD-FLI(t)=μe(tt0τ)Π(tt0).

Now because log(mTD-FLI(t)) is linear, a direct fit [8] can be used to estimate τ, and, hence, calibration is implicitly avoided by the choice of p=δ.

On the other hand, for our proposed TD-ToF method for FLI, the illumination/probing function is composed of N0 multiplexed frequencies, which yield measurements

mTD-ToF-FLI(t)=(17)1Δ|n|N0(ϕ^nh^n)ejω0nt,
and because Cp,p¯ is a band-limited approximation of ϕ [Eq. (9)], a linear fit no longer suffices. However, because of the properties of (the generalized) p(t), we solve this more complex inverse problem to estimate {d,τ} without knowing ϕ.

2. Experimental Verification of TD-ToF FLI

The setup for the TD-ToF FLI method is shown in Fig. 2(e). A 405 nm laser diode illuminates the scene. We use a precomputed, quantized p [Eq. (6)] based on a 31-bit MLS described by the code

MLS0101110110001111100110100100001,
with Δ=309.9ns and Ts=7.8120ps. The sample consists of a CdSe–CdS quantum dot sample prepared by dissolving it in hexane and polymethyl methacrylate (PMMA) onto a glass slide. This sample has a lifetime of τ=32ns, and it is located at 1.05 m from the sensor.

The reflected light is passed through a dielectric interference filter with a cut-off wavelength of 450 nm, leading to ρ=0 [Eq. (21)]. A 160×120 pixel PMD 19k-S3 lock-in sensor with custom field programmable gate array (FPGA) programming cross correlates the reflected signal to produce measurements m. The sensor has an 80 MHz modulation bandwidth operating within 90 frames per second [29]. The total cost of our system is $1200.

With N0=15, we compute

h^obs=V+m=(θτ(2πn/Δ)+θd(2πn/Δ)),
where V+ denotes the pseudo-inverse of the matrix V and h^obs is the experimentally observed phase. Since we know that the observed phase is related to the theoretical phase in Eq. (20), we estimate the parameters of interest by solving the nonlinear leastsquares problem
argmind,τn=N0n=+N0|h^(nω0)h^obs(nω0+β)+α|2,
where α and β are offset parameters to ensure a solution that is centered at origin. We use the trust-region-based [30] algorithm with the least absolute residual criterion. The nonlinear parameter estimation problem is solved using bounded constraints that are accommodated within the trust-region optimization framework. We require that the minimum and maximum lifetimes and distances are in ranges of 0 to 100 ns and 0 to 10 m, respectively.

Per-pixel fluorescent sample measurements are shown in Fig. 3, which shows both the time-domain measurement [Fig. 3(a)] and the raw phase measurement [Fig. 3(b)]. We also show the fitted phase measurement in Fig. 3(b). We assign a confidence level to each pixel to estimate the signal-to-noise ratio (SNR). We use ToF measurements from 14 different pixels imaging the same scene [Fig. 2(e)]. The results from our computation are tabulated in Table 3. In addition to the estimated lifetimes and distances, we tabulate two relevant metrics.

 

Fig. 3. TD-ToF FLI measurements. (a) Time profile of measurements mk=m(kTs) and Ts=7.8120ps based on K=3968 samples. These measurements result from convolving cross correlation of the probing function with the SRF [Eq. (17)]. (b) Phase measurements computed using h^=V+m with N0=15 and Δ=309.9ns. We estimate the distance d˜ and lifetime τ˜ parameters using nonlinear least squares fitting, which results in the fitted curve. The fitted result is shown by the red solid line. The estimated phase contribution [Eq. (20)] due to distance θd˜ and lifetime θτ˜ is also plotted.

Download Full Size | PPT Slide | PDF

Tables Icon

Table 3. TD-ToF FLI (τ=32ns and d=1.05m)

  • 1. First, the mean squared error or MSE, a measure of distortion, is defined as follows: let ν be the oracle estimate and {ν˜n}n=0N1 be N estimated values of ν. The MSE is
    MSEν=1Nn=0N1|ν˜nν|2.

    We compute log(MSE) for τ˜, d˜, and h^˜ in Table 3, where the last term is due to fitted measurements

    h^˜(nω0)=(20)tan1(nω0τ˜)2nω0d˜c,
    which are synthesized after estimating {τ˜,d˜}.

  • 2. We use the observed SNR to estimate measurement fidelity. Including additive Gaussian noise εn, we have
    h^obs(nω0)=h^(nω0)+εn.

    We define

    SNR=20(logh^logh^obsh^),
    measured in decibels, and h^2=n=0N1|h^(nω0)|2. In Table 3, we use h^=h^˜, the fitted phase function computed using estimates {τ˜,d˜}.

Based on the estimates in Table 3, the expected lifetime is τ˜=31.3142ns, and the estimated expected distance is d˜=1.0799m, both consistent with the ground truth.

B. FD-ToF FLI: Theory and Experiments

As described in Section 2.B, in the FD-ToF mode of operation, the ToF sensor probes the scene with an AMCW of form p(t)=1+p0cos(ωt). Following Eq. (3), the reflected signal is

r(t)=(3)(p*hSI)=|h^(0)|+|h^(ω)|p0cos(ωt+h^(ω)).

The reflected signal is cross correlated [Eq. (10)] at the ToF sensor to produce measurements

m(t)=(10)|h^(0)|+|h^(ω)|p022cos(ωth^(ω)).

To estimate the phase and amplitude, we utilize the method from Section 2.B. Noting, however, that the scene transfer function is not constant, we express z explicitly as a function of the modulation frequency ω0: z=z(ω0). The amplitude and phase estimates are

|h^(ω0)||z(ω0)|/p02Amplitude Estimateandh^(ω0)(20)z(ω0)Phase Estimate,
respectively. Note that the estimation of amplitude |h^(ω0)| requires the knowledge of p0. However, this is not the case for phase estimation, which is prescribed by h^(ω0) and is devoid of p0. Again, our method is blind in that we can directly estimate h^(ω0) from z(ω0) even if we do not know anything about p0.

Note in passing that multiple frequency measurements can be used to estimate multiple depths [16,20,21,31], for which

h(t)=k=0K1ρkδ(t2dk/c)Fourierh^(ω)=k=0K1ρkejω(2dkc).

Based on the image formation model for the SRF in Eq. (13), we will next show how multiple frequency measurements of the form {z(kω0)}k=1K can be used alternately to estimate the parameters of interest, that is, {d,τ} in the context of FLI.

1. Experimental Verification of FD-ToF FLI

Using the same physical setup as that in Section 3.A.2, we move the sample to d=2.5m, and we set ω0/2πf0=1MHz and acquire equispaced ToF measurements

{z(kf0)}k=1K,k=1,2,,40.

The amplitudes and phases of z for k=20, 30, and 40 are shown in Fig. 4(a). The effects of fluorescence are clearly visible in both amplitude and phase. The dielectric filter eliminates all nonfluorescent emission, leading to a noisy background signal. The fluorescing quantum dot causes an increase in the measured phase. The increased phase measurement at the sample location is due to the presence of θτ term in Eq. (20). In context of the experimental setup described in Fig. 2(e), we mark the background pixel (xb,yb) as well as the fluorescent pixel (xf,yf) in Fig. 4(a). The average phase of the background pixel is noted to be z(xb,yb)(40f0)=4.1625rad. This amounts to a depth of 2.4843 m, which is consistent with the experimental setup where d=2.5m. On the other hand, the average phase at the location of the quantum dot is observed to be z(xf,yf)(40f0)=5.5822rad. This relates to an erroneous depth of 3.3316 m, which is a result of multipath interference [20].

 

Fig. 4. FD-ToF FLI measurements of a 61×66 pixel patch of a 120×160 pixel sensor image. The size of the scene is approximately 2.3in2. (a) Multifrequency measurements of τ=32ns quantum-dot-based fluorescent sample. We show phase and amplitude images, that is, {z(10f0),z(20f0),z(30f0),z(40f0)} (in radians, [0,2π]) and {|z(10f0)|,|z(20f0)|,|z(30f0)|,|z(40f0)|} (in decibels), respectively. The base modulation frequency for the experiment is f0=ω0/2π=1MHz. The phase at the background pixel is recorded to be z(xb,yb)(40f0)=4.1625rad, which amounts to a depth of 2.48 m, which is consistent with the experimental setup. At the location of the fluorescent sample, we recorded a higher phase value z(xf,yf)(40f0)=5.5822, which is attributed to the fluorescence phenomenon. (b) Multifrequency raw phase measurements {z(x,y)(kf0)}k=1K=40 for four pixels. The measurements confirm with the theoretical hypothesis of Eq. (20), as well as the fitted phase obtained by Eq. (35). The estimated phase contribution [Eq. (20)] due to distance θd˜ and lifetime θτ˜ is also plotted.

Download Full Size | PPT Slide | PDF

By using ToF phase measurements [Eq. (34)] for 10 different pixel locations corresponding to (xf,yf), we solve for the nonlinear least squares problem:

argmind,τk=1K=40|z(kω0)+θτ(kω0)+θd(kω0)|2.

As before in Section 3.A.2, we use the trust-region-based algorithm with the least absolute residual criterion. As a result of fitting, we estimate d˜ and τ˜ for each pixel. The estimated distance and lifetime values together with log(MSE) for τ˜, d˜, and h^˜ for all the pixels are tabulated in Table 4. The average lifetime and distance is estimated to be

{d˜,τ˜}={2.4961m,31.024ns}.

Tables Icon

Table 4. FD-ToF FLI (τ=32ns and d=2.5m)

Both the measured phase {z(kf0)}k=1K=40 and the fit obtained by Eq. (35) for four pixels are plotted in Fig. 4(b).

4. DISCUSSION

A. ToF Sensors for Microscopy

The method generalizes to microscopy modes. A salient feature about the technique is that it is a local, per-pixel calculation, so that the lateral scale of the problem does not influence the technique. Thus, our proof-of-principle demonstration can be extended to integration with microscopy, both wide-field and point-scanning techniques, provided there is no pixel crosstalk. In fact, our current system is not aberration-corrected, and the resulting model mismatch makes reconstruction more challenging. A well-calibrated microscopy setup should alleviate this mismatch and improve results.

B. Nanosecond Range Lifetime Sensing

For biological imaging, lifetimes are of the order of nanoseconds. Recent work suggests that current ToF sensors (with bandwidths of tens of megahertz) are optimal for recovering lifetimes under 5 ns [32]. For example, a 3 ns lifetime is optimally estimated by a 30 MHz signal ([32], p. 378). Further, a similar ToF setup estimated lifetimes of the order of 4 ns [12] (though it did not simultaneously estimate distance).

The important difference here is that the present approach simultaneously estimates lifetime and distance based on the same measurement. Numerical simulations in Fig. 5 indicate that our method is suitable for recovering a 4 ns lifetime. We simulate a system bandwidth of 40 MHz (half the experimental bandwidth [29]). Numerically, we compare the estimation accuracy of τ1=4ns and τ2=32ns with d=2.5m. We do so by studying the upper bounds on the MSE linked with estimation of the set of parameters of interest, {τ1,d} and {τ2,d}. For a fixed ToF sensor bandwidth, we vary the number of measurements [Eq. (33)] by sampling with equispaced frequencies from 0 to 40 MHz. We use four different step sizes, f0(k) (corresponding to N(k) samples), specified by

f0(1)=0.10N(1)=401,f0(2)=0.25N(2)=161,f0(3)=0.50N(3)=81,f0(4)=1.00N(4)=41.

 

Fig. 5. Estimation accuracy of τ1=4ns and τ2=32ns. (a) We plot 2000 estimated values of lifetime as a function of SNR ranging from 0 to 60 dB. As the SNR increases, the estimates cluster around the oracle estimates of τ=4ns and τ=32ns, respectively. (b) We plot the MSEτ on log scale as a function of SNR. After 15 dB, we note a consistent linear relationship [Eq. (36)] between SNR and the log(MSEτ).

Download Full Size | PPT Slide | PDF

f0(k) is measured in megahertz. For example, the step size in experiments in Section 3.B.1 was f0=1MHz. For each f0(k), we generate the measurement vector,

h^obs(k)(nω0)=(30)h^(nω0)+εn,n=0,,N(k)1,
where εn represents additive white Gaussian noise for a SNR ranging from 0 to 60 dB. Each SNR value is averaged over 2000 realizations. In Fig. 5(a) we plot the 2000 estimated values of the lifetime parameters for each value of SNR. Intuitively, as the SNR increases, the estimates cluster around the oracle estimate of τ1=4ns and τ2=32ns, respectively.

In Fig. 5(b), we plot the MSEτ on log scale as a function of SNR. Note that the MSEτ [Eq. (28)] is the average over all 2000 trials. After 15 dB, we note a consistent linear relationship of the form

log(MSEτ)logK0λlog(SNR).

As the number of measurements N increases, the log(MSEτ) drops consistently. This observation is consistent throughout all of our experiments. In fact, we note that MSEτ1<MSEτ2, implying that the 4 ns sample may be estimated with higher accuracy when compared to the 32 ns sample. As noted in Table 4, the operational SNR of our system is around 45 dB, implying that the lifetime parameter can be estimated with sufficient accuracy, as plotted in Fig. 5(b).

Thus, although our demonstration here operates at a lower time scale (Fig. 1) than is typical in practice, this is not a fundamental limitation of the method. The reason is that we compensate for lower time resolution by utilizing a computationally different method of inversion. Indeed, appropriate modeling and prior information lend themselves naturally to ToF sensing and offer a path toward superresolution [21,33].

Of course, calculation of the theoretical lower bounds (on lifetimes and distances) requires calculation of the Cramer–Rao bound. Though beyond the scope of the present work, this limit is ultimately dictated by noise. For instance, the Cramer–Rao lower bound for distance estimation is derived in [21] and obeys a version of the law in Eq. (36). For typical systems, we expect from Fig. 5 that the method is suitable for current application needs.

C. Experimental and Computational Precision

The current optical ToF technology allows for range estimation in millimeter precision. The variation in estimated lifetime is due to sample inhomogeneity, nonuniform lighting [34], and potential model mismatch from lens aberration.

The measurement precision here is indicated in Fig. 6, which shows cross sections of the four phase plots in Fig. 4(a). The phase contribution the first 50 pixels is due to the background. We mark the average phase value on the y axis. Using θd(f)=4πfd/c, we estimate the distance d given θd(f) and f={10,20,30,40}MHz. At each modulation frequency, the estimated distance is d={2.56,2.52,2.47,2.48}m, whereas the actual distance is 2.5 m. The variability in the distance estimates is mainly because, across modulation frequencies, the phase-frequency relation θd(f)=4πfd/c may not be strictly linear due to distortions.

 

Fig. 6. Measured phase as a function of the modulation frequency. We plot the measured phase corresponding to a 120 pixel cross section of the 160×120 ToF sensor image for modulation frequencies f={10,20,30,40}MHz. The first 50 phase measurements are due to the background pixel, which is at a depth d=2.5m. The dashed line marks the average phase value over the first 50 pixels. The average phase value for each modulation frequency leads to a distance estimate: d={2.56,2.52,2.47,2.48} in meters.

Download Full Size | PPT Slide | PDF

5. CONCLUSIONS

In conclusion, we have demonstrated a FLI alternative that is based on cost-effective ToF sensors. We simultaneously estimate the lifetime and the distance of the sample from the sensor. Unlike existing methods [8,12], our approach is calibration-free and requires no prior information on the experimental path length and, thus, allows for faster acquisition time. Furthermore, our technique is blind in that we do not assume the knowledge of the illumination waveform. Overall, our system shows promise for two-dimensional imaging, and can be generalized to volumes [35]. Because the technique is modular, it can be implemented with other computational imaging techniques to create a new platform for wide-field FLI.

The method offers new possibilities for open questions. The case of multiple lifetime imaging [10,13] is interesting and is yet to be explored in the context of ToF sensors, both theoretically as well as experimentally. Comparison with other FLI techniques [36] and fundamental resolution limits for simultaneous estimation of lifetime and depth information will allow for better understanding of the applicability of ToF sensors for bio-imaging tasks such as tumor detection [2,3] and fluorescence tomography [4,5].

Funding

Massachusetts Institute of Technology Media Laboratory Consortia (2746038); USAR Laboratory and USAR Office (W9911NF-13-D-0001).

Acknowledgment

We thank R. Whyte and A. Das for discussions about the experimental setup at the initial stages of this collaboration.

REFERENCES

1. H. He, B. K. Nunnally, L.-C. Li, and L. B. McGown, “On-the-fly fluorescence lifetime detection of dye-labeled DNA primers for multiplex analysis,” Anal. Chem. 70, 3413–3418 (1998). [CrossRef]  

2. Y. Sun, J. Phipps, D. S. Elson, H. Stoy, S. Tinling, J. Meier, B. Poirier, F. S. Chuang, D. G. Farwell, and L. Marcu, “Fluorescence lifetime imaging microscopy: in vivo application to diagnosis of oral carcinoma,” Opt. Lett. 34, 2081–2083 (2009). [CrossRef]  

3. R. Cicchi, D. Massi, S. Sestini, P. Carli, V. De Giorgi, T. Lotti, and F. Pavone, “Multidimensional non-linear laser imaging of basal cell carcinoma,” Opt. Express 15, 10135–10148 (2007). [CrossRef]  

4. A. T. Kumar, S. B. Raymond, B. J. Bacskai, and D. A. Boas, “Comparison of frequency-domain and time-domain fluorescence lifetime tomography,” Opt. Lett. 33, 470–472 (2008). [CrossRef]  

5. A. T. Kumar, S. B. Raymond, A. K. Dunn, B. J. Bacskai, and D. A. Boas, “A time domain fluorescence tomography system for small animal imaging,” IEEE Trans. Med. Imaging 27, 1152–1163 (2008). [CrossRef]  

6. J. Rao, A. Dragulescu-Andrasi, and H. Yao, “Fluorescence imaging in vivo: recent advances,” Curr. Opin. Biotechnol. 18, 17–25 (2007). [CrossRef]  

7. B. J. Bacskai, J. Skoch, G. A. Hickey, R. Allen, and B. T. Hyman, “Fluorescence resonance energy transfer determinations using multiphoton fluorescence lifetime imaging microscopy to characterize amyloid-beta plaques,” J. Biomed. Opt. 8, 368–375 (2003). [CrossRef]  

8. J. Lakowicz, Principles of Fluorescence Microscopy (Springer, 1982).

9. D. R. Yankelevich, D. Ma, J. Liu, Y. Sun, Y. Sun, J. Bec, D. S. Elson, and L. Marcu, “Design and evaluation of a device for fast multispectral time-resolved fluorescence spectroscopy and imaging,” Rev. Sci. Instrum. 85, 034303 (2014). [CrossRef]  

10. E. Gratton, M. Limkeman, J. R. Lakowicz, B. P. Maliwal, H. Cherek, and G. Laczko, “Resolution of mixtures of fluorophores using variable-frequency phase and modulation data,” Biophys. J. 46, 479–486 (1984). [CrossRef]  

11. G.-J. Kremers, E. B. Van Munster, J. Goedhart, and T. W. Gadella, “Quantitative lifetime unmixing of multiexponentially decaying fluorophores using single-frequency fluorescence lifetime imaging microscopy,” Biophys. J. 95, 378–389 (2008). [CrossRef]  

12. A. Esposito, T. Oggier, H. Gerritsen, F. Lustenberger, and F. Wouters, “All-solid-state lock-in imaging for wide-field fluorescence lifetime sensing,” Opt. Express 13, 9812–9821 (2005). [CrossRef]  

13. A. Squire, P. J. Verveer, and P. I. H. Bastiaens, “Multiple frequency fluorescence lifetime imaging microscopy,” J. Microsc. 197, 136–149 (2000). [CrossRef]  

14. J. C. K. Chan, E. D. Diebold, B. W. Buckley, S. Mao, N. Akbari, and B. Jalali, “Digitally synthesized beat frequency-multiplexed fluorescence lifetime spectroscopy,” Biomed. Opt. Express 5, 4428–4436 (2014). [CrossRef]  

15. Q. S. Hanley, V. Subramaniam, D. J. Arndt-Jovin, and T. M. Jovin, “Fluorescence lifetime imaging: multi-point calibration, minimum resolvable differences, and artifact suppression,” Cytometry 43, 248–260 (2001). [CrossRef]  

16. A. Bhandari, M. Feigin, S. Izadi, C. Rhemann, M. Schmidt, and R. Raskar, “Resolving multipath interference in Kinect: An inverse problem approach,” in IEEE Sensors (IEEE, 2014), pp. 614–617.

17. F. Heide, L. Xiao, W. Heidrich, and M. B. Hullin, “Diffuse mirrors: 3D reconstruction from diffuse indirect illumination using inexpensive time-of-flight sensors,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014), pp. 3222–3229.

18. A. Kadambi, A. Bhandari, R. Whyte, A. Dorrington, and R. Raskar, “Demultiplexing illumination via low cost sensing and nanosecond coding,” in IEEE International Conference on Computational Photography (ICCP) (2014).

19. H. Qiao, J. Lin, Y. Liu, M. B. Hullin, and Q. Dai, “Resolving transient time profile in ToF imaging via log-sum sparse regularization,” Opt. Lett. 40, 918–921 (2015). [CrossRef]  

20. A. Bhandari, A. Kadambi, R. Whyte, C. Barsi, M. Feigin, A. Dorrington, and R. Raskar, “Resolving multipath interference in time-of-flight imaging via modulation frequency diversity and sparse regularization,” Opt. Lett. 39, 1705–1708 (2014). [CrossRef]  

21. A. Bhandari, A. Kadambi, and R. Raskar, “Sparse linear operator identification without sparse regularization? Applications to mixed pixel problem in time-of-flight/range imaging,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2014), pp. 365–369.

22. A. Kadambi, R. Whyte, A. Bhandari, L. Streeter, C. Barsi, A. Dorrington, and R. Raskar, “Coded time of flight cameras: sparse deconvolution to address multipath interference and recover time profiles,” ACM Trans. Graph. 32, 167 (2013). [CrossRef]  

23. F. Heide, M. B. Hullin, J. Gregson, and W. Heidrich, “Low-budget transient imaging using photonic mixer devices,” ACM Trans. Graph. 32, 1 (2013).

24. F. Heide, L. Xiao, A. Kolb, M. B. Hullin, and W. Heidrich, “Imaging in scattering media using correlation image sensors and sparse convolutional coding,” Opt. Express 22, 26338–26350 (2014). [CrossRef]  

25. S. Foix, G. Alenya, and C. Torras, “Lock-in time-of-flight (ToF) cameras: a survey,” IEEE Sens. J. 11, 1917–1926 (2011). [CrossRef]  

26. S. Y. Kung, Kernel Methods and Machine Learning (Cambridge University, 2014).

27. S. W. Golomb and G. Gong, Signal Design for Good Correlation: For Wireless Communication, Cryptography, and Radar (Cambridge University, 2005).

28. I. Yamaguchi and T. Zhang, “Phase-shifting digital holography,” Opt. Lett. 22, 1268–1270 (1997). [CrossRef]  

29. PMD Technologies, “pmd PhotonICs 19k–S3: Specs and reference design,” (2014).

30. T. F. Coleman and Y. Li, “An interior trust region approach for nonlinear minimization subject to bounds,” SIAM J. Optim. 6, 418–445 (1996). [CrossRef]  

31. A. Bhandari, A. Bourquard, S. Izadi, and R. Raskar, “Blind transmitted and reflected image separation using depth diversity and time-of-flight sensors,” in Computational Optical Sensing and Imaging (Optical Society of America, 2015), paper CT4F-2.

32. J. Philip and K. Carlsson, “Theoretical investigation of the signal-to-noise ratio in fluorescence lifetime imaging,” J. Opt. Soc. Am. A 20, 368–379 (2003). [CrossRef]  

33. S. Schuon, C. Theobalt, J. Davis, and S. Thrun, “Lidarboost: Depth superresolution for ToF 3D shape scanning,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2009), pp. 343–350.

34. D. Lefloch, R. Nair, F. Lenzen, H. Schäfer, L. Streeter, M. J. Cree, R. Koch, and A. Kolb, “Technical foundation and calibration methods for time-of-flight cameras,” in Time-of-Flight and Depth Imaging: Sensors, Algorithms, and Applications (Springer, 2013), pp. 3–24.

35. G. Satat, B. Heshmat, C. Barsi, D. Raviv, O. Chen, M. G. Bawendi, and R. Raskar, “Locating and classifying fluorescent tags behind turbid layers using time-resolved inversion,” Nat. Commun. 6, 6796 (2015). [CrossRef]  

36. M. A. Digman, V. R. Caiolfa, M. Zamai, and E. Gratton, “The phasor approach to fluorescence lifetime imaging analysis,” Biophys. J. 94, L14–L16 (2008). [CrossRef]  

References

  • View by:
  • |
  • |
  • |

  1. H. He, B. K. Nunnally, L.-C. Li, and L. B. McGown, “On-the-fly fluorescence lifetime detection of dye-labeled DNA primers for multiplex analysis,” Anal. Chem. 70, 3413–3418 (1998).
    [Crossref]
  2. Y. Sun, J. Phipps, D. S. Elson, H. Stoy, S. Tinling, J. Meier, B. Poirier, F. S. Chuang, D. G. Farwell, and L. Marcu, “Fluorescence lifetime imaging microscopy: in vivo application to diagnosis of oral carcinoma,” Opt. Lett. 34, 2081–2083 (2009).
    [Crossref]
  3. R. Cicchi, D. Massi, S. Sestini, P. Carli, V. De Giorgi, T. Lotti, and F. Pavone, “Multidimensional non-linear laser imaging of basal cell carcinoma,” Opt. Express 15, 10135–10148 (2007).
    [Crossref]
  4. A. T. Kumar, S. B. Raymond, B. J. Bacskai, and D. A. Boas, “Comparison of frequency-domain and time-domain fluorescence lifetime tomography,” Opt. Lett. 33, 470–472 (2008).
    [Crossref]
  5. A. T. Kumar, S. B. Raymond, A. K. Dunn, B. J. Bacskai, and D. A. Boas, “A time domain fluorescence tomography system for small animal imaging,” IEEE Trans. Med. Imaging 27, 1152–1163 (2008).
    [Crossref]
  6. J. Rao, A. Dragulescu-Andrasi, and H. Yao, “Fluorescence imaging in vivo: recent advances,” Curr. Opin. Biotechnol. 18, 17–25 (2007).
    [Crossref]
  7. B. J. Bacskai, J. Skoch, G. A. Hickey, R. Allen, and B. T. Hyman, “Fluorescence resonance energy transfer determinations using multiphoton fluorescence lifetime imaging microscopy to characterize amyloid-beta plaques,” J. Biomed. Opt. 8, 368–375 (2003).
    [Crossref]
  8. J. Lakowicz, Principles of Fluorescence Microscopy (Springer, 1982).
  9. D. R. Yankelevich, D. Ma, J. Liu, Y. Sun, Y. Sun, J. Bec, D. S. Elson, and L. Marcu, “Design and evaluation of a device for fast multispectral time-resolved fluorescence spectroscopy and imaging,” Rev. Sci. Instrum. 85, 034303 (2014).
    [Crossref]
  10. E. Gratton, M. Limkeman, J. R. Lakowicz, B. P. Maliwal, H. Cherek, and G. Laczko, “Resolution of mixtures of fluorophores using variable-frequency phase and modulation data,” Biophys. J. 46, 479–486 (1984).
    [Crossref]
  11. G.-J. Kremers, E. B. Van Munster, J. Goedhart, and T. W. Gadella, “Quantitative lifetime unmixing of multiexponentially decaying fluorophores using single-frequency fluorescence lifetime imaging microscopy,” Biophys. J. 95, 378–389 (2008).
    [Crossref]
  12. A. Esposito, T. Oggier, H. Gerritsen, F. Lustenberger, and F. Wouters, “All-solid-state lock-in imaging for wide-field fluorescence lifetime sensing,” Opt. Express 13, 9812–9821 (2005).
    [Crossref]
  13. A. Squire, P. J. Verveer, and P. I. H. Bastiaens, “Multiple frequency fluorescence lifetime imaging microscopy,” J. Microsc. 197, 136–149 (2000).
    [Crossref]
  14. J. C. K. Chan, E. D. Diebold, B. W. Buckley, S. Mao, N. Akbari, and B. Jalali, “Digitally synthesized beat frequency-multiplexed fluorescence lifetime spectroscopy,” Biomed. Opt. Express 5, 4428–4436 (2014).
    [Crossref]
  15. Q. S. Hanley, V. Subramaniam, D. J. Arndt-Jovin, and T. M. Jovin, “Fluorescence lifetime imaging: multi-point calibration, minimum resolvable differences, and artifact suppression,” Cytometry 43, 248–260 (2001).
    [Crossref]
  16. A. Bhandari, M. Feigin, S. Izadi, C. Rhemann, M. Schmidt, and R. Raskar, “Resolving multipath interference in Kinect: An inverse problem approach,” in IEEE Sensors (IEEE, 2014), pp. 614–617.
  17. F. Heide, L. Xiao, W. Heidrich, and M. B. Hullin, “Diffuse mirrors: 3D reconstruction from diffuse indirect illumination using inexpensive time-of-flight sensors,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014), pp. 3222–3229.
  18. A. Kadambi, A. Bhandari, R. Whyte, A. Dorrington, and R. Raskar, “Demultiplexing illumination via low cost sensing and nanosecond coding,” in IEEE International Conference on Computational Photography (ICCP) (2014).
  19. H. Qiao, J. Lin, Y. Liu, M. B. Hullin, and Q. Dai, “Resolving transient time profile in ToF imaging via log-sum sparse regularization,” Opt. Lett. 40, 918–921 (2015).
    [Crossref]
  20. A. Bhandari, A. Kadambi, R. Whyte, C. Barsi, M. Feigin, A. Dorrington, and R. Raskar, “Resolving multipath interference in time-of-flight imaging via modulation frequency diversity and sparse regularization,” Opt. Lett. 39, 1705–1708 (2014).
    [Crossref]
  21. A. Bhandari, A. Kadambi, and R. Raskar, “Sparse linear operator identification without sparse regularization? Applications to mixed pixel problem in time-of-flight/range imaging,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2014), pp. 365–369.
  22. A. Kadambi, R. Whyte, A. Bhandari, L. Streeter, C. Barsi, A. Dorrington, and R. Raskar, “Coded time of flight cameras: sparse deconvolution to address multipath interference and recover time profiles,” ACM Trans. Graph. 32, 167 (2013).
    [Crossref]
  23. F. Heide, M. B. Hullin, J. Gregson, and W. Heidrich, “Low-budget transient imaging using photonic mixer devices,” ACM Trans. Graph. 32, 1 (2013).
  24. F. Heide, L. Xiao, A. Kolb, M. B. Hullin, and W. Heidrich, “Imaging in scattering media using correlation image sensors and sparse convolutional coding,” Opt. Express 22, 26338–26350 (2014).
    [Crossref]
  25. S. Foix, G. Alenya, and C. Torras, “Lock-in time-of-flight (ToF) cameras: a survey,” IEEE Sens. J. 11, 1917–1926 (2011).
    [Crossref]
  26. S. Y. Kung, Kernel Methods and Machine Learning (Cambridge University, 2014).
  27. S. W. Golomb and G. Gong, Signal Design for Good Correlation: For Wireless Communication, Cryptography, and Radar (Cambridge University, 2005).
  28. I. Yamaguchi and T. Zhang, “Phase-shifting digital holography,” Opt. Lett. 22, 1268–1270 (1997).
    [Crossref]
  29. PMD Technologies, “pmd PhotonICs 19k–S3: Specs and reference design,” (2014).
  30. T. F. Coleman and Y. Li, “An interior trust region approach for nonlinear minimization subject to bounds,” SIAM J. Optim. 6, 418–445 (1996).
    [Crossref]
  31. A. Bhandari, A. Bourquard, S. Izadi, and R. Raskar, “Blind transmitted and reflected image separation using depth diversity and time-of-flight sensors,” in Computational Optical Sensing and Imaging (Optical Society of America, 2015), paper CT4F-2.
  32. J. Philip and K. Carlsson, “Theoretical investigation of the signal-to-noise ratio in fluorescence lifetime imaging,” J. Opt. Soc. Am. A 20, 368–379 (2003).
    [Crossref]
  33. S. Schuon, C. Theobalt, J. Davis, and S. Thrun, “Lidarboost: Depth superresolution for ToF 3D shape scanning,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2009), pp. 343–350.
  34. D. Lefloch, R. Nair, F. Lenzen, H. Schäfer, L. Streeter, M. J. Cree, R. Koch, and A. Kolb, “Technical foundation and calibration methods for time-of-flight cameras,” in Time-of-Flight and Depth Imaging: Sensors, Algorithms, and Applications (Springer, 2013), pp. 3–24.
  35. G. Satat, B. Heshmat, C. Barsi, D. Raviv, O. Chen, M. G. Bawendi, and R. Raskar, “Locating and classifying fluorescent tags behind turbid layers using time-resolved inversion,” Nat. Commun. 6, 6796 (2015).
    [Crossref]
  36. M. A. Digman, V. R. Caiolfa, M. Zamai, and E. Gratton, “The phasor approach to fluorescence lifetime imaging analysis,” Biophys. J. 94, L14–L16 (2008).
    [Crossref]

2015 (2)

H. Qiao, J. Lin, Y. Liu, M. B. Hullin, and Q. Dai, “Resolving transient time profile in ToF imaging via log-sum sparse regularization,” Opt. Lett. 40, 918–921 (2015).
[Crossref]

G. Satat, B. Heshmat, C. Barsi, D. Raviv, O. Chen, M. G. Bawendi, and R. Raskar, “Locating and classifying fluorescent tags behind turbid layers using time-resolved inversion,” Nat. Commun. 6, 6796 (2015).
[Crossref]

2014 (4)

2013 (2)

A. Kadambi, R. Whyte, A. Bhandari, L. Streeter, C. Barsi, A. Dorrington, and R. Raskar, “Coded time of flight cameras: sparse deconvolution to address multipath interference and recover time profiles,” ACM Trans. Graph. 32, 167 (2013).
[Crossref]

F. Heide, M. B. Hullin, J. Gregson, and W. Heidrich, “Low-budget transient imaging using photonic mixer devices,” ACM Trans. Graph. 32, 1 (2013).

2011 (1)

S. Foix, G. Alenya, and C. Torras, “Lock-in time-of-flight (ToF) cameras: a survey,” IEEE Sens. J. 11, 1917–1926 (2011).
[Crossref]

2009 (1)

2008 (4)

A. T. Kumar, S. B. Raymond, B. J. Bacskai, and D. A. Boas, “Comparison of frequency-domain and time-domain fluorescence lifetime tomography,” Opt. Lett. 33, 470–472 (2008).
[Crossref]

A. T. Kumar, S. B. Raymond, A. K. Dunn, B. J. Bacskai, and D. A. Boas, “A time domain fluorescence tomography system for small animal imaging,” IEEE Trans. Med. Imaging 27, 1152–1163 (2008).
[Crossref]

G.-J. Kremers, E. B. Van Munster, J. Goedhart, and T. W. Gadella, “Quantitative lifetime unmixing of multiexponentially decaying fluorophores using single-frequency fluorescence lifetime imaging microscopy,” Biophys. J. 95, 378–389 (2008).
[Crossref]

M. A. Digman, V. R. Caiolfa, M. Zamai, and E. Gratton, “The phasor approach to fluorescence lifetime imaging analysis,” Biophys. J. 94, L14–L16 (2008).
[Crossref]

2007 (2)

2005 (1)

2003 (2)

B. J. Bacskai, J. Skoch, G. A. Hickey, R. Allen, and B. T. Hyman, “Fluorescence resonance energy transfer determinations using multiphoton fluorescence lifetime imaging microscopy to characterize amyloid-beta plaques,” J. Biomed. Opt. 8, 368–375 (2003).
[Crossref]

J. Philip and K. Carlsson, “Theoretical investigation of the signal-to-noise ratio in fluorescence lifetime imaging,” J. Opt. Soc. Am. A 20, 368–379 (2003).
[Crossref]

2001 (1)

Q. S. Hanley, V. Subramaniam, D. J. Arndt-Jovin, and T. M. Jovin, “Fluorescence lifetime imaging: multi-point calibration, minimum resolvable differences, and artifact suppression,” Cytometry 43, 248–260 (2001).
[Crossref]

2000 (1)

A. Squire, P. J. Verveer, and P. I. H. Bastiaens, “Multiple frequency fluorescence lifetime imaging microscopy,” J. Microsc. 197, 136–149 (2000).
[Crossref]

1998 (1)

H. He, B. K. Nunnally, L.-C. Li, and L. B. McGown, “On-the-fly fluorescence lifetime detection of dye-labeled DNA primers for multiplex analysis,” Anal. Chem. 70, 3413–3418 (1998).
[Crossref]

1997 (1)

1996 (1)

T. F. Coleman and Y. Li, “An interior trust region approach for nonlinear minimization subject to bounds,” SIAM J. Optim. 6, 418–445 (1996).
[Crossref]

1984 (1)

E. Gratton, M. Limkeman, J. R. Lakowicz, B. P. Maliwal, H. Cherek, and G. Laczko, “Resolution of mixtures of fluorophores using variable-frequency phase and modulation data,” Biophys. J. 46, 479–486 (1984).
[Crossref]

Akbari, N.

Alenya, G.

S. Foix, G. Alenya, and C. Torras, “Lock-in time-of-flight (ToF) cameras: a survey,” IEEE Sens. J. 11, 1917–1926 (2011).
[Crossref]

Allen, R.

B. J. Bacskai, J. Skoch, G. A. Hickey, R. Allen, and B. T. Hyman, “Fluorescence resonance energy transfer determinations using multiphoton fluorescence lifetime imaging microscopy to characterize amyloid-beta plaques,” J. Biomed. Opt. 8, 368–375 (2003).
[Crossref]

Arndt-Jovin, D. J.

Q. S. Hanley, V. Subramaniam, D. J. Arndt-Jovin, and T. M. Jovin, “Fluorescence lifetime imaging: multi-point calibration, minimum resolvable differences, and artifact suppression,” Cytometry 43, 248–260 (2001).
[Crossref]

Bacskai, B. J.

A. T. Kumar, S. B. Raymond, A. K. Dunn, B. J. Bacskai, and D. A. Boas, “A time domain fluorescence tomography system for small animal imaging,” IEEE Trans. Med. Imaging 27, 1152–1163 (2008).
[Crossref]

A. T. Kumar, S. B. Raymond, B. J. Bacskai, and D. A. Boas, “Comparison of frequency-domain and time-domain fluorescence lifetime tomography,” Opt. Lett. 33, 470–472 (2008).
[Crossref]

B. J. Bacskai, J. Skoch, G. A. Hickey, R. Allen, and B. T. Hyman, “Fluorescence resonance energy transfer determinations using multiphoton fluorescence lifetime imaging microscopy to characterize amyloid-beta plaques,” J. Biomed. Opt. 8, 368–375 (2003).
[Crossref]

Barsi, C.

G. Satat, B. Heshmat, C. Barsi, D. Raviv, O. Chen, M. G. Bawendi, and R. Raskar, “Locating and classifying fluorescent tags behind turbid layers using time-resolved inversion,” Nat. Commun. 6, 6796 (2015).
[Crossref]

A. Bhandari, A. Kadambi, R. Whyte, C. Barsi, M. Feigin, A. Dorrington, and R. Raskar, “Resolving multipath interference in time-of-flight imaging via modulation frequency diversity and sparse regularization,” Opt. Lett. 39, 1705–1708 (2014).
[Crossref]

A. Kadambi, R. Whyte, A. Bhandari, L. Streeter, C. Barsi, A. Dorrington, and R. Raskar, “Coded time of flight cameras: sparse deconvolution to address multipath interference and recover time profiles,” ACM Trans. Graph. 32, 167 (2013).
[Crossref]

Bastiaens, P. I. H.

A. Squire, P. J. Verveer, and P. I. H. Bastiaens, “Multiple frequency fluorescence lifetime imaging microscopy,” J. Microsc. 197, 136–149 (2000).
[Crossref]

Bawendi, M. G.

G. Satat, B. Heshmat, C. Barsi, D. Raviv, O. Chen, M. G. Bawendi, and R. Raskar, “Locating and classifying fluorescent tags behind turbid layers using time-resolved inversion,” Nat. Commun. 6, 6796 (2015).
[Crossref]

Bec, J.

D. R. Yankelevich, D. Ma, J. Liu, Y. Sun, Y. Sun, J. Bec, D. S. Elson, and L. Marcu, “Design and evaluation of a device for fast multispectral time-resolved fluorescence spectroscopy and imaging,” Rev. Sci. Instrum. 85, 034303 (2014).
[Crossref]

Bhandari, A.

A. Bhandari, A. Kadambi, R. Whyte, C. Barsi, M. Feigin, A. Dorrington, and R. Raskar, “Resolving multipath interference in time-of-flight imaging via modulation frequency diversity and sparse regularization,” Opt. Lett. 39, 1705–1708 (2014).
[Crossref]

A. Kadambi, R. Whyte, A. Bhandari, L. Streeter, C. Barsi, A. Dorrington, and R. Raskar, “Coded time of flight cameras: sparse deconvolution to address multipath interference and recover time profiles,” ACM Trans. Graph. 32, 167 (2013).
[Crossref]

A. Bhandari, A. Kadambi, and R. Raskar, “Sparse linear operator identification without sparse regularization? Applications to mixed pixel problem in time-of-flight/range imaging,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2014), pp. 365–369.

A. Bhandari, M. Feigin, S. Izadi, C. Rhemann, M. Schmidt, and R. Raskar, “Resolving multipath interference in Kinect: An inverse problem approach,” in IEEE Sensors (IEEE, 2014), pp. 614–617.

A. Kadambi, A. Bhandari, R. Whyte, A. Dorrington, and R. Raskar, “Demultiplexing illumination via low cost sensing and nanosecond coding,” in IEEE International Conference on Computational Photography (ICCP) (2014).

A. Bhandari, A. Bourquard, S. Izadi, and R. Raskar, “Blind transmitted and reflected image separation using depth diversity and time-of-flight sensors,” in Computational Optical Sensing and Imaging (Optical Society of America, 2015), paper CT4F-2.

Boas, D. A.

A. T. Kumar, S. B. Raymond, B. J. Bacskai, and D. A. Boas, “Comparison of frequency-domain and time-domain fluorescence lifetime tomography,” Opt. Lett. 33, 470–472 (2008).
[Crossref]

A. T. Kumar, S. B. Raymond, A. K. Dunn, B. J. Bacskai, and D. A. Boas, “A time domain fluorescence tomography system for small animal imaging,” IEEE Trans. Med. Imaging 27, 1152–1163 (2008).
[Crossref]

Bourquard, A.

A. Bhandari, A. Bourquard, S. Izadi, and R. Raskar, “Blind transmitted and reflected image separation using depth diversity and time-of-flight sensors,” in Computational Optical Sensing and Imaging (Optical Society of America, 2015), paper CT4F-2.

Buckley, B. W.

Caiolfa, V. R.

M. A. Digman, V. R. Caiolfa, M. Zamai, and E. Gratton, “The phasor approach to fluorescence lifetime imaging analysis,” Biophys. J. 94, L14–L16 (2008).
[Crossref]

Carli, P.

Carlsson, K.

Chan, J. C. K.

Chen, O.

G. Satat, B. Heshmat, C. Barsi, D. Raviv, O. Chen, M. G. Bawendi, and R. Raskar, “Locating and classifying fluorescent tags behind turbid layers using time-resolved inversion,” Nat. Commun. 6, 6796 (2015).
[Crossref]

Cherek, H.

E. Gratton, M. Limkeman, J. R. Lakowicz, B. P. Maliwal, H. Cherek, and G. Laczko, “Resolution of mixtures of fluorophores using variable-frequency phase and modulation data,” Biophys. J. 46, 479–486 (1984).
[Crossref]

Chuang, F. S.

Cicchi, R.

Coleman, T. F.

T. F. Coleman and Y. Li, “An interior trust region approach for nonlinear minimization subject to bounds,” SIAM J. Optim. 6, 418–445 (1996).
[Crossref]

Cree, M. J.

D. Lefloch, R. Nair, F. Lenzen, H. Schäfer, L. Streeter, M. J. Cree, R. Koch, and A. Kolb, “Technical foundation and calibration methods for time-of-flight cameras,” in Time-of-Flight and Depth Imaging: Sensors, Algorithms, and Applications (Springer, 2013), pp. 3–24.

Dai, Q.

Davis, J.

S. Schuon, C. Theobalt, J. Davis, and S. Thrun, “Lidarboost: Depth superresolution for ToF 3D shape scanning,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2009), pp. 343–350.

De Giorgi, V.

Diebold, E. D.

Digman, M. A.

M. A. Digman, V. R. Caiolfa, M. Zamai, and E. Gratton, “The phasor approach to fluorescence lifetime imaging analysis,” Biophys. J. 94, L14–L16 (2008).
[Crossref]

Dorrington, A.

A. Bhandari, A. Kadambi, R. Whyte, C. Barsi, M. Feigin, A. Dorrington, and R. Raskar, “Resolving multipath interference in time-of-flight imaging via modulation frequency diversity and sparse regularization,” Opt. Lett. 39, 1705–1708 (2014).
[Crossref]

A. Kadambi, R. Whyte, A. Bhandari, L. Streeter, C. Barsi, A. Dorrington, and R. Raskar, “Coded time of flight cameras: sparse deconvolution to address multipath interference and recover time profiles,” ACM Trans. Graph. 32, 167 (2013).
[Crossref]

A. Kadambi, A. Bhandari, R. Whyte, A. Dorrington, and R. Raskar, “Demultiplexing illumination via low cost sensing and nanosecond coding,” in IEEE International Conference on Computational Photography (ICCP) (2014).

Dragulescu-Andrasi, A.

J. Rao, A. Dragulescu-Andrasi, and H. Yao, “Fluorescence imaging in vivo: recent advances,” Curr. Opin. Biotechnol. 18, 17–25 (2007).
[Crossref]

Dunn, A. K.

A. T. Kumar, S. B. Raymond, A. K. Dunn, B. J. Bacskai, and D. A. Boas, “A time domain fluorescence tomography system for small animal imaging,” IEEE Trans. Med. Imaging 27, 1152–1163 (2008).
[Crossref]

Elson, D. S.

D. R. Yankelevich, D. Ma, J. Liu, Y. Sun, Y. Sun, J. Bec, D. S. Elson, and L. Marcu, “Design and evaluation of a device for fast multispectral time-resolved fluorescence spectroscopy and imaging,” Rev. Sci. Instrum. 85, 034303 (2014).
[Crossref]

Y. Sun, J. Phipps, D. S. Elson, H. Stoy, S. Tinling, J. Meier, B. Poirier, F. S. Chuang, D. G. Farwell, and L. Marcu, “Fluorescence lifetime imaging microscopy: in vivo application to diagnosis of oral carcinoma,” Opt. Lett. 34, 2081–2083 (2009).
[Crossref]

Esposito, A.

Farwell, D. G.

Feigin, M.

A. Bhandari, A. Kadambi, R. Whyte, C. Barsi, M. Feigin, A. Dorrington, and R. Raskar, “Resolving multipath interference in time-of-flight imaging via modulation frequency diversity and sparse regularization,” Opt. Lett. 39, 1705–1708 (2014).
[Crossref]

A. Bhandari, M. Feigin, S. Izadi, C. Rhemann, M. Schmidt, and R. Raskar, “Resolving multipath interference in Kinect: An inverse problem approach,” in IEEE Sensors (IEEE, 2014), pp. 614–617.

Foix, S.

S. Foix, G. Alenya, and C. Torras, “Lock-in time-of-flight (ToF) cameras: a survey,” IEEE Sens. J. 11, 1917–1926 (2011).
[Crossref]

Gadella, T. W.

G.-J. Kremers, E. B. Van Munster, J. Goedhart, and T. W. Gadella, “Quantitative lifetime unmixing of multiexponentially decaying fluorophores using single-frequency fluorescence lifetime imaging microscopy,” Biophys. J. 95, 378–389 (2008).
[Crossref]

Gerritsen, H.

Goedhart, J.

G.-J. Kremers, E. B. Van Munster, J. Goedhart, and T. W. Gadella, “Quantitative lifetime unmixing of multiexponentially decaying fluorophores using single-frequency fluorescence lifetime imaging microscopy,” Biophys. J. 95, 378–389 (2008).
[Crossref]

Golomb, S. W.

S. W. Golomb and G. Gong, Signal Design for Good Correlation: For Wireless Communication, Cryptography, and Radar (Cambridge University, 2005).

Gong, G.

S. W. Golomb and G. Gong, Signal Design for Good Correlation: For Wireless Communication, Cryptography, and Radar (Cambridge University, 2005).

Gratton, E.

M. A. Digman, V. R. Caiolfa, M. Zamai, and E. Gratton, “The phasor approach to fluorescence lifetime imaging analysis,” Biophys. J. 94, L14–L16 (2008).
[Crossref]

E. Gratton, M. Limkeman, J. R. Lakowicz, B. P. Maliwal, H. Cherek, and G. Laczko, “Resolution of mixtures of fluorophores using variable-frequency phase and modulation data,” Biophys. J. 46, 479–486 (1984).
[Crossref]

Gregson, J.

F. Heide, M. B. Hullin, J. Gregson, and W. Heidrich, “Low-budget transient imaging using photonic mixer devices,” ACM Trans. Graph. 32, 1 (2013).

Hanley, Q. S.

Q. S. Hanley, V. Subramaniam, D. J. Arndt-Jovin, and T. M. Jovin, “Fluorescence lifetime imaging: multi-point calibration, minimum resolvable differences, and artifact suppression,” Cytometry 43, 248–260 (2001).
[Crossref]

He, H.

H. He, B. K. Nunnally, L.-C. Li, and L. B. McGown, “On-the-fly fluorescence lifetime detection of dye-labeled DNA primers for multiplex analysis,” Anal. Chem. 70, 3413–3418 (1998).
[Crossref]

Heide, F.

F. Heide, L. Xiao, A. Kolb, M. B. Hullin, and W. Heidrich, “Imaging in scattering media using correlation image sensors and sparse convolutional coding,” Opt. Express 22, 26338–26350 (2014).
[Crossref]

F. Heide, M. B. Hullin, J. Gregson, and W. Heidrich, “Low-budget transient imaging using photonic mixer devices,” ACM Trans. Graph. 32, 1 (2013).

F. Heide, L. Xiao, W. Heidrich, and M. B. Hullin, “Diffuse mirrors: 3D reconstruction from diffuse indirect illumination using inexpensive time-of-flight sensors,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014), pp. 3222–3229.

Heidrich, W.

F. Heide, L. Xiao, A. Kolb, M. B. Hullin, and W. Heidrich, “Imaging in scattering media using correlation image sensors and sparse convolutional coding,” Opt. Express 22, 26338–26350 (2014).
[Crossref]

F. Heide, M. B. Hullin, J. Gregson, and W. Heidrich, “Low-budget transient imaging using photonic mixer devices,” ACM Trans. Graph. 32, 1 (2013).

F. Heide, L. Xiao, W. Heidrich, and M. B. Hullin, “Diffuse mirrors: 3D reconstruction from diffuse indirect illumination using inexpensive time-of-flight sensors,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014), pp. 3222–3229.

Heshmat, B.

G. Satat, B. Heshmat, C. Barsi, D. Raviv, O. Chen, M. G. Bawendi, and R. Raskar, “Locating and classifying fluorescent tags behind turbid layers using time-resolved inversion,” Nat. Commun. 6, 6796 (2015).
[Crossref]

Hickey, G. A.

B. J. Bacskai, J. Skoch, G. A. Hickey, R. Allen, and B. T. Hyman, “Fluorescence resonance energy transfer determinations using multiphoton fluorescence lifetime imaging microscopy to characterize amyloid-beta plaques,” J. Biomed. Opt. 8, 368–375 (2003).
[Crossref]

Hullin, M. B.

H. Qiao, J. Lin, Y. Liu, M. B. Hullin, and Q. Dai, “Resolving transient time profile in ToF imaging via log-sum sparse regularization,” Opt. Lett. 40, 918–921 (2015).
[Crossref]

F. Heide, L. Xiao, A. Kolb, M. B. Hullin, and W. Heidrich, “Imaging in scattering media using correlation image sensors and sparse convolutional coding,” Opt. Express 22, 26338–26350 (2014).
[Crossref]

F. Heide, M. B. Hullin, J. Gregson, and W. Heidrich, “Low-budget transient imaging using photonic mixer devices,” ACM Trans. Graph. 32, 1 (2013).

F. Heide, L. Xiao, W. Heidrich, and M. B. Hullin, “Diffuse mirrors: 3D reconstruction from diffuse indirect illumination using inexpensive time-of-flight sensors,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014), pp. 3222–3229.

Hyman, B. T.

B. J. Bacskai, J. Skoch, G. A. Hickey, R. Allen, and B. T. Hyman, “Fluorescence resonance energy transfer determinations using multiphoton fluorescence lifetime imaging microscopy to characterize amyloid-beta plaques,” J. Biomed. Opt. 8, 368–375 (2003).
[Crossref]

Izadi, S.

A. Bhandari, M. Feigin, S. Izadi, C. Rhemann, M. Schmidt, and R. Raskar, “Resolving multipath interference in Kinect: An inverse problem approach,” in IEEE Sensors (IEEE, 2014), pp. 614–617.

A. Bhandari, A. Bourquard, S. Izadi, and R. Raskar, “Blind transmitted and reflected image separation using depth diversity and time-of-flight sensors,” in Computational Optical Sensing and Imaging (Optical Society of America, 2015), paper CT4F-2.

Jalali, B.

Jovin, T. M.

Q. S. Hanley, V. Subramaniam, D. J. Arndt-Jovin, and T. M. Jovin, “Fluorescence lifetime imaging: multi-point calibration, minimum resolvable differences, and artifact suppression,” Cytometry 43, 248–260 (2001).
[Crossref]

Kadambi, A.

A. Bhandari, A. Kadambi, R. Whyte, C. Barsi, M. Feigin, A. Dorrington, and R. Raskar, “Resolving multipath interference in time-of-flight imaging via modulation frequency diversity and sparse regularization,” Opt. Lett. 39, 1705–1708 (2014).
[Crossref]

A. Kadambi, R. Whyte, A. Bhandari, L. Streeter, C. Barsi, A. Dorrington, and R. Raskar, “Coded time of flight cameras: sparse deconvolution to address multipath interference and recover time profiles,” ACM Trans. Graph. 32, 167 (2013).
[Crossref]

A. Bhandari, A. Kadambi, and R. Raskar, “Sparse linear operator identification without sparse regularization? Applications to mixed pixel problem in time-of-flight/range imaging,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2014), pp. 365–369.

A. Kadambi, A. Bhandari, R. Whyte, A. Dorrington, and R. Raskar, “Demultiplexing illumination via low cost sensing and nanosecond coding,” in IEEE International Conference on Computational Photography (ICCP) (2014).

Koch, R.

D. Lefloch, R. Nair, F. Lenzen, H. Schäfer, L. Streeter, M. J. Cree, R. Koch, and A. Kolb, “Technical foundation and calibration methods for time-of-flight cameras,” in Time-of-Flight and Depth Imaging: Sensors, Algorithms, and Applications (Springer, 2013), pp. 3–24.

Kolb, A.

F. Heide, L. Xiao, A. Kolb, M. B. Hullin, and W. Heidrich, “Imaging in scattering media using correlation image sensors and sparse convolutional coding,” Opt. Express 22, 26338–26350 (2014).
[Crossref]

D. Lefloch, R. Nair, F. Lenzen, H. Schäfer, L. Streeter, M. J. Cree, R. Koch, and A. Kolb, “Technical foundation and calibration methods for time-of-flight cameras,” in Time-of-Flight and Depth Imaging: Sensors, Algorithms, and Applications (Springer, 2013), pp. 3–24.

Kremers, G.-J.

G.-J. Kremers, E. B. Van Munster, J. Goedhart, and T. W. Gadella, “Quantitative lifetime unmixing of multiexponentially decaying fluorophores using single-frequency fluorescence lifetime imaging microscopy,” Biophys. J. 95, 378–389 (2008).
[Crossref]

Kumar, A. T.

A. T. Kumar, S. B. Raymond, B. J. Bacskai, and D. A. Boas, “Comparison of frequency-domain and time-domain fluorescence lifetime tomography,” Opt. Lett. 33, 470–472 (2008).
[Crossref]

A. T. Kumar, S. B. Raymond, A. K. Dunn, B. J. Bacskai, and D. A. Boas, “A time domain fluorescence tomography system for small animal imaging,” IEEE Trans. Med. Imaging 27, 1152–1163 (2008).
[Crossref]

Kung, S. Y.

S. Y. Kung, Kernel Methods and Machine Learning (Cambridge University, 2014).

Laczko, G.

E. Gratton, M. Limkeman, J. R. Lakowicz, B. P. Maliwal, H. Cherek, and G. Laczko, “Resolution of mixtures of fluorophores using variable-frequency phase and modulation data,” Biophys. J. 46, 479–486 (1984).
[Crossref]

Lakowicz, J.

J. Lakowicz, Principles of Fluorescence Microscopy (Springer, 1982).

Lakowicz, J. R.

E. Gratton, M. Limkeman, J. R. Lakowicz, B. P. Maliwal, H. Cherek, and G. Laczko, “Resolution of mixtures of fluorophores using variable-frequency phase and modulation data,” Biophys. J. 46, 479–486 (1984).
[Crossref]

Lefloch, D.

D. Lefloch, R. Nair, F. Lenzen, H. Schäfer, L. Streeter, M. J. Cree, R. Koch, and A. Kolb, “Technical foundation and calibration methods for time-of-flight cameras,” in Time-of-Flight and Depth Imaging: Sensors, Algorithms, and Applications (Springer, 2013), pp. 3–24.

Lenzen, F.

D. Lefloch, R. Nair, F. Lenzen, H. Schäfer, L. Streeter, M. J. Cree, R. Koch, and A. Kolb, “Technical foundation and calibration methods for time-of-flight cameras,” in Time-of-Flight and Depth Imaging: Sensors, Algorithms, and Applications (Springer, 2013), pp. 3–24.

Li, L.-C.

H. He, B. K. Nunnally, L.-C. Li, and L. B. McGown, “On-the-fly fluorescence lifetime detection of dye-labeled DNA primers for multiplex analysis,” Anal. Chem. 70, 3413–3418 (1998).
[Crossref]

Li, Y.

T. F. Coleman and Y. Li, “An interior trust region approach for nonlinear minimization subject to bounds,” SIAM J. Optim. 6, 418–445 (1996).
[Crossref]

Limkeman, M.

E. Gratton, M. Limkeman, J. R. Lakowicz, B. P. Maliwal, H. Cherek, and G. Laczko, “Resolution of mixtures of fluorophores using variable-frequency phase and modulation data,” Biophys. J. 46, 479–486 (1984).
[Crossref]

Lin, J.

Liu, J.

D. R. Yankelevich, D. Ma, J. Liu, Y. Sun, Y. Sun, J. Bec, D. S. Elson, and L. Marcu, “Design and evaluation of a device for fast multispectral time-resolved fluorescence spectroscopy and imaging,” Rev. Sci. Instrum. 85, 034303 (2014).
[Crossref]

Liu, Y.

Lotti, T.

Lustenberger, F.

Ma, D.

D. R. Yankelevich, D. Ma, J. Liu, Y. Sun, Y. Sun, J. Bec, D. S. Elson, and L. Marcu, “Design and evaluation of a device for fast multispectral time-resolved fluorescence spectroscopy and imaging,” Rev. Sci. Instrum. 85, 034303 (2014).
[Crossref]

Maliwal, B. P.

E. Gratton, M. Limkeman, J. R. Lakowicz, B. P. Maliwal, H. Cherek, and G. Laczko, “Resolution of mixtures of fluorophores using variable-frequency phase and modulation data,” Biophys. J. 46, 479–486 (1984).
[Crossref]

Mao, S.

Marcu, L.

D. R. Yankelevich, D. Ma, J. Liu, Y. Sun, Y. Sun, J. Bec, D. S. Elson, and L. Marcu, “Design and evaluation of a device for fast multispectral time-resolved fluorescence spectroscopy and imaging,” Rev. Sci. Instrum. 85, 034303 (2014).
[Crossref]

Y. Sun, J. Phipps, D. S. Elson, H. Stoy, S. Tinling, J. Meier, B. Poirier, F. S. Chuang, D. G. Farwell, and L. Marcu, “Fluorescence lifetime imaging microscopy: in vivo application to diagnosis of oral carcinoma,” Opt. Lett. 34, 2081–2083 (2009).
[Crossref]

Massi, D.

McGown, L. B.

H. He, B. K. Nunnally, L.-C. Li, and L. B. McGown, “On-the-fly fluorescence lifetime detection of dye-labeled DNA primers for multiplex analysis,” Anal. Chem. 70, 3413–3418 (1998).
[Crossref]

Meier, J.

Nair, R.

D. Lefloch, R. Nair, F. Lenzen, H. Schäfer, L. Streeter, M. J. Cree, R. Koch, and A. Kolb, “Technical foundation and calibration methods for time-of-flight cameras,” in Time-of-Flight and Depth Imaging: Sensors, Algorithms, and Applications (Springer, 2013), pp. 3–24.

Nunnally, B. K.

H. He, B. K. Nunnally, L.-C. Li, and L. B. McGown, “On-the-fly fluorescence lifetime detection of dye-labeled DNA primers for multiplex analysis,” Anal. Chem. 70, 3413–3418 (1998).
[Crossref]

Oggier, T.

Pavone, F.

Philip, J.

Phipps, J.

Poirier, B.

Qiao, H.

Rao, J.

J. Rao, A. Dragulescu-Andrasi, and H. Yao, “Fluorescence imaging in vivo: recent advances,” Curr. Opin. Biotechnol. 18, 17–25 (2007).
[Crossref]

Raskar, R.

G. Satat, B. Heshmat, C. Barsi, D. Raviv, O. Chen, M. G. Bawendi, and R. Raskar, “Locating and classifying fluorescent tags behind turbid layers using time-resolved inversion,” Nat. Commun. 6, 6796 (2015).
[Crossref]

A. Bhandari, A. Kadambi, R. Whyte, C. Barsi, M. Feigin, A. Dorrington, and R. Raskar, “Resolving multipath interference in time-of-flight imaging via modulation frequency diversity and sparse regularization,” Opt. Lett. 39, 1705–1708 (2014).
[Crossref]

A. Kadambi, R. Whyte, A. Bhandari, L. Streeter, C. Barsi, A. Dorrington, and R. Raskar, “Coded time of flight cameras: sparse deconvolution to address multipath interference and recover time profiles,” ACM Trans. Graph. 32, 167 (2013).
[Crossref]

A. Bhandari, A. Bourquard, S. Izadi, and R. Raskar, “Blind transmitted and reflected image separation using depth diversity and time-of-flight sensors,” in Computational Optical Sensing and Imaging (Optical Society of America, 2015), paper CT4F-2.

A. Bhandari, A. Kadambi, and R. Raskar, “Sparse linear operator identification without sparse regularization? Applications to mixed pixel problem in time-of-flight/range imaging,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2014), pp. 365–369.

A. Bhandari, M. Feigin, S. Izadi, C. Rhemann, M. Schmidt, and R. Raskar, “Resolving multipath interference in Kinect: An inverse problem approach,” in IEEE Sensors (IEEE, 2014), pp. 614–617.

A. Kadambi, A. Bhandari, R. Whyte, A. Dorrington, and R. Raskar, “Demultiplexing illumination via low cost sensing and nanosecond coding,” in IEEE International Conference on Computational Photography (ICCP) (2014).

Raviv, D.

G. Satat, B. Heshmat, C. Barsi, D. Raviv, O. Chen, M. G. Bawendi, and R. Raskar, “Locating and classifying fluorescent tags behind turbid layers using time-resolved inversion,” Nat. Commun. 6, 6796 (2015).
[Crossref]

Raymond, S. B.

A. T. Kumar, S. B. Raymond, B. J. Bacskai, and D. A. Boas, “Comparison of frequency-domain and time-domain fluorescence lifetime tomography,” Opt. Lett. 33, 470–472 (2008).
[Crossref]

A. T. Kumar, S. B. Raymond, A. K. Dunn, B. J. Bacskai, and D. A. Boas, “A time domain fluorescence tomography system for small animal imaging,” IEEE Trans. Med. Imaging 27, 1152–1163 (2008).
[Crossref]

Rhemann, C.

A. Bhandari, M. Feigin, S. Izadi, C. Rhemann, M. Schmidt, and R. Raskar, “Resolving multipath interference in Kinect: An inverse problem approach,” in IEEE Sensors (IEEE, 2014), pp. 614–617.

Satat, G.

G. Satat, B. Heshmat, C. Barsi, D. Raviv, O. Chen, M. G. Bawendi, and R. Raskar, “Locating and classifying fluorescent tags behind turbid layers using time-resolved inversion,” Nat. Commun. 6, 6796 (2015).
[Crossref]

Schäfer, H.

D. Lefloch, R. Nair, F. Lenzen, H. Schäfer, L. Streeter, M. J. Cree, R. Koch, and A. Kolb, “Technical foundation and calibration methods for time-of-flight cameras,” in Time-of-Flight and Depth Imaging: Sensors, Algorithms, and Applications (Springer, 2013), pp. 3–24.

Schmidt, M.

A. Bhandari, M. Feigin, S. Izadi, C. Rhemann, M. Schmidt, and R. Raskar, “Resolving multipath interference in Kinect: An inverse problem approach,” in IEEE Sensors (IEEE, 2014), pp. 614–617.

Schuon, S.

S. Schuon, C. Theobalt, J. Davis, and S. Thrun, “Lidarboost: Depth superresolution for ToF 3D shape scanning,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2009), pp. 343–350.

Sestini, S.

Skoch, J.

B. J. Bacskai, J. Skoch, G. A. Hickey, R. Allen, and B. T. Hyman, “Fluorescence resonance energy transfer determinations using multiphoton fluorescence lifetime imaging microscopy to characterize amyloid-beta plaques,” J. Biomed. Opt. 8, 368–375 (2003).
[Crossref]

Squire, A.

A. Squire, P. J. Verveer, and P. I. H. Bastiaens, “Multiple frequency fluorescence lifetime imaging microscopy,” J. Microsc. 197, 136–149 (2000).
[Crossref]

Stoy, H.

Streeter, L.

A. Kadambi, R. Whyte, A. Bhandari, L. Streeter, C. Barsi, A. Dorrington, and R. Raskar, “Coded time of flight cameras: sparse deconvolution to address multipath interference and recover time profiles,” ACM Trans. Graph. 32, 167 (2013).
[Crossref]

D. Lefloch, R. Nair, F. Lenzen, H. Schäfer, L. Streeter, M. J. Cree, R. Koch, and A. Kolb, “Technical foundation and calibration methods for time-of-flight cameras,” in Time-of-Flight and Depth Imaging: Sensors, Algorithms, and Applications (Springer, 2013), pp. 3–24.

Subramaniam, V.

Q. S. Hanley, V. Subramaniam, D. J. Arndt-Jovin, and T. M. Jovin, “Fluorescence lifetime imaging: multi-point calibration, minimum resolvable differences, and artifact suppression,” Cytometry 43, 248–260 (2001).
[Crossref]

Sun, Y.

D. R. Yankelevich, D. Ma, J. Liu, Y. Sun, Y. Sun, J. Bec, D. S. Elson, and L. Marcu, “Design and evaluation of a device for fast multispectral time-resolved fluorescence spectroscopy and imaging,” Rev. Sci. Instrum. 85, 034303 (2014).
[Crossref]

D. R. Yankelevich, D. Ma, J. Liu, Y. Sun, Y. Sun, J. Bec, D. S. Elson, and L. Marcu, “Design and evaluation of a device for fast multispectral time-resolved fluorescence spectroscopy and imaging,” Rev. Sci. Instrum. 85, 034303 (2014).
[Crossref]

Y. Sun, J. Phipps, D. S. Elson, H. Stoy, S. Tinling, J. Meier, B. Poirier, F. S. Chuang, D. G. Farwell, and L. Marcu, “Fluorescence lifetime imaging microscopy: in vivo application to diagnosis of oral carcinoma,” Opt. Lett. 34, 2081–2083 (2009).
[Crossref]

Theobalt, C.

S. Schuon, C. Theobalt, J. Davis, and S. Thrun, “Lidarboost: Depth superresolution for ToF 3D shape scanning,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2009), pp. 343–350.

Thrun, S.

S. Schuon, C. Theobalt, J. Davis, and S. Thrun, “Lidarboost: Depth superresolution for ToF 3D shape scanning,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2009), pp. 343–350.

Tinling, S.

Torras, C.

S. Foix, G. Alenya, and C. Torras, “Lock-in time-of-flight (ToF) cameras: a survey,” IEEE Sens. J. 11, 1917–1926 (2011).
[Crossref]

Van Munster, E. B.

G.-J. Kremers, E. B. Van Munster, J. Goedhart, and T. W. Gadella, “Quantitative lifetime unmixing of multiexponentially decaying fluorophores using single-frequency fluorescence lifetime imaging microscopy,” Biophys. J. 95, 378–389 (2008).
[Crossref]

Verveer, P. J.

A. Squire, P. J. Verveer, and P. I. H. Bastiaens, “Multiple frequency fluorescence lifetime imaging microscopy,” J. Microsc. 197, 136–149 (2000).
[Crossref]

Whyte, R.

A. Bhandari, A. Kadambi, R. Whyte, C. Barsi, M. Feigin, A. Dorrington, and R. Raskar, “Resolving multipath interference in time-of-flight imaging via modulation frequency diversity and sparse regularization,” Opt. Lett. 39, 1705–1708 (2014).
[Crossref]

A. Kadambi, R. Whyte, A. Bhandari, L. Streeter, C. Barsi, A. Dorrington, and R. Raskar, “Coded time of flight cameras: sparse deconvolution to address multipath interference and recover time profiles,” ACM Trans. Graph. 32, 167 (2013).
[Crossref]

A. Kadambi, A. Bhandari, R. Whyte, A. Dorrington, and R. Raskar, “Demultiplexing illumination via low cost sensing and nanosecond coding,” in IEEE International Conference on Computational Photography (ICCP) (2014).

Wouters, F.

Xiao, L.

F. Heide, L. Xiao, A. Kolb, M. B. Hullin, and W. Heidrich, “Imaging in scattering media using correlation image sensors and sparse convolutional coding,” Opt. Express 22, 26338–26350 (2014).
[Crossref]

F. Heide, L. Xiao, W. Heidrich, and M. B. Hullin, “Diffuse mirrors: 3D reconstruction from diffuse indirect illumination using inexpensive time-of-flight sensors,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014), pp. 3222–3229.

Yamaguchi, I.

Yankelevich, D. R.

D. R. Yankelevich, D. Ma, J. Liu, Y. Sun, Y. Sun, J. Bec, D. S. Elson, and L. Marcu, “Design and evaluation of a device for fast multispectral time-resolved fluorescence spectroscopy and imaging,” Rev. Sci. Instrum. 85, 034303 (2014).
[Crossref]

Yao, H.

J. Rao, A. Dragulescu-Andrasi, and H. Yao, “Fluorescence imaging in vivo: recent advances,” Curr. Opin. Biotechnol. 18, 17–25 (2007).
[Crossref]

Zamai, M.

M. A. Digman, V. R. Caiolfa, M. Zamai, and E. Gratton, “The phasor approach to fluorescence lifetime imaging analysis,” Biophys. J. 94, L14–L16 (2008).
[Crossref]

Zhang, T.

ACM Trans. Graph. (2)

A. Kadambi, R. Whyte, A. Bhandari, L. Streeter, C. Barsi, A. Dorrington, and R. Raskar, “Coded time of flight cameras: sparse deconvolution to address multipath interference and recover time profiles,” ACM Trans. Graph. 32, 167 (2013).
[Crossref]

F. Heide, M. B. Hullin, J. Gregson, and W. Heidrich, “Low-budget transient imaging using photonic mixer devices,” ACM Trans. Graph. 32, 1 (2013).

Anal. Chem. (1)

H. He, B. K. Nunnally, L.-C. Li, and L. B. McGown, “On-the-fly fluorescence lifetime detection of dye-labeled DNA primers for multiplex analysis,” Anal. Chem. 70, 3413–3418 (1998).
[Crossref]

Biomed. Opt. Express (1)

Biophys. J. (3)

E. Gratton, M. Limkeman, J. R. Lakowicz, B. P. Maliwal, H. Cherek, and G. Laczko, “Resolution of mixtures of fluorophores using variable-frequency phase and modulation data,” Biophys. J. 46, 479–486 (1984).
[Crossref]

G.-J. Kremers, E. B. Van Munster, J. Goedhart, and T. W. Gadella, “Quantitative lifetime unmixing of multiexponentially decaying fluorophores using single-frequency fluorescence lifetime imaging microscopy,” Biophys. J. 95, 378–389 (2008).
[Crossref]

M. A. Digman, V. R. Caiolfa, M. Zamai, and E. Gratton, “The phasor approach to fluorescence lifetime imaging analysis,” Biophys. J. 94, L14–L16 (2008).
[Crossref]

Curr. Opin. Biotechnol. (1)

J. Rao, A. Dragulescu-Andrasi, and H. Yao, “Fluorescence imaging in vivo: recent advances,” Curr. Opin. Biotechnol. 18, 17–25 (2007).
[Crossref]

Cytometry (1)

Q. S. Hanley, V. Subramaniam, D. J. Arndt-Jovin, and T. M. Jovin, “Fluorescence lifetime imaging: multi-point calibration, minimum resolvable differences, and artifact suppression,” Cytometry 43, 248–260 (2001).
[Crossref]

IEEE Sens. J. (1)

S. Foix, G. Alenya, and C. Torras, “Lock-in time-of-flight (ToF) cameras: a survey,” IEEE Sens. J. 11, 1917–1926 (2011).
[Crossref]

IEEE Trans. Med. Imaging (1)

A. T. Kumar, S. B. Raymond, A. K. Dunn, B. J. Bacskai, and D. A. Boas, “A time domain fluorescence tomography system for small animal imaging,” IEEE Trans. Med. Imaging 27, 1152–1163 (2008).
[Crossref]

J. Biomed. Opt. (1)

B. J. Bacskai, J. Skoch, G. A. Hickey, R. Allen, and B. T. Hyman, “Fluorescence resonance energy transfer determinations using multiphoton fluorescence lifetime imaging microscopy to characterize amyloid-beta plaques,” J. Biomed. Opt. 8, 368–375 (2003).
[Crossref]

J. Microsc. (1)

A. Squire, P. J. Verveer, and P. I. H. Bastiaens, “Multiple frequency fluorescence lifetime imaging microscopy,” J. Microsc. 197, 136–149 (2000).
[Crossref]

J. Opt. Soc. Am. A (1)

Nat. Commun. (1)

G. Satat, B. Heshmat, C. Barsi, D. Raviv, O. Chen, M. G. Bawendi, and R. Raskar, “Locating and classifying fluorescent tags behind turbid layers using time-resolved inversion,” Nat. Commun. 6, 6796 (2015).
[Crossref]

Opt. Express (3)

Opt. Lett. (5)

Rev. Sci. Instrum. (1)

D. R. Yankelevich, D. Ma, J. Liu, Y. Sun, Y. Sun, J. Bec, D. S. Elson, and L. Marcu, “Design and evaluation of a device for fast multispectral time-resolved fluorescence spectroscopy and imaging,” Rev. Sci. Instrum. 85, 034303 (2014).
[Crossref]

SIAM J. Optim. (1)

T. F. Coleman and Y. Li, “An interior trust region approach for nonlinear minimization subject to bounds,” SIAM J. Optim. 6, 418–445 (1996).
[Crossref]

Other (11)

A. Bhandari, A. Bourquard, S. Izadi, and R. Raskar, “Blind transmitted and reflected image separation using depth diversity and time-of-flight sensors,” in Computational Optical Sensing and Imaging (Optical Society of America, 2015), paper CT4F-2.

PMD Technologies, “pmd PhotonICs 19k–S3: Specs and reference design,” (2014).

S. Y. Kung, Kernel Methods and Machine Learning (Cambridge University, 2014).

S. W. Golomb and G. Gong, Signal Design for Good Correlation: For Wireless Communication, Cryptography, and Radar (Cambridge University, 2005).

S. Schuon, C. Theobalt, J. Davis, and S. Thrun, “Lidarboost: Depth superresolution for ToF 3D shape scanning,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2009), pp. 343–350.

D. Lefloch, R. Nair, F. Lenzen, H. Schäfer, L. Streeter, M. J. Cree, R. Koch, and A. Kolb, “Technical foundation and calibration methods for time-of-flight cameras,” in Time-of-Flight and Depth Imaging: Sensors, Algorithms, and Applications (Springer, 2013), pp. 3–24.

J. Lakowicz, Principles of Fluorescence Microscopy (Springer, 1982).

A. Bhandari, A. Kadambi, and R. Raskar, “Sparse linear operator identification without sparse regularization? Applications to mixed pixel problem in time-of-flight/range imaging,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2014), pp. 365–369.

A. Bhandari, M. Feigin, S. Izadi, C. Rhemann, M. Schmidt, and R. Raskar, “Resolving multipath interference in Kinect: An inverse problem approach,” in IEEE Sensors (IEEE, 2014), pp. 614–617.

F. Heide, L. Xiao, W. Heidrich, and M. B. Hullin, “Diffuse mirrors: 3D reconstruction from diffuse indirect illumination using inexpensive time-of-flight sensors,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014), pp. 3222–3229.

A. Kadambi, A. Bhandari, R. Whyte, A. Dorrington, and R. Raskar, “Demultiplexing illumination via low cost sensing and nanosecond coding,” in IEEE International Conference on Computational Photography (ICCP) (2014).

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (6)

Fig. 1.
Fig. 1. Experimental cross-correlated probing function [Eq. (8)] (blue line) and its Fourier series approximation (red dashed–dotted line) [Eq. (9)], with N0=30. Since the time-domain signal Cp,p¯ lasts for Δ=309.902ns, the corresponding fundamental Fourier harmonic is ω0/2π=3.2268MHz. Consequently, Fourier coefficient N0=30 corresponds to 96.80MHz. Inset: Fourier series coefficients ϕ^n,n=N0,,N0.
Fig. 2.
Fig. 2. ToF depth imaging and its link with FLI. For the SRF h(t,t)=ρδ(tt2d/c), (a) and (b) compare TD- and FD-ToF principles. (a) Probing and reflected signal for TD-ToF. The time delay is proportional to distance d. (b) Probing and reflected signal for FD-ToF. The phase is proportional to distance d and modulation frequency ω0. For the SRF h(t,t)=hDepth(t,t)+hSample(t,t) in Eq. (13), (c) and (d) compare the TD-ToF and FD-ToF methods for FLI. (c) Probing and reflected signal for TD-ToF FLI. The time delay is proportional to distance d and the waveform shape is linked with lifetime τ. (d) Probing and reflected signal for FD-ToF FLI. The phase is proportional to distance d, modulation frequency ω0, and lifetime τ. (e) Experimental setup for FLI estimation via ToF.
Fig. 3.
Fig. 3. TD-ToF FLI measurements. (a) Time profile of measurements mk=m(kTs) and Ts=7.8120ps based on K=3968 samples. These measurements result from convolving cross correlation of the probing function with the SRF [Eq. (17)]. (b) Phase measurements computed using h^=V+m with N0=15 and Δ=309.9ns. We estimate the distance d˜ and lifetime τ˜ parameters using nonlinear least squares fitting, which results in the fitted curve. The fitted result is shown by the red solid line. The estimated phase contribution [Eq. (20)] due to distance θd˜ and lifetime θτ˜ is also plotted.
Fig. 4.
Fig. 4. FD-ToF FLI measurements of a 61×66 pixel patch of a 120×160 pixel sensor image. The size of the scene is approximately 2.3in2. (a) Multifrequency measurements of τ=32ns quantum-dot-based fluorescent sample. We show phase and amplitude images, that is, {z(10f0),z(20f0),z(30f0),z(40f0)} (in radians, [0,2π]) and {|z(10f0)|,|z(20f0)|,|z(30f0)|,|z(40f0)|} (in decibels), respectively. The base modulation frequency for the experiment is f0=ω0/2π=1MHz. The phase at the background pixel is recorded to be z(xb,yb)(40f0)=4.1625rad, which amounts to a depth of 2.48 m, which is consistent with the experimental setup. At the location of the fluorescent sample, we recorded a higher phase value z(xf,yf)(40f0)=5.5822, which is attributed to the fluorescence phenomenon. (b) Multifrequency raw phase measurements {z(x,y)(kf0)}k=1K=40 for four pixels. The measurements confirm with the theoretical hypothesis of Eq. (20), as well as the fitted phase obtained by Eq. (35). The estimated phase contribution [Eq. (20)] due to distance θd˜ and lifetime θτ˜ is also plotted.
Fig. 5.
Fig. 5. Estimation accuracy of τ1=4ns and τ2=32ns. (a) We plot 2000 estimated values of lifetime as a function of SNR ranging from 0 to 60 dB. As the SNR increases, the estimates cluster around the oracle estimates of τ=4ns and τ=32ns, respectively. (b) We plot the MSEτ on log scale as a function of SNR. After 15 dB, we note a consistent linear relationship [Eq. (36)] between SNR and the log(MSEτ).
Fig. 6.
Fig. 6. Measured phase as a function of the modulation frequency. We plot the measured phase corresponding to a 120 pixel cross section of the 160×120 ToF sensor image for modulation frequencies f={10,20,30,40}MHz. The first 50 phase measurements are due to the background pixel, which is at a depth d=2.5m. The dashed line marks the average phase value over the first 50 pixels. The average phase value for each modulation frequency leads to a distance estimate: d={2.56,2.52,2.47,2.48} in meters.

Tables (4)

Tables Icon

Table 1. TD- and FD-ToF Depth Imaging

Tables Icon

Table 2. TD- and FD-ToF FLI

Tables Icon

Table 3. TD-ToF FLI (τ=32ns and d=1.05m)

Tables Icon

Table 4. FD-ToF FLI (τ=32ns and d=2.5m)

Equations (49)

Equations on this page are rendered with MathJax. Learn more.

r(t)=p(t)h(t,t)dt.
m(t)=p¯(t)r(tt)dt,
h(t,t)=hSI(tt),
r(t)=(p*hSI)(t)=p(t)hSI(tt)h(t,t)dt.
m(t)=(p*p¯*hSI)(t)(ϕ*hSI)(t),
h(t,t)=ρδ(tt2d/c),
m(t)=ρϕ(t)(tt0)ρ(p*p¯)(tt0).
t˜0=maxtm(t)=maxtϕ(tt0),t0=2dc.
p(t)=1Δn=p^nejnω0t,ω0=2π/Δ,
p^n=0Δp(t)ejnω0tdt,
ϕ(t)=(p*p¯)(t)=n=ϕ^nejω0nt.
ϕ^n=p^np^n*=|p^n|2.
Cp,p¯(t)1Δ|n|N0ϕ^nejnω0t,
p(t)=1+p0cos(ωt),
r(t)=ρp(tt0)=ρ(1+p0cos(ω(tt0))).
m(t)=limB12BB+Bp¯(t)r(tt)dt
=ρ(1+p022cos(ω(t+t0))),
z=(m0m2)+j(m3m1).
ρ˜=|z|p02andd˜=c2ωz.
h(t,t)=hDepth(t,t)+hSample(t,t),
hDepth(t,t)=ρδ(tt2d/c),
hSample(t,t)=μe(tt2d/cτ)Π(tt2d/c).
m(t)=(4)(p*p¯*hSI)(t)(9)(Cp,p¯*hSI)(t)Cp,p¯*(δ(t2d/c)*(ρδ(t)+μetτΠ(t))).
m(t)1Δ|n|N0(ϕ^nh^n)ejω0nt,
h^(ω)=h(t)ejωtdt(Fourier Transform)
=ρejω(2dc)+μτ1+jωτejω(2dc).
|h^(ω)|=(ρ+μτ)2+(ωρτ)21+(ωτ)2,h^(ω)=tan1(ωμτ2ρ+μτ+ρ(ωτ)2)2dcω.
h^(ω)=θτ(ω)θd(ω),
θτ(ω)=tan1(ωτ2τ+ρ(1+(ωτ)2))andθd(ω)=2dω/c.
m=VDϕ^h^,
mTD-FLI(t)=μe(tt0τ)Π(tt0).
mTD-ToF-FLI(t)=(17)1Δ|n|N0(ϕ^nh^n)ejω0nt,
MLS0101110110001111100110100100001,
h^obs=V+m=(θτ(2πn/Δ)+θd(2πn/Δ)),
argmind,τn=N0n=+N0|h^(nω0)h^obs(nω0+β)+α|2,
MSEν=1Nn=0N1|ν˜nν|2.
h^˜(nω0)=(20)tan1(nω0τ˜)2nω0d˜c,
h^obs(nω0)=h^(nω0)+εn.
SNR=20(logh^logh^obsh^),
r(t)=(3)(p*hSI)=|h^(0)|+|h^(ω)|p0cos(ωt+h^(ω)).
m(t)=(10)|h^(0)|+|h^(ω)|p022cos(ωth^(ω)).
|h^(ω0)||z(ω0)|/p02Amplitude Estimateandh^(ω0)(20)z(ω0)Phase Estimate,
h(t)=k=0K1ρkδ(t2dk/c)Fourierh^(ω)=k=0K1ρkejω(2dkc).
{z(kf0)}k=1K,k=1,2,,40.
argmind,τk=1K=40|z(kω0)+θτ(kω0)+θd(kω0)|2.
{d˜,τ˜}={2.4961m,31.024ns}.
f0(1)=0.10N(1)=401,f0(2)=0.25N(2)=161,f0(3)=0.50N(3)=81,f0(4)=1.00N(4)=41.
h^obs(k)(nω0)=(30)h^(nω0)+εn,n=0,,N(k)1,
log(MSEτ)logK0λlog(SNR).

Metrics