Open Access
Add to BibSonomyAbstract
We demonstrated the use of the analog self electro-optic effect device (SEED) as part of an artificial retina chip for the detection and estimation of local motion. The characterization was performed by comparing our chip to biological and computational models and to other artificial retina chips. Its main unique feature is the optical output, since most chips have electrical output. By combining the response of the chip with temporal information about the input image, it is possible to estimate the velocity perpendicular to an edge, including its direction.
©2010 Optical Society of America
1. Introduction
Local motion detection is an early vision process performed by the retinas of many animals, including humans [1]. It is also very important in image processing. Computational and biological models are quite different, but the goal is the same: to obtain a 2-dimensional velocity vector field of the input image. It is only possible to detect the velocity of edges, since we cannot detect movement on a constant color and intensity image profile. Due to the aperture problem, the information about the velocity along the edge is lost, regardless of the local motion detection technique. Local motion detection is thus an ill-posed problem.
There are basically two local motion detection techniques: the correlation or Reichardt local motion detector and the spatial-temporal gradient local motion detector. The first one was proposed by Reichardt in 1957 [2] to explain the behavior of biological retinas. It has many variations, but all of them consist of at least two photodetectors, a delay line and a non-linear interaction (e.g. multiplication) between the response of a photodetector and the delayed response of another [3]. The second one is a mathematical/computational calculation of the velocity based on the image intensity function I(x,y,t), where x and y are the spatial coordinates and t the time. Assuming that the illumination over the image is constant, i.e. dI/dt = 0:
and in 1-D:So, by calculating the spatial and temporal image variations it is possible to obtain the image velocity.Delbrück implemented a correlation-based chip [4]. He also developed a time, but not spatial, derivative chip [5]. Deutschmann made a one-dimension spatial-temporal gradient chip [6]. Yamada proposed a network composed of electrical circuits which display an optical flow by using the motion signals at pixels and adapt themselves to local velocities of a background image [7]. Cassinelli et al made a prototype where they demonstrated motion detection at a video rate in a sequence of gray-level images [8]. All of them use complementary metal oxide semiconductor (CMOS) very large scale integrated (VLSI) technology and they have electrical output.
The potential of AlGaAs/GaAs technology and the performance of their modulators make them suitable for this application. Especially, the self electro-optic effect devices (SEED) that have been used in applications such as differential and direction-sensitive detection [9], directional coupler [10], edges detection [11, 12], and cross-differentiator image processing [13, 14]. In reference [8] a theoretical modeling of a hybrid CMOS-SEED technology showed an improvement in their prototypes as motion detector. In this paper we demonstrated this new functionality of the SEED as a local motion detector of the gradient type. The SEED chip uses GaAs multiple quantum wells (MQW) with Al0.3Ga0.7As barriers and it has both optical input and output, which allows subsequent processing by concatenating different stages of smart SEED arrays.
2. Experimental setup
To focus the image and the reference beam at the same time on the SEED and read only the modulated beam, we used the apparatus described in Fig. 1 . The polarizing beam splitter (PBS) allows the passage of the beam if it is at a particular polarization and causes a total internal reflection for polarization perpendicular to it.
Fig. 1 Experimental apparatus to estimate local derivative of a incident image at 780 nm. The set of polarizing beam splitter (PBS2) with 50:50 mirror and wave plates λ/41 and λ/42 allow the image beam (green line) and the reference beam (red line) illuminate simultaneously the SEED.
The reference beam at 850 nm (red line) after being corrected and attenuated is divided by the binary phase grate (BPG) and circularly polarized by the λ/41 wave plate. As the circular polarization is a linear combination of polarization in perpendicular directions, the beam is transmitted by PBS2 and part is reflected. The transmitted part of the beam is polarized now, because the PBS2 transmits light only in a certain polarization. It passes once through the λ/42 wave plate shining on the SEED where it suffers the modulation. It is reflected back and passes again through the λ/42 wave plate. This double passage makes the beam now with a polarization delayed by 90° with respect to the initial polarization. This new polarization is precisely the one that suffers total internal reflection in PBS2. That way the beam is reflected by PBS2 and can be read by conventional photodetectors connected to a lock-in amplifier.
For the image beam at 780 nm (green line) an optical fiber is used as a spatial filter, which is polarized in the direction that the PBS2 causes total reflection. The reflected beam by the PBS2 passes through λ/41, reflects in the 50:50 mirror and passes again through the λ/41. A double passage at λ/41 makes the beam to be transmitted by the PBS2. As the reference beam, the image focusing on the SEED suffers a double passage through λ/42 and goes toward the detectors. As we do not want to measure the input image, but the modulated beams, we filtered it with a 850 nm filter. The hybrid beam splitter transmits 60% and reflects 40% of the light incident. That allows the use of a CCD camera to visualize the SEED (which is illuminated by a LED) and the beams.
3. SEED device
The SEED is a p-i-n diode with MQW in the intrinsic region [15]. It consists of heterostructures layers of 90 Å GaAs wells with 35 Å barriers of Al0,3Ga0,7As grown by molecular beam epitaxy. For this experiment we used a symmetric one with a pair of quantum wells. Figure 2 shows a picture of whole array where we show, in detail, a single pixel used in this experiment. The photodetectors (labeled as 1 and 2) and the modulators (labeled as A and B) are actually the same struc.ture integrated as a single SEED. This is possible because the quantum-well modulator operate as conventional photodetectors for wavelengths much shorter than the exciton peak. Each pixel is then composed of two quantum-well modulators.
Fig. 2 Picture of the self-linearized differential device used to estimate the local motion. In the detail: the same structure is used simultaneously as quantum-well modulators (A and B) and as conventional photodetectors (1 and 2). The photodetector and the modulator are the same structure illuminated by different wavelength and therefore having a different response to each one of them. For the image one ate 780 nm it operates as a conventional photodiode and for the reference beam at 850 nm it operates as a quantum well modulator.
The optoelectronic circuit of the single pixel described in Fig. 2 is showed in Fig. 3 . The input image (Pa1 and Pa2) modulates the two reference beams (PinA and PinB). The output of a single pixel (PoutB - PoutA) is proportional to the difference of intensities in two adjacent points of the input image, i.e., the first order spatial derivative of the input image on that point. The output is an analog bipolar (positive and negative) value, or zero, as it was demonstrated by De Souza [16].
Fig. 3 Self-linearized differential circuit where a pair of quantum well works simultaneously as modulators (A and B) and conventional photodetectors (1 and 2). Pa1 and Pa2 are the input image. PinA and PinB are the input reference beams used to make the measurement. PoutA and PoutB are the modulated output beams of each SEED. Ic is the difference between the currents of the photodetectors; i1 and i2 are the photocurrent in the modulators B and A, respectively. The output of the whole circuit is the difference between the modulated beams (PoutB - PoutA) which is proportional to Ic.
The input image is a laser beam small enough to cover a single modulator. The beam is positioned on each modulator, one at a time. This mimics a spatial gradient of the image intensity profile, similar to what an edge would produce. The output of the pixel is measured by two reference beams over the modulators. The wavelength of the input image is different from the wavelength of the reference beams, and only the wavelength of the beams is measured, resulting in a clean measurement.
The chip is expected to give positive and negative responses, depending on which of the modulators has the laser beam on it. This allows it to be used as an image intensity gradient estimator.
4. Results
Figure 4 shows the results when the image beam is positioned on each of the modulators. The apparatus is calibrated so that there is a null response when there is no image beam on the photodetectors (modulators), i.e., PoutB - PoutA = 0. The image beam is positioned first on photodetectors 1 (modulator B - top) showing a positive response and later on photodetectors 2 (modulator A - bottom) showing a negative response, as expected, for five different measurements. The results show also that this new functionality of the SEED is suitable to estimate local derivative (dI/dx), as shown in Eq. (2), which is necessary to calculate the image velocity. The variation between the two outputs from the two quantum-well modulators is due to the non-equal power of the reference beams what can be fixed by a careful design of the binary phase grating.
Fig. 4 Measurements of the difference between the two reference beams (PoutB - PoutA) when a single image is illuminating first the photodetector 1, and modulator B, (blue curve at the top) giving positive response and later the photodetector 2, and modulator A, (red curve at the bottom) giving negative responses, as expected, in all cases. The small variation between the two average values (red and blue lines) is due to the unequal power in the reference beams.
Another important point is the signal to noise ratio (SNR). Since our signal is obtained by taking the difference between two output beams (PoutB - PoutA) and both originate from the same laser what eliminate this source of noise increasing the SNR. So, the variation in the signal amplitude in both cases was mainly related to the image what we believe to be due to fiber coupling.
5. Conclusion
We demonstrated that the SEED can be used as a component of a gradient based direction selective local motion detector. Its main unique feature is the optical output, since most chips have electrical output. By combining the response of the chip with temporal information about the input image, it is possible to estimate the velocity perpendicular to an edge, including its direction.
Acknowledgments
We would like to thank Celso Jose Vianna Barbosa, Farshad Yazdani and Janio Figueiredo de Aquino for helpful discussions and experimental aid. We thank CAPES and FINATEC for financial support and Prof. David A.B. Miller for several donations.
References and Links
1. T. Poggio, V. Torre, and C. Koch, “Computational vision and regularization theory,” Nature 317(6035), 314–319 (1985). [CrossRef] [PubMed]
2. W. Reichardt, “Autokorrelationsauswertung als Funktionsprinzip des Zentralnervensystems,” Z. Naturforsch. B 12b, 447 (1957).
3. A. Borst and M. Egelhaff, “Principles of visual motion detection,” Trends in Neuro-Science 12(8), 297–306 (1989). [CrossRef]
4. T. Delbruck, “Silicon retina with correlation-based, velocity-tuned pixels,” IEEE Trans. Neural Netw. 4(3), 529–541 (1993). [CrossRef] [PubMed]
5. T. Delbruck and C. A. Mead, “Time-derivative adaptive silicon photoreceptor array,” SPIE Infrared Sensors: Detectors, Electron, Signal Proc. 1541, 92 (1991).
6. R. A. Deutschmann and C. Koch, “An analog VLSI velocity sensor using the gradient method,” Proc. IEEE International Symposium on Circuits and Systems 6, 649 (1998) (ISCAS).
7. H. Yamada, K. Nishioy, M. Ohtaniz, H. Yonezux, and Y. Furukawa, “A network of analog metal oxide semiconductor circuits for motion detection with local adaptation to a background image,” Opt. Rev. 11(5), 320–327 (2004). [CrossRef]
8. A. Cassinelli, P. Chavel, and M. P. Y. Desmulliez, “Dedicated optoelectronic stochastic parallel processor for real-time image processing: motion-detection demonstration and design of a hybrid complementary-metal-oxide semiconductor- self-electro-optic-device-based prototype,” Appl. Opt. 40(35), 6479–6491 (2001). [CrossRef]
9. J. F. Aquino and E. A. De Souza, “Differential and direction-sensitive detector based on self-electro-optic effect in GaAs multiple quantum well,” Electron. Lett. 41(24), 1350 (2005). [CrossRef]
10. E. P. Keyworth, M. Cada, J. M. Glinski, A. J. Springthorpe, C. Rolland, and K. O. Hill, “Multiple quantum well directional coupler as a self-electro-optic effect device,” Electron. Lett. 26(24), 2011–2013 (1990). [CrossRef]
11. C. J. Vianna and E. A. De Souza, “Detecting Edges Using an Analog Electrooptic Device,” J. Microw. and Optoelectron. 5, 30 (2006).
12. C. J. Vianna and E. A. De Souza, “An Electrooptic Multiple-Quantum-Well Device for Image Processing,” IEEE J. Quantum Electron. 45(6), 603–608 (2009). [CrossRef]
13. F. Yazdani and E. A. De Souza, “Cross-differentiator image processor based on self-electro-optic effect device,” Electron. Lett. 43(14), 771 (2007). [CrossRef]
14. F. Yazdani and E. A. De Souza, “Operating point optimization of self-linearized differential quantum well electroabsorptive modulator,” Microw. Opt. Technol. Lett. 52(1), 1–4 (2010). [CrossRef]
15. D. A. B. Miller, D. S. Chemla, T. C. Damen, T. H. Wood, C. A. Burrus, A. C. Gossard, and W. Wiegmann, “The Quantum Well Self-Electrooptic Effect Device: Optoelectronic Bistability and Oscillation, and Self-Linearized Modulation,” IEEE J. Quantum Electron. 21(9), 1462–1476 (1985). [CrossRef]
16. E. A. De Souza, L. Carraresi, and D. A. B. Miller, “Linear image differentiation by use of analog differential self-electro-optic effect devices,” Opt. Lett. 19(22), 1882 (1994). [CrossRef] [PubMed]
