Skip to main content
No Access

A vision sensor system with a real-time multi-scale filtering function

Published Online:pp 248-258https://doi.org/10.1504/IJMA.2014.066367

We developed a compact and energy-efficient vision sensor system that separates an image into a set of spatial frequency bands at 50 fps. The vision sensor system comprises a photo-sensor array, a metal-oxide-semiconductor (MOS)-based resistive network, and a field-programmable gate array (FPGA). To apply multiple spatial filters efficiently, which is required for the separation of an image, we employed a MOS-based resistive network, whose strengths are instantaneous filtering, configurable filter size, and low power consumption. A digital circuit for controlling the filter size of the resistive network was programmed in the FPGA; this circuit changes the filter size four times in a single frame sampling period. This control scheme generates four filtered images from a single resistive network. This system was applied to edge extraction of a photograph or a movie of natural scenes, and it was successful in extracting edges and separating them by spatial frequencies in real time, e.g., the outline and stripe patterns of a zebra.

Keywords

robotic vision, vision sensor, resistive network, multiple spatial filtering

References

  • 1. K.A. Boahen, A.G. Andreou, '‘A contrast sensitive silicon retina with reciprocal synapses’' Advances in Neural Information Processing Systems 4 (Proc. of NIPS 1991) (1992) Google Scholar
  • 2. V. Bonato, E. Marques, G.A. Constantinides, '‘A parallel hardware architecture for scale and rotation invariant feature detection’' CircuitsandSystemsforVideoTechnology, IEEE Transactions on (2008) Google Scholar
  • 3. R. Deriche, '‘Fast algorithms for low-level vision’' Pattern Analysis and Machine Intelligence, IEEE Transactions on (1990) Google Scholar
  • 4. E. Elboher, M. Werman, '‘Cosine integral images for fast spatial and range filtering’' (2011) Google Scholar
  • 5. G. Farnebäck, C.F. Westin, '‘Improving Deriche-style recursive Gaussian filters’' Journal of Mathematical Imaging and Vision (2006) Google Scholar
  • 6. J. Fernandez-Berni, R. Carmona-Galan, L. Carranza-González, '‘FLIP-Q: a QCIF resolution focal-plane array for low-power image processing’' Solid-State Circuits, IEEE Journal of (2011) Google Scholar
  • 7. F.C. Huang, S.Y. Huang, J.W. Ker, Y.C. Chen, '‘High-performance SIFT hardware accelerator for, real-time image feature extraction’' Circuits and Systems for Video Technology, IEEE Transactions on (2012) Google Scholar
  • 8. S. Kameda, T. Yagi, '‘An analog VLSI chip emulating sustained and transient response channels of the vertebrate retina’' Neural Networks, IEEE Transactions on (2003) Google Scholar
  • 9. Y. Kim, J. Park, H.J. Yoo, '‘A 34.1 fps scale-space processor with two-dimensional cache for real-time object recognition’' (2013) Google Scholar
  • 10. C. Koch, B. Mathur, '‘Neuromorphic vision chips’' IEEE Spectrum (1996) Google Scholar
  • 11. T.D. Lamb, '‘Spatial properties of horizontal cell responses in the turtle retina’' The Journal of Physiology (1976) Google Scholar
  • 12. S. Lee, J. Oh, J. Park, J. Kwon, M. Kim, H.J. Yoo, '‘A 345 mW heterogeneous many-core processor with an intelligent inference engine for robust object recognition’' Solid-State Circuits, IEEE Journal of (2011) Google Scholar
  • 13. T. Lindeberg, '‘Scale-space theory: a basic tool for analysing structures at different scales’' in Proc. of Journal of Applied Statistics (1994) Google Scholar
  • 14. T. Lindeberg, '‘Feature detection with automatic scale selection’' International Journal of Computer Vision (1998) Google Scholar
  • 15. D.G. Lowe, '‘Distinctive image features from scale invariant keypoints’' International Journal of Computer Vision (IJCV) (2004) Google Scholar
  • 16. D. Marr, Vision (1982) Google Scholar
  • 17. D. Marr, E. Hildreth, '‘Theory of edge detection’' Proceedings of the Royal Society of London, Series B., Biological Sciences (1980) Google Scholar
  • 18. D. Marr, T. Poggio, '‘A computational theory of human stereo vision’' Proceedings of the Royal Society of London (1979) Google Scholar
  • 19. C. Mead, Analog VLSI and Neural Systems (1989) Google Scholar
  • 20. C. Mead, M.A. Mahowald, '‘A silicon model of early visual processing’' Neural Networks (1988) Google Scholar
  • 21. K.I. Naka, W.A.H. Rushton, '‘The generation and spread of S-potentials in fish (Cyprinidae)’' The Journal of Physiology (1967) Google Scholar
  • 22. D.S. Park, J.H. Kim, H.S. Kim, J.H. Park, J.K. Shin, M. Lee, '‘A foveated-structure CMOS retina chip for edge detection with local light adaptation’' Sensors and Actuators A: Physical (2003) Google Scholar
  • 23. J.A. Robinson, '‘Efficient Gaussian filtering using cascaded prefix sums’' (2012) Google Scholar
  • 24. K. Shimonomura, S. Kameda, A. Iwata, T. Yagi, '‘Wide-dynamic range APS-based silicon retina with brightness constancy’' Neural Networks, IEEE Transactions on (2011) Google Scholar
  • 25. S.N. Sinha, J.M. Frahm, M. Pollefeys, Y. Genc, '‘GPU-based video feature tracking and matching’' (2006) Google Scholar
  • 26. K. Sugimoto, S. Kamata, '‘Fast Gaussian filter with second-order shift property of DCT-V’' Proc. of IEEE International Conference on Image Processing (ICIP2013) (2013) Google Scholar
  • 27. M. Unser, A. Aldroubi, M. Eden, '‘B-spline, signal processing. II. Efficiency design and applications’' Signal Processing, IEEE Transactions on (1993) Google Scholar
  • 28. G.S. Van der Wal, J.O. Sinniger, '‘Real time pyramid transform architecture’' (1985) Google Scholar
  • 29. W.M. Wells, '‘Efficient synthesis of Gaussian filters by cascaded uniform filters’' Pattern Analysis and Machine Intelligence, IEEE Transactions on (1986) Google Scholar
  • 30. A.P. Witkin, '‘Scale-space filtering’' (1983) Google Scholar
  • 31. C.Y. Wu, C.F. Chiu, '‘A new structure of the 2-D silicon retina’' Solid-StateCircuits, IEEE Journal of (1995) Google Scholar
  • 32. S. Yasukawa, H Okuno, T. and Yagi, '‘Detection of scale-invariant key points employing a resistive network’' (2012) Google Scholar
  • 33. S. Yasukawa, H. Okuno, S. Kameda, T. Yagi, '‘High-speed multiple spatial band-pass filtering using a resistive network’' (2013) Google Scholar
  • 34. S.M.A. Zeinolabedin, N. Karimi, S. Samavi, '‘Low computational complexity hardware implementation of Laplacian Pyramid’' (2010) Google Scholar
  • 35. Z. Zhou, B. Pain, E.R. Fossum, '‘CMOS active, pixel sensor with on-chip successive approximation, analog-to-digital converter’' Electron Devices, IEEE Transactions on (1997) Google Scholar