Computer Vision for Human-Machine Interaction [pdf]

Keith Waters, Jim Rehg, Maria Loughlin, Sing Bing Kang and Demetri Terzopolous

Digital Equipment Corporation

Cambridge Research Laboratory
Technical report CRL 96/5


Computer vision-based sensing of people enables a new class of public multi-user computer interfaces. Computing resources in public spaces, such as automated, information dispensing kiosks represent interfaces that differs from the conventional desktop environment and correspondingly, require a user-interface metaphor quite unlike the traditional WIMP interface. This paper describes a prototype public computer interface, which employs color and stereo tracking to sense the users activity and an animated, speaking agent to attract attention and communicate through visual and audio modalities.

wallable macrodevice
Figure 1. A Wallable macro-device

A similar technology using CV and a vertically mounted camera, was developed as a wallable macro-device for whole person sensing infront of a screen.

A Wallable macro device is a large-scale appliance such as a kiosk, touch-panel, or touch-screen that can be place on or in the wall itself. The form-factor of such applications demand novel forms of user interaction that breaks from the traditional mouse and keyboard. Large-scale appliances necessitate interactions that focus on human modes of interaction and behavior such as orientation, pose and location. Human detection techniques are used to identify the location of the individual in relation to the screen and a synthetic talking face to provide audio-visual feedback to create a unique form of interaction.


imagebased touchscreen imagebased touchscreen
Figure 2. Image-based touchscreen

On a smaller scale the basic techniques originated in an image-based touch screen. Patent [pdf]

interactive kiosk screen
Figure 3. Kiosk interactive screen


  1. Waters, K., Iannucci, A. R. A Wallable Macrodevice, CRL Technical Report, 99/6, September 1999
  2. US Patent: 6,256,046, Method and Apparatus for Visual Sensing of Humans for Active Public Interfaces, Waters, K. Loughlin, M., Rehg, J. and Kang, S. Issued: July 3rd 2001
  3. US Patent: 6,052,132,Technique for Providing a Computer Generated Face Having Coordinated Eye and Head Movement, Christian, A., Avery, B., and Waters, K. Issued: April 18th 2000