Gesture Recognition by Data Fusion of Time-of-Flight and Color Cameras

Piercarlo Dondi, Luca Lombardi, Marco Porta

Keywords: gesture recognition, human-computer interaction, time-of-flight camera.

Abstract: In the last years numerous applications of Human-Computer Interaction have exploited the capabilities of Time-of-Flight cameras for achieving more and more comfortable and precise interactions. In particular, gesture recognition is one of the most active fields. This work presents a new method for interacting with a virtual object in a 3D space. Our approach is based on the fusion of depth data, supplied by a ToF camera, with color information, supplied by a HD webcam. The hand detection procedure does not require any learning phase and is able to concurrently manage gestures of two hands. The system is robust to the presence in the scene of other objects or people, thanks to the use of the Kalman filter for maintaining the tracking of the hands.