Welcome to the Computer Vision and Multimedia Lab website.GET IN TOUCH
Netytar is a gaze-based Virtual Digital Musical Instrument (Virtual DMI) developed by the CVMLab - in particular, by Nicola Davanzo for his Master's Thesis in Computer Engineering (with the supervision of professors Mauro Mosconi and Marco Porta, and with the help of Dr. Piercarlo Dondi).
Playing music with the eyes through an isomorphic interface, by Nicola Davanzo, Piercarlo Dondi, Mauro Mosconi and Marco Porta.
Abstract: Playing music with the eyes is a challenging task. In this paper, we propose a virtual digital musical instrument, usable by both motor-impaired and able-bodied people, controlled through an eye tracker and a "switch". Musically speaking, the layout of the graphical interface is isomorphic, since the harmonic relations between notes have the same geometrical shape regardless of the key signature of the music piece. Four main design principles guided our choices, namely: (1) Minimization of eye movements, especially in case of large note intervals; (2) Use of a grid layout where "nodes" (keys) are connected each other through segments (employed as guides for the gaze); (3) No need for smoothing filters or time thresholds; and (4) Strategic use of color to facilitate gaze shifts. Preliminary tests, also involving another eye-controlled musical instrument, have shown that the developed system allows "correct" execution of music pieces even when characterized by complex melodies.
Netytar, with its source code, can be downloaded from a GitHub repository.
Please notice that, at the moment, the use of the Tobii EyeX eye tracker is mandatory.