Research

Topics

Computer Vision and Pattern Recognition. Acquiring, processing, analyzing and understanding digital images for applications such as object detection, segmentation, vision based learning tasks. The goal is to extract information from images, that can come from video sequences, from multiple cameras, from medical scanners, or from other different sensors. Methods for the extraction of high-dimensional data from the real world in order to produce information for decisions tasks are applied. Sub-domains such as scene reconstruction, event detection, video tracking, object recognition, writing analysis of relics, proteomics, and image restoration are studied. Computer graphics principles and 3D modeling techniques are implemented to enhance cultural heritage accessibility and promotion. Approaches and strategies of pattern recognition techniques are used to improve aspects in the field of bioinformatics and computational biology, in particular for modeling and comparing proteins 3D structures in proteomics. The technology of a vehicle-based mobile mapping system is applied to maintain an updated transportation database for road and railway inventory, for recognition of signs and tunnel inspection.

Deep Learning. Deep learning models are applied in a variety of tasks. In particular, deep reinforcement learning for collaborative robotics, focus on incremental autonomous learning, experience transfer, and robust avoidance strategy, with the virtualization of a real‐world robot to learn reaching a target while avoiding obstacles in a simulation environment. Another topic, investigated with recurrent neural networks, is fall detection. Accidental falls have an enormous human cost, especially for elderly people, so there is need for automatic fall detection techniques for timely warnings. We use “smart” wearable devices and an innovative technique: deep learning on embedded. The implementation challenge is due to limited computing and memory resources and to the necessity of battery life for continuous use (24x7). Datasets with simulated falls by volunteers have been collected: seven carry positions, seventeen different activities, forty volunteers, over five thousands tracks. Manual annotations on videos, basic for training, have been done. The most modern deep learning techniques are also used for the diagnosis of breast cancer metastases in lymph nodes. Breast cancer diagnosis is usually made through a visual analysis of slides containing sentinel lymph node biopsy. The research is focused on automatic screening of digital slides and automatic segmentation of injuries. Other projects have applied deep learning to bioinformatics for the prediction of glycemic rate in patients with T1 diabetes (with recurrent neural networks) and we are working on algorithmic support with deep learning for the development of artificial retina.

Digital Humanities. Digital technologies applied for analysis, promotion, dissemination, and accessibility of cultural heritage: 3D scan and modeling of historical musical instruments, augmented reality and interactive applications for museums, virtual thematic reconstruction of ancient cities and monuments, anastylosis of frescoes, 3D printing, production of tactile images, image processing for preventive conservation and restoration of artwork and analysis of manuscripts. In particular, since 2014 is active a collaboration with the Arvedi Laboratory of Non-Invasive Diagnostics for the application of artificial vision and 3D modeling techniques in the study of historical musical instruments preserved in the Violin Museum of Cremona. In this context, a collaboration with the Universitè Paris Sud has recently begun, focused on the creation of new algorithms for the identification of alterations on the surface of historical violins. Another relevant project is “Pavia Digital City”: in the educational context of the Computer Vision course for the Master’s degree in Computer Engineering at the University of Pavia, 3D computer graphics have been taught using a new teaching strategy, Experiential Learning, with a focus on activities applied to local cultural assets, successfully increasing the students’ involvement and promoting synergies and partnerships between the world of education and the local community. Under the direction of Prof. Virginio Cantoni, the project includes the activities completed during the academic years 2015–2016 and 2016–2017 for the virtual reconstruction of Pavia in the XVI century and 2017–18, 2018–19, and 2019–2020 for the virtual reconstruction of the Ark of St. Augustine. For further details, please visit the dedicated Digital Humanities Projects page.

Human Computer Interaction. Our studies are exploiting gaze and gesture interaction. We use an eye tracker - a device for measuring eye positions and eye movement and so detecting the user's gaze direction - and Kinect - a motion sensing input device - to design new interfaces. Interfaces operated through the eyes are of great help for people with severe disabilities, allowing them to use their gaze to identify, or even move, objects on the screen, as well as to write. We have developed: Eye-S, a system that allows input to be provided to the computer through a pure eye-based approach; Netytar, a gaze-based Virtual Digital Musical Instrument (Virtual DMI), usable by both motor-impaired and able-bodied people, controlled through an eye tracker and a "switch", to play music with the eyes; and, recently, a Gaze-Based Web Browser. But eye-tracking is also studied and applied in several contexts besides that of an input means for interfaces. In our research we consider it both for implementing explicit/implicit interfaces and as a helpful means for the evaluation of web sites, usability issues, information presentation modes and visual interactions in general, useful, for example, for interactive and more involving museum experiences. We developed e5Learning, an e-learning environment where eye tracking is used to observe user behavior, so as to adapt content presentation in real-time. Also, we are studying the effectiveness of existing and new RSVP (Rapid Serial Visual Presentation) image visualization methods, which involve strong eye activity. Soft Biometrics, Automotive, Assistive and Persuasive Technologies are other examples of fields for application.

Social Media Analysis. Social media analysis is useful in developing and enhancing a successful social media strategy and marketing campaign, reaching the right audience, at the right time, and on the right channel. The most valuable data - a very large amount of information to gather and analyse - must be collected in order to draw actionable conclusions. Network analysis, social commerce, customer segmentation, malware detection, social media mining, sentiment analysis, and opinion mining are studied in this framework. We are also studying adversarial machine learning methods used to attack image classifiers.

Brief history of the lab