Computer Vision and Multimedia Lab

Welcome to the Computer Vision and Multimedia Lab website.

Image processing and 3D modeling of historical musical instruments /
Elaborazione di immagini e modellazione 3D di strumenti musicali storici

Activity in progress since 2014 in collaboration with Arvedi Laboratory of Non Invasive Diagnostics of University of Pavia, located inside Museo del Violino of Cremona (Italy). The objective is the analysis of surface and morfology of historical musical instruments, with a special focus on violins and other string musical instruments held in the museum. Histrorical violins are a very unique kind of artworks since are both preserved in museums and also played. The multiple restorations occurred during centuries to maintain the instruments in use have created a very complex surface, hard to correctly interpret.

Main tracks of this project include:

analysis of UV Induced Fluorescence (UVF) images
3D scan and modelling
Human-Comptuter Interaction (HCI) applications for scientific dissemination

Analysis of UV Induced Fluorescence (UVIFL) images
UV Induced Flurescence (UVF or UVIFL) photography is a non-invasive analytical technique, widely used in Cultural Heritage studies, which allow to see details of a surface not perceivable with visible light, such as restorations, retouches or alterations. In particular, in the case of historical musical instrument, UVF images are used to decide where to apply more precise but slower diagnostic techniques, like XRF or FTIR spectroscopy. The analysis of UVF images is traditionally a naked eye examination, leading to a risk of human errors, expecially if we want to compare fluorescence colors of different images. For improving this phase, we desinged various image processing algorithms that can highglight the distribution of varnishes on the entire surface of the violins, identify worn-out areas, and allow comparisons among instruments. A tool that implements all these methods (UVAnalyzer) was also developed to give scientists and restorers a user-friendly access to them. Currently, we are working on multi-temporal analysis of UVFIL images (see reference DATASET), with the final goal to produce new methods for an efficient constant monitoring of the state of conservation of the violins (preventive conservation). This last part was conducted in collaboration with Université Paris-Sud (France).

3D scan and modelling
The digitalization of artworks is now a standard practice in many museums and archaeological sites. A faithful 3D model can be very useful for studying the morphology of the original object without the risk of damages. This is particularly true in the case of historical violins, since a continuous use of caliber for taking measures can ruin the varnishes. However, the 3D scan of historical violins is not straightforward but has various problems mainly due to their complex morphology and the reflectivity of the varnishes. After a series of tests conducted in Arvedi laboratory with sample violins, we defined an ad-hoc protocol for the acquisition of 3D models of violins using an high resolution laser scanner. The protocol was then tested for acquiring various instruments held in "Museo del Violino". The obtained 3D models allowed to study and compare dimensions and shapes of violins of different authors and periods. The experiments now continue on other kinds of historical instruments in collaboration with various institutions and museums, such as hunt horns from Castello Sforzesco of Milan (Italy) and popular instruments from "Museo del Paesaggio Sonoro" of Riva Presso Chiero(Torino, Italy).

HCI applications for scientific dissemination kinect
Digital technologies for enhancing the fruition of museums and exhibitions have the main aim to connect people, culture, and technologies. Interactive and immersive applications proved to be very helpful to present cultural information to a large public; in fact, many museums or temporary exhibitions around the world now adopt serious games or virtual and augmented reality applications to present their collections. In this context, we designed various applications that can be used to show the results of scientific examinations on violins or other artworks in an appealing and comprehensible way even to people non-expert in the field. We considered both gestural and gaze interaction, respectively with kinect and eye-tracker. Now, we are also studying the use of Augmented Reality to present both 3D models acquired and scietific data coming from the various non-invasive examinations performed.

Related publications

International journals

Piercarlo Dondi, Luca Lombardi, Marco Porta, Tommaso Rovetta, Claudia Invernizzi, Marco Malagodi (2019). What do luthiers look at? An eye tracking study on the identification of meaningful areas in historical violins, Multimedia Tools and Applications, Vol. 78, Issue 14, pp. 19115-19139, DOI: 10.1007/s11042-019-7276-2
Piercarlo Dondi, Luca Lombardi, Irene Rocca, Marco Malagodi, Maurizio Licchelli (2018). Multimodal workflow for the creation of interactive presentations of 360 spin images of historical violins, Multimedia Tools and Applications, Springer Science+Business Media, LLC, part of Springer Nature 2018, Springer US, November 2018, Volume 77, Issue 21, pp. 28309–28332, Print ISSN: 1380-7501, Online ISSN: 1573-7721, DOI:10.1007/s11042-018-6046-x
Piercarlo Dondi, Luca Lombardi, Claudia Invernizzi, Tommaso Rovetta, Marco Malagodi and Maurizio Licchelli, (2017). Automatic analysis of UV induced fluorescence imagery of historical violins, Journal on Computing and Cultural Heritage (JOCCH), Volume 10 Issue 2, March 2017, ACM New York, NY, USA, DOI:10.1145/3051472.
Piercarlo Dondi, Luca Lombardi, Marco Malagodi, Maurizio Licchelli, (2017). 3D modelling and measurements of historical violins, ACTA IMEKO, Vol 6, No3, Sep. 2017, pp. 29-34, DOI:10.21014/acta_imeko.v6i3.455.
Piercarlo Dondi, Luca Lombardi, Marco Malagodi, Maurizio Licchelli, (2016). Automatic identification of varnish wear on historical instruments: The case of Antonio Stradivari violins, Journal of Cultural Heritage, Vol 22 (Nov-Dec 2016), pp. 968-973, Elsevier, DOI:10.1016/j.culher.2016.05.010.
Piercarlo Dondi, Luca Lombardi, Marco Malagodi, Maurizio Licchelli, and Fausto Cacciatori, (2016). Color-based automatic detection of worn out varnishes on Stradivari's “Scotland University” violin back plate, Color Research & Application, Special Issue: Bridging Science with Art, Volume 41, Issue 3, pages 313–316, article first published online: 29 DEC 2015, Wiley Periodicals, Inc. Col Res Appl, DOI:10.1002/col.22012.

International conferences

Alireza Rezaei, Sylvie Le Hégarat-Mascle, Emanuel Aldea, Piercarlo Dondi, Marco Malagodi (2021). One step clustering based on a-contrario framework for detection of alterations in historical violins, in 25th International Conference on Pattern Recognition (ICPR2020), pp. 9348-9355, IEEE, DOI:10.1109/ICPR48806.2021.9412129.
Piercarlo Dondi, Luca Lombardi, Marco Malagodi, Maurizio Licchelli (2021). Stylistic Classification of Historical Violins: A Deep Learning Approach, in Del Bimbo A. et al. (eds.), Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2020. Lecture Notes in Computer Science, vol 12667, Springer, Cham, pp. 112-125, DOI:10.1007/978-3-030-68787-8_8.
Michela Albano, Giacomo Fiocco, Piercarlo Dondi, Francesca Tasso, Valentina Ricetti, Daniela Comelli, Maurizio Licchelli, Claudio Canevari, Marco Malagodi (2020). Preliminary study of the shapes and materials of two natural horns of the end of the 17th century, in MetroArcheo2020, International Conference on Metrology for Archaeology and Cultural Heritage, Trento, Italy, 22-24 October 2020, pp. 454-459.
Piercarlo Dondi, Luca Lombardi, Marco Malagodi, Maurizio Licchelli (2019). Segmentation of Multi-temporal UV-Induced Fluorescence Images of Historical Violins, in Cristani M., Prati A., Lanz O., Messelodi S., Sebe N. (eds.), New Trends in Image Analysis and Processing, ICIAP 2019. Lecture Notes in Computer Science, vol 11808, pp. 81-91, Springer, Cham, print ISBN: 978-3-030-30753-0, online ISBN: 978-3-030-30754-7, DOI:10.1007/978-3-030-30754-7_9
Alireza Rezaei, Emanuel Aldea, Piercarlo Dondi, Marco Malagodi, Sylvie Le Hégarat (2019). Detecting alterations in historical violins with optical monitoring, Proc. SPIE 11172, Fourteenth International Conference on Quality Control by Artificial Vision, 1117210 (16 July 2019), DOI:10.1117/12.2521702
Giacomo Patrucco, Filiberto Chiabrando, Piercarlo Dondi, and Marco Malagodi (2018). Image and Range-Based 3D Acquisition and Modeling of Popular Musical Instruments, Proceedings from the Document Academy, Vol. 5, Iss. 2, Article 9.
Piercarlo Dondi, Luca Lombardi, Marco Malagodi, Maurizio Licchelli, (2016). Measuring Stradivari violin “Cremonese” (1715) by 3D Modeling, in Proocedings of 2nd IMEKO International Conference of Metrology of Archaeology and Cultural Heritage (MetroArcheo 2016), pp. 29-33, IMEKO, Torino 19-21 October 2016, ISBN: 978-92-990075-4-9.
Piercarlo Dondi, Luca Lombardi, Marco Malagodi, Maurizio Licchelli, Tommaso Rovetta, Claudia Invernizzi (2015). An Interactive Tool for Speed up the Analysis of UV Images of Stradivari Violins, New Trends in Image Analysis and Processing, Proceedings of ICIAP 2015 International Workshops, Genoa, Italy, September 7-8, 2015, Volume 9281 of the series Lecture Notes in Computer Science, Springer International Publishing, pp 103-110, Print ISBN: 978-3-319-23221-8, Online ISBN: 978-3-319-23222-5.
Dondi Piercarlo, Invernizzi Claudia, Licchelli Maurizio, Lombardi Luca, Malagodi Marco, Rovetta Tommaso (2015). Semi-automatic system for UV images analysis of historical musical instruments, Proc. SPIE 9527, Optics for Arts, Architecture, and Archaeology V, 95270H, DOI:10.1117/12.2184744.

Others

Tiziana Cavaleri, Tommaso Rovetta, Giacomo Fiocco, Piercarlo Dondi, Marco Malagodi, Monica Gulmini, Anna Piccirillo, Marco Pisani, Massimo Zucco, Marco Gargano (2020). A new imaging method of fluorescence induced by multispectral UV for studying historical musical instruments coatings, in Conservation 360°: UV-Vis Fluorescence imaging techniques, Editorial Universitat Politècnica de València.
Piercarlo Dondi, Giusj Valentina Fichera, Claudia Invernizzi, Maurizio Licchelli, Marco Malagodi, Tommaso Rovetta, (2017) The Messie model, research on materials and study of moulds, in Alf G., Cacciatori F., "The Absolute Stradivari, the Messie violin 1716/2016", Cremona, Fondazione Museo del Violino Antonio Stradivari, 2016, pp.89-112, ISBN: 8890917970
Tommaso Rovetta, Alessandra Chiapparini , Piercarlo Dondi, Giusj Valentina Fichera, Claudia Invernizzi, Marco Malagodi, Maurizio Licchelli, (2016). Metodi analitici non invasivi per lo studio dei materiali degli strumenti musicali antichi, in Archeomatica, anno VII #2, pag. 20-25, July 2016.
Marco Malagodi, Tommaso Rovetta, Piercarlo Dondi, Claudia Invernizzi, Giusj Valentina Fichera, Maurizio Licchelli (2015). La materia del Cremonese, in Il Cremonese 1715-2015, 300° anniversario, Fondazione Museo del Violino Antonio Stradivari Cremona.