A machine learning framework for enhancing digital experiences in cultural heritage
Journal of Enterprise Information Management
ISSN: 1741-0398
Article publication date: 15 June 2020
Issue publication date: 24 April 2023
Abstract
Purpose
Digital tools have been used to document cultural heritage with high-quality imaging and metadata. However, some of the historical assets are totally or partially unlabeled and some are physically damaged, which decreases their attractiveness and induces loss of value. This paper introduces a new framework that aims at tackling the cultural data enrichment challenge using machine learning.
Design/methodology/approach
This framework focuses on the automatic annotation and metadata completion through new deep learning classification and annotation methods. It also addresses issues related to physically damaged heritage objects through a new image reconstruction approach based on supervised and unsupervised learning.
Findings
The authors evaluate approaches on a data set of cultural objects collected from various cultural institutions around the world. For annotation and classification part of this study, the authors proposed and implemented a hierarchical multimodal classifier that improves the quality of annotation and increases the accuracy of the model, thanks to the introduction of multitask multimodal learning. Regarding cultural data visual reconstruction, the proposed clustering-based method, which combines supervised and unsupervised learning is found to yield better quality completion than existing inpainting frameworks.
Originality/value
This research work is original in sense that it proposes new approaches for the cultural data enrichment, and to the authors’ knowledge, none of the existing enrichment approaches focus on providing an integrated framework based on machine learning to solve current challenges in cultural heritage. These challenges, which are identified by the authors are related to metadata annotation and visual reconstruction.
Keywords
Acknowledgements
This publication was made possible by NPRP grant 9-181-1-036 from the Qatar National Research Fund (a member of Qatar Foundation). The statements made herein are solely the responsibility of the authors (www.ceproqha.qa). The authors would also like to thank the Museum of Islamic Art (MIA), the MIA Multimedia team, Mr. Marc Pelletreau, the art curators and the management staff of the Museum of Islamic Art, Doha, Qatar, for their help and contribution in the data acquisition.
Citation
Belhi, A., Bouras, A., Al-Ali, A.K. and Foufou, S. (2023), "A machine learning framework for enhancing digital experiences in cultural heritage", Journal of Enterprise Information Management, Vol. 36 No. 3, pp. 734-746. https://doi.org/10.1108/JEIM-02-2020-0059
Publisher
:Emerald Publishing Limited
Copyright © 2020, Emerald Publishing Limited