Decay detection in historic buildings through image-based deep learning




built heritage, historic buildings, decay detection, deep learning, Mask R-CNN


Nowadays, built heritage condition assessment is realized through on-site or photo-aided visual inspections, reporting pathologies manually on drawings, photographs, notes. The knowledge of the state of conservation goes through subjective and time or cost consuming procedures. This is even relevant for a historic building characterized by geometrical and morphological complexity and huge extension, or at risk of collapse. In this context, advancements in the field of Computer Vision and Artificial Intelligence provide an opportunity to address these criticalities. The proposed methodology is based on a Mask R-CNN model, for the detection of decay morphologies on built heritages, and, particularly on historic buildings. The experimentation has been carried out and validated on a highly heterogeneous dataset of images of historic buildings, representative of the regional Architectural Heritage, such as: castles, monasteries, noble buildings, rural buildings. The outcomes highlighted the significance of this remote, non-invasive inspection technique, in support of the technicians in the preliminary knowledge of the building state of conservation, and, most of all, in the decay mapping of some particular classes of alterations (moist area, biological colonization).


Download data is not yet available.

Author Biographies

Silvana Bruno, Polytechnic University of Bari

DICATECh, Department of Civil, Environmental, Land, Construction and Chemistry

Rosella Alessia Galantucci, Polytechnic University of Bari

DICATECh, Department of Civil, Environmental, Land, Construction and Chemistry

Antonella Musicco, Polytechnic University of Bari

DICATECh, Department of Civil, Environmental, Land, Construction and Chemistry


Girshick, R. (2015) ‘Fast R-CNN’, Proceedings of the IEEE International Conference on Computer Vision, 2015 Inter, pp. 1440–1448.

Hatir, M. E., Barstuğan, M. and İnce, İ. (2020) ‘Deep learning-based weathering type recognition in historical stone monuments’, Journal of Cultural Heritage, 45, pp. 193–203.

He, K. et al. (2016) ‘Deep residual learning for image recognition’, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016-Decem, pp. 770–778.

He, K. et al. (2017) ‘Mask R-CNN’, in 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2980–2988.

ICOMOS ISCS. (2008) Illustrated glossary on stone deterioration patterns.

Json (no date).

Kalfarisi, R., Wu, Z. Y. and Soh, K. (2020) ‘Crack Detection and Segmentation Using Deep Learning with 3D Reality Mesh Model for Quantitative Assessment and Integrated Visualization’, Journal of Computing in Civil Engineering, 34(3), pp. 1–20.

Khandelwal, R. (2019). Computer vision: instance segmentation with mask R-CNN. Dostupné z:”

keras (no date).

Kim, B. and Cho, S. (2019) ‘Image-based concrete crack assessment using mask and region-based convolutional neural network’, Structural Control and Health Monitoring, 26(8), pp. 1–15.

Li, X. et al. (2019) ‘Weighted feature pyramid networks for object detection’, Proceedings - 2019 IEEE Intl Conf on Parallel and Distributed Processing with Applications, Big Data and Cloud Computing, Sustainable Computing and Communications, Social Computing and Networking,

Lin, T. Y. et al. (2014) ‘Microsoft COCO: Common objects in context’, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 8693 LNCS(PART 5), pp. 740–755.

Liu, Z. et al. (2019) ‘Computer vision-based concrete crack detection using U-net fully convolutional networks’, Automation in Construction. Elsevier, 104(January), pp. 129–139.

Mask R-CNN library (no date).

Mishra, M. (2021) ‘Machine learning techniques for structural health monitoring of heritage buildings: A state-of- the-art review and case studies’, Journal of Cultural Heritage, 47, pp. 227–245.

Odemakinde, E. (no date) Mask R-CNN: A Beginner’s Guide.

OpenCV (no date).

Perez, H., Tah, J. H. M. and Mosavi, A. (2019) ‘Deep Learning for Detecting Building Defects Using’, Sensors, 19(16), p. 3556.

Ren, S. et al. (2017) ‘Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks’, IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), pp. 1137–1149.

Renu Khandelwal (2019) Computer Vision: Instance Segmentation with Mask R-CNN.

Sagar, V. and Jain, S. J. (2018) ‘Yield Estimation using faster R-CNN’, International Research Journal in GlobalEngineering and Sciences., 3(1), pp. 110–116.

Scikit image (no date).

TensorFlow (no date).

UNI (2006) ‘UNI 11182 Beni culturali - Materiali lapidei naturali e artificiali - Descrizione della forma di alterazione - Termini e definizioni’.

Wu, Z. Y. et al. (2020) ‘Applying deep convolutional neural network with 3D reality mesh model for water tank crack detection and evaluation’, Urban Water Journal, 17(8), pp. 682–695.

Xu, X. et al. (2022) ‘Crack Detection and Comparison Study Based on Faster R-CNN and Mask R-CNN’, Sensors, 22(3).




How to Cite

Bruno, S., Galantucci, R. A. . and Musicco, A. (2023) “Decay detection in historic buildings through image-based deep learning”, VITRUVIO - International Journal of Architectural Technology and Sustainability, 8, pp. 6–17. doi: 10.4995/vitruvio-ijats.2023.18662.