Decay detection in historic buildings through image-based deep learning

Authors

DOI:

https://doi.org/10.4995/vitruvio-ijats.2023.18662

Keywords:

built heritage, historic buildings, decay detection, deep learning, Mask R-CNN

Abstract

Nowadays, built heritage condition assessment is realized through on-site or photo-aided visual inspections, reporting pathologies manually on drawings, photographs, notes. The knowledge of the state of conservation goes through subjective and time or cost consuming procedures. This is even relevant for a historic building characterized by geometrical and morphological complexity and huge extension, or at risk of collapse. In this context, advancements in the field of Computer Vision and Artificial Intelligence provide an opportunity to address these criticalities. The proposed methodology is based on a Mask R-CNN model, for the detection of decay morphologies on built heritages, and, particularly on historic buildings. The experimentation has been carried out and validated on a highly heterogeneous dataset of images of historic buildings, representative of the regional Architectural Heritage, such as: castles, monasteries, noble buildings, rural buildings. The outcomes highlighted the significance of this remote, non-invasive inspection technique, in support of the technicians in the preliminary knowledge of the building state of conservation, and, most of all, in the decay mapping of some particular classes of alterations (moist area, biological colonization).

Downloads

Download data is not yet available.

Author Biographies

Silvana Bruno, Polytechnic University of Bari

DICATECh, Department of Civil, Environmental, Land, Construction and Chemistry

Rosella Alessia Galantucci, Polytechnic University of Bari

DICATECh, Department of Civil, Environmental, Land, Construction and Chemistry

Antonella Musicco, Polytechnic University of Bari

DICATECh, Department of Civil, Environmental, Land, Construction and Chemistry

References

Girshick, R. (2015) ‘Fast R-CNN’, Proceedings of the IEEE International Conference on Computer Vision, 2015 Inter, pp. 1440–1448. https://doi.org/10.1109/ICCV.2015.169

Hatir, M. E., Barstuğan, M. and İnce, İ. (2020) ‘Deep learning-based weathering type recognition in historical stone monuments’, Journal of Cultural Heritage, 45, pp. 193–203. https://doi.org/10.1016/j.culher.2020.04.008

He, K. et al. (2016) ‘Deep residual learning for image recognition’, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016-Decem, pp. 770–778. https://doi.org/10.1109/CVPR.2016.90

He, K. et al. (2017) ‘Mask R-CNN’, in 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2980–2988. https://doi.org/10.1109/ICCV.2017.322

ICOMOS ISCS. (2008) Illustrated glossary on stone deterioration patterns.

Json (no date). https://www.json.org/json-en.html

Kalfarisi, R., Wu, Z. Y. and Soh, K. (2020) ‘Crack Detection and Segmentation Using Deep Learning with 3D Reality Mesh Model for Quantitative Assessment and Integrated Visualization’, Journal of Computing in Civil Engineering, 34(3), pp. 1–20. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000890

Khandelwal, R. (2019). Computer vision: instance segmentation with mask R-CNN. Dostupné z:” https://towardsdatascience.com/computer-vision-instancesegmentation-with-mask-r-cnn-7983502fcad1.

keras (no date). https://keras.io/

Kim, B. and Cho, S. (2019) ‘Image-based concrete crack assessment using mask and region-based convolutional neural network’, Structural Control and Health Monitoring, 26(8), pp. 1–15. https://doi.org/10.1002/stc.2381

Li, X. et al. (2019) ‘Weighted feature pyramid networks for object detection’, Proceedings - 2019 IEEE Intl Conf on Parallel and Distributed Processing with Applications, Big Data and Cloud Computing, Sustainable Computing and Communications, Social Computing and Networking, https://doi.org/10.1109/ISPA-BDCloud-SustainCom-SocialCom48970.2019.00217

Lin, T. Y. et al. (2014) ‘Microsoft COCO: Common objects in context’, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 8693 LNCS(PART 5), pp. 740–755. https://doi.org/10.1007/978-3-319-10602-1_48

Liu, Z. et al. (2019) ‘Computer vision-based concrete crack detection using U-net fully convolutional networks’, Automation in Construction. Elsevier, 104(January), pp. 129–139. https://doi.org/10.1016/j.autcon.2019.04.005

Mask R-CNN library (no date). https://github.com/matterport/Mask_RCNN

Mishra, M. (2021) ‘Machine learning techniques for structural health monitoring of heritage buildings: A state-of- the-art review and case studies’, Journal of Cultural Heritage, 47, pp. 227–245. https://doi.org/10.1016/j.culher.2020.09.005

Odemakinde, E. (no date) Mask R-CNN: A Beginner’s Guide.

OpenCV (no date). https://opencv.org/

Perez, H., Tah, J. H. M. and Mosavi, A. (2019) ‘Deep Learning for Detecting Building Defects Using’, Sensors, 19(16), p. 3556. https://doi.org/10.3390/s19163556

Ren, S. et al. (2017) ‘Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks’, IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), pp. 1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031

Renu Khandelwal (2019) Computer Vision: Instance Segmentation with Mask R-CNN.

Sagar, V. and Jain, S. J. (2018) ‘Yield Estimation using faster R-CNN’, International Research Journal in GlobalEngineering and Sciences., 3(1), pp. 110–116.

Scikit image (no date). https://scikit-image.org/

TensorFlow (no date). https://www.tensorflow.org

UNI (2006) ‘UNI 11182 Beni culturali - Materiali lapidei naturali e artificiali - Descrizione della forma di alterazione - Termini e definizioni’.

Wu, Z. Y. et al. (2020) ‘Applying deep convolutional neural network with 3D reality mesh model for water tank crack detection and evaluation’, Urban Water Journal, 17(8), pp. 682–695. https://doi.org/10.1080/1573062X.2020.1758166

Xu, X. et al. (2022) ‘Crack Detection and Comparison Study Based on Faster R-CNN and Mask R-CNN’, Sensors, 22(3). https://doi.org/10.3390/s22031215

Downloads

Published

2023-04-04

How to Cite

Bruno, S., Galantucci, R. A. . and Musicco, A. (2023) “Decay detection in historic buildings through image-based deep learning”, VITRUVIO - International Journal of Architectural Technology and Sustainability, 8, pp. 6–17. doi: 10.4995/vitruvio-ijats.2023.18662.