Decay detection in historic buildings through image-based deep learning
Keywords:built heritage, historic buildings, decay detection, deep learning, Mask R-CNN
Nowadays, built heritage condition assessment is realized through on-site or photo-aided visual inspections, reporting pathologies manually on drawings, photographs, notes. The knowledge of the state of conservation goes through subjective and time or cost consuming procedures. This is even relevant for a historic building characterized by geometrical and morphological complexity and huge extension, or at risk of collapse. In this context, advancements in the field of Computer Vision and Artificial Intelligence provide an opportunity to address these criticalities. The proposed methodology is based on a Mask R-CNN model, for the detection of decay morphologies on built heritages, and, particularly on historic buildings. The experimentation has been carried out and validated on a highly heterogeneous dataset of images of historic buildings, representative of the regional Architectural Heritage, such as: castles, monasteries, noble buildings, rural buildings. The outcomes highlighted the significance of this remote, non-invasive inspection technique, in support of the technicians in the preliminary knowledge of the building state of conservation, and, most of all, in the decay mapping of some particular classes of alterations (moist area, biological colonization).
Girshick, R. (2015) ‘Fast R-CNN’, Proceedings of the IEEE International Conference on Computer Vision, 2015 Inter, pp. 1440–1448. https://doi.org/10.1109/ICCV.2015.169
Hatir, M. E., Barstuğan, M. and İnce, İ. (2020) ‘Deep learning-based weathering type recognition in historical stone monuments’, Journal of Cultural Heritage, 45, pp. 193–203. https://doi.org/10.1016/j.culher.2020.04.008
He, K. et al. (2016) ‘Deep residual learning for image recognition’, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016-Decem, pp. 770–778. https://doi.org/10.1109/CVPR.2016.90
He, K. et al. (2017) ‘Mask R-CNN’, in 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2980–2988. https://doi.org/10.1109/ICCV.2017.322
ICOMOS ISCS. (2008) Illustrated glossary on stone deterioration patterns.
Json (no date). https://www.json.org/json-en.html
Kalfarisi, R., Wu, Z. Y. and Soh, K. (2020) ‘Crack Detection and Segmentation Using Deep Learning with 3D Reality Mesh Model for Quantitative Assessment and Integrated Visualization’, Journal of Computing in Civil Engineering, 34(3), pp. 1–20. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000890
Khandelwal, R. (2019). Computer vision: instance segmentation with mask R-CNN. Dostupné z:” https://towardsdatascience.com/computer-vision-instancesegmentation-with-mask-r-cnn-7983502fcad1.
keras (no date). https://keras.io/
Kim, B. and Cho, S. (2019) ‘Image-based concrete crack assessment using mask and region-based convolutional neural network’, Structural Control and Health Monitoring, 26(8), pp. 1–15. https://doi.org/10.1002/stc.2381
Li, X. et al. (2019) ‘Weighted feature pyramid networks for object detection’, Proceedings - 2019 IEEE Intl Conf on Parallel and Distributed Processing with Applications, Big Data and Cloud Computing, Sustainable Computing and Communications, Social Computing and Networking, https://doi.org/10.1109/ISPA-BDCloud-SustainCom-SocialCom48970.2019.00217
Lin, T. Y. et al. (2014) ‘Microsoft COCO: Common objects in context’, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 8693 LNCS(PART 5), pp. 740–755. https://doi.org/10.1007/978-3-319-10602-1_48
Liu, Z. et al. (2019) ‘Computer vision-based concrete crack detection using U-net fully convolutional networks’, Automation in Construction. Elsevier, 104(January), pp. 129–139. https://doi.org/10.1016/j.autcon.2019.04.005
Mask R-CNN library (no date). https://github.com/matterport/Mask_RCNN
Mishra, M. (2021) ‘Machine learning techniques for structural health monitoring of heritage buildings: A state-of- the-art review and case studies’, Journal of Cultural Heritage, 47, pp. 227–245. https://doi.org/10.1016/j.culher.2020.09.005
Odemakinde, E. (no date) Mask R-CNN: A Beginner’s Guide.
OpenCV (no date). https://opencv.org/
Perez, H., Tah, J. H. M. and Mosavi, A. (2019) ‘Deep Learning for Detecting Building Defects Using’, Sensors, 19(16), p. 3556. https://doi.org/10.3390/s19163556
Ren, S. et al. (2017) ‘Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks’, IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), pp. 1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031
Renu Khandelwal (2019) Computer Vision: Instance Segmentation with Mask R-CNN.
Sagar, V. and Jain, S. J. (2018) ‘Yield Estimation using faster R-CNN’, International Research Journal in GlobalEngineering and Sciences., 3(1), pp. 110–116.
Scikit image (no date). https://scikit-image.org/
TensorFlow (no date). https://www.tensorflow.org
UNI (2006) ‘UNI 11182 Beni culturali - Materiali lapidei naturali e artificiali - Descrizione della forma di alterazione - Termini e definizioni’.
Wu, Z. Y. et al. (2020) ‘Applying deep convolutional neural network with 3D reality mesh model for water tank crack detection and evaluation’, Urban Water Journal, 17(8), pp. 682–695. https://doi.org/10.1080/1573062X.2020.1758166
Xu, X. et al. (2022) ‘Crack Detection and Comparison Study Based on Faster R-CNN and Mask R-CNN’, Sensors, 22(3). https://doi.org/10.3390/s22031215
How to Cite
Copyright (c) 2023 VITRUVIO - International Journal of Architectural Technology and Sustainability
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
This journal is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License