Self-adaptive colour calibration pipeline for deep underwater image restoration: a reference-based evaluation
Submitted: 2025-06-20
|Accepted: 2025-10-15
|Published: 2025-10-22
Copyright (c) 2025 Virtual Archaeology Review
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Downloads
Keywords:
structure from motion (SfM), multi-view stereo (MVS), underwater (UW) colour restoration, feedforward neural networks (FNNs), colour charts, quantitative evaluation
Supporting agencies:
Abstract:
Building upon our previous work on underwater (UW) image colour restoration, this paper advances the self-adaptive colour calibration (SACC)pipeline, by introducing a systematic, reference-based quantitative evaluation. UWimagery is notoriously affected by wavelength-dependent light absorption, scattering, and colour attenuation; these factors distort the visual appearance of submerged objects and complicate scientific documentation. Traditional model-based correction techniques often rely on well-calibrated equipment or prior environmental data; many artificial intelligence (AI)-driven approaches suffer from dataset bias and limited generalisability. The work explained below addresses these challenges by providing a flexible pipeline that restores colours without requiring extensive environmental metadata; this methodology is particularly well-suited for archival and in-field datasets. New datasets acquired from the 18th-century Nissia shipwreck site, located off Protaras in Cyprus, were employed to evaluate the pipeline under real-world excavation conditions. These datasets, captured with a rigid camera–strobe configuration, include reference regions with colour charts; these provide reliable ‘ground-truth’(GT)values for objective comparison. Structure from Motion (SfM) and Multi-View Stereo (MVS) were used to derive depth maps, which, together with red-green-blue (RGB)values, were input to a Feedforward Neural Network (FNN). The network was trained using Adam and RMSprop optimisers, enabling the restoration of perceptually accurate colours across varying conditions. Three metrics, modified R², Euclidean Distance, and the D metric, were applied to assess performance. The evaluation is constrained by the limited number of images containing colour charts; even so, this limitation reflects authentic field conditions, where divers must prioritise site documentation over calibration. Overall, this study validates the robustness of the SACC pipeline and introduces a replicable framework for evaluating UW image colour restoration. Beyond methodological advances, the findings highlight the potential of the pipeline to support cultural heritage documentation, archaeological interpretation, and long-term preservation of submerged sites.
References:
Aggarwal, C. C. (2018). Teaching deep learners to generalize. In C. C. Aggarwal (Ed.), Neural networks and deep learning (pp. 169–216). Cham, Switzerland: Springer. https://doi.org/10.1007/978-3-319-94463-0_4
Aggarwal, C. C. (2023). Neural Networks and Deep Learning (2nd ed.). Cham, Switzerland: Springer. https://doi.org/10.1007/978-3-031-29642-0
Akkaynak, D., & Treibitz, T. (2018). A revised underwater image formation model. In Proceedings of the IEEE/CVF Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 6723–6732). Los Alamitos, CA: IEEE Computer Society. https://doi.org/10.1109/CVPR.2018.00703
Akkaynak, D., & Treibitz, T. (2019). Sea-thru: A method for removing water from underwater images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 1682–1691). Los Alamitos, CA: IEEE Computer Society. https://doi.org/10.1109/CVPR.2019.00178
Akkaynak, D., Treibitz, T., Shlesinger, T., Tamir, R., Loya, Y., & Iluz, D. (2017). What is the space of attenuation coefficients in underwater computer vision? In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 568–577). Los Alamitos, CA: IEEE Computer Society. https://doi.org/10.1109/CVPR.2017.68
Ancuti, C., Ancuti, C. O., Haber, T., & Bekaert, P. (2012). Enhancing underwater images and videos by fusion. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 81–88). Los Alamitos, CA: IEEE Computer Society. https://doi.org/10.1109/CVPR.2012.6247661
Ancuti, C. O., Ancuti, C., De Vleeschouwer, C., Neumann, L., & Garcia, R. (2017). Color transfer for underwater dehazing and depth estimation. In Proceedings of the IEEE International Conference on Image Processing (ICIP) (pp. 695–699). Los Alamitos, CA: IEEE. https://doi.org/10.1109/ICIP.2017.8296370
Bekerman, Y., Avidan, S., & Treibitz, T. (2020). Unveiling optical properties in underwater images. In Proceedings of the IEEE International Conference on Computational Photography (ICCP) (pp. 1–12). Los Alamitos, CA: IEEE. https://doi.org/10.1109/iccp48838.2020.9105267
Bianco, G., Muzzupappa, M., Bruno, F., Garcia, R., & Neumann, L. (2015). A new color correction method for underwater imaging. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 40(5/W5), 25–32. https://doi.org/10.5194/isprsarchives-XL-5-W5-25-2015
Bryson, M., Johnson-Roberson, M., Pizarro, O., & Williams, S. B. (2013). Colour-consistent structure-from-motion models using underwater imagery. In Proceedings of Robotics: Science and Systems VIII (pp. 33–40). Berlin, Germany: MIT Press. https://doi.org/10.7551/mitpress/9816.003.0010
Bryson, M., Johnson-Roberson, M., Pizarro, O., & Williams, S. B. (2016). True color correction of autonomous underwater vehicle imagery. Journal of Field Robotics, 33(6), 853–874. https://doi.org/10.1002/rob.21629
Chao, L., & Wang, M. (2010). Removal of water scattering. In Proceedings of the 2010 International Conference on Computer Engineering and Technology (ICCET) (Vol. 2, pp. 35–39). Piscataway, NJ: IEEE. https://doi.org/10.1109/ICCET.2010.5485339
Chiang, J. Y., & Chen, Y. C. (2012). Underwater image enhancement by wavelength compensation and dehazing. IEEE Transactions on Image Processing, 21(4), 1756–1769. https://doi.org/10.1109/TIP.2011.2179666
Corchs, S., & Schettini, R. (2010). Underwater image processing: State of the art of restoration and image enhancement methods. EURASIP Journal on Advances in Signal Processing, 2010, 746052. https://doi.org/10.1155/2010/746052
Demesticha, S. (2011). The 4th-century BC Mazotos shipwreck, Cyprus: A preliminary report. International Journal of Nautical Archaeology, 40(1), 39–59. https://doi.org/10.1111/j.1095-9270.2010.00269.x
Demesticha, S., Skarlatos, D., & Neophytou, A. (2014). The 4th-century BC shipwreck at Mazotos, Cyprus: New techniques and methodologies in the 3D mapping of shipwreck excavations. Journal of Field Archaeology, 39(2), 134–150. https://doi.org/10.1179/0093469014Z.00000000077
DGK Color Tools. (n.d.). WDKK waterproof color chart. B&H Photo Video. Retrieved November 25, 2024, from https://www.bhphotovideo.com/c/product/1014565-REG/dgk_color_tools_wdkk_bubble_bag_1.html
Doležal, M., Vlachos, M., Secci, M., Demesticha, S., Skarlatos, D., & Liarokapis, F. (2019). Understanding underwater photogrammetry for maritime archaeology through immersive virtual reality. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLII-2/W10, 85–91. https://doi.org/10.5194/isprs-archives-XLII-2-W10-85-2019
Ertan, Z., Korkut, B., Gördük, G., Kulavuz, B., Bakırman, T., & Bayram, B. (2024). Enhancement of underwater images with artificial intelligence. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLVIII-4/W9-2024, 149–156. https://doi.org/10.5194/isprs-archives-XLVIII-4-W9-2024-149-2024
Fabbri, C., Islam, M. J., & Sattar, J. (2018). Enhancing underwater imagery using generative adversarial networks. Proceedings of the IEEE International Conference on Robotics and Automation (pp. 7159–7165). IEEE. https://doi.org/10.1109/ICRA.2018.8460552
Géron, A. (2019). Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow: Concepts, tools, and techniques to build intelligent systems (2nd ed.). Sebastopol, CA: O’Reilly Media.
Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. Cambridge, MA: MIT Press.
Goodfellow, I., Pouget-Abadie, J., Mirza, B., Xu, B., Warde-Farley, D., Ozair, S., … Bengio, Y. (2014). Generative adversarial nets. In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, & K. Q. Weinberger (Eds.), Advances in neural information processing systems (Vol. 27, pp. 2672–2680). Red Hook, NY: Curran Associates, Inc. Retrieved from https://proceedings.neurips.cc/paper_files/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf
Hashisho, Y., Albadawi, M., Krause, T., & von Lukas, U. F. (2019). Underwater color restoration using U-Net denoising autoencoder [Preprint]. arXiv. http://arxiv.org/abs/1905.09000
Hastie, T., Tibshirani, R., & Friedman, J. (2001). The elements of statistical learning: Data mining, inference, and prediction. New York, NY: Springer.
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012). Improving neural networks by preventing co-adaptation of feature detectors [Preprint] http://arxiv.org/abs/1207.0580
Hou, W., Gray, D. J., Weidemann, A. D., Fournier, G. R., & Forand, J. L. (2007). Automated underwater image restoration and retrieval of related optical properties. In Proceedings of the IEEE International Geoscience and Remote Sensing Symposium (IGARSS) (pp. 1889–1892). IEEE. https://doi.org/10.1109/IGARSS.2007.4423193
Isola, P., Zhu, J. Y., Zhou, T., & Efros, A. A. (2017). Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017) (pp. 5967–5976). IEEE. https://doi.org/10.1109/CVPR.2017.632
Jerlov, N. G., Koczy, F. F., & Schooner, A. (1951). Photographic measurements of daylight in deep water. Gothenburg, Sweden: Elanders Boktr.
Jordan, M. I., & Mitchell, T. M. (2015). Machine learning: Trends, perspectives, and prospects. Science, 349(6245), 255–260. https://doi.org/10.1126/science.aaa8415
Kingma, D. P., & Ba, J. L. (2015). Adam: A method for stochastic optimization [Preprint]. arXiv. http://arxiv.org/abs/1412.6980
Klemen, I. (2021). Underwater image-based 3D reconstruction with quality estimation (PhD thesis). University of Girona, Girona, Spain.
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. In F. Pereira, C. J. C. Burges, L. Bottou, & K. Q. Weinberger (Eds.), Advances in neural information processing systems (Vol. 25, pp. 1097–1105). Red Hook, NY: Curran Associates, Inc. Retrieved from https://proceedings.neurips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf
Lee, J., Park, J. K., Baik, S., & Lee, K. M. (2024). Rethinking RGB color representation for image restoration models [Preprint]. arXiv. https://arxiv.org/abs/2402.03399
Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., … Shi, W. (2017). Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017) (pp. 105–114). IEEE. https://doi.org/10.1109/CVPR.2017.19
Li, C., Guo, C., Ren, W., Cong, R., Hou, J., Kwong, S., & Tao, D. (2020). An underwater image enhancement benchmark dataset and beyond. IEEE Transactions on Image Processing, 29, 4376–4389. https://doi.org/10.1109/TIP.2019.2955241
Li, C. Y., & Cavallaro, A. (2022). On the limits of perceptual quality measures for enhanced underwater images [Preprint]. arXiv. https://arxiv.org/abs/2207.05470
Li, J., Skinner, K. A., Eustice, R. M., & Johnson-Roberson, M. (2018). WaterGAN: Unsupervised generative network to enable real-time color correction of monocular underwater images. IEEE Robotics and Automation Letters, 3(1), 387–394. https://doi.org/10.1109/LRA.2017.2730363
Lu, J., Li, N., Zhang, S., Yu, Z., Zheng, H., & Zheng, B. (2019). Multi-scale adversarial network for underwater image restoration. Optics and Laser Technology, 110, 105–113. https://doi.org/10.1016/j.optlastec.2018.05.048
Morel, A., Gentili, B., Claustre, H., Babin, M., Bricaud, A., Ras, J., & Tièche, F. (2007). Optical properties of the “clearest” natural waters. Limnology and Oceanography, 52(1), 217–229. https://doi.org/10.4319/lo.2007.52.1.0217
Mu, D., Li, H., Liu, H., Dong, L., & Zhang, G. (2023). Underwater image enhancement using a mixed generative adversarial network. IET Image Processing, 17(4), 1149–1160. https://doi.org/10.1049/ipr2.12702
Müller, A. C., & Guido, S. (2016). Introduction to machine learning with Python: A guide for data scientists. Sebastopol, CA: O’Reilly Media.
Murphy, K. P. (2013). Machine learning: A probabilistic perspective. Cambridge, MA: MIT Press.
University of Cyprus. (n.d.). Nissia shipwreck – Multimedia applications for Cypriot maritime cultural heritage. Retrieved November 14, 2024, from https://www.ucy.ac.cy/marecyprium/case-studies/nissia-shipwreck/
Nurtantio Andono, P., Purnama, I. K. E., & Hariadi, M. (2013). Underwater image enhancement using adaptive filtering for enhanced SIFT-based image matching. Journal of Theoretical and Applied Information Technology, 52(3), 273–280.
Panetta, K., Gao, C., & Agaian, S. (2016). Human-visual-system-inspired underwater image quality measures. IEEE Journal of Oceanic Engineering, 41(3), 541–551. https://doi.org/10.1109/JOE.2015.2469915
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., … Duchesnay, É. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830.
Pyle, D. (1999). Data preparation for data mining. Burlington, MA: Morgan Kaufmann Publishers.
Ren, S., He, K., Girshick, R., & Sun, J. (2017). Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031
Schechner, Y. Y., & Karpel, N. (2005). Recovery of underwater visibility and structure by polarization analysis. IEEE Journal of Oceanic Engineering, 30(3), 570–587. https://doi.org/10.1109/JOE.2005.850871
Solonenko, M. G., & Mobley, C. D. (2015). Inherent optical properties of Jerlov water types. Applied Optics, 54(17), 5392–5401. https://doi.org/10.1364/ao.54.005392
Torres-Méndez, L. A., & Dudek, G. (2005). Color correction of underwater images for aquatic robot inspection. In R. Siegwart & I. Nourbakhsh (Eds.), Field and Service Robotics (pp. 60–73). Berlin, Germany: Springer. https://doi.org/10.1007/11585978_5
Vlachos, M., Calantropio, A., Skarlatos, D., & Chiabrando, F. (2022). Modelling colour absorption of underwater images using SfM-MVS generated depth maps. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLIII(B2-2), 959–966. https://doi.org/10.5194/isprs-archives-XLIII-B2-2022-959-2022
Vlachos, M., & Skarlatos, D. (2021). An extensive literature review on underwater image colour correction. Sensors, 21(17), 5690. https://doi.org/10.3390/s21175690
Vlachos, M., & Skarlatos, D. (2024). Self-adaptive colour calibration of deep underwater images using FNN and SfM-MVS-generated depth maps. Remote Sensing, 16(7), 1279. https://doi.org/10.3390/rs16071279
Wang, N., Zhou, Y., Han, F., Zhu, H., & Zheng, Y. (2019). UWGAN: Underwater GAN for real-world underwater color restoration and dehazing. arXiv preprint arXiv:1907.03552, 1–10. https://arxiv.org/abs/1912.10269
Wang, Y., Li, N., Li, Z., Gu, Z., Zheng, H., Zheng, B., & Sun, M. (2018). An imaging-inspired no-reference underwater color image quality assessment metric. Computers and Electrical Engineering, 70, 904–913. https://doi.org/10.1016/j.compeleceng.2017.12.006
Wang, Y., Song, W., Fortino, G., Qi, L. Z., Zhang, W., & Liotta, A. (2019). An experimental-based review of image enhancement and image restoration methods for underwater imaging. IEEE Access, 7, 140233–140251. https://doi.org/10.1109/ACCESS.2019.2932130
Peng, Y.-T., Zhao, X., & Cosman, P. C. (2015). Single underwater image enhancement using depth estimation based on blurriness. Proceedings of the IEEE International Conference on Image Processing (ICIP), 2–6. https://doi.org/10.1109/ICIP.2015.7351749
Yang, M., & Sowmya, A. (2015). An underwater color image quality evaluation metric. IEEE Transactions on Image Processing, 24(12), 6062–6071. https://doi.org/10.1109/TIP.2015.2491020
Zhao, X., Jin, T., & Qu, S. (2015). Deriving inherent optical properties from background color and underwater image enhancement. Ocean Engineering, 94, 163–172. https://doi.org/10.1016/j.oceaneng.2014.11.036
Zhu, J. Y., Park, T., Isola, P., & Efros, A. A. (2017). Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2242–2251. https://doi.org/10.1109/ICCV.2017.244
