Deep learning for agricultural land use classification from Sentinel-2

M. Campos-Taberner, F.J. García-Haro, B. Martínez, M.A. Gilabert

Abstract

The use of deep learning techniques for remote sensing applications has recently increased. These algorithms have proven to be successful in estimation of parameters and classification of images. However, little effort has been made to make them understandable, leading to their implementation as “black boxes”. This work aims to evaluate the performance and clarify the operation of a deep learning algorithm, based on a bi-directional recurrent network of long short-term memory (2-BiLSTM). The land use classification in the Valencian Community based on Sentinel-2 image time series in the framework of the common agricultural policy (CAP) is used as an example. It is verified that the accuracy of the deep learning techniques is superior (98.6 % overall success) to that other algorithms such as decision trees (DT), k-nearest neighbors (k-NN), neural networks (NN), support vector machines (SVM) and random forests (RF). The performance of the classifier has been studied as a function of time and of the predictors used. It is concluded that, in the study area, the most relevant information used by the network in the classification are the images corresponding to summer and the spectral and spatial information derived from the red and near infrared bands. These results open the door to new studies in the field of the explainable deep learning in remote sensing applications.


Keywords

deep learning; BiLSTM; classification; time series; Sentinel-2

Full Text:

PDF

References

Baraldi, A., Parmiggiani, F. 1995. An investigation of the textural characteristics associated with gray level cooccurrence matrix statistical parameters. IEEE Transactions on Geoscience and Remote Sensing, 33(2), 293-304. https://doi.org/10.1109/36.377929

Bengio, Y., Simard, P., Frasconi, P. 1994. Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks, 5(2), 157-166. https://doi.org/10.1109/72.279181

Breiman, L., Friedman, J., Olshen, R.A., Stone, C.J. 1984. Classification and regression trees. Taylor & Francis: London, UK.

Breiman, L. 2001. Random forests. Machine learning, 45(1), 5-32. https://doi.org/10.1023/A:1010933404324

Campos-Taberner, M., Romero-Soriano, A., Gatta, C., Camps-Valls, G., Lagrange, A., Le Saux, B., Beaupère, A., Boulch, A., Chan-Hon-Tong, A., Herbin, S., Randrianarivo, H., Ferecatu, M., Shimoni, M., Moser, G., Tuia, D. 2016. Processing of extremely high-resolution Lidar and RGB data: outcome of the 2015 IEEE GRSS data fusion contest–part a: 2-D contest. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 9(12), 5547-5559. https://doi.org/10.1109/JSTARS.2016.2569162

Campos-Taberner, M., García-Haro, F.J., Martínez, B., Sánchez-Ruíz, S., Gilabert, M.A. 2019a. A Copernicus Sentinel-1 and Sentinel-2 Classification Framework for the 2020+ European Common Agricultural Policy: A Case Study in València (Spain). Agronomy, 9(9), 556. https://doi.org/10.3390/agronomy9090556

Campos-Taberner, M., García-Haro, F.J., Martínez, B., Sánchez-Ruiz, S., Gilabert, M.A. 2019b. Evaluación del potencial de Sentinel-2 para actualizar el SIGPAC de la Comunitat Valenciana. En: XVIII Congreso de la Asociación Española de Teledetección. Valladolid, España, 24-27, septiembre. pp 11-14.

Camps-Valls, G., Tuia, D., Bruzzone, L., Benediktsson, J.A. 2013. Advances in hyperspectral image classification: Earth monitoring with statistical learning methods. IEEE Signal Processing Magazine, 31(1), 45-54. https://doi.org/10.1109/MSP.2013.2279179

Chuvieco, E. 2008. Teledetección Ambiental. La observación de la Tierra desde el espacio. Madrid: Ariel.

Cover, T., Hart, P. 1967. Nearest neighbor pattern classification. IEEE Transactions on Information Theory, 13(1), 21-27. https://doi.org/10.1109/TIT.1967.1053964

Estrada, J., Sánchez, H., Hernanz, L., Checa, M.J., Roman, D. 2017. Enabling the Use of Sentinel-2 and LiDAR Data for Common Agriculture Policy Funds Assignment. ISPRS International Journal of Geo-Information, 6(8), 255. https://doi.org/10.3390/ijgi6080255

Friedl, M.A., McIver, D.K., Hodges, J.C., Zhang, X.Y., Muchoney, D., Strahler, A.H., Baccini, A. 2002. Global land cover mapping from MODIS: algorithms and early results. Remote sensing of Environment, 83(1-2), 287-302. https://doi.org/10.1016/S0034-4257(02)00078-0

Graves, A., Mohamed, A.R., Hinton, G. 2013. Speech recognition with deep recurrent neural networks. En 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 6645-6649. https://doi.org/10.1109/ICASSP.2013.6638947

Gilabert, M.A., González-Piqueras, J., García-Haro, J., 1997. Acerca de los índices de vegetación. Revista de teledetección, 8, 1-10. Disponible en http://www.aet.org.es/?q=revista8-4

González-Guerrero, O., y Pons, X., 2020. The 2017 Land Use/Land Cover Map of Catalonia based on Sentinel-2 images and auxiliary data. Revista de Teledetección, 55, 81-92. https://doi.org/10.4995/raet.2020.13112

Gregrio, A., Jansen, J. 2000. Land cover classification system (LCCS); Classification concepts and user manual for software version 2. Roma: FAO.

Griffiths, P., Nendel, C., Hostert, P. 2019. Intra-annual reflectance composites from Sentinel-2 and Landsat for national-scale crop and land cover mapping. Remote Sensing of Environment, 220, 135-151. https://doi.org/10.1016/j.rse.2018.10.031

Gunning, D., Stefik, M., Choi, J., Miller, T., Stumpf, S., Yang, G.Z. 2019. XAI—Explainable artificial intelligence. Science Robotics, 4(37). https://doi.org/10.1126/scirobotics.aay7120

Haralick, R.M., Shanmugam, K., Dinstein, I.H. 1973. Textural features for image classification. IEEE Transactions on systems, man, and cybernetics, 3(6), 610-621. https://doi.org/10.1109/TSMC.1973.4309314

Haykin, S. 1994. Neural networks: a comprehensive foundation. River: Prentice Hall.

Immitzer, M., Vuolo, F., Atzberger, C. 2016. First experience with Sentinel-2 data for crop and tree species classifications in central Europe. Remote Sensing, 8(3), 166. https://doi.org/10.3390/rs8030166

Hochreiter, S., Schmidhuber, J. 1997. Long short-term memory. Neural computation, 9(8), 1735-1780. https://doi.org/10.1162/neco.1997.9.8.1735

Kussul, N., Lavreniuk, M., Skakun, S., Shelestov, A. 2017. Deep learning classification of land cover and crop types using remote sensing data. IEEE Geoscience and Remote Sensing Letters, 14(5), 778-782. https://doi.org/10.1109/LGRS.2017.2681128

Ma, L., Liu, Y., Zhang, X., Ye, Y., Yin, G., Johnson, B.A. 2019. Deep learning in remote sensing applications: A meta-analysis and review. ISPRS Journal of Photogrammetry and Remote Sensing, 152, 166-177. https://doi.org/10.1016/j.isprsjprs.2019.04.015

Maggiori, E., Tarabalka, Y., Charpiat, G., Alliez, P. 2016. Convolutional neural networks for large-scale remotesensing image classification. IEEE Transactions on Geoscience and Remote Sensing, 55(2), 645-657. https://doi.org/10.1109/TGRS.2016.2612821

Mikolov, T., Kombrink, S., Burget, L., Černocký, J., Khudanpur, S. 2011. Extensions of recurrent neural network language model. En 2011 IEEE International Conference on acoustics, speech and signal processing (ICASSP). Praga, República Checa, 22-27 Mayo. pp. 5528-5531. https://doi.org/10.1109/ICASSP.2011.5947611

Montavon, G., Samek, W., Müller, K.R. 2018. Methods for interpreting and understanding deep neural networks. Digital Signal Processing, 73, 1-15. https://doi.org/10.1016/j.dsp.2017.10.011

Mou, L., Ghamisi, P., Zhu, X.X. 2017. Deep recurrent neural networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 55(7), 3639-3655. https://doi.org/10.1109/TGRS.2016.2636241

Mountrakis, G., Im, J., Ogole, C. 2011. Support vector machines in remote sensing: A review. ISPRS Journal of Photogrammetry and Remote Sensing, 66(3), 247- 259. https://doi.org/10.1016/j.isprsjprs.2010.11.001

Ndikumana, E., Ho Tong Minh, D., Baghdadi, N., Courault, D., Hossard, L. 2018. Deep recurrent neural network for agricultural classification using multitemporal SAR Sentinel-1 for Camargue, France. Remote Sensing, 10(8), 1217. https://doi.org/10.3390/rs10081217

Reichstein, M., Camps-Valls, G., Stevens, B., Jung, M., Denzler, J., Carvalhais, N. 2019. Deep learning and process understanding for data-driven Earth system science. Nature, 566(7743), 195-204. https://doi.org/10.1038/s41586-019-0912-1

Ruiz, L.A., Almonacid-Caballer, J., Crespo-Peremarch, P., Recio, J.A., Pardo-Pascual, J.E., SánchezGarcía, E. 2020. Automated classification of crop types and condition in a mediterranean area using a fine-tuned convolutional neural network. En The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences. Niza, Francia, 31 Agosto - 2 Septiembre (en línea). pp. 1061-1068. https://doi.org/10.5194/isprs-archivesXLIII-B3-2020-1061-2020

Samek, W., Montavon, G., Vedaldi, A., Hansen, L. K., Muller, K.R. (Eds.). 2020. Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. Cham: Springer Nature. https://doi.org/10.1007/978-3-030-28954-6

Schmedtmann, J., Campagnolo, M.L. 2015. Reliable crop identification with satellite imagery in the context of common agriculture policy subsidy control. Remote Sensing, 7(7), 9325-9346. https://doi.org/10.3390/rs70709325

Schuster, M., Paliwal, K.K. 1997. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11), 2673-2681. https://doi.org/10.1109/78.650093

Sitokonstantinou, V., Papoutsis, I., Kontoes, C., Lafarga Arnal, A., Armesto Andrés, A.P., Garraza Zurbano, J.A. 2018. Scalable parcel-based crop identification scheme using sentinel-2 data time-series for the monitoring of the common agricultural policy. Remote Sensing, 10(6), 911. https://doi.org/10.3390/rs10060911

Story, M., Congalton, R.G. 1986. Accuracy assessment: a user’s perspective. Photogrammetric Engineering and Remote Sensing, 52(3), 397-399.

Tapsall, B., Milenov, P., Tasdemir, K. 2010. Analysis of RapidEye imagery for annual landcover mapping as an aid to European Union (EU) common agricultural polic. En ISPRS Technical Commission VII Symposium – 100 Years ISPRS. Viena, Austria, 5-7 Julio. pp. 568-573.

Van Den Oord, A., Kalchbrenner, N., Kavukcuoglu, K. 2016. Pixel recurrent neural networks. En Proceedings of the 33rd International Conference on International Conference on Machine Learning. New York, EEUU., 20-22 Junio. pp. 1747-1756.

Vuolo, F., Neuwirth, M., Immitzer, M., Atzberger, C., Ng, W.T. 2018. How much does multi-temporal Sentinel-2 data improve crop type classification? International Journal of Applied Earth Observation and Geoinformation, 72, 122-130. https://doi.org/10.1016/j.jag.2018.06.007

Wardlow, B.D., Egbert, S.L., Kastens, J.H. 2007. Analysis of time-series MODIS 250 m vegetation index data for crop classification in the US Central Great Plains. Remote Sensing of Environment, 108(3), 290-310. https://doi.org/10.1016/j.rse.2006.11.021

Watson, R.T., Noble, I.R., Bolin, B., Ravindranath, N.H., Verardo, D.J., Dokken, D.J. 2000. Land use, land-use change and forestry: a special report of the Intergovernmental Panel on Climate Change. Cambridge: Cambridge University Press.

Zhan, X., Defries, R., Townshend, J.R.G., Dimiceli, C., Hansen, M., Huang, C., Sohlberg, R. 2000. The 250 m global land cover change product from the Moderate Resolution Imaging Spectroradiometer of NASA’s Earth Observing System. International Journal of Remote Sensing, 21(6-7), 1433-1460. https://doi.org/10.1080/014311600210254

Zhang, L., Zhang, L., Du, B. 2016. Deep learning for remote sensing data: A technical tutorial on the state of the art. IEEE Geoscience and Remote Sensing Magazine, 4(2), 22-40. https://doi.org/10.1109/MGRS.2016.2540798

Zhong, L., Hu, L., Zhou, H. 2019. Deep learning based multi-temporal crop classification. Remote sensing of environment, 221, 430-443. https://doi.org/10.1016/j.rse.2018.11.032

Zhu, Z., Woodcock, C.E. 2014. Continuous change detection and classification of land cover using all available Landsat data. Remote Sensing of Environment, 144, 152-171. https://doi.org/10.1016/j.rse.2014.01.011

Zhu, X.X., Tuia, D., Mou, L., Xia, G.S., Zhang, L., Xu, F., Fraundorfer, F. 2017. Deep learning in remote sensing: A comprehensive review and list of resources. IEEE Geoscience and Remote Sensing Magazine, 5(4), 8-36. https://doi.org/10.1109/MGRS.2017.2762307

Abstract Views

1360
Metrics Loading ...

Metrics powered by PLOS ALM


 

Cited-By (articles included in Crossref)

This journal is a Crossref Cited-by Linking member. This list shows the references that citing the article automatically, if there are. For more information about the system please visit Crossref site

1. Land Use Classification of VHR Images for Mapping Small-Sized Abandoned Citrus Plots by Using Spectral and Textural Information
Sergio Morell-Monzó, María-Teresa Sebastiá-Frasquet, Javier Estornell
Remote Sensing  vol: 13  issue: 4  first page: 681  year: 2021  
doi: 10.3390/rs13040681



 

This journal is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International

Universitat Politècnica de València

Official Journal of the Spanish Association of Remote Sensing

e-ISSN: 1988-8740    ISSN: 1133-0953           https://doi.org/10.4995/raet