131-Learning and analyzing spatio-temporal objects from satellite time series

  • Ph.D.
  • Full-time
  • Entry Level
  • Master’s Degree, MA/MS/MSc
  • Data Sciences

Mission

On March 7, 2017, the European Space Agency (ESA) put its latest satellite Sentinel-2B into orbit. The two Sentinel-2 satellites are capturing images of all emerged surfaces every 2 to 5 days at a high temporal resolution, which makes it possible to monitor the evolution of land surfaces on a global scale. Satellite image time series (SITS) extracted from Sentinel-2 or high-resolution satellites such as Pléiades are useful for many applications such as land cover mapping, crop type mapping, or disaster risk management.
Due to their volume and complexity, the analysis of these spatio-spectro-temporal datacubes requires automatic tools. Recent advances have been marked by the use of deep learning to make the most of the temporal structure of SITS: 1D convolutions [4], recurrent networks [5], or attention-based architectures [7]. There were also several attempts to jointly exploit the spatial and temporal dimensions of SITS by the means of deep learning [6,7]. Although these approaches have proven their efficiency, they suffer from 2 main issues: (i) they require a vast amount of high-quality labelled data, and (ii) they ignore previous trends in remote sensing, especially contributions from object-based image analysis (OBIA) [1]. Ensuring a convergence between these two distinct paradigms would allow embedding more structural and semantic information in the process [4].
Since only a few attempts exist to jointly use temporal relationships between satellite images and their intrinsic spatial structure in deep learning, the Ph.D. aims at developing novel deep learning architectures for the generation of spatio-temporal objects with no or limited supervision. It will be composed of 2 main objectives: (i) developing new techniques to structure raw SITS data into spatio-temporal objects, and (ii) analysing spatio-temporal objects.
First, we will consider the task of extracting objects from SITS as either a temporal sequence of 2D objects or directly 3D objects with no or a few supervision. For this task, we will propose new unsupervised deep learning strategies that take inspiration from self-supervised strategies [2] and go much further than the segmentation networks introduced recently [8]. We will also consider the case where a weak reference is available as we know that prior knowledge can be used to guide the extraction of objects. Compared to computer vision algorithms used for instance segmentation such as Mask-RCNN, the novelty will be to consider both spatial and temporal structures of SITS to deal with the lack of quality labelled reference data at the object level.
Second, we will develop new methods to analyze the produced spatio-temporal objects. We will still consider the deep learning framework as a methodology to perform object-based time series analysis. To do so, we will represent objects as nodes in a spatio-temporal graph, such as Graph CNNs and their formulation in the spatio-temporal domain [9]. This representation will be then used for classical applications such as land cover mapping or clustering.
[1] Blaschke, T., Lang, S., & Hay, G. (Eds.). (2008). Object-based image analysis: spatial concepts for knowledge-driven remote sensing applications. Springer Science& Business Media.
[2] Jing, L., & Tian, Y. (2020). Self-supervised visual feature learning with deep neural networks: A survey. IEEE Transac. on Pat. Anal. and Mach. Intel.
[3] Mboga, N., Georganos, S., Grippa, T., Lennert, M., Vanhuysse, S., & Wolff, E. (2019). Fully convolutional networks and geographic object-based image analysis for the classification of VHR imagery. Remote Sensing, 11(5), 597.
[4] Pelletier, C., Webb, G. I., & Petitjean, F. (2019). Temporal convolutional neural network for the classification of satellite image time series. Remote Sensing, 11(5), 523.
[5] Ruβwurm, M., & Korner, M. (2017). Temporal vegetation modelling using long short-term memory networks for crop identification from medium-resolution multi-spectral satellite images. In Proc. of the IEEE Conf. on CV and PR Workshop (p. 11-19).
[6] Ruβwurm, M., & Korner, M. (2018). Multi-temporal land cover classification with sequential recurrent encoders. ISPRS Int. Jour. of Geo-Information, 7(4), 129.
[7] Saint Fare Garnot, V., Landrieu, L., Giordano, S., & Chehata, N. (2020). Satellite image time series classification with pixel-set encoders and temporal self-attention. In Proc. of the IEEE/CVF Conf. on CV and PR (p. 12325-12334).
[8] Xia, X., & Kulis, B. (2017). W-Net: A deep model for fully unsupervised image segmentation. preprint arXiv:1711.08506.
[9] Yu, B., Yin, H., & Zhu, Z. (2017). Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. In Pro. of the 27 Int. Joint Conf. on AI (IJCAI-18) (pp. 3634-3640)

For more Information, contact Directeur de thèse : sebastien.lefevre@univ-ubs.fr


about the topics and the co-financial partner (found by the lab !). Then, prepare a resumé, a recent transcript and a reference letter from your M2 supervisor/ engineering school director and you will be ready to apply online !

Profile

We are looking for a candidate who hold a Master in computer science, data science, image and signal processing or any other relevant fields. The candidate should have strong data analysis, machine learning, and image processing skills and be familiar with deep learning techniques. TS/he should also have excellent programming skills in at least one language (C/C++, Python, etc.). Knowledge of time series analysis and remote sensing techniques will be appreciated. Good communication skills (at least in English) are required.

Message from PhD team

CNES will inform about the status of your application in mid-June. More details on CNES website : https://cnes.fr/en/web/CNES-en/10685-st-doctoral-grants.php