Presentation and context. This PhD is part of the RELEO project, whose aim is to develop new representation learning strategies to build a Large Earth Observation Model, able to predict Essential Climate Variables from multi-modal remote sensing data. RELEO is part of ANITI-2, the follow-on of the Interdisciplinary Artificial Intelligence Institute in the frame of the French ANR “AI Clusters”. A critical question is how to represent Earth obsevation data while accounting for the uncertainties, than can be caused by the measurements themselves or that are predicted by physical models. The need for uncertainty quantification leads us towards deep generative models, that learn a probabilistic mapping between the data and a latent variable whose distribution is usually given beforehand. While a huge literature is devoted to generative representation learning, no existing techniques are fully adapted to remote sensing data because of several bottlenecks.

We aim here at tackling two of them: multi-modality and interpretability. Indeed, using such large representation models in scientific applications requires to combine different data sources (e.g., optical images, SAR, infrared). And to be able to quantify the uncertainties and interpret the results, those models should

be aware of the physics that relates the different modalities and the essential variables. We intend to achieve both goals with conditional generative models informed by physics.

Objectives of the PhD. The PhD will focus on conditional generative models, and will address the following questions.

1. What generative models offer the best trade-off between scalability and expressivity? Typically, state-of-the-art diffusion models are too complex, require too much processing in inference and too many images to be trained. Lighter architectures such as hierarchical VAEs [1] or normalizing flow VAEs [2]

are easier to train and seem complex enough to model the uncertainties. Another direction could be to use pre-trained foundation models such as [3], combined with simple generative models such as vanilla VAEs.

2. What kind of conditionning can we add in the model to account for multi-modality? We will first investigate the conditioning of one modality w.r.t. the others, and in a second step the conditioning w.r.t. physical parameters that describe the modality (e.g., wavelength).

3. How to assess the quality of the representation, in terms of fidelity, diversity, quality of the uncertainties for real applications? 

The proposed methodology will be set up on real remote sensing datasets of moderate size already used in the RELEO project, for instance Sentinel 1 and Sentinel 2 time series. It will be evaluated on real use-cases such as land cover classification or change detection. 

Integration within the RELEO project. The PhD will interact with the other work-packages in the RELEO project. In particular, WP1 will propose a multi-modal embedding on which the generative model will be trained. The developed conditional generative model will need to be compatible with physical decoders, according to the ongoing developments of WP3. And several use-cases and datasets will be proposed by WP6.

References

[1] C. K. Sønderby, T. Raiko, et al. Ladder variational autoencoders. Proc. NeurIPS, 2016.

[2] D. Rezende and S. Mohamed. Variational inference with normalizing flows. Proc. ICML, 2015.

[3] D. Hong, et al. SpectralGPT: Spectral remote sensing foundation model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.

[4] Y. Zérah, S. Valero, and J. Inglada. Physics-constrained deep learning for biophysical parameter retrieval from Sentinel-2 images: Inversion of the PROSAIL model. Remote Sensing of Environment, 312, 2024.

=================

For more Information about the topics and the co-financial partner (found by the lab !); 

contact Directeur de thèse - thomas.oberlin@isae-supaero.fr

Then, prepare a resume, a recent transcript and a reference letter from your M2 supervisor/ engineering school director and you will be ready to apply online  before March 14th, 2025 Midnight Paris time !