Thesis : Knowledge-Augmented Vision-Language Pretraining for Generalist Surgical Scene Understanding

23 septembre 2026
14h

PhD Thesis Defense: Kun Yuan

Date and Time: Wednesday, 23 September 2026, 2:00 PM

Location: Salle Hygie, Institut Hospitalo-Universitaire (IHU) Strasbourg, 1 place de l'Hôpital (Nouvel Hôpital Civil)

The defense will be held in English.

Jury :

Director: Prof. Nicolas Padoy, University of Strasbourg, France
Co-Director: Prof. Nassir Navab, Technical University of Munich, Germany
Reviewer: Prof. Gregory D. Hager, Johns Hopkins University, USA
Reviewer: Prof. Qi Dou, The Chinese University of Hong Kong, Hong Kong
Examiner: Prof. Cédric Wemmert, University of Strasbourg, France
Examiner: Dr. Vinkle Srivastav, University of Strasbourg, France

  • Dr. Sandrine Voros, Université Grenoble Alpes
  • Prof. Purang Abolmaesumi, The University of British Columbia
  • Prof. Sophia Bano, University College London

Abstract:

Artificial intelligence has the potential to transform computer-assisted surgery by enabling comprehensive surgical scene understanding, clinical decision support, and intelligent surgical assistance. However, existing surgical AI systems often rely on task-specific models and extensive manual annotations, limiting their scalability, generalizability, and adaptability to diverse clinical environments.

This thesis investigates knowledge-augmented vision-language pretraining as a foundation for generalist surgical scene understanding, with language serving as a unified interface for representation learning, interaction, and reasoning.

We first establish surgical vision-language pretraining frameworks that learn transferable representations from large-scale surgical videos and textual descriptions. We then investigate how linguistic and procedural knowledge can enhance representation learning by addressing noisy textual supervision and incorporating surgical context. Building upon these representations, we explore efficient foundation-model adaptation strategies to support diverse downstream tasks under limited annotations and domain shifts. Finally, we extend these approaches to multimodal understanding and interactive applications, demonstrating their potential across different surgical procedures and medical imaging domains.

More broadly, this thesis demonstrates how integrating visual information with linguistic and procedural knowledge can advance surgical AI beyond isolated, task-specific solutions toward scalable, adaptable, and generalist foundation models capable of supporting diverse clinical applications.

En collaboration étroite avec

Logo INRIA
Logo INSERM
Logo IHU
Logo Hôpitaux Universitaires de Strasbourg

Avec le soutien de nos partenaires

Logo France 2030
Logo Investissements d'avenir
Logo ANR
Logo Union européenne
Logo Région Grand Est
Logo Institut Carnot