Thesis Proposals

Deep-learning systems can be accurate on average and still fail on individual medical images. The cause of failure matters: an examination may be degraded by motion or noise, the image may be intrinsically ambiguous even for expert readers, or the case may differ from the training distribution. Current uncertainty-quantification methods usually return a confidence or risk score that is useful for error detection, calibration, selective prediction, or out-of-distribution (OOD) detection. However, the same level of predictive uncertainty can arise from different failure sources, which may require different responses. Goal and objectives The thesis will investigate whether uncertainty can be made failure-source-aware and actionable. The objective is to test whether observable uncertainty signals contain enough information to distinguish clinically meaningful failure factors beyond generic predictive risk. - Build a benchmark that separates acquisition degradation, reader disagreement / intrinsic ambiguity, and distribution shift - Compare standard uncertainty signals: softmax confidence/entropy, calibration scores, deep ensembles, Monte-Carlo dropout, test-time augmentation, and representation/OOD scores. - Evaluate failure-source attribution at matched predictive risk: cases with similar error risk should still be distinguishable by the underlying factor. - Test whether factor-aware decisions improve simple intervention policies, such as reacquisition/quality control, expert review, or OOD referral, under the same intervention budget. Project environment, supervision and international mobility SafeMed is a collaborative research project between GIPSA-lab in Grenoble and the Bio-AI Lab at UiT – The Arctic University of Norway in Tromsø. The activities are based in GIPSA-lab, Grenoble, and are expected to start in February 2027.

Context: The increasing integration of advanced technologies in various industries necessitates robust validation methods for computer vision systems. A recent initiative aims to leverage generative AI to create a controllable simulator that can enhance the validation of face recognition systems. This approach addresses challenges such as environmental lighting variability and changes in user appearance, which are critical for ensuring the accuracy and reliability of biometric systems. Project description: This project will explore the use of generative AI as a controllable simulator for validating computer vision systems, with a particular focus on face recognition technologies. The research aims to evaluate the effectiveness of generative models by examining various aspects such as semantic fidelity, which includes identity and scene coherence; geometric fidelity, focusing on pose and multi-view consistency; and photometric fidelity, which analyzes lighting and lens effects. Additionally, the project will assess physical fidelity through the control of camera parameters, ground-truth reliability by measuring the accuracy of measurements, and dataset quality by emphasizing diversity and scalability. Objectives: • To evaluate the effectiveness of genAI in creating controllable synthetic data for computer vision validation. • To establish a comprehensive evaluation framework based on the proposed taxonomy. • To generate a dataset that meets requirements for effective validation of face recognition systems. • To explore potential applications of the findings in other industries, including automotive and beyond.

Despite advancements in visual fidelity, current generative text-to-image models struggle with precise control over generated images, especially in response to detailed prompts and directional cues. ​ While current generative text-to-image latent diffusion models have reached unprecedented results in terms of visual fidelity, there are still open issues to be addressed in exerting precise control over the generated images. On the one hand, generative models have difficulty in creating correct images when the textual prompt contains many details and often with object placement and spatial awareness. Recent text-to-image latent diffusion models have shown substantial improvements in prompt following, yet still struggle with the use of words such as “left” or “behind”. One possible reason lies in the inherent limitations of the text embedding employed to condition the generation process, that fails to learn sufficiently detailed and disentangled representation. We are developing a new set of architectures based on scene-graph-to-image conditioning, to validate if a structured input overcomes some of the limitations of unstructured text. The proposed architecture exploit graph neural networks instead of visual language models, such as CLIP, to guide the generative process. ​ Thesis topics available in this context follow three research lines. First, investigate the impact of data quality on scene-graph-to-image generation. Particularly relevant is the adaptation of text-to-image benchmark to the SG-to-image setting by automatically translating text captions to SG, solving a bottleneck of the current approach. The thesis activities will involve one or more of the following:​ i) develop or train LLMs to convert textual descriptions to scene graph representations​; ii) create and extend data pipelines, including data augmentation, for training generative models​, iii) validate experimentally the impact of data quality on the generative models through established benchmarks for visual reasoning. ​ ​The second topic investigates the impact of different architectural choices and their impact on the quality of the generated images. The thesis activities will involve one or more of the following:​ i) develop variants of the proposed architecture to, e.g., understand the properties of different encoding, GNN architectures, in collaboration with other students​; ii) perform ablation studies​ and iii) adapt the pipeline to other benchmarks, e.g., for visual and spatial reasoning​. The third topic will compare layout-free with layout-based methods for SG-to-image generation. Layout-based methods condition the image generation on a spatial layout formed by bounding boxes of objects. It forces stronger constraints on the spatial arrangements of objects, but disregard relationships and attributes. ​ The thesis activities will involve one or more of the following:​ i) automatic layout generation from scene graph or textual description; ​ii) Implementing and testing existing layout-to-image architectures on our datasets;​ and iii) extending our current SG-to-image pipeline integrating bounding boxes in the scene graph representation.

Objective: Design, extend, and evaluate prototypical Logic Tensor Networks (LTNs) across diverse tasks and embedding geometries, with a particular focus on hyperbolic and other non-Euclidean representation spaces.​ Context: Logic Tensor Networks offer a principled neuro-symbolic framework for integrating first-order logic constraints with learned representations. Recent work has introduced prototypical variants of LTNs, where concepts are grounded through prototype-based representations — drawing inspiration from prototypical networks in few-shot learning. However, existing investigations remain limited in scope: they typically target a narrow set of tasks and rely exclusively on Euclidean embedding spaces. This leaves open several important questions: How do prototypical LTNs behave across tasks with different structural properties (e.g., hierarchical classification, relational reasoning, few-shot learning)? Can hyperbolic embeddings — known to better capture hierarchical and tree-like structure — improve concept grounding and logical consistency in LTNs? How do different embedding geometries interact with the satisfaction of logical axioms and the interpretability of learned prototypes?​ Thesis Activities: The goal of this thesis is to systematically extend and evaluate prototypical LTNs along two main axes. Depending on the candidate background and predisposition, the problem can be tackled either from a theoretical or experimental standpoint. Starting from an analysis of the current literature, techniques to be investigated include: ​ - adapting and testing the framework on a range of tasks beyond those considered in prior work, including hierarchical multi-label classification, relational learning, and low-data regimes​; the extension of prototypical learning to other neuro-symbolic frameworks, such as probabilistic circuits or NeuPSL, will also be considered. - replacing standard Euclidean embedding space will be replaced with alternative geometries — most notably hyperbolic space — and the effects on prototype quality, logical satisfiability, and downstream performance will be studied both empirically and analytically. ​ The candidate will benchmark these variants against standard LTN baselines and relevant deep learning models, with careful attention to reproducibility, robustness, hyperparameter sensitivity, and the theoretical and/or experimental properties induced by each geometric choice.​

Current social media generate a tremendous amount of visual material, that can be exploited by researchers operating in social media research, digital humanities, and marketing. However, privacy regulations impose significant restrictions to both data collection and sharing. The CAMOUFLAGE project, funded by the AI4Media European project, aims to exploit diffusion model (such as Stable Diffusion) to produce a synthetic version of an existing image, presenting equivalent visual and semantic characteristics of the original, while at the same time fully preserving the anonymity of the user who published the image. The project has a strong multidisciplinary focus and will be carried out in collaboration with researchers in visual semiotics to determine the quality and information content of the generated image. The system will be tested on a collection of real Facebook and Instagram profile pictures. Multiple thesis are available tackling three distinct, yet related, research objectives: to design and implement controllable image synthesis that retains the visual and semantic content of a target image; to determine whether the resulting synthetic images can be considered successfully anonymized, in comparison with state-of-the-art solutions based on GANs; and whether the synthetic collection has the same information content of the original image.

Deep neural networks have shown remarkable performance in the interpretation of medical images. However, they require large datasets for training, and performance may degrade when training on imbalanced datasets. Radiologists are typically trained through examples, as well as through structured taxonomies that present prototypical examples of visual features associated with benign and malignant lesions. The goal of the thesis is to design neuro-symbolic architectures, such as Logic Tensor Networks, that can incorporate such constraints during training. Experiments are foreseen on mammography and possibly other medical imaging modalities. ​ Thesis activities include design and comparison of different neuro-symbolic techniques, starting from patch-level classification and moving on towards image-level analysis. Issues to be tackled include how to align visual features computed by the network to human-interpretable concepts and how to encode the constraints in the loss. Evaluation will be performed in terms of performance, robustness, explainability and alignment with human asssessment. ​  Prerequisites:   programming skills (Python, Pytorch or other deep learning framework); good analytical and mathematical skills. Prior knowledge of neuro-symbolic techniques is not required – essential material to study on the topic will be provided. The candidate we are looking for is highly motivated and interested towards research-oriented activities. ​ ​

Digital human models are essential for applications like training and monitoring systems, where animation realism is crucial. NVIDIA Omniverse offers a powerful platform for testing these animations. This project aims to evaluate and enhance the realism of digital human animations for human monitoring micro-simulation applications. This thesis will focus on testing digital human animations in NVIDIA Omniverse for various environments. The candidate will integrate human operators using motion capture techniques with wearable suits and log synthetic sensor data, including biometric data. The project will also involve synthetic data annotation and develop a testing framework layer that integrates with an in-house Test Automation Framework (TAF). Additionally, the project will benchmark performance against Unreal Engine and Unity while incorporating animation generative models. Objectives: 1. Animation Realism Evaluation: Assess and enhance digital human animations for human monitoring scenarios. 2. Integration of Human Operators: Incorporate human operators into various simulated environments. 3. Synthetic Data Logging: Log synthetic sensor data, including biometric data, during simulations. 4. Synthetic Data Annotation: Develop methodologies for annotating synthetic data for machine learning model training. 5. Motion Capture Integration: Use wearable motion capture suits to improve animation fidelity. 6. Animation Generative Models: Integrate Generative AI to enhance animation quality and variability. 7. Benchmarking Across Platforms: Compare digital human animations in Omniverse with those in Unreal Engine and Unity. 8. Testing Framework Development: Develop a testing framework layer that integrates with the existing TAF for automated scenario testing.

Context: To enhance the effectiveness of driver monitoring systems and in-cabin monitoring technologies, a robust framework is being developed to generate unique virtual scenarios. This framework utilizes realistic digital human models and is currently being implemented in collaboration with a car manufacturer to create a comprehensive database of scenarios for testing dual-camera in-cabin monitoring systems. Description: This thesis will focus on enhancing the scenario generation processes within the framework. The candidate will work on improving libraries recall, plugin integration, and rendering techniques. Additionally, the project will involve collecting an animation database and post-processing for animation library generation. New plugins will be developed, to handle easily hand positioning, face and eye gaze animations and the lighting source fuzzing. The project will also enhance scenario fuzzing techniques and develop for the client a ready-to-use scenario database with related annotations. Objectives: 1. Enhance Scenario Generation Processes: Improve the efficiency and effectiveness of actual scenario generation process through libraries generation and plugin integration. 2. Animation Database Collection: Collect and post-process animations to create a comprehensive animation library for use in scenario generation (through the usage of wearable suite and other motion capture techniques) 3. Driver Animation Database: Improve an existing dataset for driver animations to enhance DMS capabilities, focusing on realistic responses and behaviors. 4. GenAI fine-tuning: Fine tune a GenAI model to generate driver animations from text inputs. 5. Plugin Development: Create new plugins, including: • Hand Positioning Animation Manager • Face and Eye Gaze Animation Manager • Lighting Source Fuzzing Manager • Animation Database Interface 6. Scenario Fuzzing Techniques: Enhance existing scenario fuzzing techniques to improve and guarantee variability within the operational design domain 7. Scenario Database Development: Develop a ready-to-use scenario database along with comprehensive annotations for effective testing and validation.

Objective: Design and develop AI systems capable of automatically generating coherent natural language or visual summaries from clusters of semantically related images from social media​ Context: The automatic summarization of image collections represents a challenging and emerging frontier in multimodal AI. Unlike single-image captioning, cluster-level summarization requires a system to identify recurring themes, abstract away from redundant visual details, and produce a concise, human-readable description that captures the collective content of a group of images. The goal of this project is to design generative models that can generate meaningful summaries (textual and/or visual) at the cluster level. Key challenges are enabling the model to perform cross-image reasoning — identifying what is shared, what varies, and what is most salient across the set — and incorporating existing semiotic models developed by our group.​ Activities: The thesis will include one or more of the following activities:​ - starting from an analysis of the SOTA [2], design an effective pooling or aggregation strategy that compresses visual information across a cluster into a unified representation suitable for language generation, tailoring the tool to social media listening, market research and cultural studies;​ - leverage Large Language Models (e.g., GPT-4, LLaMA) to produce fluent and abstractive summaries grounded in visual content;​ - incorporate structured knowledge, also coming from computational semiotics tools such as FRESCO [2], to improve the coherence and factual accuracy of generated summaries;​ - Investigate the application of cluster-level summarization to produce AI “personas” based on clusters of similar social media images.​ Reading material:​ [1] https://arxiv.org/abs/2503.19361​ [2] https://arxiv.org/abs/2407.03268​

Context: Social media have a profound impact on the way individuals choose to (re)present themselves in the digital era. We have developed in collaboration with the Department of Philosophy at Università di Torino, visual big data analytics tools through computational image analysis and deep learning techniques that borrow from other disciplines, such as socio-semiotics and visual semiotics.  Our goal is to extend these techniques to develop powerful, easy-to-use tools to power the next generation of cultural analytics. Open research questions to be tackled in a research thesis are varied and will be selected based on candidates skills and research interests. Examples of research questions include: - Extending the available computational pipeline (FRESCO) with new tools and techniques to extract semantic characteristics from images  (composition, content, etc.)  - Designing user-friendly data analytics and visual analytics pipelines to apply the FRESCO pipeline for strategic and cultural research (e.g., marketing, trend analysis, network analysis) - Extending FRESCO to work on multi-modal and short video collections (e.g, TikTok) - Adapting the proposed pipeline to the analysis of different types of images (advertisement, meme, artworks, AI-generated images, …). Particularly relevant is the use of the proposed pipeline to detect and quantify biases in AI-generated images   - Design, train and evaluate deep neural networks that mimic high-level semantic analysis (which emotions are solicited by a given image? Which values are expressed? what are the narrative roles in an image?)  - Investigating the applicability of Multi-modal Large Language Model (LLM) to extract high-level interpretation from images and reproduce the type of in-depth analysis performed by experts, such as semioticians Prerequisites: good knowledge of big data analytics, computer vision and deep learning frameworks, and a willingness to engage in multi-disciplinary research For more information on FRESCO see: https://arxiv.org/abs/2407.03268