Lia Morra

Context: Social media have a profound impact on the way individuals choose to (re)present themselves in the digital era. We have developed in collaboration with the Department of Philosophy at Università di Torino, visual big data analytics tools through computational image analysis and deep learning techniques that borrow from other disciplines, such as socio-semiotics and visual semiotics.  Our goal is to extend these techniques to develop powerful, easy-to-use tools to power the next generation of cultural analytics.
Open research questions to be tackled in a research thesis are varied and will be selected based on candidates skills and research interests. Examples of research questions include:
– Extending the available computational pipeline (FRESCO) with new tools and techniques to extract semantic characteristics from images  (composition, content, etc.)
– Designing user-friendly data analytics and visual analytics pipelines to apply the FRESCO pipeline for strategic and cultural research (e.g., marketing, trend analysis, network analysis)
– Extending FRESCO to work on multi-modal and short video collections (e.g, TikTok)
– Adapting the proposed pipeline to the analysis of different types of images (advertisement, meme, artworks, AI-generated images, …). Particularly relevant is the use of the proposed pipeline to detect and quantify biases in AI-generated images
– Design, train and evaluate deep neural networks that mimic high-level semantic analysis (which emotions are solicited by a given image? Which values are expressed? what are the narrative roles in an image?)
– Investigating the applicability of Multi-modal Large Language Model (LLM) to extract high-level interpretation from images and reproduce the type of in-depth analysis performed by experts, such as semioticians
Prerequisites: good knowledge of big data analytics, computer vision and deep learning frameworks, and a willingness to engage in multi-disciplinary research
For more information on FRESCO see: https://arxiv.org/abs/2407.03268

Objective: Design and develop AI systems capable of automatically generating coherent natural language or visual summaries from clusters of semantically related images from social media​
Context: The automatic summarization of image collections represents a challenging and emerging frontier in multimodal AI. Unlike single-image captioning, cluster-level summarization requires a system to identify recurring themes, abstract away from redundant visual details, and produce a concise, human-readable description that captures the collective content of a group of images. The goal of this project is to design generative models that can generate meaningful summaries (textual and/or visual) at the cluster level. Key challenges are enabling the model to perform cross-image reasoning — identifying what is shared, what varies, and what is most salient across the set — and incorporating existing semiotic models developed by our group.​
Activities: The thesis will include one or more of the following activities:​
– starting from an analysis of the SOTA [2], design an effective pooling or aggregation strategy that compresses visual information across a cluster into a unified representation suitable for language generation, tailoring the tool to social media listening, market research and cultural studies;​
– leverage Large Language Models (e.g., GPT-4, LLaMA) to produce fluent and abstractive summaries grounded in visual content;​
– incorporate structured knowledge, also coming from computational semiotics tools such as FRESCO [2], to improve the coherence and factual accuracy of generated summaries;​
– Investigate the application of cluster-level summarization to produce AI “personas” based on clusters of similar social media images.​
Reading material:​
[1] https://arxiv.org/abs/2503.19361​
[2] https://arxiv.org/abs/2407.03268​

Context:
To enhance the effectiveness of driver monitoring systems and in-cabin monitoring technologies, a robust framework is being developed to generate unique virtual scenarios. This framework utilizes realistic digital human models and is currently being implemented in collaboration with a car manufacturer to create a comprehensive database of scenarios for testing dual-camera in-cabin monitoring systems.
Description:
This thesis will focus on enhancing the scenario generation processes within the framework. The candidate will work on improving libraries recall, plugin integration, and rendering techniques. Additionally, the project will involve collecting an animation database and post-processing for animation library generation. New plugins will be developed, to handle easily hand positioning, face and eye gaze animations and the lighting source fuzzing. The project will also enhance scenario fuzzing techniques and develop for the client a ready-to-use
scenario database with related annotations.
Objectives:
1. Enhance Scenario Generation Processes: Improve the efficiency and effectiveness of actual scenario generation process through libraries generation and plugin integration.
2. Animation Database Collection: Collect and post-process animations to create a comprehensive animation library for use in scenario generation (through the usage of wearable suite and other motion capture techniques)
3. Driver Animation Database: Improve an existing dataset for driver animations to enhance DMS capabilities, focusing on realistic responses and behaviors.
4. GenAI fine-tuning: Fine tune a GenAI model to generate driver animations from text inputs.
5. Plugin Development: Create new plugins, including:
• Hand Positioning Animation Manager
• Face and Eye Gaze Animation Manager
• Lighting Source Fuzzing Manager
• Animation Database Interface
6. Scenario Fuzzing Techniques: Enhance existing scenario fuzzing techniques to improve and guarantee variability within the operational design domain
7. Scenario Database Development: Develop a ready-to-use scenario database along with comprehensive annotations for effective testing and validation.

Digital human models are essential for applications like training and monitoring systems, where animation realism is crucial. NVIDIA Omniverse offers a powerful platform for testing these animations. This project aims to evaluate and enhance the realism of digital human animations for human monitoring micro-simulation applications.
This thesis will focus on testing digital human animations in NVIDIA Omniverse for various environments. The candidate will integrate human operators using motion capture techniques with wearable suits and log synthetic sensor data, including biometric data. The project will also involve synthetic data annotation and develop a testing framework layer that integrates with an in-house Test Automation Framework (TAF). Additionally, the project will benchmark performance against Unreal Engine and Unity while incorporating animation
generative models.
Objectives:
1. Animation Realism Evaluation: Assess and enhance digital human animations for human monitoring scenarios.
2. Integration of Human Operators: Incorporate human operators into various simulated environments.
3. Synthetic Data Logging: Log synthetic sensor data, including biometric data, during simulations.
4. Synthetic Data Annotation: Develop methodologies for annotating synthetic data for machine learning model training.
5. Motion Capture Integration: Use wearable motion capture suits to improve animation fidelity.
6. Animation Generative Models: Integrate Generative AI to enhance animation quality and variability.
7. Benchmarking Across Platforms: Compare digital human animations in Omniverse with those in Unreal Engine and Unity.
8. Testing Framework Development: Develop a testing framework layer that integrates with the existing TAF for automated scenario testing.

Image interpretation tasks, such as object detection and visual relationship detection, may benefit from neural-symbolic integration in order to inject prior knowledge, e.g., from knowledge graphs,  about objects and their relationship into the training objectives of deep neural networks. The goal of this project is to design deep architectures combining convolutional neural networks and trasformers with neural-symbolic components such as Logic Tensor Networks to solve various image interpretation tasks, such as object detection, image classification or visual relationship detection. Multiple thesis are available to tackle the following open issues: i) how to integrate the knowledge into the training process of high-level semantic interpretation (scene graph generation);​ ii) how to automatically extract and compile prior knowledge from existing sources and format it for appropriate ingestion; ​ iii) how to use Large Language Models (ChatGPT) to extract prior knowledge and logical constraints;​ iv) how to characterize neural-symbolic architectures, e.g. in terms of types of features learnt, efficiency and interpretability;​ v) how to combine concept extraction with knowledge induction to close the neuro-symbolic cycle, and how to simplify knowledge acquisition through human-in-the-loop appraoches​

Deep neural networks have shown remarkable performance in the interpretation of medical images. However, they require large datasets for training, and performance may degrade when training on imbalanced datasets. Radiologists are typically trained through examples, as well as through structured taxonomies that present prototypical examples of visual features associated with benign and malignant lesions. The goal of the thesis is to design neuro-symbolic architectures, such as Logic Tensor Networks, that can incorporate such constraints during training. Experiments are foreseen on mammography and possibly other medical imaging modalities. ​
Thesis activities include design and comparison of different neuro-symbolic techniques, starting from patch-level classification and moving on towards image-level analysis. Issues to be tackled include how to align visual features computed by the network to human-interpretable concepts and how to encode the constraints in the loss. Evaluation will be performed in terms of performance, robustness, explainability and alignment with human asssessment. ​
Prerequisites:   programming skills (Python, Pytorch or other deep learning framework); good analytical and mathematical skills. Prior knowledge of neuro-symbolic techniques is not required – essential material to study on the topic will be provided. The candidate we are looking for is highly motivated and interested towards research-oriented activities. ​
​

La mancanza di dati di benchmark per la segmentazione semantica degli scenari del patrimonio digitale ostacola lo sviluppo di soluzioni di classificazione automatica in questo campo. I dati 3D del patrimonio architettonico presentano strutture complesse e classi non comuni che impediscono la semplice distribuzione di metodi disponibili sviluppati in altri campi e per altri tipi di dati. La classificazione semantica dei dati 3D del patrimonio aiuterebbe la comunità a comprendere e analizzare meglio i gemelli digitali e faciliterebbe i lavori di restauro e conservazione. Il dataset ArCH nasce dalla collaborazione di diverse università e istituti di ricerca e comprende nuvole di punti annotate e non annotate di siti del patrimonio culturale. Tuttavia, la scarsità di dati, la varietà e l’unicità delle architetture e la categorizzazione a grana fine necessaria per la segmentazione rendono questo compito particolarmente difficile per i modelli di apprendimento profondo. L’obiettivo di questo progetto è arricchire reti neurali profonde all’avanguardia per la segmentazione di nuvole di punti utilizzando tecniche neurosimboliche, come le reti Logic Tensork, per iniettare conoscenze pregresse nella rete.

Neuro-symbolic AI encompasses a broad class of architectures aiming at combining machine learning with knowledge representation and reasoning (“Good old-fashioned AI”). In particular, Logic Tensor Networks are a framework to encode a first order logic language into a trainable neural network. NeSy techniques like LTNs introduce many additional design choices, such as: How to define axiomatic prior knowledge (“how to encode the rules of Sudoku”?); How to ground predicates and connectives? How to choose between different frameworks? Comparison about existing frameworks is limited in literature, and issues in reporting, hyper-parameter selections, etc. affect this comparison.
Several thesis are available on tackling these issues. Depending on the candidate background and predisposition, the problem can be tackled either from a more theoretical or experimental standpoint. Starting from an analysis of the current literature, techniques to be investigated include:
– Analysis of the mathematical and numerical properties of different frameworks
– Comparison of existing frameworks and losses
– Application of techniques from the fields of hyper-parameter optimization (Random search, Bayesian Optimization), Neural Architectural Search (NAS), genetic algorithms and curriculum learning
Experiments will be conducted on standard benchmarks as well as applications on semantic image interpretation explored by the group.
Prerequisites:  programming skills (Python, Pytorch or other deep learning framework); good analytical skills. Prior knowledge of neuro-symbolic techniques is not required – essential material to study on the topic will be provided. The candidate we are looking for is highly motivated and interested towards research-oriented activities.

Current social media generate a tremendous amount of visual material, that can be exploited by researchers operating in social media research, digital humanities, and marketing. However, privacy regulations impose significant restrictions to both data collection and sharing. The CAMOUFLAGE project, funded by the AI4Media European project, aims to exploit diffusion model (such as Stable Diffusion) to produce a synthetic version of an existing image, presenting equivalent visual and semantic characteristics of the original, while at the same time fully preserving the anonymity of the user who published the image. The project has a strong multidisciplinary focus and will be carried out in collaboration with researchers in visual semiotics to determine the quality and information content of the generated image. The system will be tested on a collection of real Facebook and Instagram profile pictures. Multiple thesis are available tackling three distinct, yet related, research objectives: to design and implement controllable image synthesis that retains the visual and semantic content of a target image; to determine whether the resulting synthetic images can be considered successfully anonymized, in comparison with state-of-the-art solutions based on GANs; and whether the synthetic collection has the same information content of the original image.

Objective: Design, extend, and evaluate prototypical Logic Tensor Networks (LTNs) across diverse tasks and embedding geometries, with a particular focus on hyperbolic and other non-Euclidean representation spaces.​
Context: Logic Tensor Networks offer a principled neuro-symbolic framework for integrating first-order logic constraints with learned representations. Recent work has introduced prototypical variants of LTNs, where concepts are grounded through prototype-based representations — drawing inspiration from prototypical networks in few-shot learning. However, existing investigations remain limited in scope: they typically target a narrow set of tasks and rely exclusively on Euclidean embedding spaces. This leaves open several important questions: How do prototypical LTNs behave across tasks with different structural properties (e.g., hierarchical classification, relational reasoning, few-shot learning)? Can hyperbolic embeddings — known to better capture hierarchical and tree-like structure — improve concept grounding and logical consistency in LTNs? How do different embedding geometries interact with the satisfaction of logical axioms and the interpretability of learned prototypes?​
Thesis Activities: The goal of this thesis is to systematically extend and evaluate prototypical LTNs along two main axes. Depending on the candidate background and predisposition, the problem can be tackled either from a theoretical or experimental standpoint. Starting from an analysis of the current literature, techniques to be investigated include: ​
– adapting and testing the framework on a range of tasks beyond those considered in prior work, including hierarchical multi-label classification, relational learning, and low-data regimes​; the extension of prototypical learning to other neuro-symbolic frameworks, such as probabilistic circuits or NeuPSL, will also be considered.
– replacing standard Euclidean embedding space will be replaced with alternative geometries — most notably hyperbolic space — and the effects on prototype quality, logical satisfiability, and downstream performance will be studied both empirically and analytically. ​
The candidate will benchmark these variants against standard LTN baselines and relevant deep learning models, with careful attention to reproducibility, robustness, hyperparameter sensitivity, and the theoretical and/or experimental properties induced by each geometric choice.​

LLMs are powerful — until accuracy becomes non-negotiable. In regulated domains like insurance, compliance, or legal advisory, a single hallucinated value carries real financial and legal consequences. Yet current approaches to grounding LLM behavior
— prompt engineering, fine-tuning, RLHF — offer probabilistic guarantees at best. The gap between “usually correct” and “provably correct” remains wide open.
This thesis aims to close that gap by investigating neuro-symbolic architectures that enforce hard, verifiable constraints on LLM outputs — combining the generative fluency of modern GenAI with the logical rigor of symbolic reasoning. The candidate
will build and benchmark constrained pipelines capable of producing structured, traceable, auditable responses, and stress-test them against adversarial inputs designed to break compliance. There is the possibility to collaborate with the creator of
one of the core neuro-symbolic frameworks.

Despite advancements in visual fidelity, current generative text-to-image models struggle with precise control over generated images, especially in response to detailed prompts and directional cues. ​ While current generative text-to-image latent diffusion models have reached unprecedented results in terms of visual fidelity, there are still open issues to be addressed in exerting precise control over the generated images. On the one hand, generative models have difficulty in creating correct images when the textual prompt contains many details and often with object placement and spatial awareness. Recent text-to-image latent diffusion models have shown substantial improvements in prompt following, yet still struggle with the use of words such as “left” or “behind”. One possible reason lies in the inherent limitations of the text embedding employed to condition the generation process, that fails to learn sufficiently detailed and disentangled representation.
We are developing a new set of architectures based on scene-graph-to-image conditioning, to validate if a structured input overcomes some of the limitations of unstructured text. The proposed architecture exploit graph neural networks instead of visual language models, such as CLIP, to guide the generative process. ​
Thesis topics available in this context follow three research lines. First, investigate the impact of data quality on scene-graph-to-image generation. Particularly relevant is the adaptation of text-to-image benchmark to the SG-to-image setting by automatically translating text captions to SG, solving a bottleneck of the current approach. The thesis activities will involve one or more of the following:​ i) develop or train LLMs to convert textual descriptions to scene graph representations​; ii) create and extend data pipelines, including data augmentation, for training generative models​, iii) validate experimentally the impact of data quality on the generative models through established benchmarks for visual reasoning. ​
​The second topic investigates the impact of different architectural choices and their impact on the quality of the generated images. The thesis activities will involve one or more of the following:​ i) develop variants of the proposed architecture to, e.g., understand the properties of different encoding, GNN architectures, in collaboration with other students​; ii) perform ablation studies​ and iii) adapt the pipeline to other benchmarks, e.g., for visual and spatial reasoning​.
The third topic will compare layout-free with layout-based methods for SG-to-image generation. Layout-based methods condition the image generation on a spatial layout formed by bounding boxes of objects. It forces stronger constraints on the spatial arrangements of objects, but disregard relationships and attributes. ​ The thesis activities will involve one or more of the following:​ i) automatic layout generation from scene graph or textual description; ​ii) Implementing and testing existing layout-to-image architectures on our datasets;​ and iii) extending our current SG-to-image pipeline integrating bounding boxes in the scene graph representation.