Thesis Proposals

This thesis aims to evaluate and compare two different approaches for the synthetic dataset generation in the car domain: CARLA2real (based on Unreal Engine) and NVIDIA Cosmo (a physical AI based-pipeline to support the photorealistic transfer).

This thesis explores the design and development of a prototypal framework for integrating multimodal perception, reasoning models, and agent orchestration in XR applications. These capabilities should be exploited to build spatially aware, multimodal AI agents that deliver low-latency, context-aware assistance in XR experiences. The system should make use of the most recent available AI models and frameworks, such as NVIDIA XR AI, exploring three core concepts: • Ingest real-world signals from XR devices, including video, audio, depth, pose and sensor data. • Connect agents to specialized tools and services for video search and summarization (VSS), for visual AI and video understanding, and for knowledge retrieval and retrieval-augmented generation. • Support multiple AI models.

The student will implement a real-time computer vision pipeline for tracking medical manikins and surgical tools, He will develop methods for 3D object detection and pose estimation and integrate the real objects as virtual overlays into a Unity-based XR environment. The thesis will focus on algorithm development and real-time graphics rendering, with the main goal of increasing realism and interactivity in medical and surgical training simulations.

La tesi ha l'obiettivo di valutare e confrontare sistemi di motion capture basati su piattaforme inerziali. Il confronto coinvolgerà sistemi professionali e sistemi consumer.

The project aims to develop an innovative Virtual Reality (VR) system that leverages artificial intelligence (AI) to enhance motor rehabilitation following stroke. Individuals with stroke injury frequently experience upper limb motor deficits, which can significantly impact daily activities and independence. With more than £3 billion of NHS expenditure annually devoted to neurorehabilitation programmes, improving the efficiency and engagement of rehabilitation strategies is a crucial area of research. Traditional physiotherapy programmes often struggle to maintain patient motivation and consistency. VR-based rehabilitation offers an immersive and interactive environment that increases engagement and provides precise tracking of motor performance. AI technologies can be used to monitor user progress in real time, automatically adjusting the level of task difficulty and providing personalised feedback to optimise recovery outcomes. We are seeking two students for this project: Student 1: Development of the Virtual Reality rehabilitation environment. Student 2: Integration of AI algorithms for adaptive feedback and performance monitoring. Participants will engage in a VR-enhanced version of the Graded Repetitive Arm Supplementary Program (GRASP), performing functional tasks such as reaching, grasping, and manipulating virtual objects (e.g., picking up fruit or opening a door). The system will accurately track hand movements through VR controllers and provide realistic feedback to simulate object interaction. The goal is to create a motivating and data-driven rehabilitation tool that improves adherence and maximises motor recovery outcomes.

Object detection methods strongly rely on supervised deep learning approaches that require large amounts of labeled data. Data augmentation techniques can be employed to increase dataset size. Among these techniques, the generation of fully synthetic data is particularly promising, as it provides automatically labeled data, thereby saving time and effort. However, these approaches present several drawbacks, as they often require non-trivial processes, such as 2D image-to-3D model conversion or photorealistic rendering. Furthermore, these challenges are amplified in industrial environments, which often involve highly specific assets and objects that are not easily found in publicly available datasets. Therefore, this thesis aims to address these limitations by implementing an effective pipeline that enables human operators to easily generate synthetic datasets for object detection tasks in industrial scenarios. The thesis will first explore recent approaches (e.g., 2D-to-3D conversion, Generative Adversarial Networks, diffusion models) and will subsequently propose a complete pipeline, which will be evaluated on a real industrial use case. Examples of similar/useful ideas: • https://www.youtube.com/watch?v=HHzNIh72B_Y • https://www.youtube.com/watch?v=iuDv3TS-xQA • https://www.youtube.com/watch?v=dxlyCPGCvy8 • https://www.youtube.com/watch?v=pJfHiVo8GUg • https://www.youtube.com/watch?v=BuL7RroTG7s

The term "remesh" indicates a technique to automatically reconstruct the geometry of a 3D object with a different, generally more uniform, topology. The remesh technique also allows to increase or reduce the mesh topology based on the desired definition and is especially used to convert images generated through the sculpting technique. More generally, typical areas of use of remesh concern the simplification of high-definition 3D objects that need to be animated, or the simplification of 3D objects for use in gaming (e.g. props, environments, buildings, etc.). Often, these techniques require user interaction to guide the choice of remesh settings and evaluate the result obtained. This thesis proposes to investigate the remesh solutions currently available and to design and develop an innovative solution based on the most recent computer vision and artificial intelligence techniques.