Thesis Description & Objectives
Problem:
Recent research highlights that XR + LLM integration is still fragmented, lacking unified frameworks for interaction and collaboration. Human-AI collaboration benefits from complementary reasoning between humans and AI, but current systems poorly coordinate roles and interaction dynamics. Moreover, existing XR systems typically treat AI as a tool, not as a team of collaborating agents, and do not support multi-agent coordination alongside humans in shared virtual environments.
Objective:
Design, implement, and evaluate a human-centered XR framework where multiple specialized LLM agents collaboratively assist users in complex tasks within a shared virtual environment. Specific goals include: i) Enable multi-LLM collaboration; ii) Integrate human-in-the-loop interaction within XR iii) Support real-time co-presence and co-decision-making; iv) Evaluate improvements in task performance, cognitive load, and user trust.
Approach:
The proposed system will include: i) an immersive workspace based on XR for interactions; ii) multi-LLM agent system and iii) Human user as supervisor and collaborator. Possible use cases may consider, e.g., collaborative design; medical or scientific decision-making, training and education in XR labs.
Expected Impact:
Improving user performance and experience.
