This paper presents a framework for automatically generating speech-based interfaces for controlling virtual and augmented reality (AR) applications on wearable devices. Starting from a set of natural language descriptions of application functionalities and a catalog of general-purpose icons, annotated with possible implied meanings, the framework creates both vocabulary and grammar for the speech recognizer, as well as a graphic interface for the target application, where icons are expected to be capable of evoking available commands. To minimize user's cognitive load during interaction, a semantics-based optimization mechanism was used to find the best mapping between icons and functionalities and to expand the set of valid commands. The framework was evaluated by using it with see-through glasses for AR-based maintenance and repair operations. A set of experimental tests were designed to objectively and subjectively assess first-time user experience of the automatically generated interface in relation to that of a fully personalized interface. Moreover, intuitiveness of the automatically generated interface was studied by analyzing the results obtained through trained users on the same interface. Objective measurements (in terms of false positives, false negatives, task completion rate, and average number of attempts for activating functionalities) and subjective measurements (about system response accuracy, likeability, cognitive demand, annoyance, habitability, and speed) reveal that the results obtained by the first-time users and experienced users with the proposed framework's interface are very similar, and their performances are comparable with those of both the considered references.

The term Augmented Reality (AR) refers to a set of technologies and devices able to enhance and improve human perception, thus bridging the gap between real and virtual space. Physical and artificial objects are mixed together in a hybrid space where the user can move without constraints. This mediated reality is spread in our everyday life: work, study, training, relaxation, time spent traveling are just some of the moments in which you can use AR applications. This paper aims to provide an overview of current technologies and future trends of augmented reality as well as to describe the main application domains, outlining benefits and open issues

Augmented reality (AR) is a well-known technology that can be exploited to provide mass-market users an effective and customizable support in a large spectrum of personal applications, by overlapping computer-generated hints to the real world. Mobile devices, such as smartphones and tablets, are playing a key role in the exponential growth of this kind of solutions. Nonetheless, there exists some application domains that just started to take advantage from the AR systems. Maintenance, repair, and assembly have been considered as strategic fields for the application of the AR technology from the 1990s, but often only specialists using ad hoc hardware were involved in limited experimental tests. Nowadays, AR-based maintenance and repair procedures are available also for end-users on consumer electronics devices. This paper aims to explore new challenges and opportunities of this technology, by also presenting the software framework that is being developed in the EASE-R3 project by exploiting reconfigurable AR procedures and tele-assistance to overcome some of the limitations of current solutions

This paper presents a new and challenging approach to the control of mobile platforms. Natural user interfaces (NUIs) and visual computing techniques are used to control the navigation of a quadrotor in GPS-denied indoor environments. A visual odometry algorithm allows the platform to autonomously navigate the environment, whereas the user can control complex manoeuvres by gestures and body postures. This approach makes the human-computer interaction (HCI) more intuitive, usable, and receptive to the user's needs: in other words, more user-friendly and, why not, fun. The NUI presented in this paper is based on the Microsoft Kinect and users can customize the association among gestures/postures and platform commands, thus choosing the more intuitive and effective interface

Several virtual and augmented reality tasks involve users in hands-free interactions; in this case, speech-based systems are often preferred for their intuitiveness and naturalness. On the other hand, the robustness of this kind of interfaces can be an issue, thus affecting both the usability and the user experience, when they are used in noisy environments. This paper proposes a one switch interface triggered by vocal commands: three different scanning algorithms are tested. With this kind of interface users can sequentially select the desired command, thus improving the robustness of traditional speech recognition-based interfaces. Latency time is an issue for one switch interfaces, but it is shown how a bidirectional scanning algorithm based on a non binary switch can strongly mitigate this problem. Vocal one switch (VOS) interfaces are compared with a traditional multi-word speech interface both objectively (in terms of robustness and efficiency) and subjectively (user feedback).

This paper studies the opportunities coming from the use of consumer devices like smartphones and tablets to perform maintenance and assembly procedures with Augmented Reality (AR). Pros and cons are evaluated by comparing completion times and errors made while executing a maintenance procedure with an AR-based tool and paper-based instructions

Augmented Reality (AR) has been proved to be an effective tool to improve and enhance the learning experience of students. On the other hand, issues regarding the inflexibility of AR contents can strongly limit the usability of AR applications in education. This paper presents results obtained by using the AR framework designed and developed for the EASE-R3 European project and focused on the generation of maintenance procedures for machine tools. The high system flexibility allows instructors to easily make maintenance procedures suitable for the skill level of technicians to be trained. A case study is presented and results gathered so far analyzed and assessed

Augmented Reality (AR) applications are nowadays largely diffused in many fields of use, especially for entertainment, and the market of AR applications for mobile devices grows faster and faster. Moreover, new and innovative hardware for human-computer interaction has been deployed, such as the Leap Motion Controller. This paper presents some preliminary results in the design and development of a hybrid interface for hand-free augmented reality applications. The paper introduces a framework to interact with AR applications through a speech and gesture recognition-based interface. A Leap Motion Controller is mounted on top of AR glasses and a speech recognition module completes the system. Results have shown that, using the speech or the gesture recognition modules singularly, the robustness of the user interface is strongly dependent on environmental conditions. On the other hand, a combined usage of both modules can provide a more robust input.

The widespread adoption of mobile devices is giving everyone access to augmented reality systems, possibly involving a huge number of people in AR-based apps, with a pervasive social impact that cannot be neglected. AR systems are becoming affordable to everyone and especially useful in the maintenance field. This report aims to describe in a clear and accessible way the workflow to design and develop an augmented reality (AR) application for supporting maintenance procedures. The main focus of this paper is the evaluation of markerless tracking systems, as they could provide environment-independent solutions. The tests performed on a real use case outline the robustness of 3D CAD tracking with respect to other solutions

The evolution of input device technologies led to identification of the natural user interface (NUI) as the clear evolution of the human-machine interaction, following the shift from command-line interfaces (CLI) to graphical user interfaces (GUI). The design of user interfaces requires a careful mapping of complex user \"actions\" in order to make the human-computer interaction (HCI) more intuitive, usable, and receptive to the user's needs: in other words, more user-friendly and, why not, fun. NUIs constitute a direct expression of mental concepts and the naturalness and variety of gestures, compared with traditional interaction paradigms, can offer unique opportunities also for new and attracting forms of human-machine interaction. In this paper, a kinect-based NUI is presented; in particular, the proposed NUI is used to control the Ar.Drone quadrotor