MTI 2024 Article

Recently, I co-authored an open-access journal paper that was published at Multimodal Technologies and Interaction, sponsored by the MDPI. The paper is entitled “Extended Reality Educational System with Virtual Teacher Interaction for Enhanced Learning”. The paper introduces an interactive XR intelligent assistant featuring a virtual teacher that interacts dynamically with PowerPoint presentations using OpenAI’s ChatGPT API.

It incorporates multilingual speech-to-text and text-to-speech capabilities, custom lip-syncing solutions, eye gaze, head rotation and gestures. Panoramic images can be used as a sky box giving the illusion that the AI assistant is located at another location. Findings from three pilots indicate that the proposed technology has a lot of potential to be used as an additional tool for enhancing the learning process.

More information:

https://www.mdpi.com/2414-4088/8/9/83

IEEE CG&A Article

Recently, a co-authored article entitled ‘XR4ED: An Extended Reality Platform for Education’ was published at IEEE Computer Graphics and Applications. It presents the overall architecture and objectives of the EU-funded project dedicated to XR for education, called Extended Reality for Education (XR4ED). The goal is to provide a platform where educators will be able to build XR teaching experiences without the need to have programming or 3D modeling expertise.

The article describes the platform and focuses on a key aspect of collaborative and social XR, which is the use of avatars. We show initial results on a) a marketplace which is used for populating educational content into XR environments, b) an intelligent augmented reality assistant that communicates between nonplayer characters and learners, and c) self-avatars providing nonverbal communication in collaborative VR.

More information:

https://ieeexplore.ieee.org/abstract/document/10640174

ERCIM News XR5.0 Article 2024

XR applications for manufacturing are built based on a one-size-fits-all philosophy that does not consider the characteristics of individual workers. Hence, they fall short when it comes to supporting the emerging wave of Industry 5.0 (I5.0) applications that are destined to be human-centric and emphasise trustworthy human-machine collaboration. I5.0 requires XR visualisations that consider the characteristics, skills, and context of the manufacturing worker, along with the peculiarities of their interactions with machinery, automation devices, and cyber physical production systems.

In this context, the newly started XR5.0 Horizon Europe Project will build, demonstrate, and validate a novel person-centric and AI-based XR paradigm that will be tailored to the requirements and nature of I5.0 applications. The project will specify blueprints for using XR in I5.0 applications with emphasis on the development of innovative XR-made-in-Europe technology that blends with human-centric manufacturing technologies and adheres to European values (e.g. trustworthiness, security/privacy-by-design, transparency) as reflected in relevant EU regulations and policies.

More information:

https://ercim-news.ercim.eu/en137/special/xr5-0-human-centric-ai-enabled-extended-reality-applications-for-the-industry-5-0-era