AI Assesses Violinist’s Bow Movements

In a recent study, members of the Music and Machine Learning Lab of the Music Technology Group (MTG) at the Department of Information and Communication Technologies (DTIC) of UPF, apply artificial intelligence to the automatic classification of violin bow gestures according to the performer’s movement. Researchers recorded movement and audio data corresponding to seven representative bow techniques (Détaché, Martelé, Spiccato, Ricochet, Sautillé, Staccato and Bariolage) performed by a professional violinist. They obtained information about the inertial motion from the right forearm and we synchronized it with the audio recordings.

The data used in this study are available in an online public repository. After extracting the characteristics of the information concerning movement and audio, the researchers trained a system to automatically identify the different bow techniques used in playing the violin. The model can determine the different techniques studied to more than 94% accuracy. The results enable applying this work to a practical learning scenario, in which students of violin can benefit from the feedback provided by the system in real time. This study was conducted within the framework of the TELMI (Technology Enhanced Learning Performance of Musical Instrument) project.

More information:

https://blog.frontiersin.org/2019/07/29/psychology-violin-artificial-intelligence-learning/

Frontiers in ICT Article 2019

A few days ago, HCI Lab researchers in collaboration with Konica Minolta, published a paper at Frontiers in ICT entitled ‘An Interactive and Multimodal Virtual Mind Map for Future Workplace’. The paper presents multimodal VR collaborative interfaces that facilitate various types of intelligent ideation/brainstorming (or any other mostly creative activity). Participants can be located in different environments and have a common goal on a particular topic within a limited amount of time. 

Users can group (or ungroup) actions (i.e., notes belonging in a specific category) and intuitively interact with them using a combination of different modalities. Ideally, the multimodal interface should allow users to create actions and then post it on the virtual mind map using one or more intuitive methods, such as voice recognition, gesture recognition, and through other physiological or neurophysiological sources. Finally, users can access the content and assess it.

More information:

https://www.frontiersin.org/articles/10.3389/fict.2019.00014/full

US Navy Uses Magic Leap for Training

A new AR training tool developed by Magic Leap Horizons will use Magic Leap One AR headset to deliver various AR military training scenarios to the US Navy. Soldiers wearing the headset will experience a room size training environment designed to keep sailors and marines combat-ready while at sea. The system is called TRACER, which stands for ‘tactically reconfigurable artificial combat enhanced reality’, uses multiple technologies as part of the simulation, such as the Magic Leap headset tethered to a backpack processor, a simulated weapon from Haptech (formally StrikerVR) that can deliver realistic recoil through haptic feedback, hand tracking, and new software that can immerse soldiers into a multi-user AR experience.

Magic Leap Horizons originally developed TRACER as part of the US Army’s Augmented Reality Dismounted Soldier Training (ARDST) project. The Office of Naval Research saw the potential of TRACER and worked with the Naval Surface Warfare Center, the US Army Combat Capabilities Development Command, along with Magic Leap and Haptech Inc to reconfigure the project to work with sailors and marines. TRACER is built almost entirely from commercial, off-the-shelf products that anyone can purchase on the internet. What the TRACER system can do is put soldiers into extremely dangerous scenarios that feel very real but don’t have real-world consequences, which has always been one of the biggest benefits of VR and AR training.

More information:

https://vrscout.com/news/us-navy-magic-leap-combat-training/