ERCIM News XR4ED Article 2024

Recently, a co-authored article entitled ‘Extended Reality Intelligent Assistant for Education’ was published at ERCIM News. The article introduces an intelligent Extended Reality (XR) teaching assistant communicating with the users in natural language. The application uses artificial intelligence (ChatGPT) to understand and communicate with the students using realistic-looking avatars with body language. Teachers can customise the application by adding their slides to be taught or choosing a teaching topic that will be discussed by two intelligent agents.

Although the application is primarily presented in an AR environment, it offers a panoramic image mode that can optionally be used to surround the user in the XR space. This shift towards VR along the MR spectrum brings more immersion and is especially suitable for certain learning scenarios, e.g. featuring learning about historical sites, natural ecosystems, or other content that is bound to a place. The XR Intelligent Assistant has been implemented on Magic Leap 2 AR head-mounted display (HMD) and for Android for usage on phones.

More information:

https://ercim-news.ercim.eu/en137/special/extended-reality-intelligent-assistant-for-education

High-Resolution LiDAR with Low-Power Laser

Researchers at the University of Science and Technology of China (USTC) have developed a compact and lightweight single-photon LiDAR system that can be deployed in the air to generate high-resolution three-dimensional images with a low-power laser. The technology could be used for terrain mapping, environmental monitoring, and object identification. To minimize such sensors’ energy consumption, USTC researchers devised a single-photon lidar system and tested it in an airborne configuration. The single-photon lidar system is made possible by detection systems that can measure the small amounts of light given out by the laser when it is reflected. The researchers had to shrink the entire LiDAR system to develop it. It works like a regular LiDAR system when sending light pulses toward its targets.

To capture the small amounts of light reflected, the team used highly sensitive detectors called single-photon avalanche diode (SPAD) arrays, which can detect single photons. To reduce the overall system size, they also used small telescopes with an optical aperture of 47 mm as receiving optics. The time-of-flight of the photons makes it possible to determine the distance to the ground, and advanced computer algorithms help generate detailed three-dimensional images of the terrain from the sensor. To validate the new system, they conducted daytime tests onboard a small airplane in Yiwu City, Zhejiang Province. In pre-flight ground tests, the LiDAR demonstrated a resolution of nearly six inches (15 cm) from nearly a mile (1.5 km). They then implemented sub-pixel scanning and 3D deconvolution and found the resolution improved to 2.3 inches (six cm) from the same distance.

More information:

https://interestingengineering.com/innovation/single-photon-lidar-china

NeuCyber Array BMI System

The NeuCyber Array BMI System, a self-developed brain-machine interface (BMI) system from China, revealed a remarkable feat: a monkey with its hands restrained and soft electrode filaments implanted in its brain, controlled an isolated robotic arm and grasped a strawberry by simply using its thoughts. The NeuCyber Array BMI System is composed of three major components: high-throughput flexible micro electrode array, a thousand-channel high-speed neural signal acquisition system, and a generative neural decoding algorithm based on a feed forward control mechanism.

The electrodes determine the quality of the recorded brain signals and the accuracy of brain control. It makes use of a large number of electrode contacts on a thin electrode thread to enhance signal capture ability. The electrode filaments made of flexible materials needed to be viewed under a microscope. The material fits well with brain tissue. The generative neural decoding algorithm based on the feed forward control mechanism of the system can establish an accurate mapping between neural activity in the cerebral cortex and motion parameters.

More information:

https://www.chinadaily.com.cn/a/202404/26/WS662b09a1a31082fc043c4266.html