SiSi

Technology that translates spoken or written words into British Sign Language (BSL) has been developed by researchers at IBM. In particular a software system, called SiSi (Say It Sign It), was created by a group of students in the UK. SiSi brings together a number of computer technologies. A speech recognition module converts the spoken word into text, which SiSi then interprets into gestures that are used to animate an avatar which signs in BSL. The main of SiSi is to enable deaf people to have simultaneous sign language interpretations of meetings and presentations based on speech recognition to animate a digital character or avatar.

IBM says its technology will allow for interpretation in situations where a human interpreter is not available. It could also be used to provide automatic signing for television, radio and telephone calls. It is worth-mentioning that the concept has already gained the approval of the Royal National Institute for Deaf people (RNID). The students used two signing avatars developed by the University of East Anglia. One of them signs in BSL and the other uses Sign Supported English - a more direct translation using conventional syntax and grammar.

More information:

http://youtube.com/watch?v=RarMKnjqzZU

http://news.moneycentral.msn.com/ticker/article.aspx?symbol=US:IBM&feed=MW&date=20070913&id=7464419

http://news.bbc.co.uk/1/hi/technology/6993326.stm

Serious Virtual Worlds ‘07

Today I have attended the first day of the Serious Virtual Worlds ‘07, the First European Conference on the Professional Applications on Virtual Worlds (13-14 September 2007) held at Coventry TechnoCentre. The theme for this first Serious Virtual Worlds conference is ‘The Reality of the Virtual World' and takes a close look at how virtual worlds are now being used for serious professional purposes. Many organisations are now actively researching and deploying virtual worlds. Serious Virtual Worlds is a good introduction to the serious uses of virtual worlds. This was driven by the extraordinary success of virtual worlds such as ‘Second Life’ (see screenshot below) as virtual social spaces for play leads to the question ‘What is the potential for the serious uses of these worlds?’

The theme of the first day is ‘Introducing Virtual Worlds’. A number of presentations and conversations introducing virtual worlds and the 3D web from Cisco, Linden Labs, TruSim, Forterra, Giunti Labs, Pixel Learning, Caspian, Ambient Performance and Daden, closing with the launch of the Serious Games Institute’s ‘Second Life’ Island with a Cocktail Reception followed by the Conference Dinner. The theme of the second day, is ‘Serious Virtual Worlds: Action & Potential’ including live virtual world presentations and conversations from Digital Earth, Reuters, Stamford Medical School, TruSim, PA Consulting, IBM, Forterra, NPL, Logicom, and AVM.

More information:

http://www.seriousvirtualworlds.net/

Deep Exploration

Right Hemisphere's Deep Exploration Standard Edition is a software tool that allows the quick production of multimedia 3D graphics. Unlike typical multimedia 3D tools, Deep Exploration Standard Edition is a unified application that works across a number of multimedia formats and 3D graphics styles so it is easy to take advantage of existing content and avoid big changes in file formats. The main operations that Deep Exploration performs includes: (a) translate 2D and 3D graphics and multimedia files, (b) search, view, and markup 3D graphics and (c) author, render, and publish 3D images and animations.

The above screenshot shows a 3D model representing City University’s campus and it was used as part of the LOCUS project. This 3D model (or 3D map) was originally generated in 3D Studio Max and then converted into two different formats used for mobile navigation in personal digital assistants (PDAs). More specific, the 3D map was converted in VRML for the Virtual Navigator interface and to DirectX for the MD3DM VR interface.

More information:

http://www.righthemisphere.com/products/dexp/

IJAC Journal Article

Last June, the International Journal of Architectural Computing (IJAC) has published a journal paper I co-authored with colleagues from the Centre for VLSI and Computer Graphics at the University of Sussex and the Ename Center, in a Special Issue on Cultural Heritage. The title of the article is ‘Multimodal Mixed Reality Interfaces for Visualizing Digital Heritage’ and the main aim of the paper is to provide several different and interesting types of virtual heritage exhibitions by utilising Web3D, virtual and augmented reality technologies for visualizing digital heritage in an interactive manner through the use of several different input devices. A high-level diagram illustrating the technologies employed in the multimodal mixed reality system is presented below.

The novelty of the technologies employed is that they allow users to switch between three different types of visualization environments including: the web in the traditional way, but including 3D, virtual reality and augmented reality. Additionally, several different interface techniques can be employed to make exploration of the virtual museum that much more interesting. In particular the architectural diagram illustrates several interaction modes from use of SpaceMouse and gamepad through to use of a physical replica of an artefact, and simple hand manipulation of AR marker cards. In addition, several visualization scenarios are provided ranging from the familiar web page but with 3D object, a virtual gallery environment, a person using a physical replica of an artefact to control and explore the virtual artefact, and several augmented reality examples.

A draft version of the article can be downloaded from here.

Polhemus Patriot

Polhemus PATRIOT is a cost-effective solution for 6 Degree-Of-Freedom (DOF) tracking and 3D digitizing. Patriot can accommodate a wide array of indoor applications such as head tracking, biomechanical analysis and computer graphics to cursor control and stereotaxic localization. PATRIOT includes a system electronics unit (SEU), a power supply, one sensor and one source. However, it is possible to expand the system’s capabilities simply by adding an additional sensor or an optional stylus. Measuring in at only 6.75 by 6.25 by 1.75 inches (LWH), the electronics unit is compact for easy installation in any environment. PATRIOT interfaces with the host computer via RS-232 or USB 1.1 and it's fully compatible with Windows® XP, Windows 2000 and Linux®. The source and sensor contain electromagnetic coils enclosed in plastic shells. The source emits magnetic fields, which are detected by the sensor. The sensor’s position and orientation are precisely measured as it is moved. Because the sensor is completely passive, it's safe for use in any application.

PATRIOT provides dynamic, real-time measurements of position (X, Y and Z Cartesian coordinates) and orientation (azimuth, elevation and roll). PATRIOT can update data continuously, discretely (point by point), or incrementally. With the optional stylus, you can trace the outline of a physical object or collect polygon facets and get pinpoint accuracy of unlimited X, Y and Z data points. PATRIOT offers low latency and high stability to ensure precise, uninterrupted tracking at all times. It boasts an update rate of 60Hz per sensor and has a range of five feet, resolution of 0.0015 inch and 0.1 degree, and static accuracy of 0.1 inch RMS for the X, Y, Z position and .75 degrees RMS for orientation. Latency is less than 18 milliseconds for both sensors simultaneously.

More information:

http://www.polhemus.com/?page=Motion_Patriot

Sensor-based Mixed Reality

During June and July 2007, I have co-supervised with members of the Cogent research group and the department of Creative Computing, a research project regarding a mixed reality audio-visual visualisation and localisation interface. The project has been developed over a six-week period by three students and operates within a room which is equipped with fixed-location wireless sensing devices called gumstix. These nodes are multi-modal although the system makes use of the microphone only. The main objective of the project is to display a 3D representations of the audio data contained inside the room blended with 3D information. The overall architecture of the sensor-based MR interface is presented below.

This project has been designed to test at least some aspects of the mixed reality presentation system, using easily available sensors and display devices. MR presentation of the sound field occurs within a 3D computer model of the room in which the sensors are located. It can take a variety of forms from a sound 'mist' to 'objects' representing the sound, which hang in space. Computer-vision registration is achieved through the capabilities of ARTag and ARToolKit and the best available marker is selected using confidence levels. Finally, in terms of localisation, the sensors calculate the location of a sound before drawing the object in 3D space.

More information and a demo video of this work can be found at:

http://www.youtube.com/watch?v=ittMEBIPFNw

3D Mouse Navigation

3Dconnexion has made powerful 3D navigation accessible and affordable for architects, artists, students and anyone else who wants to enjoy the 3D experience. In particular, 3Dconnexion devices are ideal sensors for navigation into virtual and augmented reality environments, mainly because they provide intuitive manipulation in six degrees-of-freedom. A characteristic example is SpaceNavigator™ which is shown below.

3Dconnexion's SpaceNavigator™ is a very cheap solution to 3D navigation. It works with more than 100 of today's most popular and powerful 3D applications and it is ideal for Google Earth version 4 and Google SketchUp. An alternative and a bit more expensive solution is SpacePilot which is illustrated below.

The 3Dconnexion SpacePilot, connects the users to the 3D design process in a different way that the standard mouse does. Its optical sensor technology and ergonomic design combine to deliver unprecedented control and fewer distractions. The SpacePilot may be used in user’s non-dominant hand to position, rotate, pan and zoom a model in one single, fluid motion. It can be also used in conjunction with the standard mouse w to simultaneously edit the model or select menu items.


More information:

http://www.3dconnexion.com/

OQO model 2 Handheld PC

OQO model 2 is another ultra mobile computer (it is the main rival of the SONY VAIO UX model) which is ideal for a number of everyday applications including navigation and wayfinding, location-based services as well as many more. The model 2 comes standard with WiFi 802.11a/b/g and Bluetooth 2.0 technology, 60GB hard drive, 5-inch Wide VGA LCD display and a 1.2GHz or a 1.5GHz VIA C7M ULV processor with integrated graphics chipset.

The OQO model 2 includes a keyboard that features a track stick mouse pointer for precise cursor movement with your thumb and dedicated zoom keys for quickly changing screen magnification that allow full interactivity while zoomed, and support 1000x600 and 1200x720 interpolated modes. The main flaws of the model 2 are that it lacks a digital camera which is nowadays a standard feature with any mobile devices (smartphones, PDAs and other handhelds) and there is no built-in memory card reader.

More information:

http://www.oqo.com/products/index.html