MD3DM VR Interface

Managed Direct3D Mobile (MD3DM) is a graphics API developed by Microsoft for mobile applications. During the past few months, I have developed another customised mobile 3D graphics engine based on MD3DM and C#. The engine is operational in Pocket PCs, Smartphones, and other devices running Windows CE with the .NET Compact Framework. The graphics engine operates as a separate mode handling the output from the tracking hardware (GPS and/or digital compass) automatically providing six degrees-of-freedom to generate mobile VR applications. An example sceenshot of the MD3DM VR interface representing City University campus is shown below.

Compared to the Virtual Navigator, the advantages of MD3DM is that it takes full advantage of graphics hardware support and enables the development of high-performance three-dimensional rendering. In addition, it provides support to load binary 3D files which are much faster than VRML does. On the other hand, the major disadvantage is that it requires a semi-automatic procedure in order to prepare the binary 3D files that contain multiple textures. Using appropriate functionality of a modelling package (a feature called Render To Texture) the textured model’s coordinates were first flattened and then unwrapped on a new untextured model resulting in having just one texture and one set of UVW coordinates for a whole number of meshes.

LOCUS in Google Earth

Google Earth is one of the most successful GIS visualisation and interaction applications. The interface provides geo-referenced satellite imagery, which acts as a digital map, and users can perform geo-visualisation and pre-navigation operations. In the LOCUS project, we use the similar geo-referenced imagery to represent the topological information blended with geo-referenced 3D maps in a Virtual reality interface. To provide an alternative virtual reality representation I have imported one of LOCUS geo-referenced 3D maps into Google Earth by converting it from VRML to 3ds format.

The above screenshot illustrates four different views of egocentric and exocentric visualisation in Google Earth. The presented 3D map is very close to City University Campus and it was generated to assist us understand some of the issues involved navigation and wayfinding. The same 3D map will be also used in the user-testing of the VR/Map LOCUS interface. The interface operates on windows mobile 5.0 devices and uses GPS and a digital compass to calculate position and orientation respectively. These data are then used to transform both the geo-referenced digital maps and the 3D maps accordingly and provide the means for user-friendly navigation and wayfinding.

More information about Google Earth:

http://earth.google.com

http://www.gearthblog.com

VR and AR Gaming

In this month’s issue of the journal of Simulation & Gaming (December 1 2006, Volume 37, No. 4), I have published an article with title 'An exploration from virtual to augmented reality gaming'.

The paper first presents the requirements of modern gaming applications and proposes a classification of the most significant game design issues. Next, to realise the issues related to video and virtual reality gaming, an interactive game engine is designed. As a case study a traditional two-dimensional arcade game, called Breakout, is implemented into a virtual environment.

Collision detection is supported between the graphics components of the application based on Newton laws of physics. To test the effectiveness of our approach, a tangible platform for playing interactive three-dimensional games using video see-through augmented reality techniques is proposed.

The above screenshot illustrates how a user can manipulate the Breakout simulation using his hand in real-time. This allows getting the best viewpoint in a natural and realistic manner. Similarly, the player can move the AR Breakout closer to the camera (and vice-versa) to zoom into the game.

Moreover, a pilot study evaluated six important issues of each application including: efficiency; usefulness; realism; learning; interaction and camera movement. Initial results illustrated that players preferred the VR Breakout in terms of efficiency and the AR Breakout in terms of game play and ease of interactions. Overall, the AR version seems to have all the necessary potentials to become an alternative platform for future gaming applications.

For more information, a draft version of the article can be downloaded from here.