augmented reality and games

Augmented Reality and Games

Although my schedule this semester allows me to work from home on

Fridays, I decided to head into school for what may have been the last

presentation given by candidates of a gaming faculty position here at

Carleton (two of the other talks were covered here and here). This

talk was given by Mark Fiala, currently of the NRC, regarding his

vision of the future of augmented reality gaming.

Dr. Fiala began his talk with this loaded question: "Is gaming a valid

research area?" While it seems strange to suggest that it is, it does

in fact have a lot of value in the academic world. For instance, it

does a good job of motivating students to learn computer science in

the first place. But beyond that, one must consider the research

results beyond their application to gaming. Techniques developed for

the purpose of games have benefits in other areas as well, including

medical computing, computer aided drawing, robotics, artificial

intelligence, and even military training. Or, if that's not enough to

convince, then think of the application of games as just another way

to validate the theory.

In The World According To Mark, gaming will move more toward console

applications and away from the PC. Multi-player games will gain

momentum, and players will expect full mobility (including being able

to play outdoors). And, the big one, is that augmented reality will

play a major part in the games of the future. This means that it is an

exciting time to be a researcher in computer vision in general and AR

in particular. If one is lucky, he can ride the wave to where the

future is probably headed anyway.

So what is augmented reality, anyway? Although I wrote a bit about

augmented reality here, here's a little reminder: a bit like virtual

reality, augmented reality mixes real video with computer generated

objects and information. This could be seen as a new kind of

improvement to the human-computer interface.

Back to games. There are two main paradigms for AR games. First, the

magic lens makes use of a portable device with a front facing camera.

You see the real world as per usual on the screen with some computer

generated goodies added on top. Second, the magic mirror reflects

footage of you back at you as if looking in a mirror. You might have

some specially recognized tags strategically positioned on your body

that can be used to augment your image with just about anything. Some

examples shown included putting a scuba suit on someone and turning

another person into a robot.

The magic lens concept was used in the popular exhibit that traveled

(is still traveling?) across the US in various science museums called

"Star Wars: Where Science Meets Imagination." The photo below shows a

user interacting with the game, including a view of the augmented

portion.

(Image from here)

From the OMSI web site describing the exhibits:

Building Communities and Augmented Reality. Visitors build a

spaceport, moisture farm community and walled Jawa town. Placing

cards on a table - the physical landscape - a computer superimposes

a building on a site in virtual reality and real time.

This magic lens paradigm has a lot of potential, particularly for

tabletop games. Action and strategy titles could be shipped with a

specialized mat with the appropriate tag marker system to base the

augmentations off of. These tags can even be hidden into game art if

some cleverness is applied. Perhaps one day even World of Warcraft

will be played as a tabletop game in 3D!

To take things to the next level, imagine playing Halo in real life.

All you would need is a series of rooms or hallways marked up with

tags, and some kind of head mounted camera and display unit. You would

be able to see where obstacles were, though not necessarily as they

are in the real world. It would be like laser tag, but with animated

violence!

The final step, at least for now, would be to bring it all outside

into the world's playground. Here, of course, there are no tags, so

natural features are required. Extra sensors would be required, like

GPS and orientation. Some computer vision magic would help pinpoint

the exact location of a player once the GPS gives a rough enough idea

to retrieve local context information.

I'm happy to call gaming a research area with all these exciting

opportunities related to augmented reality. I guess we'll have to wait

and see who gets hired for this new games faculty position, but if

someone into AR and computer vision ends up on board, I will certainly

be watching their research, and maybe even working with them one day!