Augmented reality in gaming

Indecom · Nov 2008

This is something i've stumbled upon lately which i find absolutely fascinating.

Augmented reality is the act of overlaying 3d elements on top of a live video feed using special markers in attempt to augment your reality. Though this technology has existed for some time, not many have heard of such a thing, except for use in movies though thats no where near AR.

Here are a few examples of what is capable with AR:

[ame]

http://www.youtube.com/watch?v=M0drf5mHIH0[/ame]

[ame]

http://www.youtube.com/watch?v=aoI2CXX3L-g[/ame]

[ame]

http://www.youtube.com/watch?v=c_LrVqI6StY[/ame]

[ame]

http://www.youtube.com/watch?v=Lfp8id6bpDU[/ame]

Clearly you can see right off hand some of the possible implementations of such an interesting technology, marketing, virtual gps tracking and exploration, but i know most of us here at polycount arent interested in that.

What we want to see are full real awesome GAMES!!!

So knowing the possibilities of this kick ass tech, what kinds of games would you like to see come out of it, things like first person shooters, rts games, exploration, what would you like to see in your AR game?

MikeF · Nov 2008

L4D in my room.

Indecom · Nov 2008

Haha yea that would be pretty sweet. If there are programmers out there who want to play with stuff like this, look up artoolkit on google for a site with a free open source sdk for developing your own ar programs. I finally got it to compile and took me a long time to get the lighting set up right. daylight settings are best. but its fun holding onto a sheet of paper and seeing a model attached to it on screen.

aesir · Nov 2008

It's certainly going to be the future. Imagine going to a lazer tag arena with a bunch of friends, but you put on some vr glasses first. The course would already be setup to run with the programmed game with motion detectors everywhere for triggering events. Would be tiiiight.

pior · Nov 2008

Well it's been around since forever right? Eyetoy and others...
Obviously its looking cool in these simulated videos but for the checker/racing game I just don't see any practical way for the players to see what's going on or interact with stuff. Wearing VR glasses would kindof spoil the whole thing right?

aesir · Nov 2008

I dunno, do wearing normal glasses spoil real life? Tech will improve.

Indecom · Nov 2008

@pior- i think your concerns cant really be handled with ar. Something like that will require actual holograms which are far beyond the modern consumer, however currently possible to some limited extent. I think that AR gaming will help to pave the way to holographic gaming in the decades or more to come.

McGreed · Nov 2008

The problem right now is to get the real world interpeted and intergrated into the virtual world. AR tracking is not that easy and if you want proper AR, you need to map out the real world first and have some kind of locator sensors. Did some AR at university and you had to move the camera real slow to avoid it losing its tracking and even then you need to edit it. Best bet would be a special made room with sensors that tracks everything inside the room and people would need sensors on them as well. But thats all so boring, I'm still waiting for a holodeck

Ryno · Nov 2008

How is this not just CG/video compositing? I'm not getting the point here. Mapping out the real world shots and camera angles for creating of CG elements is pretty much standard fare for any composited effects shot.

rube · Nov 2008

It is CG/video compositing.. the difference is real time performance and interactivity.

Indecom · Nov 2008

@Runo- is is essentially that, except in movies and such all the camera movements are tracked and recored then later on after its filmed those camera movements are applied to the virtual camera in the 3d world. What AR does is uses something as simple as a webcam to track just a marker on the screen and while you are filming it, in realtime, it will overlay the 3d elements onto the marker without need of expensive motion tracking equipment or recording the cameras position and angle changes.

MoP · Nov 2008

My brother works on stuff like this, their company mainly uses it for GPS & traffic route real-time overlays.

Ryno · Nov 2008

Oh, OK. So it's just a real-time tracker, allowing for 3d compositing on the fly. With interactive CG stuff, possibly. That's kind of cool. Not a huge leap from what they've been doing in the post production world, but it would be cool to see consumers be able to play games based around the concept.

Indecom · Nov 2008

Yea, that is one most definite possibility with AR. I'd like to see maybe some sort of film which is laid over top of your windshield which will augment your reality and give you directions and names of roads and locations directly through your windshield.

And the gaming aspect is what i'm currently trying to work on. We know that most people with computers have webcams or can get them on the cheap, so its no worry with finding the right equipment for such an effect. The main problem is lighting and most people i admit will have a much better lighting setup than i do, so thats not even too much worry. Though on thing that i think will really help in the lighting department is to use actual light up markers, so no outside lighting will have any effect on the clarity of the markers placed in the scene.

I'm thinking of ideas currently for a simple game prototype, maybe somthing like a simple rts, or tower defense game like in the video above.

rube · Nov 2008

So far this is the most polished AR game I've seen(aside from that PS3 cadr game).. something I think I'd actually enjoy trying if only for the novelty of it.

http://vimeo.com/1320756

sebas · Nov 2008

AR sounds interesting. Related to gaming... I can see it for the Wii/casual gamer, but I think hardcore gamers expect to see more something like VR glasses and 2D/3D holograms.

Indecom · Nov 2008

Eventually i'd like to program and design for AR games that utilize compact headsets. The technology for that exists already for young children to bring story books to life in 3d, we just need to develop it further to be faster and more accurate even without markers or a great lighting setup. I mean, you could play a first person shooter game in real first person, that would kick tons of ass lol.

Indecom · Nov 2008

Playing around with the ARToolkit i was able to do this with my webcam and some hand copied markers since my printer is low on ink lol

really helps when you use the right markers, the reason why it didnt work for me very well is because i was using the wrong markers. But it works fantastically now!

Jose Carlos · Nov 2008

.... A few years back I had a phone that came with an AR game. It used the phone's camera for video capture and overlayed a crapload of mosquitoes that you then shot down. T'was fun for a little while, till I got bored shooting bugs off my brother's bald pate.

Indecom · Nov 2008

Haha that sounds like fun for a little while lol.

So i've been making progress, got some help from a friend who split up some source, and my friend paril and I managed to get this guy working, with smooth interpolated animations, he's just a simple md2 model

We're going to be making a simple ai controlled real desktop battle thingy. Gonna be pretty sweet and help pave the way for any other AR projects we might have in the future.

Michael Knubben · Nov 2008

A practical, cool use of this is using gps-enabled smartphones with a camera as a gun, and tracking down other players in a real environment to shoot them. I believe there was an Android, Google's Linux(or unix/gnu/whatever, I forget)-based mobile Operating System. In practice, it looked really fun! Imagine trying to hide the fact that you're playing by only sneaking glances are your phone every now and again to see who's playing (in a crowd who don't know that anyone's even playing a game), sneaking up to them, pulling out your phone and shooting them! It's like laser-tag, but a lot more fun, with the added element of stealth, and it being in a real environment, and not looking like a dick in a tron-costume.

There's also the Augmented Reality game where you've got a cube, and the computer maps a room 'onto' (actually it looks like you're watching the inside, ofcourse) the cube, and you've got a little man you're suppose to lead from room to room. (ah, levehead, rube posted a video already. Thanks for that!)

I agree that the examples in this thread aren't the greatest. The first one isn't aware of the room, except for a floor, and the catapult-game is cute, and very impressive in a gimmicky way, but it doesn't enhance the game, I don't believe.

Anymore examples, anyone? I've been following this kind of stuff with interest, but it's only been little bits left and right. There isn't a (quality) blog or anything that only follows this sort of stuff, is there?

Michael Knubben · Nov 2008

Here are a few more things that caught my eye in the past:
- http://www.bcrumbz.com Not a game, but using it to map out treasure hunts and whatnot for other people is still ridiculously cool.
- http://blog.tmcnet.com/blog/tom-keating/google/gps-google-android-wifi-army.asp
This is what I was talking about with the AR laser-tag. It appears to even use facial recognition so you don't need to hit the phone to score a point. I can see that being less-than-perfect, though. Still, it's a fascinating concept to me.

edit: AR seems to be on a lot of people's mind today: http://blogoscoped.com/archive/2008-11-24-n72.html
I came across that just a few minutes ago in my feed-reader.

Farfarer · Nov 2008

Came across this today on Waxy. Not really augmented reality in the "superimposing models on live feed" sense, but still pretty cool;

http://torbensko.com/projects/phd/

Indecom · Nov 2008

That video is really cool. I might look into developing with that stuff as well some day.

serge · Nov 2008

OK, someone was talking about AR Tower defense: http://cellagames.com/artd.html You have to have Symbian OS S60 3rd edition smartphone to play (like Nokia N95)

Augmented reality in gaming

Replies