augmented.org

A new air guitar experience is on the road. It started in Italy already to promote the new Disney sitcom I’m In The Band. It apparently is just another use of AR for marketing, but I have to say the idea of visualizing an air guitar with AR is pretty funny.

Pietro Sansone from the producing team says:

We had the idea of using the augmented reality software when we were briefed by Disney Television Italia to find a concept for a street communication activity to promote the new I’m In The Band series. I’m In The Band is a comedy that tells about a kid who makes his dream come true when he starts playing as a guitarist for a hard rock (pop metal, as Disney calls it) band.

Lester from AugmentedPlanet already wrote on it before, so I’d like to quote him quickly:

it looks like Disney are about to launch an augmented reality game called Concert Mayhem which is based around characters from the new I’m In The Band sitcom. The original concept for the game was developed by Seac02 and used in Italy as a marketing tool to launch the sitcom. The concept was users stand in front of a green screen, air guitar and they appear in the video. The game however has been adapted for online so shortly you’ll be able to give it a go.

It obviously works fine as some youtube videos demonstrate. But my point being is another one, we get illustrated below:

First conclusion is, that not always visual based tracking is the best choice. Or to be more precise: the concept of your product must match the current technology state of the art. Here they use feature tracking (for the T-Shirt) to augmented the guitar. Only problem is that while playing an air guitar you probably cover up your body quite frequently. This leads to a broken augmentation and to visual drawbacks. I am currently prototyping using depth cameras with AR, but I’m eagerly waiting for an easy toolkit to integrate depth sensing cameras into the AR toolkits. Thus we will be able to use the 3D for a better tracking (yeah, yeah, still waiting for Project Natal) and have the correct occlusion. You definitely want to see your picking hand in front of the strings and the guitar’s body!

The second point we see illustrated (the above video only being an arbitrary example of AR interaction) yields in my question: is AR too early for this kid and would he rather have an inflatable guitar to play with? Or is he already so much used to technology and 3D effects in his telly, that he just cares more about a paper snippet in his finger than about the cool Augmentation? The adults seem to be having a hard time, convincing him. Maybe it was just bed time. :-)

I hope that with an easier way of using this technology for artists and designers we can gain better demos and experiences, that are even more fun. Today AR is still limiting the interaction to the least common denominator: moving around an object by moving the marker. Wave or cover up a marker to get some magical interaction (sometimes you are not really sure if you really triggered it or it just happened). For a kids demo it might be enough with a waving interaction, but I don’t want to see the gaming world step down to mediocrity only to support the latest hype. Project Natal’s demo game of this kind-a dodge ball game was great fun and the guitar demo as a one-time-attraction has its justification, but either we shrink our brains back to infantile times or we still grab another interaction device to get a deeper interactive experience. Or maybe we just need to come up with better ways of interaction. Depth-Cameras for gestures?

… this leaves us again with the typical VR dilemma: the human body’s interaction works mainly with haptics and grabbing. If we want to control real objects we need to use our hands and touch the object. (Leaving speech and mimic interaction out here, also the overhyped Theremin.) Even if we had Project Natal today, we would still have the problem of a real feedback. Only audio and visual feedback is just so limiting. We either need a big leap in VR/AR research or we just accept the Generation TV’s style of observing our acts on a screen…

Anyway, this was just a TGIF’s comment. So, enjoy your weekend. Maybe go into the woods for a change and carve a bow. :-) We’ll see you next week with more great Augmented Reality!!!

The International Symposium on Mixed and Augmented Reality deadline is coming up! The biggest conference on AR will be held this year in Seoul, Korea, 13th-16th of October. But the deadline for handing in your papers and posters is May, 24th! So, if you have something up your AR-sleeve, go and submit it!

What is the ISMAR about?

The 2010 ISMAR Arts, Media and Humanities chairs invite artists, scholars, media practitioners, who can shed new light on emerging new relations within the future of Mixed and Augmented Reality. We would welcome musings, probings, discourses, insights, and imaginations to be presented in Seoul Korea October 13-16, 2010 in the form of long and short papers, art installations, panels, workshops or tutorials.

This theme of this year’s ISMAR conference is “Borderless” sparked by the convergence of Mixed and Augmented Reality, ubiquitous technology and global connectivity. In the rapidly emerging application of Mixed and Augmented Reality, how do we make sense of time and space now?

With no clear-cut boundary between what is natural and artificial, we are faced with ambiguity, transience, and contingency. How will melting the boundaries between real, virtual and imaginary realities transform our world?

In other news… (today only a quick linked overview)

Fight your phobia of cockroaches with Augmented Reality here:
http://neoacademic.com/

… which reminded me of the phobia of height and fall, that really seems to work for the user when simulated with AR: AugmentedEnvironments said: “This video shows an augmented reality application we have developed to use as an experimental testbed for evaluating what components of an AR experience contribute to a user feeling “present” or immersed. In this experiment the participants are presented with a virtual hole that drops three stories and are asked to perform tasks around this “pit.” Their heart rate, galvanic skin response, and skin temperature are measured while the participant is shown different versions of the pit. The goal is to develop quantitative and qualitative methods for measuring how immersed a user is in an AR experience and to develop guidelines for people building AR applications.” See:

Golf

Gigaputt: Play virtual golf with your iphone in your city. It’s not really AR as I would define it, but great
location based fun, I wanted to share, since it reminded me of the Dinosaur Jr. video by Spike Jonze and made me laugh, when I imagined how people toss away their phone:

Face Tracking

Neat AR Face Tracking and Avatar Generation using Unity:
http://www.youtube.com/watch?v=JoeKxnTDRZo

AR Furniture Interface

AR Furniture Planning, reloaded, showing a nice interface:

Enjoy!

Today I have the pleasure of publishing a chat I had with the CEO of tagwhat, Dave Elchoness. We decided to share some of his thoughts on AR and the mobile world with the augmented.org readers – you!

Tagwhat is a new mobile location based service for your smartphone, that allows you to tag the whole world and share it with your friends. It also combines the different social media platforms in one mobile augmented reality experience. They just launched a week ago (press release) and are now online under www.tagwhat.com. But I’ll just pass the word to Dave without further ado. Enjoy:

augmented.org: Hi Dave, thanks for taking your time. [...] To get us started: Could you shortly explain what does tagwhat do?

Dave: Thanks you, Tobias. [...] Tagwhat essentially gives each user the world to augment. With an on-line map interface and robust mobile AR, users create their own AR tags and interact with each other inside of AR. Following other users inside of Tagwhat merges their AR into your world. Tagwhat is in essence a creation and distribution system for AR.

What is Iryss’ history? Since when are you around and when did you discover AR?

Iryss was among the first developers in the world on the Layar system and created layers for Wikipedia, Yelp, Flickr, Mazda, and several others. In the fall of 2009, Iryss developed its own proprietary AR system originally designed to display the outputs of sensor technologies (like active RFID) in the cameraview of the consumer mobile device. Iryss then moved into consumer apps and quickly decided to create a much larger product which is now Tagwhat.

So, with the new product tagwhat, could you explain what is unique about it? (Compared to other location based services with an Augmented Reality mode, like Wikitude or Junaio, …)

There is the on-line map interface that allows anyone to enjoy Tagwhat and participate in the discussion and opportunities presented by the system, including creating and placing AR anywhere in the world from their desk. Second, there is the discussion and commenting inside of the AR tags. Third, there is the social aspect, meaning that like Twitter you can follow others and merge their AR into your world. We also have robust filtering and search mechanisms to ensure a meaningful, personalized experience, integrations with Facebook, Twitter and Gmail and quite a bit more. I really think we have successfully combined a strong, future looking AR system with social dynamics that will appeal to people.

So, what does an AR system add for the user? Where do you see the potential for AR in location based services?

Augmented reality adds an “inside the web” perspective that will, increasingly over time, be the way we interact with our surroundings. What we call location based services today will gradually change, but the element of location will play a role in how we use the web from now on. Now that mobile devices are equipped to offer location services, there is no turning back. This means a contextual web for everyone.

Without a turning back option and “everywhere social media”: does tagwhat address privacy issues? Can I easily define who can see what (even among my friends)?

You can choose to have a private or public account in Tagwhat. In the case of a private account, only users you accept can see your tags. Tagwhat also has a direct tag option where you can send a direct “for your eyes only” tag to people who follow you.

Can I connect to new friends etc mobile? or (today?) only via web?

Tagwhat allows you to connect to friends via Facebook, Twitter, and Gmail. You can find new people to follow by searching inside Tagwhat or choosing public tags that appear in the front page. Very soon we’ll be announcing some new functionality that will make finding people and information even easier.

Great. How do you see the future of mobile AR?

The future is mobile AR. Location based services will become one and the same with AR. As the so-called “Internet of Things” emerges, mobile AR will be viewed as a necessary technology that no one can live without.

Talking about technology: what kind of interaction paradigms do you see for the future (mobile GUIs are tiny and sometimes annoying)? Today it’s sometimes hard to get to a fast result while on the run. Too many clicks or downloads delay.

Tablets and eyewear will bring mobile AR completely into the mainstream. The interaction will be entirely natural, like reaching to shake someone’s hand.

The business model for Apps is discussed all the time. Is only ads the answer? E.g. virtual goods are getting more and more popular. Can you imagine integrating these additional bonus services, and items into your product? What do you think about this virtual good development?

I think there are numerous compelling business models for apps including ads and virtual goods. I think how these are implemented is the critical question. But as mobile AR innovators we are prepared to invent new ways of applying these business models in our unique context. I expect to see lots of innovative business models coming out of Tagwhat.

Regarding business models: will it completely stay social networking based or do you consider cooperations from companies like coffee shops or restaurant chains to add special channels?

Tagwhat has existing channels like: Wikipedia, Eat, Drink, and Foursquare, and fully intends to integrate much more compelling content from companies and organizations in the near future. Some folks at IBM are already using Tagwhat to feature some of their work. Other companies and universities are as well. Tagwhat is about information delivery in a very compelling way. Lots of content is coming.

In general, can you project the social implications by everday-AR? When is the technology and the user ready? When do people dont ask for an off-switch anymore? ;-)

We know that mobile is about to explode. As this happens the uses of mobile AR will become extremely apparent and tools like Tagwhat will become the social medium of the future. Once hardware becomes better suited to AR, there will be any turning back. People will never want to turn their AR off because they won’t want to miss the great content and opportunity all around them.

Thanks a lot, Dave.

Tobias, thanks a lot for your interest and the interview.

Dear Reader,

I’ve been quite offline lately due to the preparations for our own conference, that took place on May 6th+7th in Munich. It was a great show with two days and two floors packed with live demos and interesting talks. (Luckily Pixar didn’t claim that Stereoscopic3D is everything, but rather a small piece in making a great movie. But that’s what I expected. ;-))

We showed lots of demos and also a couple of augmented and mixed reality demos. The most simple (but still great) demo I wanted to share with you today, so that you can see the quality we get with an off-the-shelf laptop and a webcam. Spiced up with some catchy tune behind. ;-) Enjoy!

The car for instance holds about 1.000.000 polygons with dozens of shaders and was rendered with 30 fps. The laptop was a regluar Dell Latitude and the usual suspect camera (Logitech Pro9000).

As soon as the press material from the film crew is ready I hope I can show you more videos, e.g.:

• Photoshooting Set Previsualization
  The second demo I hosted is intended to be used for a photographer or film crew that needs to shoot backplates (background images of landscapes, streets, …) for commercial productions. This process is currently quite annoying for the creative guys since in 99% of the cases they won’t have a physical car (or whatever product to photograph) on site. This makes planning a shot pretty hard. The integration of the CG car will only happen later in the postproduction. With AR we can now give the live preview to the crew on location and automatically import SLR images or HD feeds and export the camera angles for postproduction.
• Virtual Photostudio
  The third AR/MR demo is a mixed reality application to set up a virtual photostudio. Markers can be touched physically and represent light sources, that can easily be re-arranged to light the object of interest without the need of touching complicated GUIs of any rendering tool. The demo makes use of full raytracing to get the light and reflections right. So, stay tuned for a video update.

So, but that’s enough talk on the stuff from RTT. Now let’s enjoy the short holiday (at least in Germany) and be back next week! To pass the time be sure to:

check out the augmented reality racing game from TU München and

take a look Project Natal’s competitor Omek InterActive with their body controls. They offer the same full body motion tracking with a depth-sensing camera and process it in real-time. There will be a SDK, so that you and me as a developer can access the system and hook it up to your specific software.

After a great 2nd regluars’ table on AR in Munich yesterday (thanks to everybody who made it!), I’m off to set up the demos to be seen from tomorrow on at the open house fair and conference in Munich – hosted by my company Realtime Technology (RTT AG).

It will be a two-day event, showing our latest technology and with great talks from Industry leaders, e.g. we have Bob Whitehill from Pixar talking about Stereoscopic 3-Dimensional Storytelling.

There will be two demos on Augmented Reality and one on Mixed Reality, which I am hosting. So drop by in the marketing lounge, if you are an attendee or wait for the reports floating in afterwards.

See you on the web next week!

- Toby