augmented.org

Since I’m as always pretty excited about projected Augmented Reality, I wanted to point again to the work at MIT MediaLab (Link to Fluid Interfaces and LuminAR currently down): their robotics arm including a camera and a projector is able to project images to the surfaces around you and allows for a gesture interaction. Imagine a small form factor, that looks like a desktop lamp, that includes all your computer hardware!

Regarding mobile Augmented Reality Interfaces, Willem Van den Eynde wrote his thesis on this topic. The demo shows a similar approach as the MIT’s 6th Sense demo (the link to the TED 2009 presentation here). I like the concept of having a wrist attached interface part, where you can drop and store snapshots or other media/augmented items. Moving a marker from your wrist to a desktop computer to transfer all taken photos or a specific one is a neat interaction detail.

A really beautiful demo was done in the Netherlands by DPI New Media for the Netherlands’ largest Energy company. It’s there to let the public come into contact with its green energy initiatives in a playful way. Players take control of the Sun, Wind, Rain and Biomass to provide the little city with electricity (DPI).

It has a shiny cartoony clay look and uses Quest3D and its AR module to get graphical effects like real-time shadows, screen-space ambient occlusion and per-pixel lighting for spotlight effects for the sun:

A great looking project, only improvements I could imagine: first, use a setup that has a better mixed reality feeling to it. I’d rather like to see a physical model landscape that gets augmented with weather effects or other changes… but with a real model below. It just is more fun than a blank table with only markers. Secondly, I’m really hoping for HMD demos soon. The interaction is just more direct and you don’t need to have the look-into-the-mirror-and-act-somewhere-else setting. But for a fair again, it’s obvious to use an easy indestructible-and-hygiene-rules-applying design. :-)

In other news…

… metaio introduces junaio glue, where you will be able to use vision based tracking on your mobile phone to glue 3D objects to any surface. Unfortunately the first concept video does not show the rendering on a device yet. But I’m sure it is only a matter of weeks until we see a running version on youtube.

… and once again 3D outdoor projections go around to impress us in the streets of Amsterdam and on two big buildings (Suntec City Towers) in Singapore.

A new air guitar experience is on the road. It started in Italy already to promote the new Disney sitcom I’m In The Band. It apparently is just another use of AR for marketing, but I have to say the idea of visualizing an air guitar with AR is pretty funny.

Pietro Sansone from the producing team says:

We had the idea of using the augmented reality software when we were briefed by Disney Television Italia to find a concept for a street communication activity to promote the new I’m In The Band series. I’m In The Band is a comedy that tells about a kid who makes his dream come true when he starts playing as a guitarist for a hard rock (pop metal, as Disney calls it) band.

Lester from AugmentedPlanet already wrote on it before, so I’d like to quote him quickly:

it looks like Disney are about to launch an augmented reality game called Concert Mayhem which is based around characters from the new I’m In The Band sitcom. The original concept for the game was developed by Seac02 and used in Italy as a marketing tool to launch the sitcom. The concept was users stand in front of a green screen, air guitar and they appear in the video. The game however has been adapted for online so shortly you’ll be able to give it a go.

It obviously works fine as some youtube videos demonstrate. But my point being is another one, we get illustrated below:

First conclusion is, that not always visual based tracking is the best choice. Or to be more precise: the concept of your product must match the current technology state of the art. Here they use feature tracking (for the T-Shirt) to augmented the guitar. Only problem is that while playing an air guitar you probably cover up your body quite frequently. This leads to a broken augmentation and to visual drawbacks. I am currently prototyping using depth cameras with AR, but I’m eagerly waiting for an easy toolkit to integrate depth sensing cameras into the AR toolkits. Thus we will be able to use the 3D for a better tracking (yeah, yeah, still waiting for Project Natal) and have the correct occlusion. You definitely want to see your picking hand in front of the strings and the guitar’s body!

The second point we see illustrated (the above video only being an arbitrary example of AR interaction) yields in my question: is AR too early for this kid and would he rather have an inflatable guitar to play with? Or is he already so much used to technology and 3D effects in his telly, that he just cares more about a paper snippet in his finger than about the cool Augmentation? The adults seem to be having a hard time, convincing him. Maybe it was just bed time. :-)

I hope that with an easier way of using this technology for artists and designers we can gain better demos and experiences, that are even more fun. Today AR is still limiting the interaction to the least common denominator: moving around an object by moving the marker. Wave or cover up a marker to get some magical interaction (sometimes you are not really sure if you really triggered it or it just happened). For a kids demo it might be enough with a waving interaction, but I don’t want to see the gaming world step down to mediocrity only to support the latest hype. Project Natal’s demo game of this kind-a dodge ball game was great fun and the guitar demo as a one-time-attraction has its justification, but either we shrink our brains back to infantile times or we still grab another interaction device to get a deeper interactive experience. Or maybe we just need to come up with better ways of interaction. Depth-Cameras for gestures?

… this leaves us again with the typical VR dilemma: the human body’s interaction works mainly with haptics and grabbing. If we want to control real objects we need to use our hands and touch the object. (Leaving speech and mimic interaction out here, also the overhyped Theremin.) Even if we had Project Natal today, we would still have the problem of a real feedback. Only audio and visual feedback is just so limiting. We either need a big leap in VR/AR research or we just accept the Generation TV’s style of observing our acts on a screen…

Anyway, this was just a TGIF’s comment. So, enjoy your weekend. Maybe go into the woods for a change and carve a bow. :-) We’ll see you next week with more great Augmented Reality!!!

The International Symposium on Mixed and Augmented Reality deadline is coming up! The biggest conference on AR will be held this year in Seoul, Korea, 13th-16th of October. But the deadline for handing in your papers and posters is May, 24th! So, if you have something up your AR-sleeve, go and submit it!

What is the ISMAR about?

The 2010 ISMAR Arts, Media and Humanities chairs invite artists, scholars, media practitioners, who can shed new light on emerging new relations within the future of Mixed and Augmented Reality. We would welcome musings, probings, discourses, insights, and imaginations to be presented in Seoul Korea October 13-16, 2010 in the form of long and short papers, art installations, panels, workshops or tutorials.

This theme of this year’s ISMAR conference is “Borderless” sparked by the convergence of Mixed and Augmented Reality, ubiquitous technology and global connectivity. In the rapidly emerging application of Mixed and Augmented Reality, how do we make sense of time and space now?

With no clear-cut boundary between what is natural and artificial, we are faced with ambiguity, transience, and contingency. How will melting the boundaries between real, virtual and imaginary realities transform our world?

In other news… (today only a quick linked overview)

Fight your phobia of cockroaches with Augmented Reality here:
http://neoacademic.com/

… which reminded me of the phobia of height and fall, that really seems to work for the user when simulated with AR: AugmentedEnvironments said: “This video shows an augmented reality application we have developed to use as an experimental testbed for evaluating what components of an AR experience contribute to a user feeling “present” or immersed. In this experiment the participants are presented with a virtual hole that drops three stories and are asked to perform tasks around this “pit.” Their heart rate, galvanic skin response, and skin temperature are measured while the participant is shown different versions of the pit. The goal is to develop quantitative and qualitative methods for measuring how immersed a user is in an AR experience and to develop guidelines for people building AR applications.” See:

Golf

Gigaputt: Play virtual golf with your iphone in your city. It’s not really AR as I would define it, but great
location based fun, I wanted to share, since it reminded me of the Dinosaur Jr. video by Spike Jonze and made me laugh, when I imagined how people toss away their phone:

Face Tracking

Neat AR Face Tracking and Avatar Generation using Unity:
http://www.youtube.com/watch?v=JoeKxnTDRZo

AR Furniture Interface

AR Furniture Planning, reloaded, showing a nice interface:

Enjoy!

Today I have the pleasure of publishing a chat I had with the CEO of tagwhat, Dave Elchoness. We decided to share some of his thoughts on AR and the mobile world with the augmented.org readers - you!

Tagwhat is a new mobile location based service for your smartphone, that allows you to tag the whole world and share it with your friends. It also combines the different social media platforms in one mobile augmented reality experience. They just launched a week ago (press release) and are now online under www.tagwhat.com. But I’ll just pass the word to Dave without further ado. Enjoy:

augmented.org: Hi Dave, thanks for taking your time. [...] To get us started: Could you shortly explain what does tagwhat do?

Dave: Thanks you, Tobias. [...] Tagwhat essentially gives each user the world to augment. With an on-line map interface and robust mobile AR, users create their own AR tags and interact with each other inside of AR. Following other users inside of Tagwhat merges their AR into your world. Tagwhat is in essence a creation and distribution system for AR.

What is Iryss’ history? Since when are you around and when did you discover AR?

Iryss was among the first developers in the world on the Layar system and created layers for Wikipedia, Yelp, Flickr, Mazda, and several others. In the fall of 2009, Iryss developed its own proprietary AR system originally designed to display the outputs of sensor technologies (like active RFID) in the cameraview of the consumer mobile device. Iryss then moved into consumer apps and quickly decided to create a much larger product which is now Tagwhat.

So, with the new product tagwhat, could you explain what is unique about it? (Compared to other location based services with an Augmented Reality mode, like Wikitude or Junaio, …)

There is the on-line map interface that allows anyone to enjoy Tagwhat and participate in the discussion and opportunities presented by the system, including creating and placing AR anywhere in the world from their desk. Second, there is the discussion and commenting inside of the AR tags. Third, there is the social aspect, meaning that like Twitter you can follow others and merge their AR into your world. We also have robust filtering and search mechanisms to ensure a meaningful, personalized experience, integrations with Facebook, Twitter and Gmail and quite a bit more. I really think we have successfully combined a strong, future looking AR system with social dynamics that will appeal to people.

So, what does an AR system add for the user? Where do you see the potential for AR in location based services?

Augmented reality adds an “inside the web” perspective that will, increasingly over time, be the way we interact with our surroundings. What we call location based services today will gradually change, but the element of location will play a role in how we use the web from now on. Now that mobile devices are equipped to offer location services, there is no turning back. This means a contextual web for everyone.

Without a turning back option and “everywhere social media”: does tagwhat address privacy issues? Can I easily define who can see what (even among my friends)?

You can choose to have a private or public account in Tagwhat. In the case of a private account, only users you accept can see your tags. Tagwhat also has a direct tag option where you can send a direct “for your eyes only” tag to people who follow you.

Can I connect to new friends etc mobile? or (today?) only via web?

Tagwhat allows you to connect to friends via Facebook, Twitter, and Gmail. You can find new people to follow by searching inside Tagwhat or choosing public tags that appear in the front page. Very soon we’ll be announcing some new functionality that will make finding people and information even easier.

Great. How do you see the future of mobile AR?

The future is mobile AR. Location based services will become one and the same with AR. As the so-called “Internet of Things” emerges, mobile AR will be viewed as a necessary technology that no one can live without.

Talking about technology: what kind of interaction paradigms do you see for the future (mobile GUIs are tiny and sometimes annoying)? Today it’s sometimes hard to get to a fast result while on the run. Too many clicks or downloads delay.

Tablets and eyewear will bring mobile AR completely into the mainstream. The interaction will be entirely natural, like reaching to shake someone’s hand.

The business model for Apps is discussed all the time. Is only ads the answer? E.g. virtual goods are getting more and more popular. Can you imagine integrating these additional bonus services, and items into your product? What do you think about this virtual good development?

I think there are numerous compelling business models for apps including ads and virtual goods. I think how these are implemented is the critical question. But as mobile AR innovators we are prepared to invent new ways of applying these business models in our unique context. I expect to see lots of innovative business models coming out of Tagwhat.

Regarding business models: will it completely stay social networking based or do you consider cooperations from companies like coffee shops or restaurant chains to add special channels?

Tagwhat has existing channels like: Wikipedia, Eat, Drink, and Foursquare, and fully intends to integrate much more compelling content from companies and organizations in the near future. Some folks at IBM are already using Tagwhat to feature some of their work. Other companies and universities are as well. Tagwhat is about information delivery in a very compelling way. Lots of content is coming.

In general, can you project the social implications by everday-AR? When is the technology and the user ready? When do people dont ask for an off-switch anymore? ;-)

We know that mobile is about to explode. As this happens the uses of mobile AR will become extremely apparent and tools like Tagwhat will become the social medium of the future. Once hardware becomes better suited to AR, there will be any turning back. People will never want to turn their AR off because they won’t want to miss the great content and opportunity all around them.

Thanks a lot, Dave.

Tobias, thanks a lot for your interest and the interview.

Dear Reader,

I’ve been quite offline lately due to the preparations for our own conference, that took place on May 6th+7th in Munich. It was a great show with two days and two floors packed with live demos and interesting talks. (Luckily Pixar didn’t claim that Stereoscopic3D is everything, but rather a small piece in making a great movie. But that’s what I expected. ;-))

We showed lots of demos and also a couple of augmented and mixed reality demos. The most simple (but still great) demo I wanted to share with you today, so that you can see the quality we get with an off-the-shelf laptop and a webcam. Spiced up with some catchy tune behind. ;-) Enjoy!

The car for instance holds about 1.000.000 polygons with dozens of shaders and was rendered with 30 fps. The laptop was a regluar Dell Latitude and the usual suspect camera (Logitech Pro9000).

As soon as the press material from the film crew is ready I hope I can show you more videos, e.g.:

• Photoshooting Set Previsualization
  The second demo I hosted is intended to be used for a photographer or film crew that needs to shoot backplates (background images of landscapes, streets, …) for commercial productions. This process is currently quite annoying for the creative guys since in 99% of the cases they won’t have a physical car (or whatever product to photograph) on site. This makes planning a shot pretty hard. The integration of the CG car will only happen later in the postproduction. With AR we can now give the live preview to the crew on location and automatically import SLR images or HD feeds and export the camera angles for postproduction.
• Virtual Photostudio
  The third AR/MR demo is a mixed reality application to set up a virtual photostudio. Markers can be touched physically and represent light sources, that can easily be re-arranged to light the object of interest without the need of touching complicated GUIs of any rendering tool. The demo makes use of full raytracing to get the light and reflections right. So, stay tuned for a video update.

So, but that’s enough talk on the stuff from RTT. Now let’s enjoy the short holiday (at least in Germany) and be back next week! To pass the time be sure to:

check out the augmented reality racing game from TU München and

take a look Project Natal’s competitor Omek InterActive with their body controls. They offer the same full body motion tracking with a depth-sensing camera and process it in real-time. There will be a SDK, so that you and me as a developer can access the system and hook it up to your specific software.

After a great 2nd regluars’ table on AR in Munich yesterday (thanks to everybody who made it!), I’m off to set up the demos to be seen from tomorrow on at the open house fair and conference in Munich - hosted by my company Realtime Technology (RTT AG).

It will be a two-day event, showing our latest technology and with great talks from Industry leaders, e.g. we have Bob Whitehill from Pixar talking about Stereoscopic 3-Dimensional Storytelling.

There will be two demos on Augmented Reality and one on Mixed Reality, which I am hosting. So drop by in the marketing lounge, if you are an attendee or wait for the reports floating in afterwards.

See you on the web next week!

- Toby

The Dutch Government started an Augmented Reality billboard campaign to raise people’s awareness on crime fighting, moral courage and on how you should act when encountering such a scenario.

A digital billboard is put up in Amsterdam and Rotterdam showing a live feed from the current spot. You can see yourself and wave into the camera/screen. Unfortunately that’s not all what’s going on: a crime/violent scene gets add to the live feed. But you can only watch - the AR is rendering you powerless to act. Even if you wanted you can’t intervene and help. You can only stare or turn away.

I think it’s a great idea to confront people with street violence problems. It ends in educating the passers-by on how they should react in real life situations. All these twisted heads and staring eyes in the video confirm to me that it definitely had an impact. Hopefully not only making people think about the technology (pre-recorded scene on a blue screen, live compositing with the video feed), but also about the social behavior and the role they were forced into and how their behavor would (or wouldn’t) differ if the scene were real… The billboard eventually switches to a short behavior code (Ask others for help, call 911, stay with the victim, …) to conclude.

With this technology you - today - get realistic augmentations and this demo is one of the very few that gives you really the feeling of being there and (almost) not being able to tell the difference between real and virtual. (In contrary to all these low-poly flat-shaded models…)

Take a look!

First of all, thanks to Dan Romescu and Willi Schroll for making this first AR business conference happen. Thanks to you and all your helpers we had a great day in Berlin with many great talks, many AR people in persona and cool demos. The setup time was the shortest I’ve ever seen for a conference and the result for an eight week preparation time was really impressive.

Summing up the conference in a few sentences is close to impossible, so I will need more lines. There were three interesting tracks of talks with all major players and start-ups, where it was actually pretty hard to decide where to go. Nice and shiny Ludwig-Erhard-Haus in Berlin, totally central, was a great location for the first summit. Maybe they have to expand next year….? ;-) The focus was clearly on the business behind AR, but no business without understanding the technology involved. So, we had quite a few videos and live demos during the talks to get our brains and fantasy started. Focus was clearly on marketing, advertisement and location based extensions for the mobile world. A bit lacking the industrial B2B area. But, that was ok. Let’s get started in a sorted way:

Robert Rice had the keynote speech at the beginning, although the volcano with the unpronounceable name kept him in NY, but with modern computer technology no problem. :) He did a great introduction and made some clear statements on the business in AR. The main issue is boiled down to: we can’t continue abusing AR for ads only. The story and the user experience is more important. AR is only a technology. There is no further use of AR only because of AR. (I totally agree.) We need to focus on the idea behind it and don’t overwhelm the user with too many information (rather adjust to a user’s choice and a push/pull-mode). He underlined not to overhype it and think about the long term impact. Gimmicks and novelty wow-factors will fail after a while. Business needs to just use the technologies to offer new values. We need to think ahead in a 5-10 years frame, rather than yak about another Flash-based ad (they are still fun, of course!). After the marketing+ads area we will enter the entertainment realm, continue to education and medicine.

Total-Immersion’s CEO Eric Gehl shares this vision of entertainment being first and also underlines that AR is now considered a solution and not longer recognized as a technology. Furthermore AR will actually save and support “older” analogue businesses to the computer age by adding value. The digital transition will offer great new possibilities and connect the users. In his eyes, the next step after web-based and mobile AR will head towards our living room and our TV sets (I can relate again).

Two presentations on gaming caught my attention: aka aki and gbanga. These talks stuck to my head quite long afterwards. On the one hand gaming is always fun and compelling, but then they made pretty good points: Matthias Sala from Gbanga reminded us to get the people engaged in the story - not in the technology. After a wow-factor we need good stories that can make the game or the app more interesting than reality. Pixels don’t matter. It’s the immersion. Look at the C64, look at the Wii… graphically not up-to-date (don’t sue me, Nintendo! I love your console!), but still great games where you can loose yourself and the computer remains as a trigger for your imagination. Gbanga, but also aka aki focus on this experience. Aka aki has some great location based games in their pipeline, augmenting the virtual world not (only) by a video feed, but rather by other cool modes: if it rains in the real world, it rains in the game, etc. I just love these ideas. Clear statement on business models by the way: besides ads, we will enter the time of virtual goods. To guys, who ebay-shop WoW-items, this might not be anything new. But this business will explode. See people showing off with their virtual farms today! A non-haptical virtual good remains a fetish for the user and connected (again) with a great story gives the added value, people are looking (and paying) for. I’ll spare you the revenue numbers here. But they are getting huge. :p

Talking about virtual good and business, almost each presentation - of course - had the business part and was stressing the way to make money with the ideas. Overall focus was clearly on marketing, ads and virtual goods. Location based ads will have bigger impact. Mark Miller from CatCap points out that we are still in the accending part of the hype cycle with AR and that it just remains a tool for our cool new ideas. E.g. the transition from black and white TV to color TV broadcasts was no investor’s decision: it just happens naturally. We need to make use of the new possibilities, but AR is not another product per se. David Blumberg from Blumberg Capital adds, that we need better (G)UIs to control the AR and that we must offer CGI quality, the customers know from computer games and movie productions. He’s aware of the timing and gives it another 2 to 3 years.

So, whew. Still can’t sum it up here all. To give you some key information on some cool demos, I’ll go with a list:

• A must see is the work from Alex Olwal, who showed some great demos during his speech. The most impressive work for me was the project involving spatial AR, projected on a transparent screen in auto-stereoscopic 3D, called “ASTOR”.
• Fjord gave us some insight on how we should design AR interfaces to get a better user experience. Important besides the GUI: don’t always focus on the visuals. AR also means audio augmentation! We could sometimes have a much bigger impact or more comfortable product when using ear buds for augmentations (e.g. pedestrian’s city guide).
• Adobe showed a lot of cool Flash/Air-based AR and explained the workflow in designing an application within 5 minutes.
• Fraunhofer FIT showed their social networking AR research, including the neat TimeWarp project. Experience the city of Cologne in different times and edutain yourself with the story of the funny helping cobolds or elves, the Cologne Heinzelmännchen. With a little bit of a better hardware in smartphones I’m sure we will get a cool game or app pretty soon. Fraunhofer is on to it!
• During their talk, TIC Mobile pointed me to the Samsung BADA smartphone. Could be a great developer’s choice for AR. Video stream access and yet-included face recognition API and other computer vision items in stock.
• Another nice live demo was shown by Tysiacslow, showing an AR gesture interface with two webcams (one IR, one RGB):

  Technical Preview v2 from tysiacSlow on Vimeo.

So, to sum it up: a great conference, which much more information, that is impossible to share in a blog. Great talks, meeting the people behind AR and a nice beer in punky Berlin to end the night in X-Berg. For me, it was a bit missing the B2B cases and industrial applications and was too much focusing on the mobile, marketing sector. A few more live demos would have been great (but with so little time and volcanos around it’s hard), but still: an awesome first conference.

Some more pix can be found here and some more here from Jan.

What did you like most at ARBCon? Do you have some further pictures or videos? Please share in the comments or on my facebook page. :-) Hopefully I will update here with some videos later on.

PS. Due to schedule conflicts I finally couldn’t make it to the ARDevCamp. Next time! (Munich Regulars’ Table due soon!)

The AR Business Conference is starting this Friday. I’ll be there in Berlin, too. So drop me a line, if you feel like chatting and discussing and don’t bump into me by accident. I’ll be blogging on the technology, the presentations and talks and also visit the ARDevCamp on Saturday. Look for the tall guy with the marker T-Shirt there. ;-) Looking forward to meeting you and all the AR folks in person again. I won’t be microblogging or twittering (you can never squeeze meaningful reviews into 160 ASCII), but expect a longer post this weekend. :-)

Cheers!

… and for the meantime: if you are looking for computer generated haptics, maybe to finally feel and touch your augmented house, car or boat for real: check out the biggest 3D printer so far.