augmented.org

Last weeks time I used extensively to go through this year’s Siggraph papers and posters. Again, great news and ideas from our fellow researchers. Today I’d like to share two of them that are related to AR and happen to have a public available videos online for all of us to enjoy.

SonalShooter

Ken Nakagaki and Yasuaki Kakehi present their sonic shooter at the conference. The title is “SonalShooter: A Spatial Augmented Reality System Using Handheld Directional Speaker with Camera”. It shows once more that AR does not only refer to visual overlays or feedback, but can also mean augmented audio. Their gun-shaped device lets users point onto objects of interest (aided with laser pointer, recognized through markers) and the gun uses directional speakers to implant the audio output into the object itself, i.e. you recognize the sound coming from that source and not from the gun!

Of course it’s not really possible to understand the concept listening to the video, but it explains the approach and technology pretty good:

SonalShooter from xlab on Vimeo.

I especially like the interaction idea of switching channels through rotation of the device (i.e. for selecting other languages).

Kinect Fusion

The inventors of the Kinect themselves (Microsoft research) have presented a great paper on 3D real-time scanning of rooms, objects and living things. You would say: yeah, well, that’s what the Kinect does, right? But this is far more advanced. Using the depth information through the IR pattern, Microsoft can create high-quality 3D models. “The system allows the user to scan a whole room and its contents within seconds. As the space is explored, new views of the scene and objects are revealed and these are fused into a single 3D model. The system continually tracks the 6DOF pose of the camera and rapidly builds a volumetric representation of arbitrary scenes.”

So, to make it easy, have a look below:

I’d say it shows us the future of devices to come, imagine having this in your AR goggles embedded: helping the cloud-based representation of the 3D world and allowing far better interaction of virtual and real objects. The video shows this on two Augmented Reality examples: first, the splash demo, that uses real-time physics to spill the virtual color over the real objects and secondly the possibility to paint onto real surfaces with your bare finger! Awesome!

ISMAR 2011

Besides the Siggraph, we sure are all waiting for at least two more conferences this year (insideAR from metaio in September) and the biggest show: the ISMAR 2011, this year to be held in Basel, Switzerland. By the way: I’ll be on location all days, blogging live from this year’s ISMAR. I hope we can meet up! Show me your inventions! :-)

To have a sneak peek on this year’s papers, I selected a paper on 3D reconstruction as well (as seen above from Microsoft) – but this time running on a mobile device! It’s called “Rapid Scene Reconstruction on Mobile Phones from Panoramic Images”:

By using a very fast and flexible algorithm a set of panoramic images is captured to form the basis of wide field-of-view images required for reliable and robust reconstruction. A cheap on-line space carving approach based on Delaunay triangulation is employed to obtain dense, polygonal, textured representations. The use of an intuitive method to capture these images, as well as the efficiency of the reconstruction approach allows for an application on recent mobile phone hardware, giving visually pleasing results almost instantly.

The system shows us where we are going and what we can expect during the next years! Calculation of the 3D geometry takes 18 seconds on the mobile device itself. We sure will see this number falling.

In other news…

Rouli found this Denno Coil-like future vision, showing us again how cool AR could be with the perfect blend of realities. Damn you science, that you develop still so slow for one measly human lifetime! Here we go:

AR Browsers News

The deployment of AR browsers is still growing and we see more and more fun and helpful features around. The guys at FARRAGO (formerly MixAR) have their first app out, giving us more social-linked AR fun to enjoy. ipad app will follow soon, too!

Staying with mobile AR browsers, we have a big update coming from metaio, too. Their browser junaio has undergone a major update (now 3.0), giving us their tagged “scan the world” functionality. It allows us to scan anything from pictures to bar codes or QR codes and get information about it from Junaio’s channels. But the information has to come from a junaio channel/metaio partner! It makes sense, but does not give us the google goggles approach of being able to scan everything and do an image search on it. (I couldn’t try it out yet, will do this weekend. So bare with me, hope I didn’t get this wrong.) Anyway, here we go with their announcement (lots of videos today, yeah!):

… and to close with HMDs!

To more pieces of news on HMDs for the AR crowd out there. Brother seems to be ready to commercialize their Airscouter HMD! Can’t wait!

… and Sony comes with a new device with HD support for both eyes and OLED technology. It’s not directly equipped for AR (no see-through or camera), but it sure looks promising, that the big players keep investing here.

Enjoy the week!

- Toby.

Tanagram has developed a concept for improved HUD systems for fire fighters. It’s the best concept video for a real life professional use case in a long time, I wanted to share with you today:

The video is titled “The future of firefighting – A HMD-AR UI concept for first responders” and uses HMD setups for firefighters to display additional information and allow for natural and quick interaction to have needed information accessible as fast as possible. It sure reminds us of other concepts and movies, but brings it together to a great scenario, that looks pretty damn feasible and useful:

In detail they describe the concept being developed for the Department of Homeland Security to explore “methods to help our firefighters leverage information displays to help them stay alive and help others”. “Tanagram, under a grant provided by the Department of Homeland Security developed a phase zero Self-contained breathing apparatus (SCBA) enhancement that leverages HMD / AR technology to display critical factors all-the-while not obscuring the firefighter’s field of view.”

They underline that it is only a first concept and more work will go into it to refine the interface, making it faster and safer to use for the firefighter. But already today you can imagine how AR can help out in scenarios, where additional information can make the difference among life and death and a split second is all it takes. The hardware and tracking development may not there yet, but it once more shows the potential of AR.

I guess it’s gonna be pretty tricky and take lots of research to have all the material fireproof for this extreme environment and to have a 100% reliability of the system (or to have an easy way out, i.e. not that due to a crash a blanked screen blocks the vision). I wonder, how much data access the firefighter really needs during a mission: it feels like a lot of information already, giving him the possibility to browse through the directory of partner views or mission maps. The fastest way – for such a extreme environment at least – seems to be having a mission commander, that coordinates and gives audio commands. Can’t wait to see more prototypes of this and even get some more feedback from user tests one day, exploring this balance and especially the way of presenting the data in a lean way to help the most.

Maybe they team up with these guys of close combat training to test their HMD setups in training scenarios. :-)

Hi fellow AR-fans!

Augmented City

Today I want to kick off with the latest video that just went online from our friends at metaio: “The Augmented City”.

Their vision aims at a fully CG/AR enriched city with perfect alignment of virtual elements to the real world. Well, of course, we all hope for that technology, but metaio is doing a pretty good job in getting the technology base on the road. They intentionally focus on their strong part of software (tracking and framework) and team up with strong content and hardware partners. So, we get closer to the demo scenario, that sets up a complete experience in a 1:1 scaled AR City like its dreamt up in so many movies or animations (like Denno Coil).

For now, we get a miniature version, showing the current technology already in action: 3D feature tracking and an improved mobile renderer. Can’t wait to see it live at their insideAR conference in 6 weeks, where they focus on telling us how the vision now gets on the road(map)!

What’s the fastest smartphone I can get until 26th? :-)

In other news…

• 13thlab has build up a neat ipad2 demo game, making also use of 3D tracking algorithms (i.e. here: SLAM) to get a better visual overlay than GPS & Galileo ever could.
• Gamesalfresco already covered it, but to mention it here is worthwhile, too: Layar gets a vision based extension, called layar vision:

  Layar Vision is an extension of the Layar platform, taking augmented reality to the next level. Layar Vision allows the creation of layers and applications that recognize real world objects and display digital experiences on top of them.

  Layar Vision uses detection, tracking and computer vision techniques to augment objects in the physical world. It can tell which objects in the real world are augmented because the visual fingerprints of the objects are preloaded into the application based on the user’s layer selection.

  Android Beta version is coming in a couple of weeks. iPhone and Android final versions should be there by the end of Q3 2011.

• Mark Skwarek has started a new project, called “The Augmented Reality Korean Unification Project“, tagged Uniting Korea with Augmented Reality to support the peace process in Korea, showing the border areas how they should be and were. No weapons, no demilitarized zone, no checkpoints, but trees and former landscape. To quote Mark: “This project tries to help the peace process by letting the Korean people see a unified Korea. This vision hopes to strengthen the people of Korea’s resolve for a peaceful unification.”

  The project is being developed by Mark Skwarek and uses erasAR technology. Mark will be travelling through Korea during August, so we will get some live footage soon. Today he shows us a teaser and demonstrator of the project.

• Besides, in another city, that has been luckily reunited quite a while ago, we have the elections coming up, bringing us augmented election posters in the streets of Berlin: http://youtu.be/lNvEGr0LuOk
• Wondering, how museums could save the money for traveling huge pieces of art around the world? Wondering how the museum would then look like if someone turned off the grid? Take a look at the Tallin Gate[way] projects! :-)
• … and to finish for today, we get another dressing room AR+Kinect+physics demo called “Fitnect” after last week’s post…
• … and a pretty scary pregnancy support with AR fetus… Maybe it’s just me, or maybe it is kind of… creepy. :-)

Nevertheless: enjoy the week!