augmented.org

Hey everyone!

Time for an all-inclusive update today, giving a wider overview of news for you to enjoy the AR weekend! Let’s start right away with a fancy picture and a new tech demo:

Interactive car windows

GM and the Future Lab joined forces to bring us a nice prototype of the augmented-car-window scenario. I’ll just have this one for itself:

Designing Real World Collaboratories with Christopher Stapleton

Christopher Stapleton gave a guest article on medicalaugmentedreality to talk about AR, medicine and innovating the innovation. Please check it out and join the open questions he gave at the end of part 1.

Hardware News

• Samsung shows their great augmented fridge glass, that makes me think of thousand other uses than displaying cooking information or the latest tweet. The (actually working) digital blinds are an awesome step. With pixel-by-pixel access for transparency switches we can have a perfect occlusion with augmented objects in see-through glasses!

• BASF and Philips show us a way to create OLED-based windows. As in the first video and the second, we get another chance to see, where this hardware is heading. A great enabler for AR to have full control over surfaces to get towards some kind of translucent surface-based AR (unlike projected or handheld-device-dependent).

• Vuzix continues to push HMD- or AR goggles-development forward, so that we can enjoy lightweight sun shades design goggles for AR. The new SMART glasses presented at CES 2012 are said to be launched for industry and consumers alike this summer. Their press release tells us a bit more about the setup, e.g. they are fully useable outdoors, are see-through and give us optionally cameras and gyros for tracking functionality. Engadget showed us already the monocole version at CES.

Software, Apps and Games

• Qualcomm unveiled at CES, that their AR platform will now be called “Vuforia”. So, don’t get confused when downloading the latest SDK Version 1.5 (Beta 1) under a new name. :-)

• The PlayStation guys have some new demos for us for their PS VITA augmented reality scenarios. Adding to the last video from December, we now get to see their soccer game presented fresh at CES.

• Besides… if you are willing to try out some stunts, without risking your live, check out “Ken Block vs you” from imagination.

• …or use browsAR to qr-code scan your shirt, redirecting your friends or not-yet-friends directly to your facebook site. Privacy is a thing of the past anyway! :-)

• …or let’s play Ogmento’s King of the Court challenge. Shortly described, it says: “NBA: King of the Court is a free to play, Location-based, Augmented Reality game for both iPhone and Android smart phones. In NBA: King of the Court , players will compete to become the “King” of virtual basketball courts placed in real-world locations around the world.”

I like the idea of combining location based “check-in” fun like foursquare with location based gaming experiences. It’s just so much more fun to be standing at a bus stop and challenging the other real people, that played the same micro game at the same spot before. Only checking into locations and becoming the major through pure physical presence… well… is a bit dull on the long run.

• … or play the aaaarrrrrrrr! AR Pirates game! A 3 coin marker system that lets you set up a pirate ship basically anywhere. I like the game played at a lake, where the ships really appear in the water. Battle your friends!

Upcoming Fairs

If you happen to be in or around Monaco next week, don’t miss the IMAGINA fair. The fair on 3D Simulation and Virtual Technology also has a special augmented reality event, with Nokia, T-Immersion and Qualcomm talking about mobile AR. Be sure to attend!

Cheers!

Hey guys,

busy days. I’m sorry for the delays. But now it’s time to pick up the loose ends and to take a good look around into 2012 and upcoming technology. While I meditate a few more hours, let me kick off with a beautiful projected augmented reality demonstration I missed to blog last year. It just came across my browser again. So, a good epic start to follow up with! :-)

Augmenting the Sydney Opera house outside and inside makes this one special. It just blows you away. The creative guys from Obscura Digital have done more of that kind lately (like the Sheikh Zayed Grand Mosque Projections), but I’ll stick with this orchestral kick-off!

Visual Design and Projections by Obscura Digital for YTSO 2011 from Obscura Digital on Vimeo.

The second video shows a bit of behind-the-scenes. Enjoy! :-) More to come again these days! It’s time!

Hey everbody,

happy new year and welcome back for more Augmented Reality fun in 2012! I’m sure we can expect more insane technology updates through the year. Major new gaming console updates to be expected – they will bring some AR news along for shizzle. Also tinier and following Moore more powerful smartphones, new tablets (ipad3 in a month?), and more technology fairs to show off the latest gizmos. To kick off the year, I’ll directly quote some news, shown at the Consumer Electronics Show 2012, that started today.

Anti-Couch-Potatoe

To start with the big player, we have Microsoft, announcing their Kinect SDK to be available for PCs officially (and as a special flavor of the SDK) starting on February, 1st. The adjusted device will cost $250 and is already said to be slashed down to $150 later 2012.

Meanwhile at CES, Microsoft shows their latest endeavors with the Kinect, using it for television programs. They show us their concept of how an interactive telly show could look like, even further forcing us to get up and not only use a remote to zap or play (thanks again, Nintendo! ;-)). Take a look at their combination of Sesame Street with the couch live view, nicely depth-keyed with the Kinect: in the video starting at 4:40. Still, this is just a ready-made concept. It’s not defined, how multiple viewers and actors at home could interact with an at-this-time broadcast show. Will Elmo be hit with 50000 coconuts from all kids?

Flying Saucer

Lot of attraction at CES is also drawn to the known AR DRONE. Now released in their 2nd version. The drone now comes with 720p HD video signals, floating to your iDevice. They show a well-done marketing video with their flying saucer, that makes me scream: I WANT TO FLY! The operation UI is always shown on an ipad, making it possible to see through the eyes of the drone directly (as before), but now with HD and easier navigation: it’s calculating relative steering information depending on the distance/direction between drone and player.
Though range and battery life are sure too be more limited than suggested. Nevertheless big fun!

See also different hands-on videos, popping up from CES. Let’s take this one, showing a bit more live streaming.

Fitting to the release they also have a new single player gaming app, using markers to be put on the ground, letting you kill those aliens once and for all! See AR.Rescue video!

So much to kick off. Let’s now augment 2012!!

PS. Munich “AR Stammtisch” again tonight!

Hey everybody!

It has been a while. I hope you enjoyed the ISMAR, its coverage, the pictures (all my pics are now available through ISMAR website) and the time beyond to digest all the cool impressions, ideas and new tech stuff! Being very busy the last weeks, I’d like to close for 2011 with awesome projected AR and some more news pieces! Enjoy!

Immersive Imaging

Coolest videos first! The guys from PlayStation have created a great show to promote their Great Films fill rooms site. A stunning combination of projected AR with a tracked moving steadicam and real props (floating through space). Look close to tell the difference between real and virtual! ;-) Holo deck or matrix… it’s up to you to decide. Enjoy the ride and be sure to follow up with part 2 and 3!

2nd part, 3rd part.

To overcome a fixed observer’s (=camera) position, they attached multiple PlayStation move controller to the camera to track the point of view and to readjust the projected image to keep the illusion. The spatial deformation happens in real-time using multiple DLPs, in-house tools, open Frameworks, OpenCV and other (unnamed) open-source software, rendering with Unity. Anyway, the details don’t matter. The result combining projected imagery with props on wires and prop-ninjas is an awesome techy eyecatcher! :-)

AR Browser World

Wikitude released their brand new developer toolkit to create your AR content. To show us their new tool called “ARchitect” they put up a video and interview, explaning the news to us. The complete written interview with the details was kindly provided to me by the wikitude team. You can download it here. For now you will need an Android phone (ver 2.2) to try it out:

At a similar point in time, metaio now released their completely free mobile SDK. As mentioned before (and announced during their insideAR conference) we can now make us of their toolset to create our own content free of charge (being limited by the feature set or by seeing watermarks). They already include their freshly presented Gravity Aware Feature Descriptors for better mobile tracking and are able to make use of Unity3D for 3D gaming needs.

… and now a happy holiday!

Finally, it’s time for Christmas and green, decorated trees. This year my office is blessed with a real tree, but if you want to take your green decoration into your bath tub or tie it to your dog, you now have almost unlimited possibilities with the nice yuletree app. Just got here in time for the holiday season!

More Christmas AR fun comes from Orange, making you dance with augmented Kings or Santas in your Malls (well, in Paris or Marseille at least). Or otherwise you are already taking part in the present scavenger hunt with your mobile gadget with the junaio AR X-Mas Special as well!

So, enjoy and merry Christmas! Stay tuned for 2012 and have a blast!

Cheers,
TOBY.

As a last demo at this year’s ISMAR I had the pleasure to meet Jan Torpus and to experience his LifeClipper3 project live. After the ISMAR started with visionary concept ideas and thinking about all senses, everyone got lost or concentrated on more daily issues like tracking, rendering or defined use cases and minor technical improvements. After a long conference I was pretty stressed out and nervous with so much input of technological considerations, I had to pull myself together to actually go to the final scheduled presentation… turns out it was the best AR experience I had during the whole conference!

I arrived at the St. Johanns Park in Basel and found myself putting on a heavy-weight backpack full of sensors and a HMD visor with video cameras, head phones, etc. With this weight I felt even worse than holding around my smartphone pointing to AR spots. An immersion seemed close to impossible.

Explaining the bold facts, LifeClipper is an outdoor AR experience, where you carry around all hardware and have an inside-out sensor-based location tracking. An advanced GPS-sensor gives your position, inertia sensors and compass give your rotation and you experience the combination of real and virtual through a stereoscopic HMD with two cameras and stereo headphones. The cameras are not used for tracking, though. You move inside a park area in Basel. This park areas has been completely scanned and remodelled in 3D space to make virtual object placement and interaction of virtual and real possible. Then I entered the mixed reality world…

A great mixed reality world, you dip in from your physical space. The best: you don’t even have to stay inside a lab or office! Going into details of the video: the different areas of the park not only result in different zones or climates with another part of the story being told and other 3D aliens or augmented objects shown. But also your perception of reality changes! You see those alien bugs, you see virtual trees, a desert or a LSD sky during a virtual night – changing the video backdrop to a dark scenario. You feel in a completely different space, you can’t tell the real trees from the virtual ones (the alternation of the reality-video-feed is the crucial part here), you move slower than usual due to aggressive deep sounds hovering above and around you. Lastly, you even leave your physical body and see the park from above floating over it (in a foggy virtual world)! This play of changing the perception of your personalized reality worked nicely. You get immersed in a true mixed reality… in the end you also switch into a 100% virtual world not noticing it – the real world is just an option. This whole design of the worlds and all the thought put into the perception of reality would be too much for this post to sum up. It is pretty interesting and can be read in the complete concept description.

Personally, I felt great during the experience. But not only. It was even uncomfortable walking closer to the big aliens hearing the dark noises, but I also felt lightweight happy, when I moved into the colorful night setting, where the real world was changed into a dreamy, positive place. When moving through virtual grass, that grew and shrinked according to my breath rythm, I felt the urge to divide it with my hands. When I moved closer to the neat alien bugs, I was afraid of stepping onto them. I almost smacked my head by walking into lanterns: the immersion worked not only visually, but emotionally! Or to rephrase: true immersion is only possible, if you are connected emotionally. It did work here completely.

The graphics didn’t really matter anymore. Abstraction versus realism didn’t matter anymore. Once I immersed, it was only about the story and the direct interaction in the real physical space. I adjusted my physical behaviour – forcing myself to move slower – and when discovering this embedded, hidden parallel universe, I immediately forgot the huge backpack I was carrying around with my for over 40 minutes! I entered my personalized reality, that I didn’t want to leave again. Not yet. This dream world was so engaging I forgot polygon counts or tracking instabilities. This is the true AR experience! I can’t wait for the day good AR goggles hit the market and an easy way is established for alternate reality game designers or every interested artist to create his/her “channels” of the mixed worlds! Time for location-bound alternation of reality!

To immerse, it totally helped to have a good combination of atmospheric audio, the visuals and interaction – actively or subtle. Without good sound it wouldn’t work. The HMD gave the ultimate feeling of being there, still leaving a bit of a dream-like feeling due to the offsets of my real hand positions to the virtual… This is one thing I’m hoping to see for future improvements: to get to interact with the virtual animals or objects through gestures. (As we could see in different ISMAR demos.) The physical connection was so important and helpful – it needs to be extended to include not only my feet, but also my hands! This physical space interaction of the real and virtual objects could also be further optimized by giving e.g. vibration feedback or giving altered virtual audio footsteps (overwriting real footsteps), etc. You could simulate a different ground (or support the moment of leaving the ground, floating). Or you could have a blindman’s stick with sensors to fake directly-world-connected feedback.

This personalized reality could possibly have even more impact when including information about the user, his friends, favorite music, etc. Artists and developers could define logics to use emotion triggers in a subtle way. For good and for bad. The personal reality within modern life could be a personal retreat or your meeting place for remotely located friends. Overall a great look into the future. Tracking and performance can easily be improved, but it was great to see, that these things didn’t even matter that much during the show. The great multi-sensoric approach and the fine mixed reality world design with simple alternations (like changing the sky color) make it work already!

It sure was a meditative escape while staying in the same physical space. Disconnecting while connecting. When I woke up, reality looked dull. I talked to Jan for a long time afterwards and when I finally left the location and passed by a spot with a park bench, where I have been during the LifeClipper-time, I thought: Haven’t I been here before? No, it must have been in a dream!

Today, I’d just want to follow up quickly (damn these days without time!) with two more demos from ISMAR 2011 in Basel. Next week I’ll conclude the demos here with a great mixed reality finish!

Augmenting Magnetic Field Lines for School Experiments

The German university of Bonn/Rhein/Sieg had the perfect educational augmented reality demo at ISMAR, where everybody just had to say: “yeah, that’s what AR is for and where AR can make the difference!”. Where metaio’s LEGO-demo is one of the best examples for industry/shops, this first demonstrator is the one for the field of education!

“We present a system for interactive magnetic field simulation in an AR-setup. The aim of this work is to investigate how AR technology can help to develop a better understanding of the concept of fields and field lines and their relationship to the magnetic forces in typical school experiments. The haptic feedback is provided by real magnets that are optically tracked. In a stereo video see-through head-mounted display, the magnets are augmented with the dynamically computed field lines.”

The magnetic field lines chance accordingly to the positions and interactions of the two (real) magnets. They are not fake, and so you even get haptic feedback (holding them) and force feedback (they actually attract and push off). It’s not really a huge success to overlay e.g. a 3D globe onto a 2D world map as a magic book (while the actual real-world globe is covered in dust in the back of the room)… but to show physical “invisible” behaviour in real-time in your view is a great interactive and immersed way of learning! I blogged on educational AR with good and real interaction between real and virtual objects before. It’s just so crucial to have a good physical interaction. This demo gives another great use case, so I just loved it and hope we will see further results from their research group!

smartAR

The second demo for today is about 3D real-time reconstruction and tracking. A coarse model of the real world was recreated using a mono-camera from a tablet device (during run-time). This way they could make the virtual object (the stuffed pink bear to say) interact with the real world (and even fall/hide behind the couch). :-) The reconstructed poly model was rather rough, but shows great possibilities to help AR with mixing the realities:

“In this demo, we will demonstrate our AR technology, named SmartAR, on mobile devices. SmartAR has two main features, object recognition capability and 3D space recognition capability which is based on a single camera SLAM, and is notably fast and robust enough for various AR applications on mobile devices such as smart phones, tablets and game consoles.”

While the minimal game (you could even interact with another pad-user moving around his or her pet animal) was just a quick demo to show the functionality, I think the reconstruction part will have another big impact and it’s a good proof to see, that it is already possible with mobile devices (not yet including structured light Kinect options or other more advanced approaches). Cool!

Have a great weekend!

Time for the 1st ISMAR demo summary! Since you can get a pretty good overview of all demos at ISMAR on their own page, it won’t make sense to quote all here, but to focus. The main interest lay in products using mobile planar tracking for ipads & alike, but also in advances in the reconstruction of the real world as polys, i.e. via the Kinect or even via monoscopic camera setups for mobile devices. Also social media and social tagging (via facial recognition) were hip. To show some, starting with two racing games:

AR Micromachines: An Augmented Reality Racing Game

Adrian Clark and Thammathip Piumsomboon from the University of Canterbury came up with this concept of AR Micromachines as a table top car racing game. Real world obstacles are being recognized in real-time via a Microsoft Kinect and combined with augmented reality graphics to create a race game. Users create their own tracks using real objects, placing coffee cups and books, and then race virtual cars head to head with realistic physics, occlusion and shadows:

Again it’s the real working concept of mixing the realities, that makes this demo so great. Another reason why the Kinect (and similar products) will have such a huge impact on further AR development. This interaction and real world collision is so crucial for a good experience! Using Kinects you no longer need stupid markers to define walls or floors (as I still had to in my old AR:Race demo ;-)). Great demo!

Sketch a race

Another neat ipad based racing demo comes from our blogger buddy from gamesalfresco – Ori inbar. Ogmento came with a sketch a race approach of creating your game world: Through the iPad camera, the game detects a hand drawn sketch on a paper (any sketch), and accurately overlays a race track on top – in real time.

Again, it’s the real world interaction I like most. The winning combination of playful real world actions to yield in a truely augmented experience. Just putting a marker there, would again only end in the same boring (well, not yet, but one day…) experience of “oops, there is something on a card, that wasn’t there before… well… ok”. Problem with these demos is often, that the real world is just another background for your game. It could most of the time be a better virtual-only game. If you chose to have the real world as an optional component, you have to make proper use of it and integrate it. That is what really happens here, when drawing the tracks! :-) Thanks, Ori.

AR Coloring Book

The coloring book demo from the HITLab and Adrian Clark and Andreas Dünser (University of Canterbury) shows the same great understanding of true AR, lets quote: users are able to color in the pages, and these pages are then recognized by the system and used to produce three dimensional scenes and textured models reflecting the artwork created by the users. This three dimensional virtual content is then overlaid on the real book pages, providing a three dimensional experience using the users own content.

It’s just such a great combination for kids to have 100T% haptics and “traditional” media to be creative and then just flick a button to bring those drawings to live in 3D! Same critic as above: just great! :-)

Audio Augmented Reality

Touching another area of AR, I’d like to shortly mention the demo called “ARA Indoor-Outoor Navigation for Pedestrians” from Mathieu Razafihamazo, Yohan Lasorsa, David Liodenot, Audrey Colbrant (INRIA), Jacques Lemordant (UJF-INRIA-LIG):

We have build a high precision (at a level of a step) navigation system using path integration, 3D audio cues, and structured environment representation which are probably the navigation aids used implicitly by visually impaired people.

Audio guides via sound-based AR are just to seldomly spread. The accuracy was amazing and I hope we soon get a google maps plug-in, audio-overlaying our smartphone mp3 tracks during the day…

BurnAR: Feel the Heat

One more… right when entering the demo room, you bumped into a black wall with markers and people waving their hands with a bulky HMD. Intrigued? As you get closer and get a short introduction from the guys from the University of South Australia and Graz (with Fairlight):

In this demo, a user will experience their own hands interacting with complex graphics simulating smoke and fire effects in the environment. A user will look through a stereo head-worn display (HWD) at their own hands which will start to smoke and interact with flames. A real-time fluid simulation running calculates the volumetric effects using the users hand as input for motion and interaction surface. The hands’ location and depth will be estimated from the stereo view delivered by the HWD’s camera pair. Overall, the immersive experience of the user’s own body interacting with the striking, high-quality graphics effects will create an exciting demo.

I have to underline this part of exciting. This – just being a demonstrator and no product – already clearly shows what is important to give an immersion within the real world: a close relation to the user (within-reach-of-your-hand augmentation) and a plausible physical behaviour. The flames going up and reacting to hands (skin-tones) only, make it clearly a winner’s demo! … and so they won the demo award! Congratulations to all involved, named at ISMAR page: Matt Swoboda (Fairlight), Thanh Nguyen (University of South Australia, Graz University of Technology), Ulrich Eck (University of South Australia), Gerhard Reitmayr, Stefan Hauswiesner, Rene Ranftl (Graz University of Technology), Christian Sandor (University of South Australia).

Thanks to all! To be continued… :-)

Keeping the visions from the keynotes in mind, we today go to the other extreme of the ISMAR conference: hardcore algorithms in the tracking competition. This is far away from visionary outlooks, but rather a necessity to create good visual overlays. We go way down to the bottom of the code, tweaking the algorithms and implementing all into the tiny mobile boxes, we carry around today.

So, what is it all about? you might ask, if you haven’t been there or even if you have been there. Let me introduce you briefly to this hardcore IT tech topic, not going to algorithmic level though! :-)

The tracking competition at ISMAR already has quite a history and was this year set up by the nice guys from TU Munich ResearchAR’s. Obviously the best team gets the prize, while each team tries to complete as many tracking tasks as possible.

But how does it work? Basically you as a team get a bunch of 3-D coordinates and you have to tell the judges to which real world object these coordinates belongs.

Let’s make an example: one coordinate lies somewhere around the mirror, i.e. one of the marbles is the target to be identified. You have to have an accurate tracking and visualization to pinpoint the correct item. Then, tell the judges which one it is and you score! Another task was to tell which candy is the chosen one, lying on a colorful table full of tiny candy bars. If your tracking is not well calibrated or accurate enough, you can only pick the right sugarcane by accident…

To achieve this goal of identifying the 14 items during each team’s 30 minutes period, they had the chance of setting up their tracking environment before the tournament (given 80 min. each): small fiducial markers on the walls of the room with known coordinates where there to help you, defining the coordinate system of your world. All markers define a single system and once you register within this system and your tracking is stable you should be able to fly to the unknown coordinates…

Now let’s introduce (a bit more techy) the teams and their approaches:

Team 1: FAR, Technische Universität München, Germany
Frieder Pankratz, et.al.

The TU Munich – as the organizers – took part non-competitively. They set up a fusion of an optical SLAM (Simultaneous Localization and Mapping) based approach using a camera on a mobile device and a smart-phone based optical outside-in tracking system.

Team 2: Millennium Three Engineering, Canada

Mark Fiala’s team used fiducial marker tracking throughout the tracking environment using a partially offline 3D registration process and online pose filtering and fusion with an inertial measurement unit. You have possibly seen their work yielding in the ARTAG library.

Team 3: VTT Technical Research Centre of Finland and Aalto University – School of Science, Finland with Alain Boyer, Petri Honkamaa, Tuomas Kantonen, Otto Korkalo, Timo Tossavainen

A client-server framework based tracking setup using iterative feature-based tracking and target visualization on the mobile client side and pose initialization and recovery on the server side. The only team using a server-based approach. All teams used smartphones nevertheless. (Though it was not mandatory.)

Team 4: Dept. of Virtual and Augmented Reality, Fraunhofer IGD, Germany
Harald Wuest, Mario Becker, Folker Wientapper

Streaming tracking system with mobile phone (image acquisition and visualization) and laptop (tracking and rendering). Optical flow based (KLT) feature tracking (frame-by-frame) using a SLAM approach for geometry acquisition.

Team 5: metaio GmbH, Germany, Thomas Olszamowski

Junaio framework featuring visual 3D markerless tracking running exclusively on a mobile device with support by built-in inertial sensors. Offline learning of an environment map with a marker-based approach which is extended online using a SLAM framework.

Competition

During the competition itself the teams were not allowed to keep any markers around inside the room (markers are still set up in the first picture and were removed during official run). All tracking had to be done visually through a RGB mono camera. Given this fact, you can imagine how tricky things can get trying to get a hold on 2D features if there is no textured floor, just reflections (due to the mirror) or way too many tiny features (candy). Low light condidition tasks, transparent plexi-glass and reflective, shiny, flowing car surfaces without edges made it even worse for the teams!

While the teams were competing live one after another, you could cut the air with a knife where the crowd was observing the final match! IGD – the winner of 2009 – didn’t handle a few tasks and VTT ran into problems losing valuable time for most targets… metaio did a good job using their very own junaio channel for the competition. Take a closer look at the tensed situation in the audience and at the battle between the two leaders metaio and IGD on flickr:

And the winner is…

… metaio! :-)

The Future

A great and exiting competition! Congratulations to the winners and lets expect some even more complicated tracking tasks next year! E.g. a rotating Volkswagen scale model…

So, enjoy the tracking and I’ll get back to you with a report on ISMAR-demos next days!

To start my reports I want to follow the concept of ISMAR, giving visionary outlooks each morning with the keynotes. Three great speakers had the chance to show their ideas, inspire us and remind us of things we might forget in everyday implementational life!

Oliver Grau started on Thursday approaching the question, if we ever will become used to immersion. Consequently, he shed much light on the history of art and of course especially on the history of illusion and immersion. He claims that we cannot have image competence in the 21st century if we don’t look back and understand how immersion was tried to be generated by humans’ work for centuries. He goes way back to classic antiquity and shows different way of immersion, e.g. the phantasmagoria and 360° panoramic exhibitions to physically stand inside another space. We must differentiate between physical immersion (e.g. a CAVE) and psychologial immersion (e.g. reading a book). Regarding the physical space we have the best examples in all world exhibitions (with a big audience and budget) with “big immersion” and “small immersions” with HMD-like personal setups. Regarding this hardware he states, that specific devices are unimportant, but we’d rather need to focus on novel illusions and the way to improve the immersion. Once the medium has become fully invisible, we can enjoy the VR/MR and that art has always played a big role in pushing these limits, driven by the fascination of extending the possibilities of immersion. Interestingly enough art tends to create a polarizing vision of the future or (digital) parallel worlds: either an utopian scenario or dystopian dark worlds to catch us emotionally.

He closes with a reminder that we need to think about ways to preserve today’s digital artworks in an adequate way. Art documentation seems to fade: if we pulled the plug, the 31st century people wouldn’t be able to understand anything about our time’s art. He underlines the need for more sustainability, better documentational structures, funding and more commitment from the pros for future generations to be able to look back at our digital steps.

Mark Bolas then continued with a great talk and a detailed look on where we are today and what we need to focus on more, if we want to mix and create our own realities better. To gain this as researchers and artists we must first fullfill the basic prerequisite: to have passion for our field and stick with our visions! Looking back at the Gartner hype cycle, we can see, that we are nevertheless still in the early adpoter phase, while games or mobile phone acceptance already matured. This highers the expectations by the public: they imagine how it could be and thus helping to improve the realization of good MR/AR. ISMAR just started to get out of this gap in understanding between researchers and customers. A good time!

The question of where we should put AR is been followed by examples on how to make use of our senses in MR. The immersion can only happen if we consider all these hidden possibilities – AR has to be much more than the direct manipulation of our vision. We need to think and look beyond planes of light and pure pixel piles. E.g. stereo speaker sound might not be sufficient to give a good immersion. We need better wave field simulations to convince the human body, that is way to sensitive to his input. E.g. light field technology may help to experience unbroken MR experiences (like having real eye contact with a remote partner, that is projected). Additionally, mixed Reality is not only about putting in cool stuff, but by the same amount we need to consider what is around us. We need to scan the real world to make the virtual work. A tactile contact between real and virtual is needed to make it work. If we have a physical bond to the virtual part, also giving us a direct way of manipulation, it starts to work. Talking about current devices like smartphones (if it’s not on a phone, it’s not AR these days… ;-)) this means to realize a connection – for example – through AR-enabled cards, I can hold in my hand – rather than having the card on the floor or just waving around my phone and seeing floating objects. The immersion only happens through a close physical relation. Limitations by displays need to be considered closely: a noticed frame breaks all illusion (cutting off virtual augmentations) and further more: we don’t even need a display to reach the user. We need to get into his/her head! Again, it has to be about creating another reality within the user. Not showing pixels.

Everyone creates his or her own reality all the time. We must support this, rather than wasting time on “token realities”: e.g. using real world posters to augment a movie clip, I’d rather watch at home without AR. In these situations we’d rather want to get rid of reality! Reality is now an optional element of our design!

If we want to integrate it, we have to preserve reality, pull it in by accepting all senses and be aware of the fact, that people are made of meat and that we need physical spaces to interact and meet. A body relation is the best start.

Adrian David Cheok then continued up on this idea of including all senses to make MR happen. With a lot of demos, he made us understand, how this could be achieved.

Though the progress in AR, tracking, mobile devices has been huge during the last 10 years, he pities, that conceptually not much has changed and that still the focus lies on the visual part and even more on tracking issues only. E.g. the magic book metaphor is still the same (and widely used) as it was decades ago. We need to reconceive and rethink our own understanding of AR to move on.

A fitting example was the comparison to a samurai fighter, who also uses all his senses to win a battle. A good samurai could still win, if he was blind:

Going through all senses he showed demos reg. smell, taste, sound (as above), touch with the huggy pajama demo and the chicken touch demo: a remote human-pet interaction system.

He stressed the part of the touch part for human interaction and thus also for mixed reality interaction metaphors. Physical interfaces just give a different feeling to us than digital-only approaches, though they might contain the same information. It just is different from the logical communication. We need to change the physical world to interact with VR/MR worlds!

To redesign the worlds we need to think ahead 10 years from today and look at all the different senses! Young people starting to work in this field of MR should begin integrating this needed multi-sensoric approaches!

After all it’s about human interaction, all sensors and virtual kisses! :-)

Great keynotes, great inspirations! Though, we need still to find the killer app ourselves to become millionaires! Glad I could hear those visionary talks and give a glimpse of it here.

Great days!

Thanks a lot ISMAR and i.e. thanks to all involved people making this great conference come alive. Now it’s time to hit the road and head back to Munich with a bag full of pictures, new ideas, visions, the warm feeling of having met so many nice people and the urge to tell everyone about it. So, expect major updates here during the next days! It wouldn’t be adequate to squeeze all into on entry here. :-)

What I liked best was the heartly and open-minded atmosphere of all AR people. Everywhere people where discussing ideas, waving around their arms in explanations and a lot of brains heating up visibly. Great keynotes gave the visions (but not the killer app ;-)), the cool demos where fun and interesting and the talks made the dusty paper work come alive. Great contributions came in the area of depth cameras (well, almost by 100% the Kinect), that really change the game for AR, I believe. The demos showed us exiting ways of how we can integrate AR into our lives, how we can enjoy it and what the current status of the rapidly evolving tracking technology is. All to be covered.

It was also amazing to see how dedicated everybody works in his or her field, especially visible in the tracking contest, where you could smell the enthusiam in the air, people starring at the screens observing, even more exited and immersed than when watching the finals of the world cup and participants squeezing that last bit of error minimization out of their code minutes before the run. The winners and the dedicated post & impressions on the tracking contest to follow, too!

Luckily parallel to the ISMAR, Basel had the SHIFT art festival running, including a special ISMAR@SHIFT exhibition with great works mixing realities, changing our perception of reality and making us dream and laugh. Impressions to follow.

If you where wise enough to sign up for the LifeClipper3 experience before, happening inside the park aside the conference dinner location, you had another chance to daydream and immerse in a parallel, yet-present MR universe. Time to dream of alternate reality games with tiny goggles and a time to think of the artistic potential and emotional suggestions made possible. Time to have a break and create your own individual reality within the hectic world. Story to follow, too.

So for now, it’s time to say FAREWELL BASEL and thank you all, who made this great conference happen and another HELLO with the best greetings to all out there I could finally meet personally!

Best,
TOBY.

PS. All the pictures I took will be published through ISMAR website next days, but partly also here and on facebook/flickr.