ISMAR – The True AR Experience
As a last demo at this year’s ISMAR I had the pleasure to meet Jan Torpus and to experience his LifeClipper3 project live. After the ISMAR started with visionary concept ideas and thinking about all senses, everyone got lost or concentrated on more daily issues like tracking, rendering or defined use cases and minor technical improvements. After a long conference I was pretty stressed out and nervous with so much input of technological considerations, I had to pull myself together to actually go to the final scheduled presentation… turns out it was the best AR experience I had during the whole conference!
I arrived at the St. Johanns Park in Basel and found myself putting on a heavy-weight backpack full of sensors and a HMD visor with video cameras, head phones, etc. With this weight I felt even worse than holding around my smartphone pointing to AR spots. An immersion seemed close to impossible.
Explaining the bold facts, LifeClipper is an outdoor AR experience, where you carry around all hardware and have an inside-out sensor-based location tracking. An advanced GPS-sensor gives your position, inertia sensors and compass give your rotation and you experience the combination of real and virtual through a stereoscopic HMD with two cameras and stereo headphones. The cameras are not used for tracking, though. You move inside a park area in Basel. This park areas has been completely scanned and remodelled in 3D space to make virtual object placement and interaction of virtual and real possible. Then I entered the mixed reality world…
A great mixed reality world, you dip in from your physical space. The best: you don’t even have to stay inside a lab or office! Going into details of the video: the different areas of the park not only result in different zones or climates with another part of the story being told and other 3D aliens or augmented objects shown. But also your perception of reality changes! You see those alien bugs, you see virtual trees, a desert or a LSD sky during a virtual night – changing the video backdrop to a dark scenario. You feel in a completely different space, you can’t tell the real trees from the virtual ones (the alternation of the reality-video-feed is the crucial part here), you move slower than usual due to aggressive deep sounds hovering above and around you. Lastly, you even leave your physical body and see the park from above floating over it (in a foggy virtual world)! This play of changing the perception of your personalized reality worked nicely. You get immersed in a true mixed reality… in the end you also switch into a 100% virtual world not noticing it – the real world is just an option. This whole design of the worlds and all the thought put into the perception of reality would be too much for this post to sum up. It is pretty interesting and can be read in the complete concept description.
Personally, I felt great during the experience. But not only. It was even uncomfortable walking closer to the big aliens hearing the dark noises, but I also felt lightweight happy, when I moved into the colorful night setting, where the real world was changed into a dreamy, positive place. When moving through virtual grass, that grew and shrinked according to my breath rythm, I felt the urge to divide it with my hands. When I moved closer to the neat alien bugs, I was afraid of stepping onto them. I almost smacked my head by walking into lanterns: the immersion worked not only visually, but emotionally! Or to rephrase: true immersion is only possible, if you are connected emotionally. It did work here completely.
The graphics didn’t really matter anymore. Abstraction versus realism didn’t matter anymore. Once I immersed, it was only about the story and the direct interaction in the real physical space. I adjusted my physical behaviour – forcing myself to move slower – and when discovering this embedded, hidden parallel universe, I immediately forgot the huge backpack I was carrying around with my for over 40 minutes! I entered my personalized reality, that I didn’t want to leave again. Not yet. This dream world was so engaging I forgot polygon counts or tracking instabilities. This is the true AR experience! I can’t wait for the day good AR goggles hit the market and an easy way is established for alternate reality game designers or every interested artist to create his/her “channels” of the mixed worlds! Time for location-bound alternation of reality!
To immerse, it totally helped to have a good combination of atmospheric audio, the visuals and interaction – actively or subtle. Without good sound it wouldn’t work. The HMD gave the ultimate feeling of being there, still leaving a bit of a dream-like feeling due to the offsets of my real hand positions to the virtual… This is one thing I’m hoping to see for future improvements: to get to interact with the virtual animals or objects through gestures. (As we could see in different ISMAR demos.) The physical connection was so important and helpful – it needs to be extended to include not only my feet, but also my hands! This physical space interaction of the real and virtual objects could also be further optimized by giving e.g. vibration feedback or giving altered virtual audio footsteps (overwriting real footsteps), etc. You could simulate a different ground (or support the moment of leaving the ground, floating). Or you could have a blindman’s stick with sensors to fake directly-world-connected feedback.
This personalized reality could possibly have even more impact when including information about the user, his friends, favorite music, etc. Artists and developers could define logics to use emotion triggers in a subtle way. For good and for bad. The personal reality within modern life could be a personal retreat or your meeting place for remotely located friends. Overall a great look into the future. Tracking and performance can easily be improved, but it was great to see, that these things didn’t even matter that much during the show. The great multi-sensoric approach and the fine mixed reality world design with simple alternations (like changing the sky color) make it work already!
It sure was a meditative escape while staying in the same physical space. Disconnecting while connecting. When I woke up, reality looked dull. I talked to Jan for a long time afterwards and when I finally left the location and passed by a spot with a park bench, where I have been during the LifeClipper-time, I thought: Haven’t I been here before? No, it must have been in a dream!