A Sign of AR Life at Apple

Apple Augmented Reality Demo seen in Stores

Is Apple finally on to something? Will they release their own augmented reality application very soon? We have further signs of life from their side.

Lately, more rumors were spreading. Analysts confirm Apple to put more forces on AR, Tim Cook seems interested, metaio and PrimeSense were shopped a while ago, patents filed and transferred to Apple. Sure, they will do something.

Let’s take a look at a marketing video, that showed up in Apple Stores lately to think about their plans. The video shows a flight over a city. On the streets we see small status messages and emoticons appear as bubbles. They pop up and go away constantly and also move through the streets, e.g. when they seem to be connected to a car or bus. Angry or happy icons might reflect the mood of the iphones walking through the streets. It seems like the idea is that you can wave your phone around with an augmented viewfinder to scan the city for tweets or iMessages and status updates of your friends and peers.

The overlay happens quite accurately during this smooth camera movement. Icons come and go with a perfect placement over moving objects in the city. Icons are sometimes aligned by 90° (possibly matching the street layout) and are sometimes freely. Some objects are clearly 2D icons from the Apple world ported into this AR view, some objects are also 3D objects (a crying emoticon) that has proper lighting and occlusion with a skyscraper in front. See it in the video below:

Dan Talmon tweeted these videos from an Apple Store in Berkeley. The videos were recorded from an iphone and ipad and clearly are marketing videos showing the idea of Apple’s approach towards an augmented window concept used on a city scale. (If it is not a fake or put in wrong context…) I haven’t been in an Apple store for a while, though it seems they are already running there for a while. Time to take a closer look, nevertheless.

Early 2000s apps like wikitude, layar or metaio’s junaio were trying the same using the GPS signal and compass of earlier generations Android and Apple phones. These location-attached information services were more of a flash in the pan back then. Very cool and fun, but useless after initial excitement. (Sorry.) You wouldn’t actually use it on a daily basis – to help your routine. Why didn’t it work back then? Will Apple be able to revive this concept?

Back then, phones were less powerful, less internet-connected (less bandwidth), GPS signals or location finding was not that accurate on most regular phones, if available at all. Battery drain sure was (and is) a major issue for these scenarios. Why should I run my camera to drain the battery if I could just use a classic top-down map view like Google Maps to show me the place or info I want to have? People are used to it and a classic app with battery left is better than a cool AR app without battery…

Will Apple make something out of this? Or is it rather a general marketing video for location-based services? After all, noone will ever see the city from that shown perspective to make sense (unless you own your own helicopter)… Will they prevail where others failed? Apple sured proved itself to make things right in the past. “Just” making a better user interface, experience, added value, etc. out of the existing pieces. Maybe they will integrate it nicely, using the twin-camera setup of the latest phone – or even further sensors in the upcoming iphone 8. Will they have an city-scale answer to Google Tango with it’s depth sensing tech inside? Apple could succeed. They are restrictive and put their foot down and rule over their own software ecosystem. They only allow well-tested and good thought-out interfaces into their operating system. This could be key: if they manage to integrate an AR view mode directly into their operating system, it could work! E.g. use twitter and once you tilt your phone to a camera position (viewfinder) it automatically switches to the AR mode overlaying the real world. Without a special app or button to press. This way it could really work to get integrated! Apple could softly force their developers and app submitters to obey these concepts. Android might be too clustered. Apple has more control to push this. Well, could be… let’s see.

Going back to the location services with AR, metaio showed their concept and initial steps of their Augmented City back in 2012. A mini scale model was augmented via their junaio app showing digital content overlaid accurately on the city’s buildings and walls. Back then, Peter Meier explained their new approach towards full 3D optical object tracking, moving away from 2D markers as used back in that time for magazine AR, etc. The demo that I reported on back then during their insideAR conference only showed a scale version of the real city trials their did in the United States back then. The concepts existed back then, only time had not yet come. Peter was saying in their own marketing video for the Augmented City back in 2012:

We need great hardware running great software as a base technology and then we need great partners with great content.

Maybe their long work on a stable AR toolkit and advanced tracking now finds its way into great hardware from Apple with great content from them and the whole ecosystem orbiting around the big fruit company. More signs like these videos above could be the heralds of a big bang to come … in 2017? It would fit.

Smartglasses as an accessory would be fun, but I doubt that Apple will do the first move here. A stable solution to get everybody hooked on mobile AR (again) – but now in their daily lives – through the new generation of handheld phones seems more logical. Let’s wait and see and enjoy the unbroken pleasant anticipation as long as it lasts.