Developers Demo Augmented Reality Improvements Coming in iOS 11.3

The iOS 11.3 update, seeded to developers and public beta testers last week, introduces ARKit 1.5, an upgraded version of the set of tools developers can use to create augmented reality apps for the iPhone and the iPad.

As it turns out, ARKit 1.5 can do a lot of neat things. It can map irregularly shaped surfaces for better detection of your surroundings, and it can also recognize and map vertical surfaces like walls and doors, so you can use AR to place and detect items on walls.

Over the course of the last week, developers have been testing out ARKit 1.5 and sharing short demo videos on Twitter, providing a look at just what will be possible with augmented reality apps when iOS 11.3 is available.


Vertical surface detection, for example, is shown off in the video below. A realistic-looking tunnel is projected on a wall, and while this doesn't have any immediate usage implications, it's a useful demo of how ARKit sees walls in iOS 11.3.

So I've been playing with the new ARKit wall detection today...what do you think?#arkit #madewitharkit #augmentedreality #iOS113 #arkit15 #ar #AugmentedReality pic.twitter.com/J9tj6sJf9S

— Ubicolor™ (@ubicolorapps) January 25, 2018

An example of how vertical plane detection can be used in augmented reality games is demonstrated in the video below, where creatures projected into open space take advantage of the area around them.

Made a weird #GGJ18 game Creature-z this weekend with @tristan_damron, @marknerys and @ericklind using #ARkit verticalPlane detection. Big up to @PIGSquad for hosting pic.twitter.com/BKRdhVKHvq

— lauri (＾ｰ^)ノ (@lingoded) January 29, 2018

Another demo adds virtual artwork to a blank wall, a concept that could potentially be used in an art gallery or museum where art is invisible without a smartphone.

Vertical plane detection is used in the video below to show a realistic-looking virtual cockatoo coming through a window and landing on a windowsill.

Look what flew in through my window today... #ARKit pic.twitter.com/MFYYQlBjP4

— Abhishek Singh (@shekitup) January 31, 2018

In addition to mapping oddly shaped spaces and recognizing vertical surfaces, ARKit 1.5 also includes image detection features that work on everything from movie posters to barcodes, as demoed below. In the future, you might be able to scan a barcode with ARKit to get a virtual popup of nutritional information, calories, and more.

In ARKit 1.5 reference images can be any image. Including barcodes!!!! #ARKit #AugmentedReality pic.twitter.com/8fwJTcSoRo

— azamsharp (@azamsharp) January 31, 2018

Image detection could be useful in settings like art galleries and museums, where visitors could use it to scan paintings and exhibits to receive more information, as shown off in the video below.

#arkit image detection 1st try: success! As @Apple said, great feature for museums. Now I need content for 2300 paintings @NationalGallery pic.twitter.com/1m3LZVVCfJ

— Romain Derrien (@LondonRom) January 28, 2018

Though not visible in the demo videos shared by developers, ARKit 1.5 also introduces a higher camera resolution, so passthrough video is 1080p rather than 720p, and there's also support for autofocus capabilities, another feature that will improve the augmented reality experience on iOS devices.

Recent data has suggested that the ARKit framework has seen only modest adoption from developers and stagnating growth since its debut in iOS 11, but improvements like ARKit 1.5 may change that in the future. Augmented reality on iOS devices is still in its infancy and it will take time for developers and users to discover the best real-world use cases for the technology.

Apps using ARKit 1.5 will be available starting this spring when iOS 11.3 is released to the public.