Halide Does Deep Dive Into iPhone SE's Software-Based Portrait Mode

Though it is a budget device with a single-lens camera, the iPhone SE features support for Portrait Mode, enabled through the powerful A13 chip in the smartphone.

iphonesehandson
It is the first of Apple's smartphones to offer Portrait Mode photos created entirely with software techniques rather than hardware, which prompted the developers behind popular iOS camera app Halide to take a deep dive into how it works.

The ‌iPhone SE‌ is equipped the same camera sensor as the iPhone 8, based on a recent teardown done by iFixit, but its camera can do more because it's using "Single Image Monocular Depth Estimation," aka generating Portrait Mode effects using a 2D image.

As Halide developer Ben Sandofsky points out, the iPhone XR is also a single-lens camera with Portrait Mode support, but the ‌iPhone XR‌ gets depth information through hardware. That's not possible on the ‌iPhone SE‌ because the older camera sensor doesn't support the feature.

Halide has discovered that unlike other iPhones, the ‌iPhone SE‌ can take a picture of another picture to attempt to develop a depth map. The app was even able to take a photo of an old slide film, adding depth effects to a 50 year old photo.

halideoldphoto

A picture of a picture and the resulting depth map from the ‌iPhone SE‌

The ‌iPhone SE‌'s Portrait Mode is somewhat limited because it only works with people, which is due to the neural network that powers the feature. When a Portrait Mode image without a person is captured, it fails in various ways because it can't create an accurate estimated depth map.

The ‌iPhone XR‌ also limited Portrait Mode to people alone, and using Portrait Mode with other objects requires upgrading to one of Apple's more expensive phones.

According to Halide, depth maps on the ‌iPhone SE‌ (or any phone with Portrait Mode) can be viewed by using the Halide app and then shooting in Depth mode. Halide's full breakdown of the ‌iPhone SE‌'s Portrait Mode can be read over on the Halide website.

Related Roundup: iPhone SE 2020
Tag: Halide
Buyer's Guide: iPhone SE (Don't Buy)
Related Forum: iPhone

Top Rated Comments

luvbug Avatar
18 months ago

Could you have used a nicer pic? lol
She's probably the author's (Halide's) grandmother. Take a break, you'll be old too, sooner than you can imagine.
Score: 21 Votes (Like | Disagree)
AngerDanger Avatar
18 months ago

Could you have used a nicer pic? lol
As you wish. :)



Attachment Image
Score: 18 Votes (Like | Disagree)
NickName99 Avatar
18 months ago

This is wildly impressive.
Agreed - this is far more impressive than most people realize. The Pixel phones and the iPhone XR achieve this with a single lens by leveraging focus pixels to generate a depth map from available 3D information. The SE doesn’t have enough focus pixels for that, so it’s relying purely on machine learning, building a depth map from 2D information.

I’m surprised it was more cost effective to do it this way rather than just make the changes necessary to fit the XR rear camera module. That really speaks to the expense of retooling manufacturing.
Score: 10 Votes (Like | Disagree)
farewelwilliams Avatar
18 months ago
Theoretically, you should be able to extract frames from a video, apply neural network from Portrait Mode on to each frame, and stitch it back into a video.

Highly likely iOS 14 will have Portrait Mode for video. Would be only enhanced with Lidar too. This will take cinematography on a smartphone to a whole new level.
Score: 9 Votes (Like | Disagree)
NickName99 Avatar
18 months ago

I had assumed the criticism was because the image is low resolution and blurry, making it a poor test image.
I‘d say the worse the quality of the photo (resolution and sharpness), the more impressive that it works.
Score: 8 Votes (Like | Disagree)
JSRinUK Avatar
18 months ago

wonder if it works on pets... or if it will in the future
Hoping that no one minds a photo of me here, but I did a test photo when playing with my dog yesterday.

I'm not sure how the portrait mode works, but I was quite impressed that both myself and my dog are in focus.



Attachment Image
Score: 6 Votes (Like | Disagree)

Top Stories

m1x mac mini screen feature

High-End 'M1X' Mac Mini With New Design and Additional Ports Expected to Launch in the 'Next Several Months'

Sunday August 22, 2021 5:59 am PDT by
Apple can be expected to launch an updated high-end Mac mini with a new design and a faster "M1X" Apple silicon processor in the "next several months," Bloomberg's Mark Gurman reports. In the latest publication of his Power On newsletter, Gurman writes that a new high-end Mac mini, which has previously been reported to feature a new design with additional ports, can be expected to replace...
mac scanner permission error

Apple Says Fix Planned for 'You Do Not Have Permission to Open the Application' Error When Using a Scanner on Mac

Saturday August 14, 2021 6:15 am PDT by
In a newly published support document on its website, Apple has acknowledged an error that some users may receive when they try to use a scanner with a Mac in the Image Capture app, Preview app, or the Printers & Scanners section of System Preferences. A screenshot of the error message from the HP Support Community When attempting to use a scanner with a Mac, Apple said users might get an...
macbookpro13large

macOS Big Sur Update Bricking Some Older MacBook Pro Models

Sunday November 15, 2020 5:33 am PST by
A large number of late 2013 and mid 2014 13-inch MacBook Pro owners are reporting that the macOS Big Sur update is bricking their machines. A MacRumors forum thread contains a significant number of users reporting the issue, and similar problems are being reported across Reddit and the Apple Support Communities, suggesting the problem is widespread. Users are reporting that during the...
original iphone

Phil Schiller Says iPhone Was 'Earth-Shattering' Ten Years Ago and Remains 'Unmatched' Today

Monday January 9, 2017 7:15 am PST by
To commemorate the tenth anniversary of the iPhone, Apple marketing chief Phil Schiller sat down with tech journalist Steven Levy for a wide-ranging interview about the smartphone's past, present, and future. The report first reflects upon the iPhone's lack of support for third-party apps in its first year. The argument inside Apple was split between whether the iPhone should be a closed...
m1 imac orange

New iMac Tidbits: Headphone Jack on Side, Ethernet Port on Power Adapter, Spatial Audio and WiFi 6 Support, No SD Card Slot

Wednesday April 21, 2021 6:38 am PDT by
Apple yesterday announced a completely redesigned 24-inch iMac with the M1 Apple silicon chip. The new iMac, the first major redesign of the Mac desktop computer since 2012, has several changes compared to the previous generation. In the aftermath of the event, a few new features and tidbits may have slipped under the radar, so we’ve compiled this list of some of the less-talked-about...
omg lightning cable comparison

Security Researcher Develops Lightning Cable With Hidden Chip to Steal Passwords

Thursday September 2, 2021 6:59 am PDT by
A normal-looking Lightning cable that can used to steal data like passwords and send it to a hacker has been developed, Vice reports. The "OMG Cable" compared to Apple's Lightning to USB cable. The "OMG Cable" works exactly like a normal Lightning to USB cable and can log keystrokes from connected Mac keyboards, iPads, and iPhones, and then send this data to a bad actor who could be over a...
Top Stories 75 Thumbnail

Top Stories: Last-Minute iPhone 13 Rumors, Apple Announces App Store Changes, and More

Saturday September 4, 2021 6:00 am PDT by
The finish line is in sight! Apple's annual iPhone event is likely just a week or so away and all eyes will be on the company as it unveils the next version of its most popular product line. With any luck, we'll also see the next-generation Apple Watch and perhaps even some new AirPods. Other news this week saw Apple making some more changes to its App Store policies in response to a...
General YouTube Feature 1

YouTube Premium and Music Surpass 50 Million Subscribers

Friday September 3, 2021 2:19 am PDT by
YouTube says it has passed 50 million subscribers for its Premium and Music subscriptions, making it the "fastest growing music subscription" service in the world, according to YouTube's global head of music, Lyor Cohen. YouTube says that it has more than 50 million paying subscribers collectively across YouTube Premium and YouTube Music. The Google-owned service says it attributes this...
iPhone 13 Dummy Thumbnail 2

Full iPhone 13 Feature Breakdown: Everything Rumors Say We Can Expect

Tuesday August 31, 2021 7:50 am PDT by
With the launch of Apple's iPhone 13 lineup believed to be just a few weeks away, we have compiled all of the coherent rumors from our coverage over the past year to build a full picture of the features and upgrades coming to the company's new smartphones. For clarity, only explicit improvements, upgrades, and new features compared to the iPhone 12 lineup are listed. It is worth noting that...
apple wallet drivers license

Apple Announces First U.S. States That Will Let You Add Your Driver's License to Your iPhone

Wednesday September 1, 2021 6:15 am PDT by
Apple today announced the first U.S. states that will be rolling out the ability for residents to add their driver's license or state ID to the Wallet app on the iPhone and Apple Watch. Arizona and Georgia will be the first states to support the feature, with Connecticut, Iowa, Kentucky, Maryland, Oklahoma, and Utah to follow, according to Apple. Apple said select TSA security checkpoints in ...