Apple's Big Push to Improve Accessibility in iOS 13, iPadOS, and macOS Catalina

Apple is introducing several accessibility-friendly features with iOS 13, iPadOS, and macOS Catalina, including Voice Control, Hover Text, and Mouse Pointer Support for iPhones and iPads.

After the company's WWDC keynote on Monday, TechCrunch's iOS accessibility expert Steven Aquino sat down with Apple's Sarah Herrlinger, director of Global Accessibility Policy & Initiatives, who offered further details on the thinking behind Apple's latest, and arguably greatest, accessibility push.


Voice Control

Herrlinger first opened up about Apple's new Voice Control feature, which earned its own slide space during Craig Federighi’s onstage presentation. Voice Control supports editing and menu navigation in both ‌macOS Catalina‌ and ‌iOS 13‌, but it was the feature's advanced dictation capabilities that Herrlinger was most eager to highlight.

For example, Herrlinger explained how you can say "show numbers" in Safari's Favorites view and little numbers, corresponding to the number of favorites you have, show up beside a website's favicon. Say TechCrunch is No. 2 in your list of favorites. If the glyph is hard to make out visually, saying "open 2" will prompt Voice Control to launch TechCrunch's page. Likewise, you can say "show grid" and a grid will appear so you perform actions such as clicking, tapping or pinching-and-zooming.

Herrlinger said Apple has worked hard to improve Voice Control's speech detection system, so that it can more adeptly parse users with different types of speech, such as those who stutter.

On iOS, the feature also utilizes attention awareness to know when a user with physical motor limitations is interacting with their device. Allaying privacy concerns, Apple says that none of the audio processed by Voice Control can be accessed by anyone else, including Apple, thanks to built-in anonymity and encryption.

Hover Text for macOS

Another feature Herrlinger was keen to demo was something called Hover Text on macOS. Described as a subset of the existing Zoom functionality, Hover Text enables the user to place the mouse pointer over a selection of text to get a bubble with the text enlarged.

Herrlinger told me the feature works system-wide, even in places like the menu bar. And yes, Hover Text is indeed customizable; users have access to a wide variety of fonts and colors to make Hover Text's "bubbles" their own. Text size can be enlarged up to 128pt, Herrlinger said. What this means is users can play with different permutations of the feature to find which label(s) work best — say, a yellow background with dark blue text set in Helvetica for the highest contrast. The possibilities are virtually endless, a testament to how rich the feature is despite its simplicity.

According to Herrlinger, Apple could have brought iOS feature Dynamic Type to the Mac, but found Hover Text accomplished the same goal of enlarging text in a way that felt better suited to its desktop operating system.

Mouse Support for iPhone and iPad

‌iPadOS‌ and ‌iOS 13‌ introduces mouse support for the first time, allowing a USB mouse to be connected to an ‌iPad‌ and ‌iPhone‌ for the first time.

Mouse support is not a standard feature, but is instead available as an AssistiveTouch option, designed for users with physical motor delays who can’t easily interact with the touchscreen itself. Apple says it works with both USB and Bluetooth mice, although the company doesn’t yet have an official compatibility list. According to developer Steve Troughton-Smith, who first discovered the feature, it also works with the Apple Magic Trackpad.

When I asked why build pointer support into a touch-based operating system, Herrlinger was unequivocal in her answer: it serves a need in the accessibility community. "This is not your old desktop cursor as the primary input method," she said.

The reality is, it's not your secondary choice, either. The bottom line is that, while Apple loves the idea of accessibility features being adopted by the mainstream, pointer support in ‌iOS 13‌ and ‌iPadOS‌ really isn't the conventional PC input mechanism at all. In this case, it's a niche feature that should suit a niche use case; it's not supposed to represent the milestone of ‌iPad‌'s productivity growth that many think it could be. Maybe that changes over time, but for now, it's the new Mac Pro of software: not for everyone, not even for most people.

According to Herrlinger, Apple recognizes that people without disabilities will use this feature. "For example, many people find value in closed captions," she said. "Our goal is to engineer for specific use cases so that we continue to bring the power of our devices to more people."


That being said, Herrlinger was quick to emphasize that mouse support should be seen in context. In other words, Apple hasn't introduced it to drastically alter the primary user input landscape of iOS, although that being said, it's not going to stop anyone outside its intended use case from plugging a mouse into their iPad Pro.

‌iOS 13‌, ‌iPadOS‌, and ‌macOS Catalina‌ are only available to registered developers at this time, but later in the summer, Apple plans to make public betas available, giving public beta testers a chance to try the software before it sees a public launch in the fall.

Related Roundups: iOS 13, iPadOS

Top Rated Comments

(View all)
Avatar
11 months ago
Why are you dragging your feet Apple? Just make it a regular mouse cursor, like it was figured out to be, by Xerox, decades ago, to be one of the fastest and most comfortable ways to interact with a computer.

Just stop being weird.

You're going to add it eventually and we all know it. Just paste in the code.

Jerks.

I can't believe a closed, locked down OS became the most popular again.

Have we learned nothing from the Windows debacle? Apple is behaving worse than Microsoft ever did.
Score: 9 Votes (Like | Disagree)
Avatar
11 months ago
I absolutely love this feature!!! The one thing that prevents me from going on vacation with an iPad alone leaving my laptop behind is that I need to be able to use Microsoft’s RD client to connect to my system at the office and do work that requires a mouse from time to time.

I can bumble my way through it with touch if it’s just adjusting a line or two of code but anything more than that and forget it on anything but a laptop or desktop with a mouse.

With mouse support I can take my iPad Pro with me and be safe in the knowledge that I’m covered if I have to do something more complex on my work PC that requires the precision of a mouse instead of touch through the RD interface! For me this one is a game changer!!!
Score: 5 Votes (Like | Disagree)
Avatar
11 months ago
Embarrassing that it took them this long. This is just a software switch to stop intentionally crippling standard bluetooth/USB interfaces.
Score: 3 Votes (Like | Disagree)
Avatar
11 months ago
I hope this works as we EXPECT IT TO, not some cockamamie Apple-exclusive-convoluted-three-finger-swipe-down-left-right-A-B-Start kind of way.
Score: 3 Votes (Like | Disagree)
Avatar
11 months ago
What is the motive for hiding ordinary features like mouse support under accessibility? Is it because of shame for backtracking, playing into the hearts of consumers that they really care about disabilities, weird rebranding, etc.? No other mouse supported OS does this.
Score: 3 Votes (Like | Disagree)
Avatar
11 months ago
Finally some innovation. :apple:
Score: 2 Votes (Like | Disagree)

Top Stories

Seemingly Unreleased Version of Logic Pro X With Live Loops Appears on Apple's Education Site [Updated]

Sunday March 29, 2020 7:23 am PDT by Hartley Charlton
Update: Apple has replaced the Logic Pro X image with an older version. Original story follows. A seemingly unreleased version of Logic Pro X has appeared on Apple's education site, as spotted by a Reddit user. The image from Apple's education products page shows a 16-inch MacBook Pro running Logic Pro X, but with a familiar interface that looks extremely similar to GarageBand's Live Loops ...

Zoom Updates iOS App to Stop Sending Data to Facebook

Friday March 27, 2020 5:35 pm PDT by Juli Clover
Zoom, a video conferencing app that many people are using at the current time to keep in touch with coworkers while working from home, was sending data to Facebook without disclosing the data sharing to customers. As of today, Zoom has updated its iOS app to remove the SDK that was providing data to Facebook through the Login with Facebook feature, according to Motherboard, the site that...

Apple Suppliers Worried About iPhone Demand, Production Ramp-Up for New iPhones Reportedly Postponed

Friday March 27, 2020 5:56 pm PDT by Juli Clover
Most of the factories in China that supply devices and components to Apple are back to churning out products, but Apple suppliers are said to be worried about how much demand there will be for the current iPhone models and the new iPhones expected in the fall. According to Reuters, a senior official at one of Apple's major supply companies said that orders for quarter ending in March are...

2020 iPad Pro Teardown Provides Closer Look at LiDAR Scanner and Confirms Incremental Update

Saturday March 28, 2020 9:56 am PDT by Hartley Charlton
iFixit today shared a video teardown of the new iPad Pro, which Apple unveiled earlier this month. iFixit found that most of the internals of the 2020 iPad Pro are the same as the 2018 model, confirming that the device is a relatively incremental update. The most notable new feature seen inside the new iPad Pro was the LiDAR scanner, which measures the distance to surrounding objects up...

Apple Watch Series 6 Could Feature Touch ID Fingerprint Sensor, Pulse Oximetry and Sleep Tracking Support

Friday March 27, 2020 11:28 am PDT by Juli Clover
The upcoming Apple Watch Series 6 set to be released this fall could include a Touch ID fingerprint sensor built into the crown of the device, according to Israeli site The Verifier, which cites "senior sources" who have worked with its staff for a "number of years" as the source of the rumor. It's not clear how the alleged Touch ID fingerprint sensor would be implemented, as the Digital...

Bloomberg: Apple's 5G iPhone Still on Schedule for Fall Launch, But Future Products Could Be Delayed

Monday March 30, 2020 2:40 am PDT by Tim Hardwick
Apple's 5G iPhone is still on track to launch within the company's typical annual fall release schedule, according to a new Bloomberg report on filed on Monday. Signs are that Apple's Chinese-centric manufacturing -- of which Hon Hai is the linchpin -- is slowly getting back on track. The next iPhones with 5G wireless capabilities remain on schedule to launch in the fall, partly because mass...

Kuo: Apple to Launch Several Macs With Arm-Based Processors in 2021, USB4 Support Coming to Macs in 2022

Thursday March 26, 2020 8:19 pm PDT by Joe Rossignol
Apple plans to launch several Mac notebooks and desktop computers with its own custom designed Arm-based processors in 2021, analyst Ming-Chi Kuo said today in a research note obtained by MacRumors. Kuo believes that Arm-based processors will significantly enhance the competitive advantage of the Mac lineup, allow Apple to refresh its Mac models without relying on Intel's processor roadmap,...

Top Stories: Hands-On With 2020 iPad Pro and MacBook Air, iOS and iPadOS 13.4 Released, iPhone 12 Delay?

Saturday March 28, 2020 6:00 am PDT by MacRumors Staff
After last week's flurry of product launches, Apple's new iPad Pro and MacBook Air have started to make their way into consumers' hands, and we've gone hands-on with both of them this week. Apple this week also released iOS and iPadOS 13.4 (as well as macOS, watchOS, and tvOS updates) with a number of new features and improvements. Subscribe to the MacRumors YouTube channel for more videos. ...

Deals: Huge Refurbished iPhone Sale Discounts iPhone 7, 8, X, XR, and XS (From $120)

Friday March 27, 2020 5:47 am PDT by Mitchel Broussard
Woot is back today with a big sale on refurbished iPhones, including markdowns on the iPhone 7, iPhone 7 Plus, iPhone 8, iPhone 8 Plus, iPhone X, iPhone XR, iPhone XS, and iPhone XS Max. Note: MacRumors is an affiliate partner with Woot. When you click a link and make a purchase, we may receive a small payment, which helps us keep the site running. As with every Woot purchase, those...

Apple Releases ProRes RAW Beta for Windows

Monday March 30, 2020 9:33 am PDT by Juli Clover
Apple today released ProRes RAW for Windows in a beta capacity (via Mark Gurman), with the software designed to allow ProRes RAW and ProRes RAW HQ video files to be watched in compatible applications on Windows machines. According to Apple, the software will let the files be played within several Adobe apps: Adobe After Effects (Beta) Adobe Media Encocder (Beta) Adobe Premiere...