Craig Federighi Explains How Scribble Feature in iPadOS 14 Was Developed

Scribble is one of the most notable new features in iPadOS 14, and in a new interview with Popular Mechanics, Apple SVP of software engineering Craig Federighi has revealed more details about how it was developed and how it works on iPad.

apple ipados 14 scribble 062220 inline
With Scribble in ‌iPadOS 14‌, you can use an ‌Apple Pencil‌ to write in any text field, and the written text is then converted to typed text. The feature means you can use the ‌‌Apple Pencil‌‌ without having to swap over to a keyboard when you need to compose an email, write a Calendar event, visit a URL, send an iMessage, or conduct a search.

Federighi explained to Popular Mechanics how Scribble's early development relied on extensive data gathering involving analyzing how people from all over the world write things down.

"When it comes to understanding [handwriting] strokes, we do data-gathering. We find people all over the world, and have them write things," says Craig Federighi, senior vice president of software engineering at Apple. "We give them a Pencil, and we have them write fast, we have them write slow, write at a tilt. All of this variation."

That methodology is distinct from the comparatively simple approach of scanning and analyzing existing handwriting. Federighi says that for Apple's tech, static examples weren't enough. They needed to see the strokes that formed each letter. "If you understand the strokes and how the strokes went down, that can be used to disambiguate what was being written."

The job of integrating stroke-based recognition with character and word prediction is all done on-device, something that machine learning and raw processing power makes possible, explained Apple's software chief.

"It's gotta be happening in real time, right now, on the device that you're holding," Federighi says. "Which means that the computational power of the device has to be such that it can do that level of processing locally."


Thanks to Scribble and ‌iPadOS 14‌, an ‌iPad‌‌ can be used almost entirely with touch and tools, and there's little need to type. If you're someone who prefers writing by hand, ‌Scribble‌ introduces tools that only enhance the experience. Check out the hands-on video above to see Scribble in action.

Related Roundups: iOS 14, iPadOS 14

Top Rated Comments

Abazigal Avatar
17 weeks ago


Everyday, thousands of MacBook screens are touched and people are bewildered that nothing happens. Touchscreens have been standard for over a decade now - it’s bizarre that there are computers costing thousands of dollars that don’t have touchscreens.

The fact that it’d be used so little didn’t stop Apple from including the touchbar - why on earth haven’t they made their products “Just Work” by including a touchscreen yet?

Seriously - have any kid who hasn’t used a Mac before use a MacBook for a few hours. I guarantee they’ll touch the screen at least once and consider it broken when nothing happens. Literally every phone, tablet, and non-Apple-or-budget laptop has a touchscreen - it’s just a given that a MacBook should, too.

I have read and reread your rationale as to why Macs ought to have touchscreens and it seems like your argument boils down to "just because".

And honestly speaking, I haven't exactly seen touchscreen PCs take the world by storm either. Yes, they are there, and I suppose having a feature you hardly use is better than not having it, but it feels like this is there purely to tick off a checkbox, not so much because it legitimately improves the end user experience.
Score: 13 Votes (Like | Disagree)
chucker23n1 Avatar
17 weeks ago


inb4 people say "steve jobs rolling in his grave" and bringing up the stylus quote...

The stylus quote was clearly a shorthand of "who wants to rely on a stylus exclusively to interact with the UI, when you already have ten fingers?". Using a stylus to write handwritten text or draw something is quite different from that.
Score: 11 Votes (Like | Disagree)
bob122989 Avatar
17 weeks ago
Newton technology all the way. I’ve been surprised they haven’t had it on iPhone and iPad since day one.
Score: 11 Votes (Like | Disagree)
Harvey Zoltan Avatar
17 weeks ago
I would not be surprised if it's Apple Pencil support and not touch support that comes to the AS Macs.
Score: 7 Votes (Like | Disagree)
ArtOfWarfare Avatar
17 weeks ago


You may well be right but many tech commentators seem to be convinced that touch screen Macs are coming. Hopefully not too much longer before we know for sure.

Everyday, thousands of MacBook screens are touched and people are bewildered that nothing happens. Touchscreens have been standard for over a decade now - it’s bizarre that there are computers costing thousands of dollars that don’t have touchscreens.

The fact that it’d be used so little didn’t stop Apple from including the touchbar - why on earth haven’t they made their products “Just Work” by including a touchscreen yet?

Seriously - have any kid who hasn’t used a Mac before use a MacBook for a few hours. I guarantee they’ll touch the screen at least once and consider it broken when nothing happens. Literally every phone, tablet, and non-Apple-or-budget laptop has a touchscreen - it’s just a given that a MacBook should, too.
Score: 6 Votes (Like | Disagree)
ouimetnick Avatar
17 weeks ago
It’s amazing how much more there is to this than just meets the eye. I generally don’t think about on device processing and learning. I’ve been using Scribble since iPadOS 14 came out and it’s really useful for converting notes into text.
Score: 6 Votes (Like | Disagree)

Top Stories

magsafecasedangle

Apple Elaborates on Potential for iPhone 12 and MagSafe Accessories to Interfere With Implantable Medical Devices

Saturday January 23, 2021 2:42 pm PST by
Since the launch of iPhone 12 models in October, Apple has acknowledged that the devices may cause electromagnetic interference with medical devices like pacemakers and defibrillators, but the company has now shared additional information. Apple added the following paragraph to a related support document today:Medical devices such as implanted pacemakers and defibrillators might contain...
Top Stories 44 Feature

Top Stories: 'Thinner and Lighter' MacBook Air, Smaller iPhone 13 Notch, iOS 14.4 Incoming

Saturday January 23, 2021 6:00 am PST by
We continued to hear a lot more about Apple's plans for its Mac lineup this week, including word of a high-end redesigned MacBook Air and the return of an SD card slot as part of the upcoming MacBook Pro redesign. It also sounds like Apple has been working on Face ID for Mac, but it won't be appearing in a redesigned iMac this year as originally planned. This week also saw rumors about the...
bloodoxygenapplewatch

Apple Watch Series 7 Rumored to Feature Blood Glucose Monitoring

Monday January 25, 2021 5:05 am PST by
The Apple Watch Series 7 will reportedly feature blood glucose monitoring via an optical sensor, according to ETNews. The report, which mainly focuses on the blood glucose capabilities of the Samsung Galaxy Watch 4, explains that Apple is intending to bring blood glucose monitoring to the upcoming Apple Watch Series 7 using a non-invasive optical sensor. Measuring blood glucose levels,...
maxresdefault

Microsoft Touts Surface Pro 7 as 'The Better Choice' Over MacBook Pro in New Ad

Saturday January 23, 2021 11:02 am PST by
Microsoft yesterday shared a new ad on YouTube titled "Microsoft Surface Pro 7: The Better Choice," in which the company compares its tablet computer to Apple's 13-inch M1 MacBook Pro, as spotted by MSPoweruser. The ad highlights the Surface Pro 7's touchscreen and included stylus as opposed to only a "little bar" (the Touch Bar) on the MacBook Pro. Other advantages of the Surface Pro 7...
airpods galaxy buds comparison

Samsung Galaxy Buds Pro vs. Apple AirPods Pro

Friday January 22, 2021 2:34 pm PST by
Samsung in January unveiled new flagship Galaxy S21 smartphones and alongside the new phones, introduced the $200 Galaxy Buds Pro, which are priced at $199 and offer Active Noise Cancellation. Subscribe to the MacRumors YouTube channel for more videos. These new Galaxy Buds Pro are clearly designed to compete with Apple's AirPods Pro, so we thought we'd compare the two sets of earbuds in our...
2021 mbp sd slot feature2

Bloomberg: Next MacBook Pro to Feature SD Card Reader

Friday January 22, 2021 7:50 am PST by
Last week, reputable analyst Ming-Chi Kuo outlined his expectations for new 14-inch and 16-inch MacBook Pro models later this year, including the return of the MagSafe charging connector, the removal of the Touch Bar, a new flat-edged design, and the return of more ports built into the notebooks for expanded connectivity. A concept of a modern MacBook Pro with an SD card reader Kuo did not...
time to walk apple watch

Apple Fitness+ Feature 'Time to Walk' Launching Soon With Audio Stories From Special Guests

Saturday January 23, 2021 7:13 pm PST by
Earlier this week, Apple seeded the watchOS 7.3 Release Candidate, which is typically the final beta version of a software update. The release notes for the update list a new "Time to Walk" feature for Apple Fitness+ subscribers, described as "an audio experience in the Workout app where guests share inspiring stories as you walk." Apple Fitness+ subscribers will be able to open the Workout...
Flat MacBook Air Feature

Bloomberg: Apple Working on 'Thinner and Lighter' High-End MacBook Air With MagSafe, Could Launch in Second Half of 2021

Friday January 22, 2021 3:34 am PST by
Apple is working on a "thinner and lighter" version of the MacBook Air that the company plans to release during the second half of this year at the earliest or in 2022, according to a new report by well-connected Bloomberg journalist Mark Gurman. It will include Apple's MagSafe charging technology and a next-generation version of the company's in-house Mac processors. Apple has discussed...
q4 2020 mac growth feature

Mac Sales Skyrocketing After M1 Launch

Tuesday January 19, 2021 3:00 am PST by
Apple's worldwide Mac shipments grew massively in the fourth quarter of 2020 after the launch of three new Macs with the M1 chip, according to new PC shipping estimates shared by Gartner. Apple shipped an estimated 6.9 million Macs, up from the 5.25 million it shipped at the same time in 2019, marking significant growth of 31.3 percent. In spite of the growth of Mac sales, Apple remained...
apple time to walk apple watch airpods 01252021 inline

Apple's New 'Time to Walk' Feature Launches Today for Fitness+ Subscribers

Monday January 25, 2021 6:30 am PST by
Apple today officially introduced Time to Walk, a new feature for Apple Fitness+ subscribers with audio stories from influential people that Apple Watch users can listen to with AirPods or other wireless headphones while walking. Time to Walk episodes will be automatically downloaded to the Apple Watch with a Fitness+ subscription, and users can start an episode directly from the Workout...