Craig Federighi Explains How Scribble Feature in iPadOS 14 Was Developed

Scribble is one of the most notable new features in iPadOS 14, and in a new interview with Popular Mechanics, Apple SVP of software engineering Craig Federighi has revealed more details about how it was developed and how it works on iPad.

apple ipados 14 scribble 062220 inline
With Scribble in iPadOS 14, you can use an ‌Apple Pencil‌ to write in any text field, and the written text is then converted to typed text. The feature means you can use the ‌‌Apple Pencil‌‌ without having to swap over to a keyboard when you need to compose an email, write a Calendar event, visit a URL, send an iMessage, or conduct a search.

Federighi explained to Popular Mechanics how Scribble's early development relied on extensive data gathering involving analyzing how people from all over the world write things down.

"When it comes to understanding [handwriting] strokes, we do data-gathering. We find people all over the world, and have them write things," says Craig Federighi, senior vice president of software engineering at Apple. "We give them a Pencil, and we have them write fast, we have them write slow, write at a tilt. All of this variation."

That methodology is distinct from the comparatively simple approach of scanning and analyzing existing handwriting. Federighi says that for Apple's tech, static examples weren't enough. They needed to see the strokes that formed each letter. "If you understand the strokes and how the strokes went down, that can be used to disambiguate what was being written."

The job of integrating stroke-based recognition with character and word prediction is all done on-device, something that machine learning and raw processing power makes possible, explained Apple's software chief.

"It's gotta be happening in real time, right now, on the device that you're holding," Federighi says. "Which means that the computational power of the device has to be such that it can do that level of processing locally."


Thanks to Scribble and iPadOS 14, an ‌iPad‌‌ can be used almost entirely with touch and tools, and there's little need to type. If you're someone who prefers writing by hand, ‌Scribble‌ introduces tools that only enhance the experience. Check out the hands-on video above to see Scribble in action.

Related Forums: iOS 14, iPad Accessories

Popular Stories

iOS 18 Siri Integrated Feature

Report: These 10 New AI Features Are Coming in iOS 18

Sunday May 26, 2024 12:57 pm PDT by
iOS 18 and macOS 15 will offer an array of new AI features such as auto-generated emojis, suggested replies to emails and messages, and more, Bloomberg's Mark Gurman reports. A significant portion of Apple's Worldwide Developers Conference (WWDC) is expected to focus on AI features. Writing his latest "Power On" newsletter, Gurman explained that Apple's AI strategy emphasizes providing...
airpods pro 2 pink

Apple Releases New AirPods Pro 2 Firmware

Tuesday May 28, 2024 11:46 am PDT by
Apple today released new firmware update for both the Lightning and USB-C versions of the AirPods Pro 2. The new firmware is version 6F7, up from the 6B34 firmware released in November. Apple does not provide details on what features might be included in the refreshed firmware beyond "bug fixes and other improvements," so it is unclear what's new in the update. Apple does not give...
new best buy blue

Best Buy's Memorial Day Sale Has Record Low Prices on iPads, MacBooks, and Much More

Friday May 24, 2024 7:12 am PDT by
Best Buy today kicked off its Memorial Day weekend sale, and it has some of the best prices we've tracked in weeks on iPads and MacBooks. Specifically, you'll find record low prices on the 5th generation iPad Air, iPad mini 6, M2 MacBook Air, and M3 MacBook Pro. Note: MacRumors is an affiliate partner with Best Buy. When you click a link and make a purchase, we may receive a small payment,...
iOS 18 WWDC 24 Feature 2

Gurman: iOS 18 Will Allow Users to Recolor App Icons and Place Them Anywhere

Sunday May 26, 2024 12:22 pm PDT by
Apple's iOS 18 update will introduce new features for further customizing the iPhone's home screen, according to Bloomberg's Mark Gurman. In the latest edition of his "Power On" newsletter, Gurman claimed that Apple will allow users to change the color of app icons in iOS 18. For example, "you can make all your social icons blue or finance-related ones green." This kind of home screen...
iPad Pro Landscape Apple Logo Feature

Apple Says Future iPads Could Feature Landscape Apple Logo

Monday May 27, 2024 6:31 am PDT by
French website Numerama interviewed three senior Apple employees about the new iPad Pro models that launched earlier this month. While the discussion did not reveal many new details, it did mention one potential change for future iPads. While the Apple logo on the back of iPads is positioned so that it appears upright in vertical orientation, the devices are often used in landscape...

Top Rated Comments

Abazigal Avatar
48 months ago

Everyday, thousands of MacBook screens are touched and people are bewildered that nothing happens. Touchscreens have been standard for over a decade now - it’s bizarre that there are computers costing thousands of dollars that don’t have touchscreens.

The fact that it’d be used so little didn’t stop Apple from including the touchbar - why on earth haven’t they made their products “Just Work” by including a touchscreen yet?

Seriously - have any kid who hasn’t used a Mac before use a MacBook for a few hours. I guarantee they’ll touch the screen at least once and consider it broken when nothing happens. Literally every phone, tablet, and non-Apple-or-budget laptop has a touchscreen - it’s just a given that a MacBook should, too.
I have read and reread your rationale as to why Macs ought to have touchscreens and it seems like your argument boils down to "just because".

And honestly speaking, I haven't exactly seen touchscreen PCs take the world by storm either. Yes, they are there, and I suppose having a feature you hardly use is better than not having it, but it feels like this is there purely to tick off a checkbox, not so much because it legitimately improves the end user experience.
Score: 13 Votes (Like | Disagree)
chucker23n1 Avatar
48 months ago

inb4 people say "steve jobs rolling in his grave" and bringing up the stylus quote...
The stylus quote was clearly a shorthand of "who wants to rely on a stylus exclusively to interact with the UI, when you already have ten fingers?". Using a stylus to write handwritten text or draw something is quite different from that.
Score: 11 Votes (Like | Disagree)
bob122989 Avatar
48 months ago
Newton technology all the way. I’ve been surprised they haven’t had it on iPhone and iPad since day one.
Score: 11 Votes (Like | Disagree)
Harvey Zoltan Avatar
48 months ago
I would not be surprised if it's Apple Pencil support and not touch support that comes to the AS Macs.
Score: 7 Votes (Like | Disagree)
ArtOfWarfare Avatar
48 months ago

You may well be right but many tech commentators seem to be convinced that touch screen Macs are coming. Hopefully not too much longer before we know for sure.
Everyday, thousands of MacBook screens are touched and people are bewildered that nothing happens. Touchscreens have been standard for over a decade now - it’s bizarre that there are computers costing thousands of dollars that don’t have touchscreens.

The fact that it’d be used so little didn’t stop Apple from including the touchbar - why on earth haven’t they made their products “Just Work” by including a touchscreen yet?

Seriously - have any kid who hasn’t used a Mac before use a MacBook for a few hours. I guarantee they’ll touch the screen at least once and consider it broken when nothing happens. Literally every phone, tablet, and non-Apple-or-budget laptop has a touchscreen - it’s just a given that a MacBook should, too.
Score: 6 Votes (Like | Disagree)
ouimetnick Avatar
48 months ago
It’s amazing how much more there is to this than just meets the eye. I generally don’t think about on device processing and learning. I’ve been using Scribble since iPadOS 14 came out and it’s really useful for converting notes into text.
Score: 6 Votes (Like | Disagree)