Apple Develops Breakthrough Method for Running LLMs on iPhones

Apple GPT in your pocket? It could be a reality sooner than you think. Apple AI researchers say they have made a key breakthrough in deploying large language models (LLMs) on iPhones and other Apple devices with limited memory by inventing an innovative flash memory utilization technique.

siri symbol iphone

LLMs and Memory Constraints

LLM-based chatbots like ChatGPT and Claude are incredibly data and memory-intensive, typically requiring vast amounts of memory to function, which is a challenge for devices like iPhones that have limited memory capacity. To tackle this issue, Apple researchers have developed a novel technique that uses flash memory – the same memory where your apps and photos live – to store the AI model's data.

Storing AI on Flash Memory

In a new research paper titled "LLM in a flash: Efficient Large Language Model Inference with Limited Memory," the authors note that flash storage is more abundant in mobile devices than the RAM traditionally used for running LLMs. Their method cleverly bypasses the limitation using two key techniques that minimize data transfer and maximize flash memory throughput:

  1. Windowing: Think of this as a recycling method. Instead of loading new data every time, the AI model reuses some of the data it already processed. This reduces the need for constant memory fetching, making the process faster and smoother.
  2. Row-Column Bundling: This technique is like reading a book in larger chunks instead of one word at a time. By grouping data more efficiently, it can be read faster from the flash memory, speeding up the AI's ability to understand and generate language.

The combination of these methods allows AI models to run up to twice the size of the iPhone's available memory, according to the paper. This translates to a 4-5 times increase in speed on standard processors (CPUs) and an impressive 20-25 times faster on graphics processors (GPUs). "This breakthrough is particularly crucial for deploying advanced LLMs in resource-limited environments, thereby expanding their applicability and accessibility," write the authors.

Faster AI on iPhone

The breakthrough in AI efficiency opens new possibilities for future iPhones, such as more advanced Siri capabilities, real-time language translation, and sophisticated AI-driven features in photography and augmented reality. The technology also sets the stage for iPhones to run complex AI assistants and chatbots on-device, something Apple is already said to be working on.

Apple's work on generative AI could eventually be incorporated into its ‌Siri‌ voice assistant. Apple in February 2023 held an AI summit and briefed employees on its large language model work. According to Bloomberg, Apple is aiming for a smarter version of Siri that's deeply integrated with AI. Apple is planning to update the way that ‌Siri‌ interacts with the Messages app, allowing users to field complex questions and auto-complete sentences more effectively. Beyond that, Apple is rumored to be planning to add AI to as many Apple apps as possible.

Apple GPT

Apple is reportedly developing its own generative AI model called "Ajax". Designed to rival the likes of OpenAI's GPT-3 and GPT-4, Ajax operates on 200 billion parameters, suggesting a high level of complexity and capability in language understanding and generation. Internally known as "Apple GPT," Ajax aims to unify machine learning development across Apple, suggesting a broader strategy to integrate AI more deeply into Apple's ecosystem.

As of the latest reports, Ajax is considered more capable than the earlier generation ChatGPT 3.5. However, it's also suggested that OpenAI's newer models may have advanced beyond Ajax's capabilities as of September 2023​.

Both The Information and analyst Jeff Pu claim that Apple will have some kind of generative AI feature available on the ‌iPhone‌ and iPad around late 2024, which is when iOS 18 will be coming out. Pu said in October that Apple is building a few hundred AI servers in 2023, with more to come in 2024. Apple will reportedly offer a combination of cloud-based AI and AI with on-device processing.

Top Rated Comments

grantishere Avatar
10 weeks ago
“Siri, when’s Apple GPT going to be released?”

“Here are the neatest pizza restaurants. Would you like directions?”
Score: 66 Votes (Like | Disagree)
Infodataset Avatar
10 weeks ago
Apple is so clever to avoid adding more ram ?
Score: 53 Votes (Like | Disagree)
Abazigal Avatar
10 weeks ago

Does this mean Siri will finally be able to “call my wife on iPhone” again after losing that ability around 2-3 years ago?
At least it didn't respond with "which one?" ?
Score: 34 Votes (Like | Disagree)
wdfly Avatar
10 weeks ago
LOL innovative invention of swapping memory to storage…… maybe they can call it something cool like “cache”.
Score: 22 Votes (Like | Disagree)
subjonas Avatar
10 weeks ago
Sounds like some significant headway. I’d definitely like as much AI done on device as possible.

Apple will probably always be behind with their LLM as long as they prioritize privacy, which I’m very ok with. But just like with big phones, they will bend if there is enough market pressure, which I suspect may eventually be the case.
Score: 21 Votes (Like | Disagree)
contacos Avatar
10 weeks ago
I'll be impressed when Siri is finally multi-lingual. I cannot believe Siri was released with iPhone 4S and it still cannot do it. It could be as simple as being "aware" what language you usually type in with contact A and automatically use Siri in language X to communicate with this person while using language Y with contact Y. All they did was "this text is in another language, do you still want me to read [the gibberish nonsense]" with iOS 16
Score: 18 Votes (Like | Disagree)

Popular Stories

Beyond iPhone 13 Better Blue Face ID Single Camera Hole

Six Reasons to Wait for Next Year's iPhone 17

Thursday February 22, 2024 4:20 am PST by
Apple's iPhone development roadmap runs several years into the future and the company is continually working with suppliers on several successive iPhone models concurrently, which is why we sometimes get rumored feature leaks so far ahead of launch. The iPhone 17 series is no different, and already we have some idea of what to expect from Apple's 2025 smartphone lineup. If you plan to skip...
iPhone Notch

Apple Explored These Notch and Dynamic Island Designs for iPhones

Friday February 23, 2024 2:05 pm PST by
With the iPhone 14 Pro models in 2022, Apple introduced the Dynamic Island, which can morph and expand to display system alerts, sports scores, and a variety of other information. The feature makes the space surrounding the front camera and Face ID sensors useful compared to the notch on older iPhone models. Apple explored a variety of ideas for the iPhone's notch area over the years before...
General Apps Messages

iOS 17.4 to Add This 'Groundbreaking' New Messaging Feature

Friday February 23, 2024 5:05 am PST by
With iOS 17.4, set to arrive in March 2024, Apple is bringing a new cryptographic security feature to iMessage called PQ3. This "groundbreaking" and "state-of-the-art" protocol provides "extensive defenses against even highly sophisticated quantum attacks," according to Apple. Let's break down what that means. Apple's iMessage service already supports end-to-end encryption, but security...
cracked glass vision pro

Handful of Apple Vision Pro Units Develop Identical Crack in Cover Glass

Friday February 23, 2024 4:46 am PST by
A small number of Apple Vision Pro owners have claimed that their headsets developed a hairline crack down the middle of the front cover glass, despite having never been dropped or mishandled. Photo of hairline crack in Vision Pro front glass (credit: Reddit user Inphenite) The first report was posted in the subreddit /r/VisionPro about 18 days ago, and several more have appeared in the last...
New Macs iPads iOS 17 4

What to Expect at Potential Apple Event in March: iPads, Macs, and More

Friday February 23, 2024 8:35 am PST by
Apple often holds its first media event of the year in March, so the company could be just weeks away from announcing new products. Below, we have outlined what to expect from a potential Apple Event this March. Past Apple Events in March Apple has held five events in March since 2015:Monday, March 9, 2015 Monday, March 21, 2016 Tuesday, March 27, 2018 Monday, March 25, 2019 Tuesday,...
iOS 17

When Will Apple Release iOS 17.4 for iPhones?

Friday February 23, 2024 11:56 am PST by
Apple has been beta testing iOS 17.4 for nearly a month now. Below, we explain when the update is likely to be released to all users. In its press release announcing major App Store changes in the EU, which were implemented in response to new regulations under the EU's Digital Markets Act, Apple confirmed that iOS 17.4 will be released at some point in March:Developers can learn about these...
Next Generation CarPlay Porsche 1

Apple Launching Next-Generation CarPlay This Year With These New Features

Wednesday February 21, 2024 2:03 pm PST by
Apple recently updated its website to confirm that the first vehicle models with next-generation CarPlay support will debut "in 2024." This wording is shown on Apple's regional websites for the U.S., Canada, Australia, New Zealand, and many other countries. The iOS 17.4 beta includes code-level references to eight new CarPlay apps:Auto Settings: This app will let you manage paired iPhones...