Apple Teams Up With NVIDIA to Speed Up AI Language Models

Apple has shared details on a collaboration with NVIDIA to greatly improve the performance of large language models (LLMs) by implementing a new text generation technique that offers substantial speed improvements for AI applications.

ml research apple
Apple earlier this year published and open-sourced Recurrent Drafter (ReDrafter), an approach that combines beam search and dynamic tree attention methods to accelerate text generation. Beam search explores multiple potential text sequences at once for better results, while tree attention organizes and removes redundant overlaps among these sequences to improve efficiency.

Apple has now integrated the technology into NVIDIA's TensorRT-LLM framework, which optimizes LLMs running on NVIDIA GPUs, where it achieved "state of the art performance," according to Apple. The integration saw the technique manage a 2.7x speed increase in tokens generated per second during testing with a production model containing tens of billions of parameters.

Apple says the improved performance not only reduces user-perceived latency but also leads to decreased GPU usage and power consumption. From Apple's Machine Learning Research blog:

"LLMs are increasingly being used to power production applications, and improving inference efficiency can both impact computational costs and reduce latency for users. With ReDrafter's novel approach to speculative decoding integrated into the NVIDIA TensorRT-LLM framework, developers can now benefit from faster token generation on NVIDIA GPUs for their production LLM applications."

Developers interested in implementing ReDrafter can find detailed information on both Apple's website and NVIDIA's developer blog.

Tag: Nvidia

Popular Stories

iphone 17 dummies sonny dickson

iPhone 17 Air Almost as Thin as Its Buttons, New Images Show

Thursday April 24, 2025 2:14 am PDT by
If you missed the video showing dummy models of Apple's all-new super thin iPhone 17 Air that's expected later this year, Sonny Dickson this morning shared some further images of the device in close alignment with the other dummy models in the iPhone 17 lineup, indicating just how thin it is likely to be in comparison. The iPhone 17 Air is expected to be around 5.5mm thick – with a thicker ...
iPhone 17 Pro Blue Feature Tighter Crop

iPhone 17 Pro Launching Later This Year With These 13 New Features

Wednesday April 23, 2025 8:31 am PDT by
While the iPhone 17 Pro and iPhone 17 Pro Max are not expected to launch until September, there are already plenty of rumors about the devices. Below, we recap key changes rumored for the iPhone 17 Pro models as of April 2025: Aluminum frame: iPhone 17 Pro models are rumored to have an aluminum frame, whereas the iPhone 15 Pro and iPhone 16 Pro models have a titanium frame, and the iPhone ...
iphone 17 air dummy unbox therapy

iPhone 17 Air's Extreme Thinness Demoed in New Video

Tuesday April 22, 2025 10:22 am PDT by
Apple plans to release an all-new super thin iPhone this year, debuting it alongside the iPhone 17, iPhone 17 Pro, and iPhone 17 Pro Max. We've seen pictures of dummy models, cases, and renders with the design, but Lewis Hilsenteger of Unbox Therapy today showed off newer dummy models that give us a better idea of just how thin the "iPhone 17 Air" will be. The iPhone 17 Air is expected to be ...
AirPods Pro 3 Mock Feature

AirPods Pro 3 Just Months Away – Here's What We Know

Friday April 18, 2025 5:16 am PDT by
Despite being more than two years old, Apple's AirPods Pro 2 still dominate the premium wireless‑earbud space, thanks to a potent mix of top‑tier audio, class‑leading noise cancellation, and Apple's habit of delivering major new features through software updates. With AirPods Pro 3 widely expected to arrive in 2025, prospective buyers now face a familiar dilemma: snap up the proven...
iPhone 17 Air Pastel Feature

iPhone 17 Air Launching Later This Year With These 16 New Features

Thursday April 24, 2025 8:24 am PDT by
While the so-called "iPhone 17 Air" is not expected to launch until September, there are already plenty of rumors about the ultra-thin device. Overall, the iPhone 17 Air sounds like a mixed bag. While the device is expected to have an impressively thin and light design, rumors indicate it will have some compromises compared to iPhone 17 Pro models, including only a single rear camera, a...
ipad air windows 11 arm

M2 iPad Air Runs Windows 11 ARM via Emulation, Thanks to EU Rules

Tuesday April 22, 2025 5:01 am PDT by
A developer has demonstrated Windows 11 ARM running on an M2 iPad Air using emulation, which has become much easier since the EU's Digital Markets Act (DMA) regulations came into effect. As spotted by Windows Latest, NTDev shared an instance of the emulation on social media and posted a video on YouTube (embedded below) demonstrating it in action. The achievement relies on new EU regulatory...
Global Close Your Rings Day Pin

Apple Stores Giving Away a Limited-Edition Pin For Free Today

Thursday April 24, 2025 10:15 am PDT by
Starting today, April 24, Apple Stores around the world are giving away a special pin for free to customers who request one, while supplies last. Photo Credit: Filip Chudzinski The enamel pin's design is inspired by the Global Close Your Rings Day award in the Activity app, which Apple Watch users can receive by closing all three Activity rings today. The limited-edition pin is the physical...
Apple Logo Spotlight Blue

White House Hits Back at Apple's Massive EU Fine

Thursday April 24, 2025 5:57 am PDT by
Apple's $570 million fine from the EU has triggered a sharp rebuke from the White House, which called the fine a form of economic extortion, Reuters reports. The fine was announced on Wednesday by the European Commission, following a formal investigation into Apple's compliance with the bloc's Digital Markets Act (DMA), a landmark piece of legislation aimed at curbing the market dominance of ...

Top Rated Comments

attohs Avatar
18 weeks ago
NVidia? Did hell freeze over again?
Score: 37 Votes (Like | Disagree)
vegetassj4 Avatar
18 weeks ago
NVIDIA and Apple??!!? Working together again?



Attachment Image
Score: 13 Votes (Like | Disagree)
Delgibbons Avatar
18 weeks ago
Can't wait to put a 5090 in my Ma....

oh.
Score: 12 Votes (Like | Disagree)
redbeard331 Avatar
18 weeks ago
Good we have to hurry this up.



Attachment Image
Score: 9 Votes (Like | Disagree)
lilkwarrior Avatar
18 weeks ago
What would be an even better collaboration would be Apple enabling Nvidia GPU options again—at least for the Mac Pro.

It would be AWESOME to be able to use Nvidia’s ray-tracing and tensor cores with my creative professional and AI problems with Titan-class/Prosumer/workstation GPUs (x90 and up) again without having to switch to my PC.

A Nvidia MPX GPU module as capable as a 5090 with no wires and Thunderbolt 5 support would be a nirvana-like outcome—especially if Microsoft, Apple, and/or Valve enables a way to dual boot to Windows on ARM and SteamOS.

While I love building a liquid-cooled PC, I and various prosumers would finally have a choice to stop buying PCs altogether
Score: 7 Votes (Like | Disagree)
Unregistered 4U Avatar
18 weeks ago

Since Apple now produces its own GPUs there is no need for hell to freeze over. Do you even remember the reason Apple and Nvidia parted ways? It was over Nvidia wanting complete access to macOS’s core. Apple said no way.
And, we’ve since had a REALLY good example (CrowdStrike) of why this would have been a baaaad idea.
Score: 4 Votes (Like | Disagree)