Apple Researchers Reveal New AI System That Can Beat GPT-4
Apple researchers have developed an artificial intelligence system named ReALM (Reference Resolution as Language Modeling) that aims to radically enhance how voice assistants understand and respond to commands.
In a research paper (via VentureBeat), Apple outlines a new system for how large language models tackle reference resolution, which involves deciphering ambiguous references to on-screen entities, as well as understanding conversational and background context. As a result, ReALM could lead to more intuitive and natural interactions with devices.
Reference resolution is an important part of natural language understanding, enabling users to use pronouns and other indirect references in conversation without confusion. For digital assistants, this capability has historically been a significant challenge, limited by the need to interpret a wide range of verbal cues and visual information. Apple's ReALM system seeks to address this by converting the complex process of reference resolution into a pure language modeling problem. In doing so, it can comprehend references to visual elements displayed on a screen and integrate this understanding into the conversational flow.
ReALM reconstructs the visual layout of a screen using textual representations. This involves parsing on-screen entities and their locations to generate a textual format that captures the screen's content and structure. Apple researchers found that this strategy, combined with specific fine-tuning of language models for reference resolution tasks, significantly outperforms traditional methods, including the capabilities of OpenAI's GPT-4.
ReALM could enable users to interact with digital assistants much more efficiently with reference to what is currently displayed on their screen without the need for precise, detailed instructions. This has the potential to make voice assistants much more useful in a variety of settings, such as helping drivers navigate infotainment systems while driving or assisting users with disabilities by providing an easier and more accurate means of indirect interaction.
Apple has now published several AI research papers. Last month, the company revealed a new method for training large language models that seamlessly integrates both text and visual information. Apple is widely expected to unveil an array of AI features at WWDC in June.
Popular Stories
Apple's iPhone 17 Pro and iPhone 17 Pro Max models will feature a number of significant display, thermal, and battery improvements, according to new late-stage rumors.
According to the Weibo leaker known as "Instant Digital," the iPhone 17 Pro models will feature displays with higher brightness, making it more suitable for use in direct sunlight for prolonged periods. The iPhone 16 Pro and...
Apple is expected to unveil the iPhone 17 series on Tuesday, September 9, and last-minute rumors about the devices continue to surface.
The latest info comes from a leaker known as Majin Bu, who has shared alleged images of Apple's Clear Case for the iPhone 17 Pro and Pro Max, or at least replicas.
Image Credit: @MajinBuOfficial
The images show three alleged changes compared to Apple's iP...
Apple will launch its new iPhone 17 series this month, and the iPhone 17 Pro models are expected to get a new design for the rear casing and the camera area. But more significant changes to the lineup are not expected until next year, when the iPhone 18 models arrive.
If you're thinking of trading in your iPhone for this year's latest, consider the following features rumored to be coming to...
Just one week before Apple is expected to unveil the iPhone 17 series, an analyst has shared new price estimates for the devices.
Here are J.P. Morgan analyst Samik Chatterjee's price estimates for the iPhone 17 series in the United States, according to 9to5Mac:
Model
Starting Price
Model
Starting Price
Change
iPhone 16
$799
iPhone 17
...
An iPhone 17 announcement is a dead cert for September 2025 – Apple has already sent out invites for an "Awe dropping" event on Tuesday, September 9 at the Apple Park campus in Cupertino, California. The timing follows Apple's trend of introducing new iPhone models annually in the fall.
At the event, Apple is expected to unveil its new-generation iPhone 17, an all-new ultra-thin iPhone 17...
Apple is preparing to release iOS 18.7 for compatible iPhone models, according to evidence of the update in the MacRumors visitor logs.
We expect iOS 18.7 to be released in September, alongside iOS 26. The update will likely include fixes for security vulnerabilities, but little else.
iOS 18.7 will be one of the final updates ever released for the iPhone XS, iPhone XS Max, and iPhone XR,...
A new survey has found that nearly seven in ten iPhone owners in the United States plan to upgrade to an iPhone 17 model, signaling strong demand ahead of Apple's expected unveiling of the devices at its September 9 keynote.
Smartphone price comparison platform SellCell surveyed over 2,000 U.S.-based iPhone users in August to assess upgrade interest and brand loyalty before Apple's event....
