Apple Researchers Reveal New AI System That Can Beat GPT-4
Apple researchers have developed an artificial intelligence system named ReALM (Reference Resolution as Language Modeling) that aims to radically enhance how voice assistants understand and respond to commands.
In a research paper (via VentureBeat), Apple outlines a new system for how large language models tackle reference resolution, which involves deciphering ambiguous references to on-screen entities, as well as understanding conversational and background context. As a result, ReALM could lead to more intuitive and natural interactions with devices.
Reference resolution is an important part of natural language understanding, enabling users to use pronouns and other indirect references in conversation without confusion. For digital assistants, this capability has historically been a significant challenge, limited by the need to interpret a wide range of verbal cues and visual information. Apple's ReALM system seeks to address this by converting the complex process of reference resolution into a pure language modeling problem. In doing so, it can comprehend references to visual elements displayed on a screen and integrate this understanding into the conversational flow.
ReALM reconstructs the visual layout of a screen using textual representations. This involves parsing on-screen entities and their locations to generate a textual format that captures the screen's content and structure. Apple researchers found that this strategy, combined with specific fine-tuning of language models for reference resolution tasks, significantly outperforms traditional methods, including the capabilities of OpenAI's GPT-4.
ReALM could enable users to interact with digital assistants much more efficiently with reference to what is currently displayed on their screen without the need for precise, detailed instructions. This has the potential to make voice assistants much more useful in a variety of settings, such as helping drivers navigate infotainment systems while driving or assisting users with disabilities by providing an easier and more accurate means of indirect interaction.
Apple has now published several AI research papers. Last month, the company revealed a new method for training large language models that seamlessly integrates both text and visual information. Apple is widely expected to unveil an array of AI features at WWDC in June.
Popular Stories
Apple is set to unveil iOS 18 during its WWDC keynote on June 10, so the software update is a little over six weeks away from being announced. Below, we recap rumored features and changes planned for the iPhone with iOS 18. iOS 18 will reportedly be the "biggest" update in the iPhone's history, with new ChatGPT-inspired generative AI features, a more customizable Home Screen, and much more....
There are widespread reports of Apple users being locked out of their Apple ID overnight for no apparent reason, requiring a password reset before they can log in again. Users say the sudden inexplicable Apple ID sign-out is occurring across multiple devices. When they attempt to sign in again they are locked out of their account and asked to reset their password in order to regain access. ...
Apple used to regularly increase the base memory of its Macs up until 2011, the same year Tim Cook was appointed CEO, charts posted on Mastodon by David Schaub show. Earlier this year, Schaub generated two charts: One showing the base memory capacities of Apple's all-in-one Macs from 1984 onwards, and a second depicting Apple's consumer laptop base RAM from 1999 onwards. Both charts were...
On this week's episode of The MacRumors Show, we discuss the announcement of Apple's upcoming "Let loose" event, where the company is widely expected to announce new iPad models and accessories. Subscribe to The MacRumors Show YouTube channel for more videos Apple's event invite shows an artistic render of an Apple Pencil, suggesting that iPads will be a focus of the event. Apple CEO Tim...
In his Power On newsletter today, Bloomberg's Mark Gurman outlined some of the new products he expects Apple to announce at its "Let Loose" event on May 7. First, Gurman now believes there is a "strong possibility" that the upcoming iPad Pro models will be equipped with Apple's next-generation M4 chip, rather than the M3 chip that debuted in the MacBook Pro and iMac six months ago. He said a ...
Apple has announced it will be holding a special event on Tuesday, May 7 at 7 a.m. Pacific Time (10 a.m. Eastern Time), with a live stream to be available on Apple.com and on YouTube as usual. The event invitation has a tagline of "Let Loose" and shows an artistic render of an Apple Pencil, suggesting that iPads will be a focus of the event. Subscribe to the MacRumors YouTube channel for more ...
Top Rated Comments
I'm going to stand on the sidelines to see what group A and group B says about this.
Siri: Tell me what you'd like to be reminded about.
Me: This.
Siri: Okay. I've added a reminder called 'this' to your reminders.