New Apple AI Model Edits Images Based on Natural Language Input

Apple researchers have released a new open-source AI model that is capable of editing images based on a user's natural language instructions (via VentureBeat).

DALL%C2%B7E apple logo image editing ai

MacRumors image made with DALL·E

Called "MGIE," which stands for MLLM-Guided Image Editing, it uses multimodal large language models (MLLMs) to interpret user requests and perform pixel-level manipulations.

The model is capable of editing various aspects of images. Global photo enhancements can include brightness, contrast, or sharpness, or the application of artistic effects like sketching. Local editing can modify the shape, size, color, or texture of specific regions or objects in an image, while Photoshop-style modifications can include cropping, resizing, rotating, and adding filters, or even changing backgrounds and blending images.

A user input for a photo of a pizza could be to "make it look more healthy." Using common sense reasoning, the model can add vegetable toppings, such as tomatoes and herbs. A global optimization input request might take the form of "add contrast to simulate more light," while a Photoshop-style modification could be made by asking the model to remove people from the background of a photo, shifting the focus of the image to the subject's facial expression.

Apple collaborated with University of California researchers to create MGIE, which was presented in a paper at the International Conference on Learning Representations (ICLR) 2024. The model is available on GitHub, and includes the code, data, and pre-trained models.

MGIE apple AI model image editing
This is Apple's second breakthrough in AI research in as many months. In late December, Apple revealed that it had made strides in deploying large language models (LLMs) on iPhones and other Apple devices with limited memory by inventing an innovative flash memory utilization technique.

For the last several months, Apple has been testing an "Apple GPT" rival that could compete with ChatGPT. According to Bloomberg's Mark Gurman, work on AI is a priority for Apple, with the company designing an "Ajax" framework for large language models.

Both The Information and analyst Jeff Pu claim that Apple will have some kind of generative AI feature available on the ‌iPhone‌ and iPad around late 2024, which is when iOS 18 will be coming out. iOS 18 is said to include an enhanced version of Siri with ChatGPT-like generative AI functionality, and has the potential to be the "biggest" software update in the iPhone's history, according to Gurman.

Popular Stories

iOS 18 Siri Integrated Feature

Report: These 10 New AI Features Are Coming in iOS 18

Sunday May 26, 2024 12:57 pm PDT by
iOS 18 and macOS 15 will offer an array of new AI features such as auto-generated emojis, suggested replies to emails and messages, and more, Bloomberg's Mark Gurman reports. A significant portion of Apple's Worldwide Developers Conference (WWDC) is expected to focus on AI features. Writing his latest "Power On" newsletter, Gurman explained that Apple's AI strategy emphasizes providing...
airpods pro 2 pink

Apple Releases New AirPods Pro 2 Firmware

Tuesday May 28, 2024 11:46 am PDT by
Apple today released new firmware update for both the Lightning and USB-C versions of the AirPods Pro 2. The new firmware is version 6F7, up from the 6B34 firmware released in November. Apple does not provide details on what features might be included in the refreshed firmware beyond "bug fixes and other improvements," so it is unclear what's new in the update. Apple does not give...
new best buy blue

Best Buy's Memorial Day Sale Has Record Low Prices on iPads, MacBooks, and Much More

Friday May 24, 2024 7:12 am PDT by
Best Buy today kicked off its Memorial Day weekend sale, and it has some of the best prices we've tracked in weeks on iPads and MacBooks. Specifically, you'll find record low prices on the 5th generation iPad Air, iPad mini 6, M2 MacBook Air, and M3 MacBook Pro. Note: MacRumors is an affiliate partner with Best Buy. When you click a link and make a purchase, we may receive a small payment,...
iOS 18 WWDC 24 Feature 2

Gurman: iOS 18 Will Allow Users to Recolor App Icons and Place Them Anywhere

Sunday May 26, 2024 12:22 pm PDT by
Apple's iOS 18 update will introduce new features for further customizing the iPhone's home screen, according to Bloomberg's Mark Gurman. In the latest edition of his "Power On" newsletter, Gurman claimed that Apple will allow users to change the color of app icons in iOS 18. For example, "you can make all your social icons blue or finance-related ones green." This kind of home screen...
iPad Pro Landscape Apple Logo Feature

Apple Says Future iPads Could Feature Landscape Apple Logo

Monday May 27, 2024 6:31 am PDT by
French website Numerama interviewed three senior Apple employees about the new iPad Pro models that launched earlier this month. While the discussion did not reveal many new details, it did mention one potential change for future iPads. While the Apple logo on the back of iPads is positioned so that it appears upright in vertical orientation, the devices are often used in landscape...

Top Rated Comments

klasma Avatar
16 weeks ago
Apple really can’t make up their minds, can they?

Attachment Image
Score: 19 Votes (Like | Disagree)
AlmightyKang Avatar
16 weeks ago
Apple as always do something because it has a benefit to the user, not just for the sake of the technology. This is the ML approach we need.

Most of the VC and AI industry are just grifting and trying to sell it as an application for everything and trying to create new markets for it.
Score: 13 Votes (Like | Disagree)
AAPLbuyback Avatar
16 weeks ago

The main imagine is kinda creepy...
Creepy is part of the allure of AI.
Score: 12 Votes (Like | Disagree)
ThatGuyInLa Avatar
16 weeks ago
And yet even then we won’t be able to ask Siri, “Turn on kitchen lights and set to 35% and light blue.”

“Play my subscriptions for YouTube on AppleTV.”
Score: 11 Votes (Like | Disagree)
kirky29 Avatar
16 weeks ago
The main imagine is kinda creepy...
Score: 10 Votes (Like | Disagree)
Jason2000 Avatar
16 weeks ago

The new development is that Apple can now do it too, which means that it’s exciting and innovative now.
It's exciting if Apple is bringing this to their photos app. I don't want to have to use Google for this type stuff.
Score: 9 Votes (Like | Disagree)