One of the biggest questions surrounding models like ChatGPT, Gemini, and Midjourney since they launched is what role, if any, they will play in everyday life. That's something Apple is trying to answer with its own take on the category, Apple Intelligence, which was officially unveiled at WWDC 2024 this week.
The company went all out for show at its launch event on Monday. That's what keynotes are all about. When Senior Vice President Craig Federighi wasn't skydiving or doing parkour tricks with the help of some Hollywood (Cupertino, to be precise) wizardry, Apple was determined to prove that its models were just as good as the competition.
The beta was just released on Monday, so the verdict on that question is still out, but the company has since shed light on some of the ways its approach to generative AI differs from others. First, its scope. Many of the most prominent companies in the field take a “bigger is better” approach to models. The goal of these systems is to act as a kind of one-stop shop for the world's information.
Meanwhile, Apple's approach in this space is based on something more pragmatic: Apple Intelligence is a more tailored approach to generative AI built specifically on top of the company's various operating systems. It's a very Apple-like approach in that it prioritizes a smooth user experience above all else.
In some ways Apple Intelligence is a branding exercise, but in other ways the company wants aspects of generative AI to blend seamlessly into the operating system, and it's perfectly fine (and even preferable) if users are completely ignorant of the underlying technology that powers these systems: Apple products have always worked that way.
Keep your models small
A key part of these efforts is creating smaller models, training the system on customized datasets designed specifically for the kinds of features users of its operating system need. It's not immediately clear how much the size of these models will affect the black-box problem, but Apple believes that at the very least, having more topic-specific models will provide more transparency into why the system makes certain decisions.
Because of the relatively limited nature of these models, Apple doesn't expect there to be much variation when telling the system to summarize text, for example. Ultimately, though, the variation per prompt will depend on the length of the text to be summarized. The operating system also has a feedback mechanism that allows users to report any issues with the generative AI system.
Apple Intelligence is much more focused than its larger models, but includes specialized “adapters” for different tasks and styles to accommodate a wide range of demands. In general, though, Apple's approach to modeling is not “bigger is better,” as it has to take into account size, speed, computational power, etc., especially when dealing with on-device models.
ChatGPT, Gemini, etc.
Given the narrow focus of Apple's model, it makes sense to open it up to third-party models like OpenAI's ChatGPT. The company trained its system specifically for the macOS/iOS experience, so there's a lot of information that's outside of its scope. If the system determines that a third-party application is better suited to respond, a system prompt will ask whether you want to share that information externally. If you don't see such a prompt, your request is being handled by Apple's in-house model.
This should work the same for all external models that Apple partners with, including Google Gemini. It's one of the rare instances where a system draws attention to its use of generative AI in this way. The decision was made to address privacy concerns, as each company has different standards when it comes to collecting and training user data.
Requiring users to opt in every time takes some of the strain off Apple, even if it adds some friction to the process. You can also opt out of using third-party platforms system-wide, but that would limit the amount of data the operating system/Siri can access. You can't opt out of Apple Intelligence all at once, though; instead, you must opt out feature by feature.
Private Cloud Computing
Meanwhile, it won't say whether the system processes certain queries on-device or via remote servers with private cloud computing — Apple's philosophy is that such disclosure is unnecessary since it holds its servers to the same privacy standards as its devices, all the way down to the first-party silicon they run on.
One way to know for sure if a query is on-device or off-device managed is to disconnect the machine from the Internet: if cloud computing is required to resolve the issue and the machine can't find a network, you'll get an error stating that the requested action can't be completed.
Apple has detailed which actions will require cloud-based processing. There are several factors at play, and these systems are constantly changing, so what requires cloud computing today may be able to run on-device tomorrow. On-device computing won't always be the faster option, as speed is one of the parameters Apple Intelligence considers when determining where to process a prompt.
However, there are some operations that always run on the device, most notably Image Playground, where the full diffusion model is stored locally. Apple has tweaked the model to generate images in three different in-house styles: animated, illustrated, and sketched. The animation style closely resembles the in-house style of another company founded by Steve Jobs. Similarly, text generation is now available in three styles: friendly, professional, and concise.
Even in this early beta stage, Image Playground is surprisingly fast to generate, often taking just a few seconds. When it comes to the question of whether to include a person in the generated image, the system requires you to input details rather than simply guessing things like ethnicity.
What will Apple do with its data sets?
Apple's models are trained on a combination of licensed datasets and publicly available crawls, the latter of which is powered by AppleBot. The company's web crawler has been around for a while and provides contextual data for apps like Spotlight, Siri, and Safari. The crawler already has an opt-out feature for publishers.
“With Applebot-Extended, web publishers can opt out of having their website content used to train Apple's underlying models that power generative AI capabilities across Apple products, including Apple Intelligence, Services and developer tools,” Apple said.
This is accomplished by building prompts into the code of your website. With the arrival of Apple Intelligence, the company has introduced a second prompt, which allows your site to be included in search results but excluded from training generative AI models.
Responsible AI
On the first day of WWDC, Apple published a white paper titled “Introducing Apple's On-Device and Server-Based Model.” The white paper highlights the principles that govern the company's AI model. In particular, Apple emphasizes the following four points:
“Empower users with intelligent tools: We identify areas where we can use AI responsibly and create tools that address the needs of specific users. We respect how users use these tools to achieve their goals.” “Represent our users: We build highly personal products with the goal of faithfully representing our users around the world. We continuously work to ensure that stereotypes and systemic biases are not perpetuated across our AI tools and models.” “Design carefully: We take precautions at every stage of the process, including design, model training, feature development, and quality assessment, to identify how our AI tools could be misused or lead to potential harm. We use user feedback to continuously and proactively improve our AI tools.” “Protect privacy: We protect user privacy with powerful in-device processing and groundbreaking infrastructure, including private cloud computing. We never use private, personal user data or user interactions when training the underlying models.”
Apple has a unique approach to its foundational model, allowing the system to be tailored to the user experience. The company has adopted this UX-first approach since the introduction of the original Mac. Providing the smoothest possible experience is in the user's interest, but it should not come at the expense of privacy.
It's a tricky balance the company will have to strike when the current set of OS betas roll out to the public this year. The ideal approach is to give end users just the information they need (or the bare minimum). Of course, there will be many people who don't care if a query runs on their machine or in the cloud, and will be happy to let the system default to whatever is most accurate and efficient.
For privacy advocates and those interested in these details, Apple should strive for as much transparency as possible for users, including, of course, for publishers who may not want to contribute their content to training these models. While there are some unavoidable black box issues at this point, where transparency can be provided, it should be provided at the users' request.