Hey everyone, welcome to TechCrunch's regular AI newsletter.
This week in AI, Apple took center stage.
At its Worldwide Developers Conference (WWDC) in Cupertino, Apple unveiled Apple Intelligence, its long-awaited ecosystem-wide generative AI effort that will power everything from an upgraded Siri to AI-generated emojis to photo editing tools that remove unwanted people and objects from photos.
The company promised that Apple Intelligence is built with safety at its core and with highly personalized experiences in mind.
“It has to understand you and be rooted in your personal context: your daily life, your relationships, your communications,” CEO Tim Cook said during his keynote address Monday. “All of this goes beyond artificial intelligence. It's personal intelligence, and it's the next big step for Apple.”
Apple Intelligence is typical Apple, hiding technical details behind apparently intuitive features (Cook never utters the words “large-scale language models”), but as someone who writes about the ins and outs of AI for a living, I wish Apple would just be a little more transparent about how the sausage is made, once and for all.
Take Apple's model training practices, for example. In a blog post, Apple revealed that it trains the AI models that power Apple Intelligence on a combination of licensed datasets and the public web. Publishers have the option to opt out of future training. But what if you're an artist who wants to know if your work was caught up in Apple's initial training? Too bad. You'll have to keep quiet.
The secrecy may be for competitive reasons, but I suspect it's also to protect Apple from legal issues, especially those related to copyright: Courts have yet to decide whether vendors like Apple have the right to train on publicly available data without paying or crediting the creators of that data — in other words, whether the fair use principle applies to generative AI.
It's a bit disappointing to see Apple, a company that often presents itself as a champion of common-sense tech policy, tacitly embrace the fair use argument. Through a veil of marketing, Apple can claim to be taking a responsible and careful approach to AI, when in reality it may be training creators' work without their permission.
Even a little clarification would go a long way. It's a shame we don't have one yet, and unless a lawsuit (or two) is filed, I don't see us getting one anytime soon.
news
Apple's key AI features: From an upgraded Siri to deeper integration with OpenAI's ChatGPT, we've rounded up the major AI features Apple announced at this week's WWDC keynote.
OpenAI makes executive hires: OpenAI this week hired Sarah Friar, former CEO of hyperlocal social network Nextdoor, as chief financial officer and Kevin Weil, who led product development at Instagram and Twitter, as chief product officer.
Mail gets even more AI-fueled: This week, Yahoo (TechCrunch's parent company) updated Yahoo Mail with new AI features, including AI-generated summaries of emails. Google recently introduced a similar generated summaries feature, but it requires a fee.
Controversial views: A recent study from Carnegie Mellon University found that not all generative AI models are created equal, especially when it comes to how they handle controversial subjects.
Sound Generator: Stability AI, the startup behind the AI-powered art generator Stable Diffusion, has released an open AI model that generates sounds and songs that it says are trained exclusively on royalty-free recordings.
Research Paper of the Week
Google believes it can build generative AI models for personal health, or at least take preliminary steps in that direction.
In a new paper featured on Google's official AI blog, Google researchers revealed details about the Personal Health Large Language Model (PH-LLM for short), a fine-tuned version of Google's Gemini model that's designed to provide recommendations for improving sleep and fitness by reading heart and respiration rate data from wearables like smartwatches.
To test whether PH-LLM could provide useful health advice, the researchers created nearly 900 case studies about sleep and fitness with participants living in the U.S. They found that the sleep advice provided by PH-LLM was close to, but not significantly better than, advice from human sleep experts.
The researchers say PH-LLM could be useful for contextualizing physiological data for “personal health applications.” Google Fit comes to mind. I wouldn't be surprised if PH-LLM eventually makes its way into new features for Fit and other fitness-focused Google apps.
Model of the Week
Apple has gone into considerable detail in a blog post detailing the new on-device and cloud-based generative AI models that make up the Apple Intelligence suite. However, despite the length of the post, very little is revealed about the models' capabilities. Here's our best interpretation:
The unnamed on-device model that Apple highlights is small in size and can certainly run offline on Apple devices like the iPhone 15 Pro and Pro Max. It contains 3 billion parameters (a “parameter” is the part of a model that essentially defines the model's skill for a problem, such as text generation), rivaling Google's on-device Gemini models, the Gemini Nano, which come in at 1.8 billion parameters and 3.25 billion parameters in size.
Meanwhile, the server model is larger (Apple doesn't say exactly how much larger). What we do know is that it's more powerful than the on-device model: In Apple's benchmarks, the on-device model performs on par with models like Microsoft's Phi-3-mini, Mistral's Mistral 7B, and Google's Gemma 7B, while Apple claims the server model “compares” with OpenAI's previous flagship GPT-3.5 Turbo model.
Apple also says that both the on-device and server models are less likely to go off track (i.e., toxic eruptions) than similarly sized models. That may be true, but this writer will reserve judgment until he has a chance to test the Apple Intelligence.
Grab Bag
This week marks the six-year anniversary of the release of GPT-1, the ancestor of OpenAI's latest flagship generative AI model, GPT-4o. Deep learning may be hitting a wall, but it's amazing how far the field has come.
Consider that it took a month to train GPT-1 on a 4.5 gigabyte text dataset (BookCorpus, which contains about 7,000 unpublished fiction books). It took 34 days to train GPT-3, which is about 1,500 times larger in terms of parameters than GPT-1 and is significantly more sophisticated in the prose it can generate and analyze. How does that scale?
What made GPT-1 revolutionary was its approach to training. Previous techniques were limited in their usefulness because they relied on large amounts of manually labeled data. (Manually labeling data is time-consuming and tedious.) But GPT-1 was different: it was trained primarily on unlabeled data, and “learned” how to perform various tasks (such as writing an essay).
Many experts believe that a paradigm shift as meaningful as GPT-1 cannot come soon enough, but the world also did not see its arrival.