Microsoft wants to bring generative AI to the forefront of Windows and the PCs that run it.
In two keynotes at this week's annual Build developer conference, the company announced a new line of Windows machines it calls Copilot+ PCs and leverages AI such as Recall to help users find apps, files, and other content. announced the generation function. What I saw in the past. Copilot, Microsoft's brand of generative AI, will soon be more deeply integrated into the Windows 11 experience. And new Microsoft Surface devices are also coming.
Here are all the major announcements from Monday and Tuesday.
volume measurement app
Microsoft is bringing Windows Volumetric Apps (basically spatially aware, interactive VR apps) to the Meta Quest headset. Microsoft says that through its partnership with Meta, it will bring Windows 365 and local PC connectivity to the Quest headset, allowing developers to extend their apps into 3D space.
Image credit: Microsoft
During Tuesday's keynote, Microsoft showed off a digitally exploded 3D view of the Xbox controller from the perspective of the Meta Quest 3 headset, a digital object that the wearer can manipulate with their hands. “We're deepening our partnership with Meta to make Windows the best-in-class experience on Quest devices,” Pavan Davuluri, Microsoft's CVP of Windows and Devices, said during the demo. .
Developers can sign up for a preview today to access Microsoft's new volumetric API.
Copilot+ PC
Image credit: Microsoft
Copilot+ PC is the flagship Windows hardware that represents Microsoft's AI-first vision. All include a specialized chip called an NPU to power AI experiences such as Recall. It also ships with a minimum of 16 GB of RAM and paired with SSD storage.
The first Copilot+ PCs will be powered by Qualcomm's Snapdragon X Elite and Plus chips. Microsoft claims these chips deliver up to 15 hours of web browsing and 20 hours of video playback battery life. Chipmakers Intel and AMD are also partnering with various manufacturers, including Acer, Asus, Dell, HP, Lenovo, and Samsung, to develop processors for Copilot+ devices.
Pricing for the Copilot+ PC starts at $999, with select units available for pre-order now.
Surface Pro and Surface Laptop
Microsoft's newly announced Surface devices, Surface Laptop and Surface Pro, focus on performance and battery.
Image credit: Microsoft
The latest Surface Laptops with 13.8-inch or 15-inch displays have been redesigned with “modern lines” and thinner screen bezels. The company says it lasts up to 22 hours on a single charge and is up to 86% faster than the Surface Laptop 5. It also supports Wi-Fi 7 and has a haptic feedback touchpad.
Image credit: Microsoft
Regarding the new Surface Pro, Microsoft claims it is up to 90% faster than the previous generation Surface Pro (Surface Pro 9), has a new OLED with HDR display, Wi-Fi 7 (and optional 5G), an upgraded ultra-wide front-facing camera, and an additional carbon fiber reinforced detachable keyboard with haptic feedback.
recollection
Windows 11's upcoming Recall feature will allow users to “remember” apps and content they accessed on their PC weeks or months ago. For example, it can help you find your girlfriend's Discord chat where she was discussing the clothes she was considering buying. Users can use her Recall timeline to “scroll back” to see what they've been working on recently, or drill down into files such as PowerPoint presentations for information that may be relevant to their search. You can also display it.
Microsoft says Recall can create associations such as colors and images, allowing users to search for virtually anything on their PC in natural language (not unlike technology from startup Rewind). Developers can improve recall by adding contextual information to their apps. And Microsoft claims that all user data associated with Recall is stored privately on the device and, importantly, is not used to train AI models.
Here's some more information from Microsoft: “Snapshots belong to you and are stored locally on your PC. You can delete individual snapshots, adjust and delete the time range in settings, or pause them at any time from the icon in the system tray on your taskbar. You can also filter apps and websites to not save.”
Image editing and live translation
Windows has more AI than ever before, and some of it is exclusive to the new Copilot+ PC.
A new feature called Super Resolution can automatically upscale and restore old photos. Copilot can also now analyze images and provide users with creative composition ideas. A feature called Cocreator allows users to generate images and also ask the AI model to modify or restyle the image according to what it draws.
Image credit: Microsoft
Live Captioning with Live Translation also translates audio passing through your PC from YouTube or local files into the language of your choice. Live Translate will initially support around 40 languages, including English, Spanish, Mandarin, and Russian.
A separate, related new feature in Microsoft Edge provides real-time video translation on sites like LinkedIn, YouTube, Coursera, Reuters, CNBC, and Bloomberg. Expected to be available in the near future, this feature supports translations from Spanish to English, English to German, Hindi, Italian, Russian, and Spanish, and audio content through both dubbing and subtitles. Translate live.
Team Copilot and Extensions
Team Copilot is the latest expansion of Microsoft's growing Copilot suite of generative AI technologies. It's integrated with Teams, the company's video conferencing app, so you can manage meeting topics and create notes that all meeting participants can co-author. It also extends to Loop and Planner, Microsoft's collaboration and planning platforms, to help you create and assign tasks, track deadlines, and notify team members when input is required.
Image credit: GitHub/Microsoft
In somewhat related Copilot news, Microsoft has launched (in private preview) Copilot Extensions, which allows developers to extend GitHub's code generation tool, GitHub Copilot, with third-party apps and skills. Launch partners include DataStax, Docker, and LambdaTest. Extensions exist on his GitHub Marketplace, but developers can also create their own private extensions to integrate with internal systems and APIs.
Windows CoPilot Runtime
Image credit: Microsoft
Powering features such as recall and super resolution is Windows Copilot Runtime. It's a collection of about 40 generative AI models that make up what Microsoft calls a “new layer” of Windows. The Windows Copilot runtime works with Semantic Index, a vector-based system local to each individual Copilot+ PC, to enable generated AI-powered apps, including third-party apps, to run without necessarily requiring an internet connection.
“[The runtime] Studio effects, live caption translation, OCR, recall based on user activity, [more]will be made available to developers in June,” Davuluri said on Tuesday.
CapCut, the popular video editor from TikTok owner ByteDance, will use Windows Copilot Runtime and the new Windows Copilot Library, an accompanying set of APIs and AI development tools, to accelerate its AI capabilities, according to Microsoft. And Meta plans to add the aforementioned Studio Effects to WhatsApp, offering features like background blur and eye contact during video calls.
Upgraded bot builder
Azure AI Studio, a toolset within Microsoft's Azure OpenAI service, allows customers to combine AI models and build apps that “reason” about their data. Developers will soon be able to create apps using pay-as-you-go inference APIs (APIs). This allows developers to access and fine-tune generative AI models hosted on Azure infrastructure. Microsoft calls this the “as-a-service model,” and it's starting with the Nixtla and Core42 models.
In the adjacent Copilot Studio product suite, Microsoft is launching the Copilot agent. The company describes it as an AI bot that can “independently tailor tasks tailored to specific roles and functions.” (Copilot Studio provides tools to connect Copilot for Microsoft 365, an AI-powered “copilot” for apps like Excel and Word, to third-party data.) Use your knowledge to navigate different types of business workflows. Learn from user feedback and ask for help when you encounter a situation you don't know how to handle.
Snapdragon development kit
Image credit: Microsoft
Qualcomm has a new development kit aimed at developers building apps for Copilot+ PCs powered by Arm chips
By the way, the $899.99 Snapdragon development kit for Windows (about the same width, height, and length as Apple's Mac Mini) includes Qualcomm's Snapdragon paired with 32GB of RAM, 512GB of storage, and tons of I/O. Equipped with X Elite chip. This development kit supports Wi-Fi 7 and Bluetooth 5.4 and can drive up to three of his 4K monitors simultaneously via various USB-C and HDMI ports.
Phi-3
Microsoft has announced Phi-3-vision, a new addition to its Phi family of generative AI models that can perform common visual analysis and reasoning tasks, such as answering questions about graphs and images. The model can read both text and images, and is efficient enough to run on mobile devices.
Phi-3-vision is available in preview, while previously announced text-only models Phi-3-mini, Phi-3-small, and Phi-3-medium are now generally available.
Partnership with Khan Academy
Microsoft will partner with Khan Academy and donate access to cloud computing infrastructure. This will enable Khan Academy to offer educators in the United States free access to Khan Academy's AI-powered tools. The two companies will also work together to explore opportunities to improve AI apps for math instruction through generative AI, Microsoft announced Tuesday.
We're launching an AI newsletter, sign up here to start receiving it in your inbox on June 5th.