Microsoft’s big bet on building a new type of AI computer

Microsoft’s new Windows on Arm push is a milestone moment. It’s taken nearly five years to get to the point where the software maker is confident it has the upper hand over the MacBook Air on performance, battery life, and app compatibility.

But behind the scenes, Microsoft has been working on something even bigger. While Microsoft has spent years working toward an Arm transition that will now play out throughout the summer and beyond, it’s the AI overhaul at the heart of this new generation of “Copilot Plus PCs” that could fundamentally change how we use Windows on a daily basis.

I recently got the chance to hear from the leaders behind Microsoft’s big AI push, and they made it clear that this isn’t just a silicon change. Microsoft has rearchitected Windows 11 to run dozens of AI models in the background, and they believe it can reshape the experience of using a PC.

“The PC is essentially going to become a new sensor for AI,” says Yusuf Mehdi, executive vice president and consumer chief marketing officer at Microsoft, in a briefing I attended recently. “With the PC as a sensor when an individual is using their PC, what they’re copying, what they’re entering, what they’re looking at… that all brought to AI can unlock a new level of context and understanding of what’s going on.”

At first, Microsoft is using these AI models to bring a handful of new features to reality. The splashiest is called Recall: it uses local AI models constantly running in the background to capture everything you do and look at on your PC so you can time travel back to something you saw on your PC or were working on.

“That becomes a superpower,” Mehdi says. “Your memory doesn’t have to be perfect … because now the PC can assist you because it knows what you’ve been doing.”

Microsoft demonstrated Recall on Monday, alongside some of the other AI features. One of the most impressive was Cocreator, a feature in Paint that’s designed to let you create images with the help of AI. On these new Copilot Plus PCs, you’ll be able to start sketching a rough outline and have AI transform your doodles into detailed images. It’s far more than just a text-to-image prompt because these local AI models can see what you’re drawing and react accordingly.

“The thing I love about Paint Cocreator is that it hints to the future of how people will use computers,” says Steven Bathiche, head of Microsoft’s applied sciences group. “Instead of this one-shot experience when you type and wait for an answer, you’re able to direct the AI and you’re in the loop yourself and directing.”

In another demo, we also saw Microsoft’s vision for using these local AI models to guide people through Minecraft, helping players craft materials and reacting to what you can see on the screen. Normally you’d have to search YouTube for guides or find a Reddit thread, but Microsoft’s demo showed AI serving as your own personal video game guide.

All of these new Windows features are made possible by the neural processing units (NPU) found in these latest Arm-powered devices. Microsoft built a new kernel that takes advantage of the NPUs and raises them up to the level of importance in Windows as the CPU and GPU, allowing AI models to run locally without bogging down the machine.

That means there’s been a lot of work on the fundamentals of Windows, which will show up first on Arm-powered devices. “We engineered Windows 11 with a real focus on AI inference and taking advantage of the ARM64 instruction set at every layer of the operating system stack,” says Windows and Surface chief Pavan Davuluri. Future AMD and Intel chips will also take advantage of this AI work soon, and we’ll even start to see Copilot Plus PCs with Nvidia GPUs to further accelerate AI tasks beyond what an NPU is capable of.

While it’s just the beginning of AI being built into Windows, I get the sense that, to Microsoft, this is just as big as Windows 95. That version of Windows debuted nearly 28 years ago and introduced the core concepts of the Start menu, taskbar, and notification area that are still key to navigating Windows today.

“It’s an amazing moment in history here. It really is about the destructive evolution of computers through the introduction of new interactive technology,” says Bathiche, head of Microsoft’s applied sciences group. He envisions interfaces for interacting with PCs that will move from commanding with a mouse and keyboard to asking an AI assistant to help you navigate Windows. “This allows us to design a new type of computer.”

Mehdi likens this shift toward local AI models and NPUs to moving from dial-up to broadband. “I think that’s going to enable some magical experiences that we’ve never been able to have before,” Mehdi says.

Microsoft doesn’t know exactly what comes next, but it’s giving developers AI tools that can handle vision, voice, displays, cameras, and language understanding and translation. It’s a “build it, and they will come approach,” much like what Apple did with the original App Store for the iPhone.

Both Intel and AMD will deliver traditional PCs soon that include the type of AI performance we’ll see from these early Copilot Plus PCs, which could complicate the transition to Arm, but it clarifies that this AI push will be available on all new PCs regardless of chips. That’s important because Microsoft’s entire AI vision will also need developers to really respond to AI being a key part of Windows now and available across a variety of hardware.

Microsoft is already lighting up some interesting new AI experiences in Windows thanks to these new chips, but it’ll be developers that really push the boundaries of what having more than 40 AI models running on a PC really means.

All of these demos have really sparked my interest in what will come next. The ability for AI models to see and understand what you’re doing on-screen will unlock even more ways to interact with and use PCs in the future.

“I think this is super exciting, and this is, in my opinion, going to change how people use computers, what people buy, and how people develop software,” says Bathiche. “This is the one for folks to pay attention to.”

Write in with any questions you have about Microsoft, and I’ll try to answer as many as I can. You can reach me via email [email protected]. If you’ve heard about some secret project Microsoft is working on you can reach me confidentially on Signal, I’m tomwarren.01 there.