Microsoft made Aion Instruct generally available on November 19, and the interesting thing about it is where it runs: not in a data centre, but in the browser. Aion is a 2.3-billion-parameter open-weights model that is now the built-in small language model in Microsoft Edge, replacing Phi-4-mini, and it comes with a way for any website to call it.

The model itself is modest by design. At 2.3 billion parameters it is smaller and faster than Phi, while delivering comparable quality on the everyday web tasks Microsoft lists: summarization, instruction following and structured outputs. It is published as open weights on Hugging Face under the name microsoft/Aion-1.0-Instruct-2.3B, and Edge downloads it only when a site or feature first needs it, and only on devices that can run it well. Users stay in control and can turn off on-device AI in Edge settings, which removes the model and stops future downloads. Microsoft says Aion is coming to Windows later this year, as the groundwork for on-device AI across Windows and Copilot+ PCs.

The bigger news for builders is the Prompt API. Until now it was a developer preview available only in Edge Canary and Dev. With the new origin trial, developers can register a domain and call the API from a live site in Microsoft Edge, testing on-device AI features with a real audience instead of a preview channel. Microsoft is asking for feedback on the API's design, the model's performance and interoperability across browsers, as the Prompt API is considered for web standardization.

Alongside it comes a developer preview of an experimental Decisions API, powered by Aion. It answers yes/no, multiple-choice or scoring questions about a piece of text in a single pass, returning probabilities and a confidence value without generating any text. The example in Microsoft's post scores a product review on topic, satisfaction and whether the customer needs a reply, all with a few lines of client-side JavaScript. It is available in Edge Canary and Dev for prototyping, and Microsoft cautions that both the API and its implementation may change.

Why it matters

Microsoft calls this hybrid intelligence: use the on-device model when it is available and well suited, and switch to a cloud provider when you need wider device coverage or more powerful inference. For an AI engineer, the on-device half is a free inference tier. Summarization, classification, routing and scoring are exactly the tasks a small model handles well, and running them in the browser means no server, no API key, no per-call bill and no user text leaving the device.

The judgement call is knowing which tasks belong on that tier and which do not. A 2.3-billion-parameter model can miss fine details and fail at complex multi-step reasoning, and Microsoft's own model card says so. The skill is the same one production systems already need: route the easy calls to the cheap model, keep the hard ones for the flagship, and measure where the line sits. The browser just made the cheap end of that routing free.

Sources: Microsoft's announcement and Windows Forum's coverage.

Go deeper

Model choice, routing and cost control are a whole module of the course. The full interactive course, 11 chapters free, no signup, is at deployedlab.dev.

← All the latest in AI