Every AI feature your app currently ships either costs you per token or depends on a server staying up. Apple wants to change that. At WWDC 2026, they announced Core AI, a new framework replacing Core ML for generative AI workloads, and the pitch is simple: run full-scale language models directly on iPhone, iPad, Mac, and Vision Pro without touching a server.

Here’s the thing — this isn’t just a nice-to-have privacy story. For a certain class of app, running inference on-device makes more sense technically and financially than making API calls. Core AI is Apple’s attempt to make that practical.

What Core AI Actually Is

Core AI is purpose-built for generative AI workloads on Apple silicon. It’s not an update to Core ML — it’s a separate framework that replaces Core ML for generative use cases, with an architecture optimised for the unified memory model that Apple silicon uses. The neural engine, GPU, and CPU can all contribute to inference, and Core AI manages that allocation automatically.

Apple ships a curated catalogue of supported open-source models at launch: Qwen, Mistral, and SAM3 are all optimised for Apple silicon and available to developers as .coreaimodel files. You don’t have to start from scratch or convert anything to get your first model running.

For models not in the catalogue, the workflow is:

  1. Train or fine-tune in PyTorch as normal
  2. Convert using coreai-torch, Apple’s new conversion tool
  3. Output a .coreaimodel file
  4. Import CoreAI in your Swift project and load the model

The conversion step handles quantisation and compilation for the target device. A model optimised for an M4 Mac will be compiled differently from one targeting an A18 iPhone.

The Developer Experience in Xcode 27

Core AI ships with Xcode 27, currently in beta for Apple Developer Program members. The standout addition is Device Hub, which lets you push models directly to physical connected devices during development — no TestFlight, no provisioning workarounds. You can profile inference latency and memory usage on the actual hardware your users will run.

The Swift API is deliberately minimal:

import CoreAI

let model = try await CoreAIModel.load("mistral-7b-instruct")
let response = try await model.generate("Summarise this support ticket: \(ticketText)")

That’s the whole integration for a basic text generation case. The model manages its own memory lifecycle, and Apple’s runtime handles offloading to disk when the device needs memory for other apps.

When On-Device Makes Sense

The honest answer is: not always. Remote API calls are faster for large model sizes, simpler to update (push a new model version without an app update), and don’t require your users to have a recent device. Core AI targets iOS 20, macOS 17, iPadOS 20, and visionOS 4 — the fall 2026 OS releases — which means you’re building for a specific hardware tier.

Where on-device wins clearly is a short list, but it’s an important list.

Privacy-sensitive processing is the obvious one. Legal document summarisation, medical notes, anything where sending text to a third-party server creates compliance problems. With Core AI, the text never leaves the device.

Offline-first apps are another genuine fit. Field service workers, aviation crew, anyone operating in environments with unreliable connectivity. An AI feature that still works when the signal drops is a real advantage.

The cost argument is also real at scale. If your app does a lot of short inference tasks — autocomplete, classification, summarisation of short inputs — the per-token economics of remote APIs can add up fast. Running those locally changes the model from variable per-request cost to a fixed development investment.

Compared to Running Ollama Locally

If you’ve been using Ollama on macOS as a local inference server, Core AI covers similar ground but with a fundamentally different integration model. Ollama runs as a local HTTP server and you make REST calls to it — it’s a great developer tool but it’s not something you ship to end users. Core AI is a framework you link against, with Apple’s runtime managing everything.

For production iOS and macOS apps, Core AI is the appropriate path. Ollama stays in the development-tool category.

What to Do Now

Core AI is in Xcode 27 beta today. If you’re building AI features into an iOS or macOS app and you’ve been relying entirely on remote APIs, it’s worth evaluating which features could move on-device.

Apple’s developer documentation at developer.apple.com/core-ai has the conversion guide and model catalogue. The WWDC 2026 session “Integrate on-device AI models into your app using Core AI” (session 326) is the best starting point — it covers the conversion pipeline and performance tuning in detail.

Production release ships with iOS 20 in autumn 2026.