How to actually use the AI chip in your Mac.

We wrote recently about the Neural Engine, the dedicated AI processor sitting inside every Apple silicon Mac, doing almost nothing. The obvious question came straight back: so how do I use it? It turns out you can, today, in about ten minutes, and on a Mac as old as 2020. Here's how, and the catch nobody mentions.

Section 01Why bother running AI on your own Mac?

Three reasons, all practical.

Privacy. It runs on your machine. Your prompts, your documents, your half-finished ideas none of it leaves the desk. If you handle client work, legal documents, patient notes or anything under NDA, this isn't a nice-to-have. It's the entire argument.

Cost. No meter running. Once it's on your Mac you can use it ten thousand times for nothing.

It always works. No rate limits, no outages, no internet.

The honest trade-off: a model small enough to run on your laptop is less capable than the frontier models in a data centre. It won't out-think the best cloud services. What it will be is private, free and always there. For summarising, drafting, tidying up notes and answering questions, that's often the better deal.

Section 02Your Mac has two AI engines

This is the thing that confuses everyone, and it's worth two minutes.

Your Mac can run an AI model on either of two pieces of silicon, and they behave completely differently.

Here's the part that surprises people: almost every popular tool uses the GPU. Ollama, LM Studio, and Apple's own MLX framework all run your model on the graphics processor. The chip Apple designed specifically for AI sits there doing nothing.

That isn't a conspiracy. The Neural Engine was designed for small, fixed-size networks, recognising a face, and transcribing speech not for the variable-length text generation a chatbot needs. Making language models run on it is genuinely hard, and for years Apple gave developers almost no way in. That was the subject of our previous piece.

So we put it on the bench.

What we measured

We ran both routes on the same machine, in the same session, watching the power draw of each part of the chip with Apple's own powermetrics tool. One 16-inch MacBook Pro (M3 Max, 36GB). First Apple's foundation model through Osaurus. Then Gemma, through the same app, on the GPU.

The result was about as clear-cut as measurement gets.

The Neural Engine reading during Gemma wasn't low. It was zero — in every sample we took, across the whole run. Meanwhile the foundation model left the GPU idling at 0.38W and put the work on the Neural Engine instead. Two engines, cleanly separated, exactly as advertised.

All in, the same Mac doing the same kind of job drew 12 watts on the Neural Engine and 29.5 watts on the GPU — 2.4× the power for the GPU route.

Put that in battery terms. A 16-inch MacBook Pro carries roughly 100 watt-hours. Chip power alone, ignoring the screen and everything else, that's about eight hours of continuous generating on the Neural Engine against three and a half on the GPU. The real numbers will be lower once the display is lit. The ratio won't move.

Three things we didn't expect

The Neural Engine isn't free. Look again at that table: while the foundation model was writing, the CPU drew 6.44W — more than the Neural Engine's 5.21W. The ANE accounted for only about 43% of the chip's power; the CPU took 54%. So the honest version isn't "the Neural Engine runs models on 5 watts". It's that the Neural Engine does its share of the work very cheaply, but something still has to feed it.

It lines up with our own earlier work — with a caveat. The Neural Engine is the same 16-core design across the whole M3 family. Back in July, running a synthetic test flat out on an M3, we measured its Neural Engine at 5.11W. Here, running Apple's actual language model on an M3 Max — a completely different workload — it drew 5.21W. Two unrelated tests, ten days apart, within 2% of each other. That's reassuring, and it suggests Apple's model works the Neural Engine close to as hard as it goes. But we'll flag the obvious limitation: those are two different chips, and Apple explicitly cautions against comparing its Neural Engine power estimates across devices. Treat it as a sanity check, not a measurement.

Power and efficiency are not the same thing — and the ranking flips. This is the finding that changed how we think about it. A chip that sips power but writes slowly can burn more energy overall, because it stays lit for longer. Independent benchmarking by the developer Daisuke Majima measured energy per token on an M4 Max, and the order inverts:

Read that bottom row twice. A hand-converted model running on the Neural Engine draws about half the GPU's power — and is the worst of the four on energy per word, because it writes slowly enough to keep the chip awake far longer. Apple's own model is the one that wins, and it wins by generating fast and cheaply.

So "the Neural Engine is more efficient" is not a safe generalisation. Apple's foundation model is efficient. The Neural Engine, in itself, is merely quiet.

One more from Majima's work, and one we'd underline: under ten minutes of continuous generation, the GPU runtimes shed half their speed or more as they heat up, while the Neural Engine held around two-thirds of its rate. Those figures were measured on an iPhone 17 Pro, not a Mac. A fanless MacBook Air is the closest thing in the Mac line to a phone's thermal situation, and we'd expect it to behave similarly,but that's our inference, not his measurement, and a 16-inch MacBook Pro with two fans is a different proposition entirely. 

And on an M5, the line blurs

Everything above describes M1 through M4, where the GPU had no dedicated AI hardware at all, matrix maths ran through the same general-purpose units that draw your windows.

The M5 changed that. Apple put a Neural Accelerator inside every GPU core, which it credits with over four times the peak GPU compute of the M4 for AI work, and it names LM Studio specifically as a beneficiary. The 16-core Neural Engine is still there, still separate, still what Apple's own model uses.

So on an M5 the picture isn't "two engines" so much as "two and a half": a Neural Engine for Apple's model, and a GPU that is now genuinely built for this rather than merely capable of it. It doesn't change any of the advice below. It does mean the gap between the two routes is narrower on the newest chips than on a 2020 M1 and that the GPU route is the one improving fastest.

Section 03Using the Neural Engine — the easy way

Until recently this was developer territory: Python environments, model conversion pipelines, Xcode. Not any more.

Osaurus is a free, open-source, MIT-licensed app built specifically for Apple silicon. It started life as a lightweight local model server and has grown into something broader — agents, persistent memory, a plugin system, a sandbox — but the part that matters here is simple: it's one of very few mainstream tools that can talk to either engine. Point it at Apple's own on-device model and the work runs on the Neural Engine. Point it at a downloaded model and it runs on the GPU via MLX.

By the project's own count it had been downloaded north of 112,000 times as of May 2026, when TechCrunch covered it. It's a real project, not a weekend experiment.

One thing to be clear about, because it bears on the privacy argument: Osaurus can also connect out to cloud providers if you ask it to. Nothing leaves your Mac unless you choose a model that lives somewhere else. On the foundation path described below, nothing leaves.

Setting it up

1. Install it. If you have Homebrew, this is the whole job:

brew install --cask osaurus

If not, download the disk image from osaurus.ai and drag the app to Applications.

2. Launch it. Spotlight (⌘ Space) → type "osaurus". Or from the terminal, osaurus ui.

3. Make sure Apple Intelligence is switched on in System Settings → Apple Intelligence & Siri. Apple's on-device model is the thing running on the Neural Engine, and it needs to be enabled and downloaded first.

4. Select the foundation model. That's Apple's system model — the Neural Engine one. In the API it's literally "model": "foundation".

That's it. No Python, no conversion, no terminal.

The requirement to check first

Osaurus itself runs on macOS 15.5 or later. But the Apple Foundation Models path needs macOS 26 (Tahoe) or newer. If you're on something older, that's your first job and Tahoe runs on every Apple Silicon Mac, including the original M1.

Worth knowing if you're reading this in autumn: macOS 27 Golden Gate is due for public release in September 2026, and it's the first version of macOS to drop Intel Macs entirely. Every Apple Silicon Mac we sell will run it. Everything in this guide applies on 26 or 27.

Connecting your own tools

Osaurus exposes OpenAI-compatible, Anthropic-compatible and Ollama-compatible endpoints on port 1337, which means anything already built to talk to ChatGPT's API can be pointed at your Mac instead, with no code changes:

curl http://127.0.0.1:1337/v1/chat/completions -H "Content-Type: application/json" -d '{"model": "foundation", "messages": [{"role": "user", "content": "Hello!"}]}'

Same request you'd send to OpenAI. Different address. Nothing leaves the building.

Section 04The ceiling nobody mentions: 4,096 tokens

Here's the limitation that will actually bite you, and it isn't power or speed.

Apple's on-device model has a fixed context window of 4,096 tokens per session and both your input and the model's reply count against it. A token is roughly three-quarters of a word, so that's about 3,000 words in total, question and answer combined.

Cross it and you don't get a gentle degradation. You get an error, and the session has to start again. Apple added tools in macOS 26.4 to let developers check how much room is left before sending, which tells you how hard the limit is: it's a budget, not a suggestion.

In practice that means the Neural Engine path is excellent at:

• Tidying up a paragraph, or rewriting an email
• Summarising a page or two of notes
• Answering a question, or a short back and forth
• Classifying, extracting, reformatting 

And unsuitable for anything where you want to paste in a contract, a long transcript or a chapter and ask questions about it. That's a GPU job, and the reason Section 06 exists.

Section 05What to actually use it for

Installing it takes ten minutes. Working out what it's for takes longer, so here's where we've found it genuinely earns its place.

The obvious one: as a private chat window. Drafting, rewriting, summarising, explaining. Nothing you type goes anywhere. If your reason for reading this was NDA'd client work, this alone is the answer.

Behind tools you already use. Because the endpoints are OpenAI-compatible, anything built to talk to ChatGPT's API can be pointed at 127.0.0.1:1337 instead. Same requests, same responses, no account, no bill.

Dictation that never leaves the Mac. Osaurus does on-device voice transcription, that runs on the Neural Engine too. A global hotkey transcribes into whatever app you're in. For anyone who takes a lot of notes, this is quietly the most-used feature.

Reading your own files. Point an agent at a working folder and it gets file, search and git tools. On macOS 26 and later there's an isolated Linux virtual machine it can run code in, so it can do real work without touching the rest of your Mac. Bear the 4,096-token limit in mind here: for anything long, switch to a GPU model.

Background jobs. This is where the power numbers stop being trivia. A model that costs 12 watts instead of 29.5 and doesn't heat the chip is one you can leave running on a schedule all day, without your laptop turning into a hotplate or your battery evaporating.

Section 06Using the GPU — when you want choice

The easy Neural Engine route has one real limitation: you get Apple's model. It's roughly a three-billion-parameter model by most estimates is capable enough for summarising, drafting and tidying text, but you can't swap it for something bigger, newer or specialised, and you can't get past that 4,096-token wall.

If you want that choice, you're on the GPU. Three ways in:

Osaurus, again

The same app. Instead of the foundation model, pick one from its built-in model library and it runs on the GPU via MLX — Apple's own machine-learning framework, which is typically the quickest option on Apple Silicon. Gemma, Qwen, Llama, GPT-OSS and others are a click away, and the download lands in ~/MLXModels.

LM Studio

A conventional Mac app with a conventional interface. Browse models, click one, start typing. It'll tell you which ones your machine can handle. If you've never done this before and want zero terminal, start here.

Ollama

The tool most people have heard of. Install from ollama.com, then:

ollama run llama3.2

It downloads the model, then you're talking to it. Type /bye to leave. Worth knowing: Ollama is a cross-platform tool that also runs on Macs, whereas Osaurus is built for Apple Silicon and nothing else. That's the argument for preferring it here.

Section 07The door Apple just opened

Everything above describes the situation as it stands for anyone who wants this working this afternoon. But the ceiling has moved, and it moved recently enough that most coverage hasn't caught up.

At WWDC26 in June, Apple announced Core AI — the successor to Core ML, and a framework built specifically for putting your own AI models on Apple silicon, on-device, with no server and no token costs. Converted PyTorch models and pre-optimised open-source ones both work, and Apple's own developer sessions are explicit that Core AI lets those models take advantage of the Neural Engine.

Apple also opened the Foundation Models framework itself, so an app can now use that same tidy interface with almost any language model rather than only Apple's — and it rebuilt the on-device model, which now accepts images as well as text.

Two things follow.

First, "Apple's model or nothing" is no longer true at the technical level. Independent developers were already running Gemma and Qwen on the Neural Engine before Core AI existed, by converting them through Core ML by hand. Core AI makes that a supported path rather than a feat.

Second, don't rush to do it yourself. Look back at the energy table in Section 02: the hand-converted route on the Neural Engine was the least efficient thing measured. Apple's model isn't just the easy option, it's currently the good one. What Core AI changes is the trajectory over the next year or two, expect the apps in Section 06 to start reaching the Neural Engine without you doing anything at all.

For now: use the foundation model on the Neural Engine, or a downloaded model on the GPU. The third road exists, it's paved, and it's not yet worth walking.

Section 08So which engine should you use?

Most people should start on the Neural Engine, because it's the shortest path to finding out whether local AI is useful to them at all. If you hit its limits, the GPU is right there in the same app.

Section 09If you go the GPU route, memory is everything

Here's what every buying guide buries. On the GPU path, the model has to fit in your memory. Not "runs better with more" has to fit. If it doesn't, macOS starts shuffling data to the SSD and performance falls off a cliff.

Apple Silicon's unified memory cuts both ways here. On a Windows PC the model has to fit in the graphics card's dedicated VRAM, typically 8 or 12GB even on expensive cards. On a Mac the GPU can use all your memory, which is why a Mac with 32GB runs models that defeat a much pricier gaming PC. But it also makes memory a hard ceiling, and on a Mac it's soldered in at the factory. You cannot add it later.

Rough arithmetic: a model compressed to 4-bit needs about 0.6GB per billion parameters, plus room for the conversation. macOS wants 3–4GB for itself. So:

Notice what's missing from that table: the chip. An M1 with 16GB runs the same models as an M4 with 16GB. The M4 answers faster — but "faster" and "possible at all" are different questions, and memory answers the second.

"But I've read you need 64GB"

You will, and from a credible source: Osaurus's own co-founder recommends at least 64GB for local models, and around 128GB for the largest ones. We don't think that's wrong, we think it's answering a different question.

That advice is aimed at running big frontier-class open models as autonomous agents, writing code and executing tasks. If that's your goal, believe him, and look at a Mac Studio rather than a laptop.

If your goal is a private assistant that summarises, drafts and answers questions, 16GB runs a genuinely useful model on the GPU, and the Neural Engine route runs happily on 8GB. Both statements are true. They're just about different jobs, and it's worth knowing which one you're buying for.

Section 10Is it fast enough?

Speed is measured in tokens per second — roughly, words per second. The bar that matters: around 20 tokens per second is faster than you can read. Past that, more speed stops mattering much for chatting, because you become the bottleneck.

That bar is low, and both routes clear it on modest hardware. In Majima's M4 Max measurements, Apple's foundation model produced around 85 tokens per second — four times reading speed, on about as much power as a bright bulb. GPU models are quicker still. Newer chips are faster again but "faster than fast enough" is a luxury, not a requirement.

Two things will genuinely slow you down, and neither is the chip:

Running out of memory. On the GPU path, if the model doesn't fit, your Mac swaps to the SSD and everything grinds. That's not a gentle slowdown, it's the difference between usable and unusable, and it's hard on a drive you can't replace.

Heat. The GPU warms up under sustained load and a fanless MacBook Air will slow itself down to cope. Fine for a question here and there; noticeable across a long session. The Neural Engine is largely immune to this, which is exactly why it exists.