What Is a Neural Engine? Apple's On-Device AI Chip Explained

Apple Silicon chips contain three kinds of processor, and the split matters because each is good at something different. The CPU handles general-purpose work one instruction at a time. The GPU handles thousands of parallel operations, which is why it suits graphics and model training. The Neural Engine is narrower than both: it does one thing, the matrix multiplication and convolution math that neural networks are made of, and it does that far more efficiently than either alternative.

That efficiency is the whole point. Running a speech recognition model on the CPU works, but it burns battery and competes with everything else you have open. The same model on the Neural Engine runs faster and draws a fraction of the power, so it can stay on continuously without you noticing.

Capability has climbed with each generation. The M1 Neural Engine performs 11 trillion operations per second (TOPS); the M4 reaches 38 trillion. Higher TOPS means larger models can run locally and respond faster, which is why a newer Mac handles on-device AI that an older one has to send to the cloud.

"Neural engine" is Apple's brand name, but the idea is not unique to Apple. Qualcomm, Intel, and AMD all ship equivalent NPUs, and phones have carried them for years. What is distinctive about Apple's is the tight integration with Core ML and the unified memory architecture, which means a model can access memory without the copy step that slows down discrete accelerators.

The practical payoff is privacy plus latency. Because the model runs on your machine, your audio and text never leave it, and there is no network round trip to wait on. That is what makes features like live dictation, on-device photo search, and offline translation feel instant rather than laggy.