AOP and the secondary coprocessors on Apple Silicon

The Apple Silicon SoC has more processors than just the P-cores, E-cores, and GPU you've seen in the Apple Silicon overview. There are at least half a dozen secondary coprocessors baked into every chip, each running its own firmware and handling a specific class of workload. This article maps them.

AOP — the Always-On Processor

The AOP (Always-On Processor) is a low-power ARM core that runs continuously, even when the main CPU clusters are powered down for sleep. Its job is to handle low-priority, low-latency tasks the main system shouldn't wake up for:

Polling sensors (accelerometer, gyroscope, ambient light, lid sensor on MacBooks).
Sampling the microphone for the "Hey Siri" hotword (on iPhone — on macOS this lives elsewhere).
Handling Bluetooth LE advertisements without waking the rest of the SoC.
Time-of-day tracking and the RTC.
Coordinating wake events (lid open, button press) — the AOP is what wakes the main system.

The AOP runs its own firmware loaded at boot, with its own minimal OS (closer to bare-metal than to SEPOS). It communicates with XNU over a mailbox similar to the SEP's.

The "always on" claim is literal: the AOP draws on the order of milliwatts and runs even when the user perceives the device as completely asleep. This is why a MacBook can wake on lid open in tens of milliseconds — the AOP saw the lid open and pinged the main system to wake.

ANE — the Apple Neural Engine

The ANE (Apple Neural Engine) is a dedicated machine-learning accelerator. On modern M-series chips, it has 16 or more cores optimized for the matrix-multiply-accumulate patterns that dominate neural network inference.

What runs on the ANE:

Core ML models with MLComputeUnits.cpuAndNeuralEngine or .all. Apple's framework dispatches eligible ops to the ANE automatically.
Face ID matching (on iPhone/iPad). The biometric template comparison runs there.
Live Text, Visual Look Up, photo subject extraction — all the on-device CV features.
Voice Memos transcription, dictation, on-device Siri (where it ships).

ANE programming is via Core ML or Apple's lower-level ANE compiler APIs (not generally exposed to third-party developers in public form). You don't write raw ANE assembly; you give it a graph.

From the kernel's perspective, the ANE is an IOKit device with its own driver (closed-source). It allocates buffers via IOSurface (see the Apple GPU article) — same unified-memory zero-copy story as the GPU.

ISP — the Image Signal Processor

The ISP is the image processing pipeline that sits between the camera sensor and the rest of the system. Its job is to turn raw sensor output into usable image data:

Demosaicing (interpolating the Bayer-filtered sensor data into RGB).
Auto-exposure, auto-white-balance, auto-focus.
Noise reduction, sharpening.
HDR fusion (multiple-exposure combining).
Smart HDR / Deep Fusion (Apple's specific HDR algorithms).

On iPhone, the ISP is doing dramatic computational photography. On Mac, where the built-in camera is simpler, the ISP still handles the basic pipeline plus features like Center Stage (which uses the ANE on top of the ISP's framing).

The ISP runs its own firmware. Driver in the kernel; ISP firmware updates ship with OS updates.

AMC — the Apple Memory Controller

The AMC is the memory controller — the part of the SoC that talks to the LPDDR5 DRAM. Not a processor in the traditional sense, but a complex piece of programmable logic that handles:

DDR command scheduling.
Refresh management.
DVFS coordination with the rest of the SoC.
Memory channel routing (Apple's M-series chips have many memory channels).

The AMC doesn't run user-replaceable firmware, but its configuration is set by the kernel at boot from device tree data.

AMX — the Matrix Coprocessor

Already covered in the AMX glossary and the Apple Silicon overview. One AMX per CPU cluster, accessed via undocumented MSR writes from the CPU. Used by Accelerate framework and Core ML's CPU fallback path.

AMX is being replaced by SME (the ARM-standard equivalent) on newer silicon.

Display Engine

The Display Engine owns:

Composing the framebuffer from multiple sources (your app's window, the menubar, system overlays).
ProMotion's variable refresh rate management.
HDR tone mapping for displays that support it.
Pixel pipeline to the actual panel driver.

The Display Engine on Apple Silicon does what was historically the GPU's compositor job — letting the GPU sleep when nothing's rendering, and pulling pre-rendered content directly to the panel.

Media Engines

Dedicated hardware encoders/decoders for video:

ProRes encoders/decoders (M-series Pro and above).
H.264 / HEVC / AV1 decoders (every M-series).
HEVC / H.264 encoders.

These are why your Mac can play 8K video while consuming a few watts — the heavy lifting is in dedicated silicon, not the CPU or GPU.

How the kernel talks to all of these

Most secondary processors expose themselves as IOKit devices. Each has:

A kernel driver (closed source for Apple's own coprocessors).
A user-facing framework (Core ML, AVFoundation, ImageCaptureCore, etc.) that wraps the IOKit interactions.
A mailbox or queue for command submission.
IOSurface-backed buffers for data exchange.

The driver-side code lives in iokit/Drivers/-style locations, mostly closed-source. The kernel infrastructure they all use (IOService, IOMemoryDescriptor, IOUserClient, IOSurface) is the same regardless of which coprocessor.

This is why every Apple Silicon feature is reachable from userspace through a clean framework — the coprocessors are all behind IOKit + a high-level API. Apps that use Core ML, AVFoundation, Vision, Speech, ARKit — all of these are dispatching to specialized hardware without realizing it.

What surprises newcomers

There are many more processors than the CPU clusters. A modern M-series SoC has 6+ programmable cores when you count the AOP, SEP, ISP firmware processor, ANE controller, GPU firmware processor, and main CPU clusters.
The AOP is always running. It's what enables sub-100ms wake-from-sleep.
None of these are usable from open-source software directly. Apple-only frameworks are the supported interface; raw access requires undocumented MSRs or private SPIs.
Power management is per-coprocessor. The kernel can independently sleep/wake the GPU, ANE, ISP, etc. via IOPM, based on activity.

What to read next

For the high-level frameworks that abstract each coprocessor:

Core ML (for ANE).
AVFoundation / Image Capture (for ISP).
Metal (for GPU).
HealthKit / CMSensorRecorder (for AOP-tracked sensors).

And for the kernel-side infrastructure they all share:

apple-oss-distributions/xnuiokit/Kernel/IOService.cppIOService — base class every coprocessor driver subclasses.View on GitHub(line —) apple-oss-distributions/xnuiokit/Kernel/IOMemoryDescriptor.cppIOMemoryDescriptor / IOSurface — the zero-copy substrate every coprocessor uses.View on GitHub(line —)

And the Apple Silicon overview for the CPU/GPU/SEP picture this builds on.