Mach IPC internals: kmsg, mqueue, vouchers

The Mach ports article covered ports, rights, and the shape of a message. This is the article one level down — what the kernel actually does between the moment a thread calls mach_msg and the moment a receiver wakes up holding the message. Three data structures matter: ipc_kmsg, ipc_mqueue, and the voucher.

The kmsg — kernel's in-flight representation

When userspace calls mach_msg(MSG_SEND, …), the kernel allocates an ipc_kmsg to represent the message while it's in flight. This is not the userspace mach_msg_header_t — it's a richer kernel structure that carries:

• The full header, copied in from userspace.
• The inline payload, copied in.
• A processed array of port descriptors — userspace passes port names (per-task numbers); the kernel resolves these to actual port pointers + reference counts.
• A processed array of out-of-line descriptors — userspace passes virtual address + size; the kernel converts these to VM map copy objects.
• Linkage pointers to thread it on the destination port's queue.
• The sender's QoS, voucher, and audit token.

apple-oss-distributions/xnuosfmk/ipc/ipc_kmsg.hThe ipc_kmsg structure — every field has a story.View on GitHub(line —) apple-oss-distributions/xnuosfmk/ipc/ipc_kmsg.cipc_kmsg_copyin / copyout / send / destroy — the full lifecycle.View on GitHub(line —)

A kmsg owns its memory. When the receiver dequeues and reads it, the kmsg is freed; on failure paths (the receiver task died, the port was deallocated, the send timed out) the kmsg is destroyed, which means releasing every port reference and unmapping every VM copy.

Copyin: the most expensive part of a send

ipc_kmsg_copyin is where the kernel does the heavy lifting. For each descriptor in the message body:

• MACH_MSG_PORT_DESCRIPTOR — the kernel looks up the named port right in the sender's ipc_space, validates the disposition (send / send-once / receive), bumps the reference count, and stores the port pointer in the kmsg.
• MACH_MSG_OOL_DESCRIPTOR — the kernel calls vm_map_copyin on the sender's VM map for the given range. This creates a "copy object" — either a real page copy (for small regions) or a virtual reference that will be lazily materialized at copyout (for large regions). The original sender pages stay alive; the kernel guarantees the receiver sees what was there at send time.
• MACH_MSG_OOL_PORTS_DESCRIPTOR — array of port rights handed over en masse. Each one goes through the same lookup-and-reference dance as a single port descriptor.

For an out-of-line descriptor over a few pages, copyin is essentially "make a VM copy object pointing at the same physical pages." Copy-on-write semantics defer the actual copy: if neither side writes, no page is duplicated.

This is why moving a 1 GB buffer between two tasks via Mach IPC is fast — the kernel doesn't copy 1 GB, it remaps it. Only writes after the message arrives trigger COW.

The mqueue — where the message waits

Every port has an embedded ipc_mqueue. Conceptually a queue of kmsgs, with a wait set and a few flags. Practically:

apple-oss-distributions/xnuosfmk/ipc/ipc_mqueue.hipc_mqueue structure — owns the message FIFO and the wait set.View on GitHub(line —) apple-oss-distributions/xnuosfmk/ipc/ipc_mqueue.cpost / receive / wakeup — the mqueue operations.View on GitHub(line —)

• Enqueue appends the kmsg to the port's FIFO and tries to wake one of the threads sleeping on the port's wait queue. If a thread is found, the kernel hands it the message directly (no second-trip dequeue needed).
• Dequeue (ipc_mqueue_receive) tries to pull the first kmsg. If empty and MACH_RCV_TIMEOUT is non-zero, it blocks the thread on the port's wait queue, then returns when woken — either with a message or with MACH_RCV_TIMED_OUT.
• Wait sets allow a single receive call to wait on a set of ports — the foundation of mach_msg-based reactor patterns. The Mach port set is added to the wait set; any port in the set with a message wakes any waiter.

The mqueue also enforces message ordering and send rights tracking — when the last send right to a port is deallocated and the queue is empty, the kernel can signal MACH_NOTIFY_NO_SENDERS to the receive-right holder, who knows no future messages can arrive.

Vouchers — the side channel most people don't notice

Every Mach message can carry a voucher in addition to its payload. A voucher is an immutable, kernel-owned object that bundles together pieces of context the receiver needs to know about the sender:

• The QoS the sender wants the work to be done at (drives the QoS-override system).
• A resource accounting bucket (used for I/O accounting, energy accounting).
• An importance boost (used for foreground-app responsiveness).
• A bank for app-extension I/O charges (used by XPC).
• A persona identifier (used in iOS for App Clips and managed identity).

apple-oss-distributions/xnuosfmk/ipc/ipc_voucher.cVoucher lifecycle — alloc, dealloc, attribute manipulation.View on GitHub(line —) apple-oss-distributions/xnuosfmk/atm/atm.cActivity Trace Manager — the resource-accounting backend a voucher carries a handle to.View on GitHub(line —)

A typical Mach round-trip looks like:

1. UI thread sends a message to a daemon. The send carries a voucher with USER_INTERACTIVE QoS + the UI's bank.
2. The daemon's mach_msg returns with the voucher attached. The runtime adopts the voucher onto the receiving thread — the thread's QoS is now USER_INTERACTIVE, its energy charges go to the UI's bank.
3. The daemon does work — possibly making nested IPC calls. Each outbound message defaults to using the adopted voucher.
4. When the daemon replies and the runtime releases the voucher, the thread reverts to its default QoS.

This is how a daemon at UTILITY QoS can correctly bill its CPU + I/O against the foreground app that requested the work, and run at the right priority while doing it. It's also the mechanism behind QoS-override propagation through IPC chains — see the scheduler article.

Special ports — host, task, thread, bootstrap

A few Mach ports are well-known by role rather than allocated by request:

• Host port — handle to the system itself. Querying host info (CPU count, memory size).
• Task port (mach_task_self) — send right to your own task. With it you can read/write your own VM, list your threads, allocate ports.
• Task port for another task (task_for_pid) — gated by entitlements; this is what dtrace, debuggers, and process-inspection tools use.
• Thread port (mach_thread_self) — same, for the calling thread.
• Bootstrap port — the per-process root of the namespace where services register names → send rights. This is the underbelly of launchd's service-name resolution.

A new task inherits its parent's bootstrap port unless told otherwise. launchd sets the bootstrap port carefully when spawning daemons, so a daemon's view of "what services exist in the world" is exactly what launchd has decided that daemon should see — the foundation of XPC service isolation.

What happens when the receiver dies mid-message

This is the gnarly case the kernel has to handle gracefully:

• Sender sends a message with an out-of-line descriptor; kernel takes a VM copy reference.
• Message lands on the port's queue. Receiver is alive but hasn't dequeued yet.
• Receiver task dies. The kernel destroys every port owned by the task, including the receive right.
• The mqueue is being torn down. Every kmsg still on the queue gets ipc_kmsg_destroy'd — which releases each port reference and unmaps each VM copy. The sender's COW objects' refcounts drop. If nothing else was referencing them, they're freed.

The send returned success the moment the message was enqueued. The sender doesn't know the receiver died; if it needs a reply, it'll time out on its own. This is why server protocols always carry a reply port — the sender can detect MACH_NOTIFY_DEAD_NAME on it and react.

What to read next

For a complete kmsg send walked from trap to wake:

apple-oss-distributions/xnuosfmk/ipc/mach_msg.cmach_msg_overwrite_trap — the kernel entry every send goes through.View on GitHub(line —) apple-oss-distributions/xnuosfmk/ipc/ipc_object.cReference counting on ports — the unsexy code that makes everything else work.View on GitHub(line —)

And re-read the Mach ports article — once you've seen the kmsg, the port descriptor format makes more sense.