Multi-tenant MQTT broker built on core NATS

Tenant isolation, MQTT 5 and free clustering, without the JVM or a licence key. Sleeping devices keep their messages when nodes die.

MQTT 3.1.1 & 5.0Shared subscriptionsApache 2.0No JVM

Why it exists

Every open-source MQTT broker that does real multi-tenancy makes you give something up.

BrokerThe catch
EMQXBusiness Source License since 5.9 — clustering more than one node needs a paid licence key.
BifroMQGenuine native multi-tenancy, but it runs on the JVM.
VerneMQApache 2.0 source, but the official packages and images are under a EULA that charges for commercial use.
RabbitMQvhosts, but no QoS 2 and no shared subscriptions.
Mosquitto, NanoMQ, FlashMQ, mochiDo not cluster at all.
mastMQTT 5 with shared subscriptions, tenant isolation with per-tenant limits, free clustering, no JVM, and a single binary you can run standalone at a customer site.

How it is put together

Core NATS moves messages

Live fan-out rides core NATS, which handles tens of millions of subjects at roughly a gigabyte of memory per million subscriptions.

A key-value store holds keys

Sessions, retained messages and offline queues live in a handful of replicated JetStream buckets and streams, kept by the core tier, so a sleeping device keeps its messages when its edge restarts. Never a consumer per subscription: consumers are Raft state machines, and a real fleet would want hundreds of thousands.

Isolation is structural

Every topic is mounted under its tenant before validation, authorization or subscription sees it, and unmounted on the way out. A client never learns the prefix exists.

Authentication, two ways

Ask an HTTP service, or verify a signed token locally. They compose: authenticate from a token while topic decisions still go to a policy server.

Where a message goes

Edges speak MQTT and keep nothing; each one joins the cores as a NATS leaf node, an ordinary outbound connection. Cores keep everything, in four replicated JetStream assets, whatever the number of devices.

How a mast cluster is put togetherDevices connect over MQTT to a tier of stateless edge nodes. Edges join three core nodes as NATS leaf nodes. The cores hold four replicated JetStream assets: the mast_qos stream, the mast_offline stream, and the mast_sessions and mast_retained key-value buckets. A queuer on each core copies messages for absent sessions from mast_qos into mast_offline.DevicesEdge tier · stateless, scales outCore tier · Raft, every asset on all threedevice · MQTT 5device · MQTT 3.1.1asleepsession keptedge-0MQTT listeners · bridgeedge-1MQTT listeners · bridgeedge-2MQTT listeners · bridgeleaf linkscore-0NATS · queuercore-1NATS · queuercore-2NATS · queuerroutes mesh the cores; JetStream replicates to all threemast_qosstream · QoS 1/2, storedbefore the PUBACKmast_offlinestream · one subject persession, newest 100mast_sessionsKV · subscriptions, andwhether anyone holds itmast_retainedKV · read with batcheddirect gets, no consumerqueueraway?1234
  1. A QoS 1 or 2 publish is written to mast_qos, replicated on three cores, before the device gets its PUBACK. QoS 0 is never stored.
  2. Live delivery rides core NATS to every node with a subscriber. Each node also reads mast_qos through one consumer of its own, so a node cut off from the core catches up when it returns.
  3. Each core’s queuer watches the sessions bucket for devices that are away, and copies their QoS 1/2 messages into mast_offline. Every writer names a message the same way, so the stream keeps one copy however many wrote it.
  4. When the device comes back, to any edge, that edge reads its queue with a direct get, delivers it, and removes only what it read. No consumer is ever created per device.

Run it

One binary, three roles. No flags are needed for the common case; the roles exist for when you outgrow it.

# all-in-one: MQTT and storage in one process, no cluster
mast

# or split the tiers once the MQTT side needs to scale
# independently of Raft
mast --role=core   # Raft and the KV buckets, stable peers, a volume
mast --role=edge   # MQTT listeners, stateless, joins core as a leaf node

You do not need the split until you are big. Up to roughly ten nodes, run every process identical and let them mesh.

A container

All-in-one, with every client in one tenant: right for a laptop, wrong for anything reachable by someone else.

docker run -p 1883:1883 ghcr.io/mastmq/mast
Image →

A binary

Static Linux builds for amd64 and arm64, with a checksums file.

# the latest release, linux/amd64
curl -sL https://api.github.com/repos/mastmq/mast/releases/latest \
  | grep -o 'https://[^"]*linux_amd64.tar.gz' | xargs curl -sL | tar xz
./mast
Releases →

Kubernetes

Standalone or clustered, with the core tier as a StatefulSet; HTTP or JWT auth, and TLS from a cert-manager Secret.

helm install mast oci://ghcr.io/mastmq/charts/mast
Helm chart →

Virtual machines

systemd units on any amd64 or arm64 host; the inventory group picks the role.

ansible-galaxy collection install \
  git+https://github.com/mastmq/ansible.git
ansible-playbook -i inventory.yml mastmq.mast.site
Ansible collection →

Under fire

Drills on three cores and three edges in Docker on one laptop (6 CPUs), with nodes killed by SIGKILL and restarted, run on 2026-11-16. Every message carries an id, and the drill counts which ones arrived.

DrillBeforeAfter
2,000 sleeping devices, every edge restarted in turn10,000 of 10,000 lostnone lost
5,000 sleeping devices, 100,000 messages, a core and two edges killednot runnone lost, 9 runs of 9
300,000 sleeping devices held by every core, then 2,000 more woken among themnot runheld in 12s at about 650 MB a core; 10,000 of 10,000 delivered
1,500 subscribers to 2,000 retained topics at once924 of 1,500 complete after 2m 10s1,500 of 1,500 in 8.5s
The other 4 drills, including the two that did not improve
DrillBeforeAfter
QoS 1, 5,000 msg/s to 5 subscribersnone lost, p99 1.4msnone lost, p99 4.1ms
QoS 1 fan-out to 2,000 subscribersnone lost, p99 110msnone lost, p99 74ms
6,000 clients connecting at once6,111 per second7,995 per second
3,000 connected devices at 2,000 msg/s; an edge and a core killed, another edge restarted89 acknowledged messages lost79 acknowledged messages lost

Two things did not get better. Every core now reads every QoS 1 and 2 message to find the ones a sleeping device needs, which roughly doubles core CPU under QoS 1 load and is where the higher p99 comes from. And a device that is still connected when its edge is killed outright loses what is published to it until it reconnects elsewhere; a graceful restart loses nothing. Delivery guarantees lists both.

Replacing EMQX

mast speaks EMQX v5’s HTTP auth wire, so an auth service written for EMQX works unchanged.

[auth]
mode = "http"

[auth.http]
wire = "emqx"          # EMQX’s request and response shape
authn_url = "http://soteria:9999/v2/auth"
authz_url = "http://soteria:9999/v2/acl"
on_error = "deny"      # EMQX_AUTHORIZATION__NO_MATCH

Topics reach the authorization endpoint unmounted, so existing ACL rules keep working. The migration guide has the full mapping and the four traps that actually fired doing it: is_superuser, multipath TCP on OpenShift, Helm’s map merging, and SCC uid ranges. Read the migration guide →

Where the project is

Early, and honest about it. The broker runs, moves messages across a cluster with tenant isolation, honours QoS 0 through 2, replays retained messages, delivers wills, and carries a persistent session from one node to another.

Gaps are tracked as issues rather than described as a roadmap. The parity label is what an MQTT client is entitled to assume and does not yet get. There is no dashboard and no management API, and no plan for a rule engine or data bridges. Open parity issues →

Know the limits before you build on it. QoS 1 and 2 hold across a cluster: a publish is stored on a replicated stream before it is acknowledged, and a node cut off from the core catches up when it returns. Two things are still best-effort: shared subscriptions whose members span nodes, and a message a node had taken when it died, which includes one for a device still connected to an edge that is killed outright. Delivery guarantees sets out what each QoS promises hop by hop, what it costs, and where it still falls short.

Repositories

Everything lives under the github.com/mastmq organization, under Apache-2.0.

mast

The broker.

charts

Helm charts.

ansible

An Ansible collection for virtual machines and bare metal.

docs

Architecture notes and operational guides.

bench

Load and latency benchmarks.

mochi

The MQTT library the broker embeds, carrying fixes upstream has not merged.