Why it exists
Every open-source MQTT broker that does real multi-tenancy makes you give something up.
| Broker | The catch |
|---|---|
| EMQX | Business Source License since 5.9 — clustering more than one node needs a paid licence key. |
| BifroMQ | Genuine native multi-tenancy, but it runs on the JVM. |
| VerneMQ | Apache 2.0 source, but the official packages and images are under a EULA that charges for commercial use. |
| RabbitMQ | vhosts, but no QoS 2 and no shared subscriptions. |
| Mosquitto, NanoMQ, FlashMQ, mochi | Do not cluster at all. |
| mast | MQTT 5 with shared subscriptions, tenant isolation with per-tenant limits, free clustering, no JVM, and a single binary you can run standalone at a customer site. |
How it is put together
Core NATS moves messages
Live fan-out rides core NATS, which handles tens of millions of subjects at roughly a gigabyte of memory per million subscriptions.
A key-value store holds keys
Sessions, retained messages and offline queues live in a handful of replicated JetStream buckets and streams, kept by the core tier, so a sleeping device keeps its messages when its edge restarts. Never a consumer per subscription: consumers are Raft state machines, and a real fleet would want hundreds of thousands.
Isolation is structural
Every topic is mounted under its tenant before validation, authorization or subscription sees it, and unmounted on the way out. A client never learns the prefix exists.
Authentication, two ways
Ask an HTTP service, or verify a signed token locally. They compose: authenticate from a token while topic decisions still go to a policy server.
Where a message goes
Edges speak MQTT and keep nothing; each one joins the cores as a NATS leaf node, an ordinary outbound connection. Cores keep everything, in four replicated JetStream assets, whatever the number of devices.
- A QoS 1 or 2 publish is written to mast_qos, replicated on three cores, before the device gets its PUBACK. QoS 0 is never stored.
- Live delivery rides core NATS to every node with a subscriber. Each node also reads mast_qos through one consumer of its own, so a node cut off from the core catches up when it returns.
- Each core’s queuer watches the sessions bucket for devices that are away, and copies their QoS 1/2 messages into mast_offline. Every writer names a message the same way, so the stream keeps one copy however many wrote it.
- When the device comes back, to any edge, that edge reads its queue with a direct get, delivers it, and removes only what it read. No consumer is ever created per device.
Run it
One binary, three roles. No flags are needed for the common case; the roles exist for when you outgrow it.
# all-in-one: MQTT and storage in one process, no cluster
mast
# or split the tiers once the MQTT side needs to scale
# independently of Raft
mast --role=core # Raft and the KV buckets, stable peers, a volume
mast --role=edge # MQTT listeners, stateless, joins core as a leaf node
You do not need the split until you are big. Up to roughly ten nodes, run every process identical and let them mesh.
A container
All-in-one, with every client in one tenant: right for a laptop, wrong for anything reachable by someone else.
docker run -p 1883:1883 ghcr.io/mastmq/mast
Image →A binary
Static Linux builds for amd64 and arm64, with a checksums file.
# the latest release, linux/amd64
curl -sL https://api.github.com/repos/mastmq/mast/releases/latest \
| grep -o 'https://[^"]*linux_amd64.tar.gz' | xargs curl -sL | tar xz
./mast
Releases →Kubernetes
Standalone or clustered, with the core tier as a StatefulSet; HTTP or JWT auth, and TLS from a cert-manager Secret.
helm install mast oci://ghcr.io/mastmq/charts/mast
Helm chart →Virtual machines
systemd units on any amd64 or arm64 host; the inventory group picks the role.
ansible-galaxy collection install \
git+https://github.com/mastmq/ansible.git
ansible-playbook -i inventory.yml mastmq.mast.site
Ansible collection →Under fire
Drills on three cores and three edges in Docker on one laptop (6 CPUs), with nodes killed by SIGKILL and restarted, run on 2026-11-16. Every message carries an id, and the drill counts which ones arrived.
| Drill | Before | After |
|---|---|---|
| 2,000 sleeping devices, every edge restarted in turn | 10,000 of 10,000 lost | none lost |
| 5,000 sleeping devices, 100,000 messages, a core and two edges killed | not run | none lost, 9 runs of 9 |
| 300,000 sleeping devices held by every core, then 2,000 more woken among them | not run | held in 12s at about 650 MB a core; 10,000 of 10,000 delivered |
| 1,500 subscribers to 2,000 retained topics at once | 924 of 1,500 complete after 2m 10s | 1,500 of 1,500 in 8.5s |
The other 4 drills, including the two that did not improve
| Drill | Before | After |
|---|---|---|
| QoS 1, 5,000 msg/s to 5 subscribers | none lost, p99 1.4ms | none lost, p99 4.1ms |
| QoS 1 fan-out to 2,000 subscribers | none lost, p99 110ms | none lost, p99 74ms |
| 6,000 clients connecting at once | 6,111 per second | 7,995 per second |
| 3,000 connected devices at 2,000 msg/s; an edge and a core killed, another edge restarted | 89 acknowledged messages lost | 79 acknowledged messages lost |
Two things did not get better. Every core now reads every QoS 1 and 2 message to find the ones a sleeping device needs, which roughly doubles core CPU under QoS 1 load and is where the higher p99 comes from. And a device that is still connected when its edge is killed outright loses what is published to it until it reconnects elsewhere; a graceful restart loses nothing. Delivery guarantees lists both.
Replacing EMQX
mast speaks EMQX v5’s HTTP auth wire, so an auth service written for EMQX works unchanged.
[auth]
mode = "http"
[auth.http]
wire = "emqx" # EMQX’s request and response shape
authn_url = "http://soteria:9999/v2/auth"
authz_url = "http://soteria:9999/v2/acl"
on_error = "deny" # EMQX_AUTHORIZATION__NO_MATCH
Topics reach the authorization endpoint unmounted, so existing ACL rules keep working. The migration guide has the full mapping and the four traps that actually fired doing it: is_superuser, multipath TCP on OpenShift, Helm’s map merging, and SCC uid ranges. Read the migration guide →
Where the project is
Early, and honest about it. The broker runs, moves messages across a cluster with tenant isolation, honours QoS 0 through 2, replays retained messages, delivers wills, and carries a persistent session from one node to another.
Gaps are tracked as issues rather than described as a roadmap. The parity label is what an MQTT client is entitled to assume and does not yet get. There is no dashboard and no management API, and no plan for a rule engine or data bridges. Open parity issues →
Know the limits before you build on it. QoS 1 and 2 hold across a cluster: a publish is stored on a replicated stream before it is acknowledged, and a node cut off from the core catches up when it returns. Two things are still best-effort: shared subscriptions whose members span nodes, and a message a node had taken when it died, which includes one for a device still connected to an edge that is killed outright. Delivery guarantees sets out what each QoS promises hop by hop, what it costs, and where it still falls short.
Repositories
Everything lives under the github.com/mastmq organization, under Apache-2.0.
