{
    "product": "Cenvero Stratum",
    "generated_at": "2026-08-03T07:18:42+00:00",
    "format": "cenvero-docs-v1",
    "document_count": 1,
    "documents": [
        {
            "slug": "clustering/overview",
            "title": "Clustering Overview",
            "category": "Clustering",
            "url": "https://www.stratum.cenvero.com/docs/clustering/overview",
            "headings": [
                {
                    "level": 1,
                    "text": "Clustering Overview"
                },
                {
                    "level": 2,
                    "text": "Which node do you talk to?"
                },
                {
                    "level": 2,
                    "text": "Ports used by clustering"
                },
                {
                    "level": 2,
                    "text": "High-availability cluster state"
                },
                {
                    "level": 2,
                    "text": "VXLAN overlay mesh"
                },
                {
                    "level": 2,
                    "text": "Forming a cluster"
                },
                {
                    "level": 2,
                    "text": "Leader election"
                },
                {
                    "level": 2,
                    "text": "Joining and leaving"
                },
                {
                    "level": 2,
                    "text": "See also"
                }
            ],
            "word_count": 1127,
            "markdown": "# Clustering Overview\n\nA Stratum cluster is a group of nodes that share network state and run a VXLAN overlay mesh so that every managed network is present on every host. Shared state is **replicated across the cluster** for high availability: one node is the elected leader and accepts writes, the others replicate it. The VXLAN overlay stretches L2 segments across physical hosts, so a workload's endpoint keeps its IP and MAC when it moves between nodes.\n\nEvery node in a cluster is the same kind of node: each hosts workloads and each can handle north-south traffic.\n\n## Which node do you talk to?\n\nThis is the first question when building anything against a cluster, and the\nanswer is not \"any of them\".\n\n**Reads work anywhere.** Every member holds the replicated state, so listing\nnetworks, endpoints, rules or leases gives the same answer on any node.\n\n**Changes belong to the node that owns the thing you are changing.** Two kinds of\nstate exist and they behave differently:\n\n| What you are changing | Where to send it |\n|---|---|\n| Something on one machine — its interfaces, its local services, its own attachments | That node |\n| Shared cluster state | The leader |\n\nA change to shared state sent to a follower is not silently swallowed, but it\ndoes not replicate either — the leader is what writes. Ask `cluster status` which\nnode is the leader before making one, and expect the answer to change over time,\nbecause leadership moves when a node restarts or becomes unreachable.\n\n> For anything automated, look up the leader rather than hard-coding a node.\n> A cluster that has failed over once will otherwise start silently dropping\n> your changes while still answering every read normally.\n\n**Changes are not instant across the cluster.** Replication is quick but not\nsimultaneous, so a write on the leader followed immediately by a read on another\nnode can miss it. If you write then verify, read back from the same node you\nwrote to.\n\n## Ports used by clustering\n\nCluster communication happens only on the management bridge. Open these ports between all cluster members on the management network:\n\n| Port | Protocol | Purpose |\n|------|----------|---------|\n| 7071 | TCP | Node-to-node control plane |\n| 7073 | TCP | Cluster state replication |\n| 7074 | UDP | Gateway HA |\n\nDo not expose these ports to untrusted networks. The management bridge is separate from the workload bridge for exactly this reason — see [Networking Overview](/docs/networking/overview).\n\n## High-availability cluster state\n\nStratum replicates cluster state across every member: the list of networks, endpoint bindings, firewall rules, load balancer VIPs, and cluster membership. State replication is persisted on each node so a restarted node rejoins with the latest state.\n\nClustering runs only when it is enabled (`cluster.enabled: true`) and **requires mutual TLS** between members — the transport refuses to run unauthenticated and there is no silent plaintext downgrade. Provision the cluster member certificates before enabling clustering.\n\nKey properties:\n\n- A cluster of N nodes tolerates `(N-1)/2` simultaneous failures and still makes progress.\n- A **3-node cluster** (2 Compute + 1 Gateway, for example) tolerates 1 failure.\n- A **5-node cluster** tolerates 2 simultaneous failures.\n- A cluster of 2 nodes has **no fault tolerance** — losing one node stalls writes.\n\n> A 1-node deployment runs in single-node mode: the agent is always the leader and there is no replication. This is fine for development and testing.\n\n## VXLAN overlay mesh\n\nEach network you create is stretched across all cluster members via a VXLAN overlay, anchored on the management interface. When an endpoint on node A sends a frame to an endpoint on node B, the overlay encapsulates the frame and sends it to node B's management IP. The receiving node decapsulates it and delivers it to the destination endpoint — transparently, as if both endpoints were on the same physical switch.\n\nEach Stratum network gets its own isolated overlay segment, so networks remain isolated even though they share the same underlay. MAC-to-node mappings are maintained in the cluster's shared state and updated in real time as endpoints attach, detach, and move between nodes. Validate cross-host forwarding against your own topology before production rollout.\n\n## Forming a cluster\n\nClustering is provisioned centrally, not with an ad-hoc CLI join. Enable it through the **panel-delivered signed configuration** (the `cluster_enabled` / `cluster_bind_addr` / `cluster_bootstrap` fields and the mutual-TLS material under `cluster_cert_dir`), read by each agent at boot. The first node bootstraps as leader; each additional node joins once its config is applied and its member certificate is trusted.\n\nA node can also be added over the node's REST API — `POST /api/v1/cluster/join` with `{ \"node_id\": \"<id>\", \"address\": \"<mgmt-ip:7073>\" }` — pointing the joining node at an existing member. The joining node syncs the full cluster state and applies it before participating in elections.\n\nCheck the cluster from any member:\n\n```bash\ncenvero-str-ctl cluster status\n```\n\n```text\nROLE      NODE-ID   PEER              STATE\nleader    cmp-01    10.0.0.11:7073    healthy\nfollower  cmp-02    10.0.0.12:7073    healthy\nfollower  gw-01     10.0.0.13:7073    healthy\n```\n\n## Leader election\n\nIf the current leader becomes unreachable, the remaining members start an election after a short timeout elapses with no signal from the leader. The node with the most up-to-date state and a majority of votes becomes the new leader. During the election window, writes are paused — existing traffic continues uninterrupted: forwarding does not depend on the leader being available, so an election is felt only by whoever is trying to make a change at that moment.\n\nYou can see the current leader at any time — `cluster status` reports the leader's id along with an `is_leader` flag for the node you ran it on:\n\n```bash\ncenvero-str-ctl cluster status\n```\n\n## Joining and leaving\n\nAdd a new node at any time by provisioning it into the cluster — through the panel-delivered config or `POST /api/v1/cluster/join`. The cluster rebalances: if the new node is a Gateway, it begins participating in HA once it has caught up with the current cluster state.\n\nTo remove a node gracefully — for maintenance or decommission — have it leave over the node's REST API:\n\n```bash\ncurl -k -X POST \"$NODE/api/v1/cluster/leave\" -H \"Authorization: Bearer $TOKEN\"\n```\n\nThis notifies the leader, which records the membership change across the cluster and adjusts the quorum size. The leaving node shuts down its cluster participation cleanly. Workloads on a node being removed should be moved to another node first — see [Moving Workloads Between Nodes](/docs/clustering/live-migration).\n\n> Do not hard-power-off a node without letting it leave the cluster first. The cluster will continue to function (assuming quorum remains), but it will count the node as a failed member until it is explicitly removed.\n\n## See also\n\n- [Gateway High Availability](/docs/clustering/high-availability) — redundant Gateways and sub-second failover.\n- [Moving Workloads Between Nodes](/docs/clustering/live-migration) — keeping an endpoint's IP/MAC when it moves between nodes.\n- [Networking Overview](/docs/networking/overview) — how the VXLAN overlay extends L2 networks.\n- [Configuration](/docs/configuration) — the cluster fields in the node config reference.\n"
        }
    ]
}