Networking Overview

Every network you define is a managed segment with its own subnet and addressing, carried on the host's workload bridge — and every usable address in it becomes an endpoint you can attach a workload to. This page describes that model: what a network gives you, how a workload joins one, and how a network is stretched across several hosts. Follow the links in each section for deeper coverage.

The two bridges

Every node owns two Linux bridges:

Bridge	Purpose
Management bridge	Carries agent control traffic and the traffic between nodes. Keep this reachable at all times.
Workload bridge	Carries all workload traffic. This is where your networks live.

The split is deliberate: workload saturation or a misconfigured network policy cannot starve the control plane.

Networks as L2 segments

A network is a named managed L2 segment on the workload bridge.

sudo cenvero-str-ctl network create \
  --name app-net \
  --cidr 10.20.0.0/24 \
  --gateway 10.20.0.1

Creating one gives you:

The segment itself — subnet, VLAN id if you gave one, and owning tenant.
An address pool covering the usable range, which network delete removes again.
An endpoint profile (IP ↔ generated MAC) for every usable host address, which you claim with network attach.
The gateway address reserved out of that range, so no endpoint is ever handed it.

What it does not do (unless you ask)

Worth knowing before you automate against it, because the difference is invisible until a workload fails to come up.

On its own, network create does not:

assign the gateway address to any interface — nothing on the host answers on it;
let the network reach the internet;
create a DHCP scope;
create a DNS zone.

The first three are one flag away — see The host gateway below. The gateway address matters if you expect workloads to route off the segment. Two workloads on the same network talk to each other without it; reaching anything else needs something actually holding that address.

The DHCP point is the one that bites. The address pool exists, but **the DHCP server will not hand out addresses for the network until a scope is bound to that pool** — see DHCP and DNS. Until then the server has nothing to offer, and per RFC 2131 a server with nothing to offer stays silent: the client retries with no reply, which looks exactly like a broken L2 path. The agent logs a warning naming the client when this happens. Give the workload a static address and it works immediately.

None of this depends on how a node is deployed. Every node runs the same DHCP and DNS servers, and every node routes — there is one kind of node.

The host gateway

Add --host-gateway and the node itself holds the network's gateway address, so a new workload can reach the node — and anything the node routes to — with no further steps:

sudo cenvero-str-ctl network create --name web --cidr 10.60.0.0/24 --gateway 10.60.0.1 --host-gateway

Two extras go with it:

--snat lets the network reach the internet through the node's own public address (masquerade — see Gateway NAT). It needs the node's uplink to be configured; without one it is refused, and nothing is created.
--dhcp hands the network's addresses out over DHCP, with the gateway as the router. A workload the node already knows is given its own endpoint address.

sudo cenvero-str-ctl network create --name web --cidr 10.60.0.0/24 --gateway 10.60.0.1 \
  --host-gateway --snat --dhcp

The network needs a gateway address (--gateway) for this. On an existing network, set it with network host-gateway; the flags you give are the whole setting, so running it again with other flags adds what is new and removes what you left out:

sudo cenvero-str-ctl network host-gateway net-953e5ae21abd5029 --snat
sudo cenvero-str-ctl network host-gateway net-953e5ae21abd5029 --off

The setting is kept with the network and shown in network show ("host_gateway": {"enabled": true, "snat": true, "dhcp": false}). It is put back after a restart — including a masquerade rule or DHCP scope removed by hand while the setting still asks for it — and everything it added goes when you turn it off or delete the network. Over the API: PUT /api/v1/networks/{id}/host-gateway with {"snat":true,"dhcp":false}, and DELETE to turn it off; or pass "host_gateway": true (and "snat", "dhcp") when creating the network.

A private network's workloads still cannot reach another tenant's private networks through the gateway: that separation is in the packet path, not in the gateway.

A single node can host many independent networks with non-overlapping subnets. A network belongs to the node you create it on: other nodes do not get a copy. To carry a subnet between nodes, build a VXLAN overlay — see Stretching a network across nodes with VXLAN below.

Worked example: two servers talking over a private network

End to end, with nothing assumed. Every command here was run on a live node and the output is what it actually printed.

1. Create the network

sudo cenvero-str-ctl network create --name doctest --cidr 10.77.0.0/24 --gateway 10.77.0.1

{ "status": "created", "network": { "id": "net-953e5ae21abd5029", "cidr": "10.77.0.0/24", "gateway": "10.77.0.1" } }

Keep the id — the next step needs it. The address pool and one endpoint profile per usable address are created with it.

2. Claim an endpoint for each server

sudo cenvero-str-ctl network attach net-953e5ae21abd5029

{ "status": "attached", "endpoint": { "ip": "10.77.0.2", "mac": "02:ce:0a:4d:00:02", "state": "bound" } }

Run it once per server. The second returns 10.77.0.3 / 02:ce:0a:4d:00:03.

Write down the IP and MAC pairs. They are the whole configuration — the MAC identifies the workload to the fabric and the IP is what it is allowed to use. Attaching also registers the pairing in the packet path, which is what permits that workload's traffic; an unregistered MAC is dropped.

3. Put each server on the workload bridge with its MAC

Configure the virtual machine's network interface to use the workload bridge and the MAC from step 2. On a hypervisor this is a per-guest setting; the exact field name varies, but it is the one that sets the guest's hardware address.

The MAC must match exactly. A guest presenting any other address is refused — that is the anti-spoofing working, not a fault.

4. Give each server its address

Configure the address from step 2 inside the guest, with prefix /24 and gateway 10.77.0.1:

10.77.0.2/24   on the first server
10.77.0.3/24   on the second

Use a static address here. DHCP will not answer for this network until a scope is bound to its pool and the node holds an address in the subnet on the workload bridge, and a client that gets no answer looks exactly like a broken connection. Once both exist the guest can use DHCP instead, and it is given exactly this endpoint address — see DHCP and DNS.

5. They can talk

# from the first server
ping 10.77.0.3

Traffic between them stays on this node and never touches your uplink.

Removing it

sudo cenvero-str-ctl network delete net-953e5ae21abd5029

This removes the network, its endpoints and its address pool together, so a deleted network leaves nothing behind.

What this network is and is not

Private, and private between customers. When networks are created for tenants (--tenant), traffic between the private networks of two different tenants is refused in the packet path, and that separation is put back in place when the node restarts. Those addresses are not reachable from outside either, so there is no way around it. Networks with no tenant, or two networks of the same tenant, are not separated from each other — that is the firewall's job.
Separated on the shared bridge too, for the workloads Stratum runs. All tenant networks on a node share the workload bridge, so two workloads could otherwise reach each other frame to frame without anything being routed. A tenant's virtual machines and attached containers are separated at their own port: nothing another tenant's workload sends reaches them directly — unicast, broadcast or multicast, any protocol — and they cannot look up another tenant's addresses. Same-tenant traffic and the node itself are unaffected. Servers you connect to the bridge yourself, as in the walk-through above, are covered by the routed separation only. See Tenants & Bandwidth for the full guarantee.
On one node. Two workloads on the same node talk over it directly. To stretch a network across several nodes, see the overlay section below.
Not routed off the segment by itself. Nothing holds the gateway address until you ask for the host gateway or configure something else to. Workloads on the network reach each other regardless.

Endpoints — how a workload joins a network

This is the model to understand before automating anything.

Creating a network does not create workloads; it creates endpoints. An endpoint is one address on that network paired with a hardware address, and one is prepared for every usable host address the moment the network exists. They sit unused until something claims them.

network 10.20.0.0/24  ─┬─ endpoint 10.20.0.2   free
                       ├─ endpoint 10.20.0.3   free
                       └─ ...

Attaching claims one. You can name the address you want or take the next free one, and you can supply the hardware address if the workload already has one of its own:

# take the next free address
sudo cenvero-str-ctl network attach <network-id>

# or claim a specific address for a workload that already has a MAC
sudo cenvero-str-ctl network attach <network-id> --ip 10.20.0.50 --mac 52:54:00:ab:01:02

Detaching returns the endpoint to the free pool, and the address becomes available again.

Two properties follow, and they are the reason the model works this way:

The address and hardware address are decided before the workload exists. You can build your network layout, firewall policy and DNS records first, then attach machines into a plan that is already in place.

The pairing is enforced on every packet. Traffic claiming to come from an endpoint must carry that endpoint's address and hardware address, so one workload cannot impersonate another. This is not a convention — it is checked in the packet path.

Because the endpoint is the unit, moving a workload between hosts is a matter of claiming the same address on the other node; the hardware address is derived from it, so the identity travels with it. See Moving Workloads Between Nodes.

Addresses, networks and endpoints — which one do you want?

Three things sound alike and are not, and picking the wrong one is the most common way to get stuck. Here is the whole distinction.

	Address pool (`ipam`)	Network	Endpoint (`network attach`)
What it is	A ledger of addresses	An L2 segment	One address bound to one hardware address
Gives you	An address	A place for workloads to live	An address and a MAC
Enforced on packets	No	—	Yes
Use it for	Tracking addresses you own	Building a segment	Putting a workload on that segment

The address pool is a ledger

ipam answers one question: which addresses are taken, and by whom.

sudo cenvero-str-ctl ipam allocate 15 testhost

{ "id": 9, "ip": "161.248.163.3", "pool_id": 15, "hostname": "testhost" }

An address and a label. No hardware address, and nothing enforced. Nothing stops a machine using that address anyway — the pool records that you assigned it, it does not police it. That is the right tool for keeping track of addresses you own. (Public addresses your provider routes to you and that you want to give to virtual machines have their own, enforced, home — see Public Addresses.)

The endpoint is an identity

network attach answers a different question: *which workload may use this address.*

sudo cenvero-str-ctl network attach net-953e5ae21abd5029

{ "ip": "10.77.0.2", "mac": "02:ce:0a:4d:00:02", "state": "bound" }

An address and a hardware address, paired — and that pairing is registered in the packet path. Traffic must carry both or it is refused. A workload cannot take an address it was not given, and cannot impersonate one that was given to another.

That enforcement is the entire difference. A pool entry is a note; an endpoint is a rule.

You do not create pools for networks

network create makes the pool for you, covering the network's usable range, and network delete removes it again. You will see it in ipam pools under the network's name. There is no step where you build one by hand.

You create a pool directly only for addresses that are not a network's — a block of public addresses, for instance, that you want to track allocation of.

So, in practice

Two workloads that need to talk? Create a network, attach an endpoint for each. You get the addresses and the hardware addresses, and the fabric enforces them.
Just recording which addresses are spoken for? Use a pool directly.
Wondering where the MAC comes from? Attaching generates one, unless the workload already has a hardware address of its own — pass --mac and it will be bound to the address instead.

What you can add to a network

A new network carries traffic and enforces endpoint identity, and nothing more. Each of these is a separate step, taken when you want it:

Addresses handed out by DHCP, or reserved to a specific workload — this needs a DHCP scope, which network create does not make. See DHCP & DNS.
A DNS zone so workloads can find each other by name — you create the zone and its records yourself.
A gateway address on the host, so workloads can route off the segment — the host gateway, which can also add internet access and DHCP in the same step.
Public addresses your provider routes to you, one per virtual machine — see Public Addresses.
Firewall policy, applied wherever traffic enters — see Firewall.
Bandwidth limits per tenant or per workload — see Tenants & Bandwidth.
Load balancing for published services — see Load Balancer.

For how a packet actually travels through a node and where it can be stopped, see How Stratum Works.

Stretching a network across nodes with VXLAN

An overlay carries a network between nodes, so two endpoints on different hosts share one subnet. Nothing sets it up for you — creating a network does not. Creating it is a two-part job, done on every node that takes part, and both parts are needed before a single packet moves.

How a frame crosses the overlay

Understanding this explains why the setup has two halves, and why a missing peer produces silence rather than an error.

A workload on node A sends a frame to a workload on node B. As far as both are concerned they are on the same flat network — same subnet, no router in between. What actually happens:

Node A wraps the frame. The original frame — addresses, contents and all — is placed inside an ordinary UDP packet addressed from node A to node B. The workload's addresses are now payload; the outer packet carries only the two nodes' addresses.
That packet crosses your existing network like any other traffic. Every switch and router between the nodes sees a normal UDP packet between two hosts. They neither know nor care that a whole frame is inside.
Node B unwraps it and delivers the original frame to the destination workload, which sees it as though it arrived over a local switch port.

Two consequences you will actually run into:

A node can only deliver to peers it has been told about. The wrapping step needs a destination — node A must know that this workload's address lives behind node B. That is what a peer entry is. Without it, node A has nowhere to send the wrapped packet, so the frame is dropped silently: no error, just traffic that never arrives. This is the single most common overlay problem, and it is why peers must be registered on every node, in both directions.

The overlay travels on your existing network, so that network must let it through. The wrapped packets are UDP on port 4789 between the nodes' own addresses. On a cloud provider that means the security group or firewall in front of each node, not just the node's own firewall — and it must be open in both directions.

Wrapping also adds bytes to every packet. If your underlying network only just accommodates a standard-size packet, a full-size frame plus its wrapper can exceed it, and the symptom is characteristic: small packets work perfectly, large transfers stall. If you see that, the overlay is fine and the size limit is the problem.

The VNI is the overlay's identifier. It must match on every participating node, and it is what keeps overlays separate — two overlays on the same nodes with different VNIs cannot see each other's traffic even though they share the same underlying network.

Create the overlay on every node that participates. The VNI identifies the overlay and must match; the subnet is the address range the overlay carries.

sudo cenvero-str-ctl vxlan create 4711 10.211.0.0/24

The response names the tunnel device it created, cnv-vx<vni>:

{"data":{"vni":4711,"subnet":"10.211.0.0/24","kernel_device":"cnv-vx4711","status":"created"},"status":"ok"}

If it reports "kernel_device": false instead, the node has no address to send encapsulated traffic from and the overlay will carry nothing — the message explains what is missing.

Then tell each node about the others. A node only delivers to peers it has been given, so every node needs an entry for every other node. A peer is its hostname, the hardware address of its tunnel device, and the underlay address its traffic arrives from (its VTEP). The VTEP must be an IP address — anything else is refused:

# Read the peer's tunnel-device address ON THAT PEER
cat /sys/class/net/cnv-vx4711/address        # e.g. ce:f0:ef:6b:18:f1

# ...then register it here, with the peer's underlay address
sudo cenvero-str-ctl vxlan peer-add 4711 node-b ce:f0:ef:6b:18:f1 203.0.113.9

The hardware address may be left empty — pass "" — when you do not know it. The node then sends broadcast and unknown traffic (which is what carries ARP) to that peer and learns the peer's addresses from the replies, instead of pointing one address at it:

sudo cenvero-str-ctl vxlan peer-add 4711 node-b "" 203.0.113.9

Repeat in the other direction on node-b, pointing back at this node. Then confirm both sides agree:

cenvero-str-ctl vxlan peers 4711
cenvero-str-ctl vxlan fdb

Peers persist and are re-applied when the agent restarts, so an overlay comes back on its own after a reboot. Removing a peer stops traffic to it immediately; deleting the overlay removes the tunnel device and every peer with it.

In a cluster, peers are shared. Add or remove a peer on any member of a cluster and every member that has created the overlay applies it — a member skips the entry that names itself — so each node is registered once rather than on every node. The overlay itself is still created on each node that takes part. A member cut off from the rest of the cluster refuses peer changes with 503 and changes nothing, because the change has to reach the cluster as a whole: make it on a member that can reach the others, or restore the network between them (see A node cut off from the others).

The underlay must allow UDP port 4789 between the nodes' addresses in both directions — on a cloud provider that means the firewall or security group in front of each node, not just the node's own firewall.

VRF & Geneve

Alongside the VXLAN overlay, Stratum manages two more kernel networking constructs directly from cenvero-str-ctl. Both are device management: the agent creates the devices, keeps a record of them on the node, and recreates them on startup — nothing is created at boot until you define it. The devices are ordinary kernel objects, visible with tools like ip link show.

VRF (virtual routing and forwarding) gives a node more than one independent routing table.

Why you would want that: normally a host has one routing table, so one destination has one answer — and two customers who both use 10.0.0.0/24 internally cannot both be routed correctly, because the address is ambiguous. A VRF removes the ambiguity by giving each its own table: the same address can route to different places depending on which VRF the traffic arrived in. It is also how you keep two upstream providers' routes from mixing on one box.

You create a VRF bound to a routing-table id and attach interfaces to it; traffic on those interfaces is then routed in that VRF's table instead of the main one. The kernel device is named cnv-vrf-<name>.

# Create a VRF "red" on routing table 100 and enslave a NIC to it
sudo cenvero-str-ctl vrf create red --table 100
sudo cenvero-str-ctl vrf attach red --iface cnv-nic-1

# Inspect, then release the interface and delete the VRF
cenvero-str-ctl vrf list
cenvero-str-ctl vrf show red
sudo cenvero-str-ctl vrf detach red --iface cnv-nic-1
sudo cenvero-str-ctl vrf delete red

Deleting a VRF releases every enslaved interface first, so none is left orphaned.

Geneve is an overlay tunnel type (RFC 8926) — a sibling of VXLAN that wraps frames the same way, described above.

When to reach for it instead of VXLAN: almost never, unless something you must interoperate with requires it. Geneve's advantage is an extensible header that can carry additional metadata alongside the frame, which matters to some hardware and to some other platforms. For stretching a Stratum network across your own nodes, use VXLAN — it is the overlay the rest of Stratum is built around and the one that is proven end-to-end across hosts. Choose Geneve when you are terminating a tunnel from equipment that speaks Geneve and not VXLAN.

A Geneve device carries a 24-bit VNI to a remote endpoint IP, with an optional outer UDP port (default 6081) and TTL; the kernel device is named cnv-gnv-<name>.

sudo cenvero-str-ctl geneve create blue --vni 42 --remote 10.0.0.2
sudo cenvero-str-ctl geneve create green --vni 7 --remote 10.0.0.3 --port 6081 --ttl 64

cenvero-str-ctl geneve list
cenvero-str-ctl geneve show blue
sudo cenvero-str-ctl geneve delete blue

Creating a Geneve device configures the tunnel endpoint on this host. Carrying packets across the overlay also requires the remote endpoint to be configured to match; validate cross-host Geneve forwarding in your own environment before relying on it.

Firewall and routing

The firewall is a dual-stack L3/L4 ACL operating at the network level. Its default action starts as allow; set it to deny so all inter-network and external traffic is blocked unless you add an explicit allow rule. See Firewall.

Traffic that leaves the fabric is routed by the node — using routes you add, or learn from upstream routers over BGP — and published services can be fronted by the L4 or L7 load balancer. See BGP Edge Routing and Load Balancer.