Docs / Networking Overview

CENVERO/DOCS

// Networking Overview

Networking Overview

Every network you define is a managed segment with its own subnet, addressing and DNS, carried on the host's workload bridge — and every usable address in it becomes an endpoint you can attach a workload to. This page describes that model: what a network gives you, how a workload joins one, and how a network is stretched across several hosts. Follow the links in each section for deeper coverage.

The two bridges

Every node owns two Linux bridges:

Bridge	Purpose
Management bridge	Carries agent control traffic, cluster communication, and the Gateway HA channel. Keep this reachable at all times.
Workload bridge	Carries all workload traffic. This is where your networks live.

The split is deliberate: workload saturation or a misconfigured network policy cannot starve the control plane.

Networks as L2 segments

A network is a named managed L2 segment on the workload bridge.

sudo cenvero-str-ctl network create \
  --name app-net \
  --cidr 10.20.0.0/24 \
  --gateway 10.20.0.1

Creating one gives you:

The segment itself — subnet, VLAN id if you gave one, and owning tenant.
An address pool covering the usable range, which network delete removes again.
An endpoint profile (IP ↔ generated MAC) for every usable host address, which

you claim with network attach.

The gateway address reserved out of that range, so no endpoint is ever handed it.

What it does not do

Worth knowing before you automate against it, because the difference is invisible until a workload fails to come up.

network create does not:

assign the gateway address to any interface — nothing on the host answers on it;
create a DHCP scope;
create a DNS zone.

The gateway address matters if you expect workloads to route off the segment. Two workloads on the same network talk to each other without it; reaching anything else needs something actually holding that address.

The DHCP point is the one that bites. The address pool exists, but **the DHCP server will not hand out addresses for the network until a scope is bound to that pool** — see DHCP and DNS. Until then the server has nothing to offer, and per RFC 2131 a server with nothing to offer stays silent: the client retries with no reply, which looks exactly like a broken L2 path. The agent logs a warning naming the client when this happens. Give the workload a static address and it works immediately.

None of this depends on how a node is deployed. Every node runs the same DHCP and DNS servers, and every node routes — there is one kind of node.

A single node can host many independent networks with non-overlapping subnets. In a cluster, the VXLAN overlay stretches each network across every node so an endpoint can move hosts without changing its IP — see Clustering Overview.

Worked example: two servers talking over a private network

End to end, with nothing assumed. Every command here was run on a live node and the output is what it actually printed.

1. Create the network

sudo cenvero-str-ctl network create --name doctest --cidr 10.77.0.0/24 --gateway 10.77.0.1

{ "status": "created", "network": { "id": "net-953e5ae21abd5029", "cidr": "10.77.0.0/24", "gateway": "10.77.0.1" } }

Keep the id — the next step needs it. The address pool and one endpoint profile per usable address are created with it.

2. Claim an endpoint for each server

sudo cenvero-str-ctl network attach net-953e5ae21abd5029

{ "status": "attached", "endpoint": { "ip": "10.77.0.2", "mac": "02:ce:0a:4d:00:02", "state": "bound" } }

Run it once per server. The second returns 10.77.0.3 / 02:ce:0a:4d:00:03.

Write down the IP and MAC pairs. They are the whole configuration — the MAC identifies the workload to the fabric and the IP is what it is allowed to use. Attaching also registers the pairing in the packet path, which is what permits that workload's traffic; an unregistered MAC is dropped.

3. Put each server on the workload bridge with its MAC

Configure the virtual machine's network interface to use the workload bridge and the MAC from step 2. On a hypervisor this is a per-guest setting; the exact field name varies, but it is the one that sets the guest's hardware address.

The MAC must match exactly. A guest presenting any other address is refused — that is the anti-spoofing working, not a fault.

4. Give each server its address

Configure the address from step 2 inside the guest, with prefix /24 and gateway 10.77.0.1:

10.77.0.2/24   on the first server
10.77.0.3/24   on the second

Use a static address. DHCP will not answer for this network until a scope is bound to its pool, and a client that gets no answer looks exactly like a broken connection — see DHCP and DNS.

5. They can talk

# from the first server
ping 10.77.0.3

Traffic between them stays on this node and never touches your uplink.

Removing it

sudo cenvero-str-ctl network delete net-953e5ae21abd5029

This removes the network, its endpoints and its address pool together, so a deleted network leaves nothing behind.

What this network is and is not

Private, and private between customers. Traffic between the private

networks of two different tenants is refused in the packet path. Those addresses are not reachable from outside either, so there is no way around it.

On one node. Two workloads on the same node talk over it directly. To

stretch a network across several nodes, see the overlay section below.

Not routed off the segment by itself. Nothing holds the gateway address

until you configure something to. Workloads on the network reach each other regardless.

Endpoints — how a workload joins a network

This is the model to understand before automating anything.

Creating a network does not create workloads; it creates endpoints. An endpoint is one address on that network paired with a hardware address, and one is prepared for every usable host address the moment the network exists. They sit unused until something claims them.

network 10.20.0.0/24  ─┬─ endpoint 10.20.0.2   free
                       ├─ endpoint 10.20.0.3   free
                       └─ ...

Attaching claims one. You can name the address you want or take the next free one, and you can supply the hardware address if the workload already has one of its own:

# take the next free address
sudo cenvero-str-ctl network attach <network-id>

# or claim a specific address for a workload that already has a MAC
sudo cenvero-str-ctl network attach <network-id> --ip 10.20.0.50 --mac 52:54:00:ab:01:02

Detaching returns the endpoint to the free pool, and the address becomes available again.

Two properties follow, and they are the reason the model works this way:

The address and hardware address are decided before the workload exists. You can build your network layout, firewall policy and DNS records first, then attach machines into a plan that is already in place.

The pairing is enforced on every packet. Traffic claiming to come from an endpoint must carry that endpoint's address and hardware address, so one workload cannot impersonate another. This is not a convention — it is checked in the packet path.

Because the endpoint is the unit, moving a workload between hosts is a matter of attaching the same endpoint elsewhere; the address travels with it.

Addresses, networks and endpoints — which one do you want?

Three things sound alike and are not, and picking the wrong one is the most common way to get stuck. Here is the whole distinction.

	Address pool (`ipam`)	Network	Endpoint (`network attach`)
What it is	A ledger of addresses	An L2 segment	One address bound to one hardware address
Gives you	An address	A place for workloads to live	An address and a MAC
Enforced on packets	No	—	Yes
Use it for	Tracking addresses you own	Building a segment	Putting a workload on that segment

The address pool is a ledger

ipam answers one question: which addresses are taken, and by whom.

sudo cenvero-str-ctl ipam allocate 15 testhost

{ "id": 9, "ip": "161.248.163.3", "pool_id": 15, "hostname": "testhost" }

An address and a label. No hardware address, and nothing enforced. Nothing stops a machine using that address anyway — the pool records that you assigned it, it does not police it. That is the right tool for keeping track of addresses you own, such as the public addresses your provider routed to you.

The endpoint is an identity

network attach answers a different question: *which workload may use this address.*

sudo cenvero-str-ctl network attach net-953e5ae21abd5029

{ "ip": "10.77.0.2", "mac": "02:ce:0a:4d:00:02", "state": "bound" }

An address and a hardware address, paired — and that pairing is registered in the packet path. Traffic must carry both or it is refused. A workload cannot take an address it was not given, and cannot impersonate one that was given to another.

That enforcement is the entire difference. A pool entry is a note; an endpoint is a rule.

You do not create pools for networks

network create makes the pool for you, covering the network's usable range, and network delete removes it again. You will see it in ipam pools under the network's name. There is no step where you build one by hand.

You create a pool directly only for addresses that are not a network's — a block of public addresses, for instance, that you want to track allocation of.

So, in practice

Two workloads that need to talk? Create a network, attach an endpoint for

each. You get the addresses and the hardware addresses, and the fabric enforces them.

Just recording which addresses are spoken for? Use a pool directly.
Wondering where the MAC comes from? Attaching generates one, unless the

workload already has a hardware address of its own — pass --mac and it will be bound to the address instead.

What runs on the network

Once a network exists it comes with services already attached, rather than needing separate ones configured:

Addresses handed out automatically, or reserved to a specific workload —

see DHCP & DNS.

A DNS zone the network answers for, so workloads can find each other by

name.

Firewall policy, applied wherever traffic enters — see

Zero-Trust Firewall.

Bandwidth limits per tenant.
Load balancing for published services — see

Load Balancer.

For how a packet actually travels through a node and where it can be stopped, see How Stratum Works.

Stretching a network across nodes with VXLAN

An overlay carries a network between nodes, so two endpoints on different hosts share one subnet. Creating it is a two-part job, and both parts are needed before a single packet moves.

How a frame crosses the overlay

Understanding this explains why the setup has two halves, and why a missing peer produces silence rather than an error.

A workload on node A sends a frame to a workload on node B. As far as both are concerned they are on the same flat network — same subnet, no router in between. What actually happens:

Node A wraps the frame. The original frame — addresses, contents and all — is placed inside an ordinary UDP packet addressed from node A to node B. The workload's addresses are now payload; the outer packet carries only the two nodes' addresses.
That packet crosses your existing network like any other traffic. Every switch and router between the nodes sees a normal UDP packet between two hosts. They neither know nor care that a whole frame is inside.
Node B unwraps it and delivers the original frame to the destination workload, which sees it as though it arrived over a local switch port.

Two consequences you will actually run into:

A node can only deliver to peers it has been told about. The wrapping step needs a destination — node A must know that this workload's address lives behind node B. That is what a peer entry is. Without it, node A has nowhere to send the wrapped packet, so the frame is dropped silently: no error, just traffic that never arrives. This is the single most common overlay problem, and it is why peers must be registered on every node, in both directions.

The overlay travels on your existing network, so that network must let it through. The wrapped packets are UDP on port 4789 between the nodes' own addresses. On a cloud provider that means the security group or firewall in front of each node, not just the node's own firewall — and it must be open in both directions.

Wrapping also adds bytes to every packet. If your underlying network only just accommodates a standard-size packet, a full-size frame plus its wrapper can exceed it, and the symptom is characteristic: small packets work perfectly, large transfers stall. If you see that, the overlay is fine and the size limit is the problem.

The VNI is the overlay's identifier. It must match on every participating node, and it is what keeps overlays separate — two overlays on the same nodes with different VNIs cannot see each other's traffic even though they share the same underlying network.

Create the overlay on every node that participates. The VNI identifies the overlay and must match; the subnet is the address range the overlay carries.

sudo cenvero-str-ctl vxlan create 4711 10.211.0.0/24

The response names the tunnel device it created, cnv-vx<vni>:

{"data":{"vni":4711,"subnet":"10.211.0.0/24","kernel_device":"cnv-vx4711","status":"created"},"status":"ok"}

If it reports "kernel_device": false instead, the node has no address to send encapsulated traffic from and the overlay will carry nothing — the message explains what is missing.

Then tell each node about the others. A node only delivers to peers it has been given, so every node needs an entry for every other node. A peer is its hostname, the address of its tunnel device, and the public address its traffic arrives from:

# Read the peer's tunnel-device address ON THAT PEER
cat /sys/class/net/cnv-vx4711/address        # e.g. ce:f0:ef:6b:18:f1

# ...then register it here, with the peer's underlay address
sudo cenvero-str-ctl vxlan peer-add 4711 node-b ce:f0:ef:6b:18:f1 203.0.113.9

Repeat in the other direction on node-b, pointing back at this node. Then confirm both sides agree:

cenvero-str-ctl vxlan peers 4711
cenvero-str-ctl vxlan fdb

Peers persist and are re-applied when the agent restarts, so an overlay comes back on its own after a reboot. Removing a peer stops traffic to it immediately; deleting the overlay removes the tunnel device and every peer with it.

The underlay must allow UDP port 4789 between the nodes' addresses in both directions — on a cloud provider that means the firewall or security group in front of each node, not just the node's own firewall.

VRF & Geneve

Alongside the VXLAN overlay, Stratum manages two more kernel networking constructs directly from cenvero-str-ctl. Both are control-plane device management: the agent creates the kernel devices, tracks them, persists them in its local database, and recreates them on startup — nothing is created at boot until you define it. The devices are ordinary kernel objects, visible with tools like ip link show.

VRF (virtual routing and forwarding) gives a node more than one independent routing table.

Why you would want that: normally a host has one routing table, so one destination has one answer — and two customers who both use 10.0.0.0/24 internally cannot both be routed correctly, because the address is ambiguous. A VRF removes the ambiguity by giving each its own table: the same address can route to different places depending on which VRF the traffic arrived in. It is also how you keep two upstream providers' routes from mixing on one box.

You create a VRF bound to a routing-table id and attach interfaces to it; traffic on those interfaces is then routed in that VRF's table instead of the main one. The kernel device is named cnv-vrf-<name>.

# Create a VRF "red" on routing table 100 and enslave a NIC to it
sudo cenvero-str-ctl vrf create red --table 100
sudo cenvero-str-ctl vrf attach red --iface cnv-nic-1

# Inspect, then release the interface and delete the VRF
cenvero-str-ctl vrf list
cenvero-str-ctl vrf show red
sudo cenvero-str-ctl vrf detach red --iface cnv-nic-1
sudo cenvero-str-ctl vrf delete red

Deleting a VRF releases every enslaved interface first, so none is left orphaned.

Geneve is an overlay tunnel type (RFC 8926) — a sibling of VXLAN that wraps frames the same way, described above.

When to reach for it instead of VXLAN: almost never, unless something you must interoperate with requires it. Geneve's advantage is an extensible header that can carry additional metadata alongside the frame, which matters to some hardware and to some other platforms. For stretching a Stratum network across your own nodes, use VXLAN — it is the path that clustering uses and the one that is proven end-to-end across hosts. Choose Geneve when you are terminating a tunnel from equipment that speaks Geneve and not VXLAN.

A Geneve device carries a 24-bit VNI to a remote endpoint IP, with an optional outer UDP port (default 6081) and TTL; the kernel device is named cnv-gnv-<name>.

sudo cenvero-str-ctl geneve create blue --vni 42 --remote 10.0.0.2
sudo cenvero-str-ctl geneve create green --vni 7 --remote 10.0.0.3 --port 6081 --ttl 64

cenvero-str-ctl geneve list
cenvero-str-ctl geneve show blue
sudo cenvero-str-ctl geneve delete blue

Creating a Geneve device configures the tunnel endpoint on this host. Carrying packets across the overlay also requires the remote endpoint to be configured to match; validate cross-host Geneve forwarding in your own environment before relying on it.

Firewall and routing

The firewall is a dual-stack L3/L4 ACL operating at the network level. For a zero-trust posture, set its default action to deny so all inter-network and external traffic is blocked unless you add an explicit allow rule. See Zero-Trust Firewall.

Traffic that exits the fabric hits the BGP speaker for north-south routing, and published services can be fronted by the L4 or L7 load balancer. See BGP Edge Routing and Load Balancer.