The Web Console

Every node whose local API is on serves a console in the browser: the node's operator sees what runs on it and manages its networks, machines, images, volumes and tenants — including each machine's screen and serial console, its graphs, and every task and change made on the node — without a command line.

The same page is also a portal for your customers: a customer who signs in with their tenant key sees only their own machines, networks and public addresses, and can start, stop and restart their machines, open their consoles and reset a password — see A portal for your customers.

Part of Compute, in early access. The console is served by the node itself; it never passes through Cenvero.

Opening it

The console lives on the same address and port as the node's local API, under /console/; the node's address on its own (https://<node>:7070/) leads there too:

https://<node>:7070/console/

The local API must be on. A node installed with the compute or suite profile has it on from the start: the installer generates the node's API token, keeps it root-only in /etc/cenvero-str/api-token, and ends by printing the console's address. Otherwise the API is off until a token is set — see API Reference (cenvero-str-ctl api-token generate). The console is on whenever the API is; there is no separate switch.
The page uses the node's own certificate. If the node's certificate is not one your browser trusts, the browser warns first; check that the address is the node's before you continue.
If you restricted the API to certain addresses (api_allowed_ips), the console is restricted to them too. In a cluster, each member's own list applies to your address, whichever member you signed in to (see Every node of a cluster).

Signing in

Operators sign in with an operator API key or the node's API token (customers sign in with their tenant key — see A portal for your customers):

sudo cenvero-str-ctl apikeys mint "web console"   # prints the key once

A key per person is better than sharing the token: the node records which key signed in, and you can revoke one person's key without affecting anyone else.

The key is sent once, in the sign-in request, over the node's encrypted connection. The browser never stores it; the node keeps it in memory for the session only.
The session is held in a cookie the page's scripts cannot read. It ends after 30 minutes without activity (the page warns you two minutes before), after 12 hours in all, when you sign out, and when the agent restarts. Typing or using the mouse in a machine's console counts as activity; a page left open on its own does not.
Revoking the key (cenvero-str-ctl apikeys revoke <id>) or changing the API token ends every session signed in with it, at its next request, and closes the machine consoles opened with it within a few seconds.
A tenant key signs in to that tenant's portal, never to the operator console.
Up to 64 operator sessions can be open at once; signing in again when they are all in use closes the one used least recently. Customers' sessions are counted apart — at most eight per customer account, where a ninth sign-in closes that account's least recently used session — so a customer signing in again and again never signs you out.
Sign-in attempts are limited per address (a burst of 5, then 10 a minute). Five wrong keys lock that address out of the API — console included — for five minutes.

What it shows

The console is laid out like a datacenter manager: a bar along the top, a tree of everything on the left, the page of what you selected in the middle, and the node's tasks along the bottom.

The top bar. Search (press Ctrl K or / from anywhere), the tasks running now, the alerts firing, the licence's state, and your sign-in.
The tree. Datacenter → the node → its Machines, Networks, Storage, Images and Tenants, each machine with its state as a shape and, when it is not the usual one, a word (stopped, suspended, degraded). Switch to the Tenants view to see each tenant's machines, networks and volumes together, and the operator's own apart. Type in the filter box to find something by name, id, address or tenant. On a phone the tree opens from the menu button.
Pages with tabs. Every object has its own address, so a reload — or a link you send someone — opens the same page and tab.

Page	Tabs	What you can do
Datacenter	Summary (nodes, totals, what needs attention — with the button that fixes it —, running tasks) · Cluster · Tasks · Log · Search	Start a machine that should be running; form, join or manage a cluster
Node	Summary (graphs of CPU, memory, disk space and network; version, licence, modules, services, self-healing checks, capacity, alerts) · Machines · Networks · Storage · Images · Tenants · Tasks	Create a machine (the wizard, below)
Machine	Summary (its state explained, graphs of CPU, memory, disk activity and network, guest agent, open consoles) · Hardware (size, network cards, volumes) · Console · Tasks · Log	Start, stop, restart, force stop; change size; add, reconnect or remove a network card; delete
Network	Summary · Addresses (which are in use, and by what) · Machines	Turn the host gateway (masquerade, DHCP) on or off; delete
Volume	Summary (size, what it is attached to, protection) · Snapshots · Backups · Tasks	Attach, detach, back up now, snapshot, grow, set how many backups to keep, revert to a snapshot, restore a backup, delete
Tenant	Summary (usage against quotas) · Machines · Networks · Volumes	Set quotas; suspend and resume; delete
Task	What it does and to what, who started it, how far it has got, when it started and ended, and its log, which follows the task while it runs	Cancel, where the operation can stop safely

Deleting anything asks you to confirm, and deleting a machine, network, volume or tenant asks you to type its name.

Changes appear as they happen. A machine that stops, a task's progress, a volume that is attached: the tree, the task panel and the page you are on follow the node as it changes, without a reload. If that live connection is interrupted the console reconnects on its own and catches up on what it missed; until it is back, pages refresh every 15 seconds. (The customer portal refreshes every 15 seconds.)

Tasks

Starting a machine, downloading an image, backing up a volume and every other longer operation is a task. The panel along the bottom lists the latest tasks with their progress — for downloads and backups, the bytes done of the total — and opens at once when you start one. Open a task to read its log as it is written, and to cancel it where the operation can stop safely. On a phone, the tasks are a page of their own.

Graphs

A node's and a machine's graphs cover the last hour as it happens (Live), or an hour, a day, a week, a month or a year. Average shows each point as the average of its time step and Peak as its highest value. Point at a graph, or focus it and use the arrow keys, to read the figures at a moment; every graph can also show its figures as a table. Times are in UTC. A break in a line is a time when nothing was recorded — the machine was stopped, or the agent was not running.

The log

The Datacenter's Log tab lists every change made to the node (and every attempt that was refused), every console sign-in and sign-out, and each task's start and end: when, who, what, on which object, from which address, and how it ended. Filter by time, person, action, object or outcome; Download saves every record the filters select, one per line; Verify the log checks that no record was changed or removed since it was written, and says so in a sentence. A machine's own Log tab shows the records about it.

Creating a machine

Create machine opens a wizard: its name and owner, the image it starts from, its size, its disk, its network cards and public address, and what it is told when it first starts (hostname and SSH keys). Sensible defaults are filled in — two vCPUs, 2 GiB of memory, a disk the image's size rounded up to the next 10 GiB, the owner's first network — and a mistake is named beside the field before anything is sent. The last step says in one sentence what will be created. Show as command gives the cenvero-str-ctl command that creates exactly the same machine (run it on the node as root), and Show as API call the same request for the API.

Every node of a cluster

On a member of a cluster (agent 1.0.0-rc.81 or later), the console manages every member, whichever one you signed in to:

The tree lists every member under Datacenter, each with its machines, networks, storage and images; tenants are shared by the members and appear once. A member that stops answering is marked, within a few seconds, with when it was last heard from (last seen 12 s ago), and its objects stay listed from its last answer. A member that has not finished joining is shown as joining, and one on a version that cannot be managed from here as update needed, with a link to its own console.
Actions on another member's machines, networks and volumes work as on the node you signed in to: the member that holds them does the work, under its own licence. A machine's screen and serial console open through the node you signed in to.
Tasks, the Log and live changes cover every member. The Log can be filtered by node, and Verify the log answers for each member, since each keeps its own log.
Datacenter › Cluster lists the members — role, whether it votes, its state (online, or when it was last seen), version, cluster address, when its identity expires, and a warning when its clock is off. It offers Make a join code (the code is shown once, with a copy button, a countdown and the command to run on the new node), the codes made so far with Revoke, and Remove… on each other member's row (type the node's name, then press Remove node). Under Advanced: giving a member a new identity, replacing the cluster's certificate authority, and Leave the cluster…, which takes the node you signed in to out of the cluster (type the cluster's name to confirm).
On a member whose plan no longer includes clustering, Datacenter › Cluster says so. It still lists the members and the join codes, with Revoke, Remove… and Leave the cluster…. Making a join code, giving a member a new identity and replacing the authority wait until the plan includes clustering again.
On a node that is not in a cluster, Datacenter › Cluster offers Form a cluster (a name and this node's address, then Form the cluster) and Join a cluster: paste a join code, check the address offered for this node, and press Next. The node asks the cluster what joining would mean, without using up the code, and the summary names the cluster and its size, the address, the tenants that will be shared, and any conflicts that stop the join. Nothing changes until you press Join the cluster.
Creating a machine asks where: it offers the members that can run machines, have Compute in their licence and are not frozen. Show as command says which node to run it on, and Show as API call addresses that node (/api/v1/nodes/<node id>/…).
A member that is not answering keeps its pages, showing what it last reported; changes to it are held back, with when it was last heard from, until it is back. Everything else keeps working.
A member that does not accept your address. Each member applies its own api_allowed_ips to your browser's address, also when you signed in to another member. If a member's list does not include it, every action and console on that member is refused with 403 forbidden: source address not allowed, and Tasks and the Log name it among the members that did not answer. Its machines and networks still appear in the tree: that is a known limitation.

A machine's screen and serial console

A running machine's Console tab opens its consoles in the browser:

Display — the machine's screen, keyboard and mouse. Buttons send Ctrl+Alt+Del, switch between fitting the window and actual size, and go full screen. Up to four people can watch at once.
Serial console — the machine's first serial port, in a terminal. One session per machine; if another session has it, Take over closes that one and connects you.

The console opens these exactly as described in Consoles: with a single-use ticket, from the node's own address. The same limits apply — a session closes after 30 minutes without input, and when the machine stops. A machine's console never outlives your sign-in: it closes at once when you sign out, and when your session ends or the key you signed in with is revoked.

Links in the serial console's output are shown as text only: the console never opens them, so nothing a machine prints can open a window or leave the page.

When the licence refuses changes

The console shows what the licence allows, with the node's own reason:

A frozen licence (or none installed): a banner says the console is read-only. Everything keeps running, and you can still look at everything and open consoles; every change is refused until the licence is renewed. In a cluster, removing a member, leaving and revoking a join code still work, so a node can always be taken out.
A module not in the plan: its pages still show what exists, but the buttons that would change it are disabled, with the reason — for example, that Storage is not included in the plan.
A module that is not available on the node (Compute on a server where the virtualization service is not installed, say): the page says why, in the node's words.

The console enforces nothing itself: every action goes to the node's API with your key, and the API applies the same checks it applies to any other client. A button the console shows as enabled can still be refused, and the console then shows the node's answer.

Security

Every change the page makes carries a per-session token and must come from the console's own page; a page on another site cannot act through your session.
The console's pages load nothing from anywhere else and cannot be framed by another site.
Sign-ins, refused sign-ins, sign-outs and sessions that end are recorded in the node's audit log (see Monitoring), and every change made through the console is recorded with the session it came through. They are also written to the node's log as audit entries (audit=true) and published as the events console.signed_in, console.sign_in_refused, console.signed_out and console.session_ended, so the event stream and webhooks carry them. They name the key that was used, never the key itself.

A portal for your customers

If you host machines for customers, each customer can have the same console, confined to their own account. You create a tenant for the customer, give their machines to that tenant, and hand the customer a tenant key; they sign in at the same address with that key.

1. Create the customer's key.

sudo cenvero-str-ctl tenant key-generate <tenant-id> --name "acme portal" --ttl 2160h

The key is printed once. Send it to the customer over a channel you trust, and give it an expiry (--ttl) so it lapses on its own if it is forgotten. A key can also be minted over the API (POST /api/v1/tenant/{id}/keys, see Tenants). Revoke it with sudo cenvero-str-ctl tenant key-revoke <key-id>: every session signed in with it ends at its next request, and the machine consoles opened with it close within a few seconds.

2. Send the customer the address. It is the node's console address, https://<node>:7070/console/. The customer's browser must be able to reach it, and must trust the node's certificate (see Opening it); if you restricted the API to certain addresses (api_allowed_ips), include the customer's.

What the customer sees and can do. The portal wears the same layout, with a tree of the customer's own machines and networks:

Page	What they see	What they can do
Overview	Their account's state (active or suspended), their limits and usage (machines, bandwidth, volumes), and their machines	—
Machines	Their own machines: state, size, addresses, network cards, open consoles, guest agent, and graphs of CPU, memory, disk activity and network	Start, stop, restart, force stop; open the display and serial console; reset a user's password through the guest agent
Networks	Their own private networks and which of their machines use them	—
Public addresses	The public addresses their machines hold	—
Tasks	What was done to their machines and volumes, by them or by you, and how it ended	—
Activity	What was done with their keys: portal sign-ins and actions on their machines	—

What stays with you. Creating and deleting machines, networks and volumes, changing a machine's size or interfaces, and changing the account's limits, bandwidth or state are not in the portal, and the node refuses them to a tenant key whatever it sends. A customer never sees another customer's machines, networks or addresses — asking for one by its id answers exactly as if it did not exist — nor your own machines, the node's licence or anything else on the node.

When you suspend the account. Suspending a tenant stops its machines. The customer can still sign in and look at everything, with a banner saying the account is suspended, but starting, stopping or restarting a machine, opening a console and resetting a password are refused until you resume the account. A console ticket issued before the suspension is refused when it is used, and a console the customer already has open closes within a few seconds.

What the customer is told when something fails. The portal answers in plain, fixed sentences — that the machine cannot be started right now, that no guest agent is running in it, and so on — never with the server's own error text. When a machine itself refuses a new password (an unknown user, say), the customer is shown the machine's own reason.

When the node's licence refuses changes. A frozen licence refuses the customer's power actions and password resets just as it refuses yours; the customer is told that changes are paused on the server and that their machines keep running. Looking and opening consoles still work.

What the customer's actions leave behind. Power actions, password resets, console sessions and portal sign-ins are recorded in the node's audit log naming the tenant key's id (cenvero-str-ctl audit list --tenant <id>), like yours; the customer can read its own records in the portal's API (GET /api/v1/tenant/{id}/audit). Consoles the customer opens appear on the machine's page in your console.

The portal shows what the customer's key allows; the node's API decides. A customer holding the key can make the same calls with any HTTP client — the API Reference lists what a tenant key may call.

Current limits

Without a cluster, one node per console: open each node's own address. Join the nodes into a cluster to manage them all from any one of them.
In a cluster, the customer portal still shows a customer only what is on the node whose tenant key they hold: tenant keys are not shared between members.
Personal accounts with passwords and roles are planned; today you sign in with an API key or the node's API token.
The customer portal carries Stratum's name; your own logo and colours are planned. Customers cannot yet create their own machines — provisioning goes through you (or your billing system over the API).
Sessions live in the agent's memory: restarting or updating the agent signs everyone out.