🏭 OpenStack for Real: Deployment Tools, High Availability, Storage, Upgrades and What to Decide First

The series ends with a cloud that you can throw away. This page is the map for the day it must not be thrown away: the real deployment tools, what high availability means here, where the storage should live, how upgrades work, what the vendors sell, and a checklist of decisions to make before you buy a single server. Read from the official documentation, not run in a lab

What the lab taught us about "not production"

The DevStack page already carried the warning that DevStack is not meant for production. Here is what we actually saw, and what each item means for a real cloud:

• One machine for everything. The database, the message queue, every API and the hypervisor lived on one host. Lose it and the whole cloud, and every server, would be gone (I did not test losing the machine).
• A reboot broke it. The br-ex bridge came back without its address, and the guests were shut off while Nova said ACTIVE (page 2). A real cloud must come back from a power cut on its own.
• Storage was a 10 GB file on one disk (page 4), with the backup service not installed at all.
• Networking was a lab trick: the public network was a bridge on the host, and I added a NAT rule by hand to give guests internet (page 3).
• Everything ran with the lab passwords (practice) over plain HTTP.

A production cloud is the answer to each of those points. The rest of the page is the menu of answers.

The shape of a real cloud

The official installation guide describes the minimum "proof of concept" as at least two machines (a controller and a compute node), each with at least two network interfaces. The roles, as it describes them:

• Controller node: identity, image, placement and the compute management services, plus the dashboard, the database, the message queue and NTP. Optional services (block storage, object storage, orchestration, telemetry) can also run there.
• Compute node: the hypervisor (typically KVM) running the instances, plus the networking agents for connectivity and security groups. You add compute nodes as you grow.
• Storage nodes (optional): block storage nodes provide the disks for instances; object storage needs at least two nodes.

The same guide offers two networking models: provider networks (mostly layer 2 with VLANs, and your physical network does the routing, with no self-service networks) and self-service networks (adds routing and overlays such as VXLAN, so users can build their own networks as on page 5). Which one you want is a decision about who is allowed to build networks, not a technical detail.

Do not install it by hand: the deployment tools

Nobody should type a production cloud in by hand. The OpenStack project itself ships several deployment projects. These are the ones I read about:

Kolla-Ansible deploys OpenStack as containers, driven by Ansible. Its own description: "production-ready containers and deployment tools for operating OpenStack clouds", "highly opinionated out of the box" but customizable. Facts from its quickstart:

• The test minimum it states: 8 GB of RAM, 40 GB of disk and 2 network interfaces.
• Supported container base distributions: Debian, Rocky (recommended for newcomers) and Ubuntu (24.04 recommended).
• The workflow is four commands: kolla-ansible bootstrap-servers, then prechecks, then deploy, then post-deploy (which writes the credentials file). You describe the machines in an inventory file and the settings in globals.yml.
• Two network settings you always have to get right: network_interface (the default interface for the management-type networks) and neutron_external_interface (the interface for external networks, which "should be active without IP address"). The kolla_internal_vip_address is the virtual IP for management traffic, handled by Keepalived.

OpenStack-Ansible deploys OpenStack from Ansible playbooks and roles. Its documentation has a dedicated production environment guide, and covers scaling, backup and restore and upgrades. Its example production layout (an example, not a rule) is:

• 3 infrastructure (control plane) hosts running Galera (the database), RabbitMQ (the message queue), Keystone, Horizon and the rest,
• 2 compute hosts, 1 NFS storage device behind Glance and Cinder and 1 log host,
• separate bridges for management, the overlay network, provider networks and storage, each on its own VLAN, with bonded pairs of network cards on every host,
• a virtual IP held by HAProxy and Keepalived on the infrastructure hosts.

(It is fun that the example uses bonded NICs: see our bonding page for what that means on a Linux host.)

OpenStack-Helm deploys OpenStack on Kubernetes with Helm charts. The pages I read list Kubernetes prerequisites and chart options but do not make a maturity statement, and the version shown was a development one (2026.1.1.dev...), so I can say what it is, not whether it suits you.

I only read the front pages and one page of detail for each, so the choice between them is yours to research: how they treat upgrades, which one your team already knows (Ansible or Kubernetes), and what each supports.

High availability: what it means

A single controller is a single point of failure. The HA guide builds on a few ideas:

• Quorum is "the minimal number of nodes that must be functional in a cluster of redundant nodes in order for the cluster to remain functional". The guide recommends floor(n/2) + 1, to prevent split-brain: the situation where a network split leaves two groups each believing they are in charge.
• That is why clusters have an odd number of nodes. With seven nodes the quorum is four, so up to three can fail. (This also explains why the production example above has three control hosts and not two: the arithmetic works out to survive one.)
• Active/passive keeps a spare that is brought up on failure (a virtual IP moves, with something like Pacemaker watching). Active/active runs everything at once, so users are unlikely to notice a failure.
• Stateless services (the API processes) scale out behind a load balancer. Stateful services (the database) must keep identical state on every copy, or have a passive standby.

Put it together with the tools above: the API services are copied across the control hosts behind HAProxy and a Keepalived virtual IP, and the database and message queue, the two stateful pieces we saw on page 1, are clustered (Galera and RabbitMQ in the OpenStack-Ansible example). What the HA guide says about specific database and queue settings was not on the pages I read, so follow the guide for your release when you build it.

And note what HA of the control plane does not do: it keeps the API alive. A guest on a failed compute node is a separate problem. The guide's section for compute nodes is where that lives, and I did not read it.

Storage: where the data really lives

On page 4 the volumes were thin LVs in a file on one disk. In production the usual answer is Ceph. The Cinder documentation describes it as "a massively scalable, open source, distributed storage system" with object, block and file storage, running on commodity hardware with self-healing. The facts that matter for planning:

• The RBD driver lets KVM and QEMU use Ceph block devices for volumes, and with the right access the images pool lets Cinder create a volume from an image with a copy-on-write clone: far faster than copying.
• Version rule: Cinder supports "the current Ceph active stable releases plus the two prior releases", and mixing client and server versions is unsupported.
• By default Cinder expects exclusive management of its pool (rbd_pool, default rbd).

The alternatives named in the documentation I read are NFS (the OpenStack-Ansible example uses one NFS device, a simple but single storage device) and vendor storage arrays through Cinder drivers. Backups need the Cinder backup service, which was not installed in our lab and broke in a very specific way (page 4): in production, install it and test a restore.

Upgrades: the part people forget

OpenStack releases every six months. The upgrade rules from the release documentation:

• Every other release is a SLURP release (Skip Level Upgrade Release Process). You can upgrade between consecutive releases, or directly between SLURP releases, skipping the one in between. That makes a yearly cycle possible.
• Upgrades from one non-SLURP release to another get no testing or support guarantees.
• Going further than two SLURP cycles needs "an FFU style arrangement" (fast-forward upgrade), which means intermediate stops.
• Support: deployers are expected to support the most recent SLURP release and the one before it, which is about 18 months during the non-SLURP cycles. Each release is maintained for roughly 12 months.

Nova's documented rolling upgrade order: database sync first (nova-manage api_db sync and then nova-manage db sync for the cells), then Placement, then the control plane services, then the compute nodes in batches, with RPC version pinning ([upgrade_levels] compute=auto) so old and new compute nodes can work together. The documentation's goal is "no VM downtime" and "no downtime during upgrades of the Nova control plane" (some API actions on a node being upgraded may be delayed).

With Kolla-Ansible, the documented upgrade is: back up /etc/kolla, upgrade the tool, update the inventory, globals.yml and passwords.yml per the release notes, kolla-ansible pull the new images, prechecks, then kolla-ansible upgrade. Two of its warnings are worth remembering: do not use Ansible's --limit during upgrades (known bugs), and for skip-level upgrades RabbitMQ does not support jumping two versions in one upgrade, so an intermediate step is needed.

None of this was tested here. The practical lesson is the same on every cloud: rehearse the upgrade on a copy, and read the release notes for your release first.

What to buy instead of building

You do not have to run it yourself. The OpenStack marketplace lists distributions (23 when I looked), among them Red Hat OpenStack Platform, Canonical OpenStack (based on Sunbeam, which the page I read says is hosted under the OpenInfra Foundation, with commercial services around it), Mirantis OpenStack for Kubernetes, Debian (the packages), OSISM and Yaook, plus many telecom-oriented and regional products. I did not evaluate any of them, and a listing is not a recommendation. What a distribution gives you is someone else to call, tested combinations and support, for money; what it costs you is choice and sometimes lock-in. Another route is a public cloud built on OpenStack operated by a provider: you then use the API from page 3 without running anything.

Security: start from the guide

The OpenStack Security Guide has a checklist per service: Identity, Dashboard, Compute, Block Storage, Shared File Systems and Networking. I only saw the section titles, not the items, so I cannot summarize them honestly. Use them as your hardening list, the way the site's hardening pages work for a Linux host. Whatever else you decide, the lab's practices (shared passwords, no TLS) must go: Kolla-Ansible, for one, has settings for TLS on the internal and external endpoints, but the page I read did not detail them.

Ten decisions before you order hardware

1. Why a private cloud at all? (cost at scale, sovereignty, existing hardware, API access). If the answer is "to try it", you are already done: keep DevStack.
2. Who operates it, and is that a full-time job? Plan for people, not just servers.
3. Size: how many guests, how big, how much storage, how fast it grows. (The one test minimum I can quote is Kolla's 8 GB and 40 GB, which is a test minimum.)
4. Control plane: three nodes for quorum, behind a virtual IP.
5. Deployment tool: Kolla-Ansible, OpenStack-Ansible, Helm on Kubernetes, or a vendor's.
6. Networking model: provider only, or self-service tenant networks. Physical design: separate management, storage and overlay networks, bonded NICs, VLANs.
7. Storage: Ceph, arrays or NFS, and who restores a backup at 3 a.m.
8. Upgrade policy: SLURP yearly or every six months, and a test environment to rehearse on.
9. Security: TLS everywhere, the security guide checklists, who gets admin.
10. Exit plan: how you would move to another cloud (the Terraform page is the first step: everything in a file).

What this page did not cover

• Anything tested. No production tool was installed or run.
• Sizing numbers for real workloads, hardware recommendations, CPU overcommit settings.
• Monitoring and logging (OpenStack has telemetry services; none were read for this page), capacity planning and billing.
• Compute failover (evacuating guests from a dead node), multi-region and availability zones, Octavia load balancers, Ironic (bare metal), Magnum (Kubernetes), Manila and Swift.
• Costs, licensing and vendor comparisons.

The cheat sheet

Roles        controller (API, DB, queue) + compute (KVM) + storage; at least 2 NICs per node
Deploy       Kolla-Ansible (containers) | OpenStack-Ansible | OpenStack-Helm (Kubernetes) | a vendor
Kolla flow   bootstrap-servers -> prechecks -> deploy -> post-deploy   (inventory + globals.yml)
HA           3 control nodes, quorum = floor(n/2)+1, VIP via HAProxy + Keepalived, clustered DB + queue
Storage      Ceph RBD (Cinder, Glance, Nova), or NFS / arrays; install and TEST backups
Upgrades     every 6 months; SLURP (every other) can skip; rolling: DB sync, Placement, control, computes
Kolla upgr.  back up /etc/kolla -> pull -> prechecks -> upgrade   (no --limit; RabbitMQ one version at a time)
Networking   provider (VLAN) or self-service (overlay); separate management / storage / overlay nets
Security     the Security Guide checklists: Identity, Dashboard, Compute, Block Storage, Shared FS, Networking

← Automate Your OpenStack Cloud with Terraform: A Whole Network and Server from One File · ↑ All tutorials