I Don't Know Rust But I Ran Oxide's Control Plane Anyway

I am a self-taught cloud engineer who lives in AWS. Lambda, DynamoDB, CloudFormation, Terraform. That has been my world so far. But I am always curious about what else is out there, and recently I came across Omicron, the open source control plane for the Oxide rack. I did what I always do when I want to understand something: I tried to run it. I do not have an Oxide rack and I do not have Helios, their operating system, but Omicron ships with a simulated mode that runs on Linux. That was enough.

What is Oxide?

For decades enterprises have had two options for running compute and both required compromise.

Traditional on-prem gives you ownership but at a cost. You are juggling eight or more vendors across compute, networking, storage, and orchestration. The integrations are brittle, the operations are complex, and when something breaks everyone points fingers at someone else.

The public cloud gives you speed and integration but at a different cost. You never own anything. You rent forever. The bills are unpredictable, egress fees are punishing, and you are locked into proprietary APIs that make it harder to leave every year.

Oxide is trying to solve both problems at once. They build a server rack where the hardware, the operating system, and the software are all designed together by the same team. The pitch is that you get the economics and governance of on-prem infrastructure with the developer experience of the public cloud. One purchase, no subscriptions, no surprise bills, setup in an afternoon.

The brain of that rack is a piece of software called Omicron.

What is Omicron?

Omicron is the control plane for the Oxide rack. Every API call you make to create an instance, a VPC, or a disk goes through Omicron. It is not one service. It is several components that work together to manage the full lifecycle of resources on the rack. If you live in AWS like I do, most of these have a familiar equivalent.

Component	What it does	AWS equivalent
Nexus	Main orchestrator — receives every external API request and coordinates the rest of the control plane	AWS control plane
Sled Agent	Runs on each physical server — executes what Nexus decides locally	Hypervisor layer on each EC2 host
CockroachDB	Stores all state durably — every project, VPC, and instance you create	RDS (for internal bookkeeping)
ClickHouse	Handles telemetry via Oximeter — columnar DB for time series metrics	CloudWatch
DNS servers	Internal service discovery and external DNS for workload reachability	Route 53 private hosted zones

On a real Oxide rack all of these run on Oxide hardware under Helios, their illumos based OS. But Omicron ships with a simulated mode that runs on Linux. That is the door I walked through.

Why EC2 and Not My Laptop

I already knew my laptop was not going to cut it. I have 16GB of RAM and I had already tried running a local LLM on it and watched it consume 98% of my memory before giving up. A massive Rust workspace with dozens of crates was not going to go any better. So I moved to EC2 before even trying.

I launched an m5.xlarge on Ubuntu 22.04 with a 100GB EBS volume. The m5.xlarge has 4 vCPUs and 16GB of RAM which gives the Rust compiler enough room to parallelize the build without running out of memory. I picked a fixed instance type instead of a burstable t3 because compilation is sustained CPU work. Burstable instances throttle when you exhaust your CPU credits and a throttled build on a massive workspace is a bad time.

Getting It Running

After cloning the repo I ran two commands to set up the environment and pull down prerequisites like CockroachDB and ClickHouse binaries:

source env.sh
./tools/install_builder_prerequisites.sh

Then I built the entire workspace:

cargo build

I expected this to take around 90 minutes based on what I had read about large Rust workspaces. It finished in under 30. Once the build was done I started the full stack with a single command:

cargo xtask omicron-dev run-all

That one command started CockroachDB, ClickHouse, Nexus, a simulated Sled Agent, internal DNS, external DNS, and the management gateway. All of it came up together and printed out every endpoint I needed to start interacting with the system.

Talking to the Control Plane

With the stack running I authenticated against Nexus using a username and password and started making API calls with curl. No special tooling. Just HTTP.

I created a project first:

curl -b /tmp/cookies.txt -X POST http://127.0.0.1:12220/v1/projects \
  -H "Content-Type: application/json" \
  -d '{"name": "my-first-project", "description": "testing omicron"}'

Nexus returned a UUID, a creation timestamp, and a modification timestamp. The project was now stored in CockroachDB.

Then I created a VPC inside that project:

curl -b /tmp/cookies.txt -X POST \
  "http://127.0.0.1:12220/v1/vpcs?project=my-first-project" \
  -H "Content-Type: application/json" \
  -d '{"name": "my-first-vpc", "description": "my first vpc", "dns_name": "my-first-vpc"}'

This is where something clicked for me. I did not ask for any subnets. I just created a VPC. But when I immediately queried the subnets for that VPC there was already a default subnet sitting there:

{
  "name": "default",
  "description": "The default subnet for my-first-vpc",
  "ipv4_block": "172.30.0.0/22",
  "ipv6_block": "fd74:c44d:2834::/64"
}

Oxide created it automatically. It assigned a proper IPv4 block and carved a /64 out of the VPC IPv6 prefix without me asking. On AWS I would have created the VPC, then created the subnet, chosen the CIDR block, associated it with a route table, and thought about availability zones. Here the control plane made a sensible decision and handled it.

That pattern kept showing up. Oxide built the complexity in so you do not have to bolt it on yourself.

Creating an Instance

Before I could create an instance I had to set up an IP pool. In simulated mode there is no pre-configured networking because there is no real rack underneath. On a real Oxide rack an operator configures IP pools during initial setup so the system knows what addresses it is allowed to hand out. In simulated mode I had to do that myself.

I created a pool, added an address range to it, and linked it to my silo:

curl -b /tmp/cookies.txt -X POST \
  "http://127.0.0.1:12220/v1/system/ip-pools" \
  -H "Content-Type: application/json" \
  -d '{"name": "default", "description": "default ip pool"}'

curl -b /tmp/cookies.txt -X POST \
  "http://127.0.0.1:12220/v1/system/ip-pools/default/ranges/add" \
  -H "Content-Type: application/json" \
  -d '{"first": "172.20.0.1", "last": "172.20.0.254"}'

curl -b /tmp/cookies.txt -X POST \
  "http://127.0.0.1:12220/v1/system/ip-pools/default/silos" \
  -H "Content-Type: application/json" \
  -d '{"silo": "test-suite-silo", "is_default": true}'

Then I created the instance:

curl -b /tmp/cookies.txt -X POST \
  "http://127.0.0.1:12220/v1/instances?project=my-first-project" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "my-first-instance",
    "description": "test instance",
    "hostname": "my-first-instance",
    "ncpus": 1,
    "memory": 1073741824,
    "network_interfaces": {"type": "default_dual_stack"}
  }'

Nexus returned the instance with a UUID, the project it belonged to, the CPU and memory allocation, and a run state of stopped. It existed in the control plane and was ready to start.

What This Actually Taught Me

Running Omicron in simulated mode means the Sled Agent is not managing real hardware. But the control plane logic in Nexus is the same code that runs in production. The state management, the API behavior, the way components register with each other and coordinate — all of it is real.

What I came away understanding is what a control plane actually does at a deeper level. It is not just an API wrapper around some servers. It is the thing responsible for maintaining desired state across an entire system, making intelligent decisions about resources, storing everything durably so it survives failures, and surfacing a coherent interface so the operator does not have to think about the complexity underneath.

That last part is what Oxide is really selling. They describe their support philosophy as transparent and open systems with rich telemetry so support issues do not disappear into a void with vendors pointing fingers at each other. Having now seen how the components are structured and how observable the system is, I believe it.

I ran all of this with curl commands and an EC2 instance that cost me less than ten dollars. The code is on GitHub. Go run it yourself.

Resources

• Oxide Computer
• Omicron on GitHub
• Oxide API Documentation

---

Want to see my other projects? Check out my portfolio or connect with me on LinkedIn.