Getting started with Cloud Hypervisor

Note: This post is still a work in progress. Figured I would share it anyway.

VMs are really cool, but they can be a bit of a pain to setup properly. While I was learning about them I had a mess of shell commands and docs that I was referring to, so I thought I’d write a quick guide in case others found it useful or I ever need to refer to it again.

In this post we will be using Cloud Hypervisor to run a VM with a HTTP server.

Prerequisites

A Linux machine with KVM support

This guide assumes you have a Linux machine with KVM support. If you don’t, you can get one on AWS or GCP. I am using a n1-standard-2 instance on GCP.

If you have an apple silicon machine with an M3 or later chip, you can use nested virtualization inside a Lima VM. Earlier chips don’t support this (I found this out by wasting 4 hours trying to get it to work).

To check the status of KVM, you can run

kvm-ok
# Should return something like:
# INFO: /dev/kvm exists
# KVM acceleration can be used

Some dependencies

Many apt packages that are necessary.

sudo apt-get install -y \
  build-essential mtools cloud-image-utils libssl-dev whois

Installation

We need to install the cloud-hypervisor binary on the machine

wget https://github.com/cloud-hypervisor/cloud-hypervisor/releases/latest/download/cloud-hypervisor
chmod +x cloud-hypervisor
sudo setcap cap_net_admin+ep ./cloud-hypervisor
sudo mv cloud-hypervisor /usr/local/bin/

Test with

cloud-hypervisor --version

Cloud init

We need to create a cloud-init ISO with a default user and password for the Ubuntu image. It also setups up some basic networking.

Clone the cloud-hypervisor repo and cd into it.

git clone https://github.com/cloud-hypervisor/cloud-hypervisor.git
cd cloud-hypervisor

Run the script to create the cloud init image ./scripts/create-cloud-init.sh

./scripts/create-cloud-init.sh

You now have a /tmp/ubuntu-cloudinit.img file.

Note: You can change the Ubuntu password by modifying the test_data/cloud-init/ubuntu/local/user-data file. The password is generated with mkpasswd --method=SHA-512 --rounds=4096 "your_desired_password"

Kernel

We need a kernel to use for the VM. We can build one from source. Or we can use a pre-built one.

To avoid building from source, we can use grab the built kernel from the cloud-hypervisor linux repo. This is using the v6.2 release.

wget https://github.com/cloud-hypervisor/linux/releases/download/ch-release-v6.2-20240908/vmlinux

Note: I don’t love this way of getting a kernel. I’d like to find a better way that has more options and doesn’t require building from source.

Disk image

Get a pre-built Ubuntu image and convert it to a raw disk image. This is using a Ubuntu 24.04 image.

wget https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
qemu-img convert -p -f qcow2 -O raw noble-server-cloudimg-amd64.img noble-server-cloudimg-amd64.raw

You can get a specific ubuntu image from the ubuntu cloud images page.

Boot the VM

cloud-hypervisor \
	--kernel vmlinux \
	--disk path=noble-server-cloudimg-amd64.raw path=/tmp/ubuntu-cloudinit.img \
	--cmdline "console=hvc0 root=/dev/vda1 rw" \
	--cpus boot=4 \
	--memory size=1024M \
	--net "tap=,mac=,ip=,mask="

You should see the VM boot up and be prompted for the Ubuntu username and password. With the default cloud-init image, the username is cloud and the password is cloud123.

You should be now have a shell into a fresh Ubuntu 24.04 instance.

uname -a

You can shutdown the VM with

# From inside the VM
sudo shutdown -h now

# From the host
pkill cloud-hypervisor

Networking

Section still in progress. It doesn’t work yet.

cloud-hypervisor automatically creates a tap interface for the VM if --net is specified.

Now to configure the VM to use the tap interface, you can modify the cloud-init

vim cloud-hypervisor/test_data/cloud-init/ubuntu/local/network-config

and add

version: 2
ethernets:
    ens4:
        addresses:
            - 192.168.100.2/24
        routes:
            - to: 0.0.0.0/0
              via: 192.168.100.1
        nameservers:
            addresses:
                - 8.8.8.8
                - 8.8.4.4
        dhcp4: false

Note: You might need to change the ens4 to the correct interface name for your machine, such as eth0.

Re-generate the cloud-init ISO with

pushd cloud-hypervisor && ./scripts/create-cloud-init.sh && popd

Now lets re-boot the VM with the updated network config.

cloud-hypervisor \
    --kernel vmlinux \
    --disk path=noble-server-cloudimg-amd64.raw path=/tmp/ubuntu-cloudinit.img \
    --cmdline "console=hvc0 root=/dev/vda1 rw" \
    --cpus boot=4 \
    --memory size=1024M \
    --net "tap=tap0,mac=,ip=192.168.100.2,mask=255.255.255.0"