Building your own EC2 on Debian

Introduction

I know the title is hyperbole, but I've been messing around with libvirt and KVM again after about 5 years of break. After my terrible experience years ago (generating SSL certificates isn't fun when you just want to mess around), I've now got it working to where I can spin up an ubuntu instance in about 5 minutes. And I learned something as well! So let's get started.

Prerequisites

Obviously, you'll need a KVM-enable system, as well as a working libvirt installation. Basically, install all the right packages for your distribution, start the processes, set up the network bridge, and you're off (theoretically). I used the debian wiki and the libvirt website for installation instructions. Note that this includes the client programs that we'll use, virsh and virt-install.

apt install libvirt-daemon bridge-utils libvirt-clients virtinst 
systemctl enable libvirtd
systemctl start libvirtd

Networking Config

Also be sure to add a bridge to your networking setup, as described by the libvirt link above. But in short, you just edit your /etc/network/interfaces file and create a new bridge. Then, do all the network configuration you did on eth0 before on the bridge (probably br0) now. My network configuration file now looks like this (minus loopback configuration, which doesn't change):

iface eth0 inet manual

iface br0 inet dhcp
    bridge_ports eth0

auto br0

While you can just use the libvirt default network, that uses NAT, which means you have to mess with port forwarding to connect to the VMs. While this can work using some sort of hook, I prefer my VMs just magically appearing in my home network. Anyways, reboot (maybe), and you should be all set.

Baby's first VM: Using the Installer

First, if you're on a remote machine, you can set $LIBVIRT_DEFAULT_URI to connect to your virtualisation host by default with all the libvirt tools. I use the ssh transport with a url like 'qemu+ssh//root@<hostname>/system', but libvirt supports other transports as well. If you've set this, you should be able to connect via virt-manager as well.

In fact, you can try this now using virt-install:

UBUNTU_LOCATION= http://archive.ubuntu.com/ubuntu/dists/eoan/main/installer-amd64/
virt-install -n test-instance-1 --ram 1024 --disk size=5G --os-type linux \
--os-variant ubuntu19.10 --location $UBUNTU_LOCATION

Running this should download some stuff, allocate some other stuff, and finally pop up a window with a graphical Ubuntu installer. This is fine, but we want to automate this.

If you like, you can go through the steps and install ubuntu on your new VM. But we could've done this with VirtualBox or whatever. We want to use the cloud images, so we don't have to install anything.

You should probably destroy your new instance via virsh destroy test-instance-1 && virsh undefine test-instance-1 –remove-all-storage to get rid of this VM.

Using Ubuntu Cloud Images

There is another type of image for Ubuntu, called the cloud image. This cloud image is a filesystem image that is bootable directly, and configures itself via cloud-init, without needing any installation.

First things first: Get a KVM cloud image from http://cloud-images.ubuntu.com/ (I used the Eoan Ermine KVM image) and upload it to the libvirt storage (called a "pool"). You should have a "default" pool in your installation. You can do this from virsh as follows:

URL=https://cloud-images.ubuntu.com/eoan/current/eoan-server-cloudimg-amd64-disk-kvm.img
wget -O eoan-server-amd64.img "$URL"
virsh vol-create-as default default eoan-server-amd64.img \
$(du -b eoan-server-amd64.img) --format qcow2
virsh vol-upload eoan-server-amd64.img eoan-server-amd64.img --pool default

This is now the base image for your future cloud installations. Using qcow2 magic, libvirt can spawn new VMs from this image without copying it multiple times, by just saving the differences to the base image. We are almost done now, except for the cloud-init configuration. cloud-init supports a stupendous amount of customisation, but for now, we just want to run a cloud image which allows us to log in via SSH. For this, create a directory somewhere, and create two files in this directory called user-data and meta-data. meta-data contains the instance metainformation, while user-data will take care of any specific device configuration that you want to apply. The contents should be as follows:

meta-data

instance-id: ubuntu-1
local-hostname: ubuntu-1

user-data

#cloud-config

users:
  - default
  - name: <username>
    ssh-authorized-keys:
      - <your ssh key>
    sudo: ALL=(ALL) NOPASSWD:ALL
    groups: sudo,users
    shell: /bin/bash

power_state:
  mode: reboot

Now, while in this directory, run python -m http.server 8000. This will serve these files via HTTP, which means we've basically created our own little metadata service. The reboot command in the configuration is for somewhat arcane reasons: Since we configure our device hostname over the network, cloud-init doesn't know the hostname until it has network connectivity. It will thus have the default hostname of "ubuntu" on first boot. After the metadata has been retrieved, the hostname will be set, but by this time, the DHCP server has already received the "old" hostname. So we reboot, and since metadat is stored, our hostname will be configured correctly after reboot.

Starting the Cloud Image

So, after all this configuration, we're finally able to start our shiny cloud image. Remember to have our "metadata service" running and serving the cloud-init files. Now, run the monster command below to create a new instance with 5 gigabytes of disk space and 2 gigabyte of RAM. Networking will be handled via the bridge you set up above called br0.

virt-install -n instance-1 --ram 2048 --os-type linux --os-variant ubuntu19.10 \
--disk size=5,pool=fast,backing_store=<imagepath>/eoan-server-amd64.img --import \
--network bridge=br0 --virt-type kvm --graphics none \
--sysinfo 'system.serial=ds=nocloud-net;s=http://<yourip>:8000/'

Two variables here: <imagepath> is the location where your uploaded image ended up on the server, while <yourip> is the IP your little metadata store is listening.

You can find <imagepath> via virssh vol-dumpxml eoan-server-amd64.img –pool default. As far as I know, there is no easier way to specify the path of the image we want to clone. Virsh should support this, but I ran into permission errors when trying to create a volume with a backing volume manually via vol-create-as or vol-clone.

The "–sysinfo" stanza provides the location of the metadata service to cloud-init, which will proceed to fetch the files, read the configuration, and finally apply it.

Wait for the reboot, and you should be able to SSH into your new instance under it's hostname. Enjoy!

Automation

Well, not quite as convenient as EC2, but you can see how these components could easily be implemented into a singlse shell command to create the metadata file, serve it, and run virt-install to create the VM. cloud-init has lots of modules, and the next step would probably involve some configuration management software like Chef or Puppet.

Date: 2020-03-13 Fri 00:00

Author: Jan Seeger

Email: sitecomments@thenybble.de

Validate