Signal9Signal9 — home
online
all lessons

Linux+ Cloud and Virtualization: KVM/libvirt, cloud-init, and Hybrid Operation

Intermediate30 min7 steps
What you'll learn
  • Describe the KVM + libvirt stack and manage VMs with virsh and virt-install
  • Decide when a workload wants a VM (isolation) versus a container (density)
  • Explain how cloud-init provisions an instance on first boot, and re-test it with cloud-init clean
  • Pull instance identity and secrets from platform metadata at 169.254.169.254 instead of baking them into the image
  • Apply the cattle-not-pets mindset to run the same automation on-prem and in the cloud
  1. Why a server should be disposable

    Frame the whole lesson around the cattle-not-pets mindset before any tooling.

    For decades, a server was a pet. You named it apollo, you logged in over months to hand-tune its config, and when it got sick you nursed it back to health because nobody remembered exactly how it was built. That works until apollo dies at 3 a.m. and you have no way to rebuild it.

    The cloud-era model is cattle. Each instance is one of an interchangeable herd: it's described entirely by an image plus a definition file (infrastructure-as-code), so if one misbehaves you don't nurse it — you destroy it and stamp out an identical replacement from the same recipe.

    This lesson is the toolset that makes cattle possible on Linux: KVM/libvirt to run virtual machines, cloud-init to provision them automatically on first boot, and instance metadata to feed each one its identity and secrets at runtime.

  2. The Linux virtualization stack: KVM + libvirt

    Name the layers of the stack and what each one is responsible for.

    Linux virtualization is a layered stack, and the exam expects you to know which layer does what:

    • KVM (Kernel-based Virtual Machine) is a module inside the Linux kernel that turns the kernel itself into a Type-1 (bare-metal) hypervisor — guest VMs run directly on the hardware's CPU virtualization extensions, not on top of a host OS.
    • QEMU emulates the virtual hardware each guest sees (disk, network card, display).
    • libvirt is the management layer — a stable API and daemon (libvirtd) that sits in front of KVM/QEMU so you have one consistent toolset regardless of the hypervisor underneath.
    • `virsh` is the command-line client you type at, and `virt-install` is the tool that builds a brand-new VM.

    Think of it like an airport: KVM is the runway and control tower built into the ground itself, QEMU is the individual aircraft, and libvirt is the booking-and-dispatch system you actually interact with — you never talk to the runway directly.

    Check yourself

    Which layer of the stack gives you one consistent management API across hypervisors — KVM, QEMU, or libvirt?

  3. Driving VMs with virsh

    Inspect and control the VM lifecycle with the everyday libvirt verbs.

    virsh is to VMs what systemctl is to services: a single command with a verb set you learn once. Start by looking before you touch — list what exists and its state.

    Example — not runnable on your lab box

    virsh list --all
     Id   Name        State
    ----------------------------
     3    web01       running
     -    db01        shut off
    list
    Show domains (libvirt's word for a VM/guest).
    --all
    Include stopped VMs too. Without it, only running domains appear — so a shut-off VM looks like it doesn't exist.
    Two VMs are defined. web01 is running and has a live Id (3); db01 is shut off and shows - for Id because a stopped domain has no runtime id. The classic beginner trap is running plain virsh list, seeing only web01, and concluding db01 was never created.

    Example — not runnable on your lab box

    virsh start db01
    Domain 'db01' started
    start
    Boot a defined-but-stopped domain. Power-on equivalent.
    shutdown
    Ask the guest OS to power off gracefully (sends ACPI signal — needs a cooperating guest).
    destroy
    Force-stop immediately, like yanking the power cord. Does NOT delete the VM despite the scary name.
    start powers on the guest. The mirror command is virsh shutdown db01, which asks the guest to halt cleanly — prefer it over destroy, which is a hard power-cut you reach for only when the guest is wedged. Re-run virsh list --all and db01 flips to running.

    Check yourself

    You ran virsh list and your VM is missing, but you're sure you created it. What flag shows stopped VMs too?

  4. Building a VM with virt-install — and when NOT to

    Provision a new VM, and decide VM-vs-container by isolation versus density.

    virsh manages VMs that already exist; `virt-install` stamps out new ones. It defines the virtual hardware and points the guest at an OS source so it can boot and install.

    Example — not runnable on your lab box

    virt-install --name web02 --memory 2048 --vcpus 2 --disk size=20 --os-variant ubuntu22.04 --cloud-init
    Starting install...
    Domain creation completed.
    --name
    The domain name you'll later pass to virsh.
    --memory
    RAM for the guest, in MB (2048 = 2 GB).
    --vcpus
    Number of virtual CPUs the guest sees.
    --disk size=20
    Create a 20 GB virtual disk for the guest's storage.
    --os-variant
    Tells libvirt which OS this is so it picks optimal virtual hardware/drivers.
    --cloud-init
    Attach cloud-init data so the VM provisions itself on first boot (next step).
    This single command defines a 2-vCPU, 2 GB, 20 GB VM and registers it with libvirt — afterward virsh list --all shows web02. Crucially, every line here is declarative: the same flags rebuild an identical VM on any KVM host. That's cattle, expressed as a command.

    But should this even be a VM? A VM carries its own full kernel and OS, which is heavy. A container shares the host's kernel and ships only the app and its libraries, which is light. The choice is a single trade-off:

    • VM → isolation. Its own kernel means a compromised or crashing guest can't easily reach the host or its neighbours. Pick a VM for hostile multi-tenant workloads, for a different OS/kernel than the host, or when strong security boundaries are required.
    • Container → density. Sharing the kernel means near-zero overhead, so you pack far more workloads per host and they start in milliseconds. Pick a container for many instances of your own stateless services.

    Check yourself

    You need to run 200 instances of your own stateless web service as cheaply as possible. VM or container?

  5. cloud-init: provisioning on first boot

    Explain what cloud-init does, when it runs, and how to re-test it.

    A freshly stamped image is generic and identical to every other copy. cloud-init is what turns that generic image into this specific instance — and it does it automatically on the instance's very first boot.

    On that first boot cloud-init reads user-data (a config you supply, usually YAML) and applies it: it sets the hostname, creates users, installs their SSH keys, installs packages, writes files, and runs arbitrary scripts. Think of it as the instance's new-hire onboarding: the laptop comes off the shelf identical for everyone, and onboarding personalizes it — accounts, access badges, software — on day one.

    Example — not runnable on your lab box

    cat user-data.yaml
    #cloud-config
    hostname: web02
    users:
      - name: deploy
        sudo: ALL=(ALL) NOPASSWD:ALL
        ssh_authorized_keys:
          - ssh-ed25519 AAAAC3Nz... deploy@laptop
    packages:
      - nginx
    runcmd:
      - systemctl enable --now nginx
    The #cloud-config first line is required — it's the marker cloud-init looks for. This file sets the hostname, creates a passwordless-sudo deploy user with an SSH key (no password ever set), installs nginx, and enables it. Notice there's no secret in here that the image itself stores — the SSH public key is safe to ship; the matching private key never leaves the operator's laptop.

    Example — not runnable on your lab box

    sudo cloud-init clean --logs
    clean
    Wipe cloud-init's state markers so it treats the NEXT boot as a first boot and re-runs provisioning.
    --logs
    Also remove cloud-init's logs, for a fully clean slate when testing.
    Silent success. After clean, the next reboot makes cloud-init run its full first-boot sequence again against your current user-data — the supported way to iterate on a provisioning config without rebuilding the image from scratch.

    Check yourself

    You changed your cloud-init user-data and rebooted, but the instance is unchanged. Which command makes it re-run provisioning on the next boot?

  6. Where identity and secrets come from: instance metadata

    Pull config and credentials from the platform at runtime instead of baking them in.

    How does cloud-init know it should be web02 and not web57? It asks the platform, not the image. Every cloud (and many on-prem setups) exposes an instance metadata service at a fixed link-local address: `169.254.169.254`. Link-local means it's only reachable from the instance itself and never routes off the box — so it's a private channel between the running instance and the platform.

    The instance queries that endpoint at boot to learn who it is — its hostname, its user-data, and short-lived credentials the platform mints for it. The principle: identity and secrets come FROM the platform AT runtime; they are never baked into the image.

    curl -s http://169.254.169.254/latest/meta-data/
    hostname
    instance-id
    local-ipv4
    public-ipv4
    iam/
    -s
    Silent: suppress curl's progress meter so only the response body prints.
    The metadata endpoint returns a directory-like listing of what this instance can ask about itself. Drilling in — e.g. curl -s http://169.254.169.254/latest/meta-data/hostname — returns the value for that one key. The iam/ branch is where the platform hands out temporary, auto-rotating credentials, so the instance gets access without any long-lived key stored on disk.

    Check yourself

    At what well-known link-local IP does an instance reach its cloud metadata service?

  7. Try it in your live terminal

    Detect your own environment with safe, read-only probes on the real sandbox.

    You have a real root shell. These are read-only probes — they only report what's present, they change nothing. Run them in order to see which parts of the stack this box has. It's fine if a tool isn't installed; the || echo fallback just tells you so instead of erroring.

    systemd-detect-virt 2>/dev/null
    Reports whether you're inside a VM or container and which technology (e.g. kvm, lxc, docker) — or none on bare metal. This is the fastest way to answer "am I virtualized, and how?".
    virsh --version 2>/dev/null || echo 'no libvirt'
    If libvirt is installed you get a version number, confirming this box can host KVM guests; otherwise the fallback prints no libvirt. The 2>/dev/null hides the error message so the fallback reads cleanly.
    cloud-init --version 2>/dev/null || echo 'no cloud-init'
    Prints the cloud-init version if this image is cloud-provisionable, else no cloud-init. On a real cloud instance you'd also run cloud-init status to see whether first-boot provisioning finished.
Recap
  • KVM is the in-kernel Type-1 hypervisor; libvirt is the management layer you drive with virsh (list/start/shutdown) and virt-install.
  • VM = isolation (its own kernel), container = density (shared kernel) — choose on which boundary the workload needs.
  • cloud-init provisions an instance from user-data on first boot only; reset with cloud-init clean to re-test.
  • Identity and secrets come from the platform at runtime via metadata at 169.254.169.254 — never baked into the image.
  • Treat instances as reproducible cattle (image + IaC), so the same automation runs on-prem and in the cloud.
VM vs container — which boundary do you need?
DimensionVirtual machine (KVM)Container
KernelIts own, full guest kernelShares the host kernel
IsolationStrong — hardware-enforced boundaryWeaker — process-level on shared kernel
Density / overheadHeavier; fewer per host; boots in secondsLight; many per host; starts in ms
Manage withvirsh, virt-install (libvirt)container runtime (see containers lesson)
Pick whenHostile tenants, different OS, hard isolationMany copies of your own stateless service