Cyberus Linux Image-based Systems

This document guides you through the process of creating an image-based system. After an introduction to what image-based systems are, we will build a simple image from scratch. Then, we will show how to refactor an existing NixOS configuration. Finally, we show how to update an image-based system.

We assume that you are familiar with configuring NixOS-based systems and have a basic understanding of disk partitioning.

Background

We call systems that are booting from a read-only /nix/store partition an image-based system. These systems have a couple of advantages:

• They offer a high degree of boot security via Secure Boot and a chain of trust all the way to the root filesystem using TPM2-backed disk encryption.
• They are reproducibly-built from a NixOS configuration, so you are sure what code is actually running.
• They naturally fit the A/B update model and allow for automatic rollbacks and, if desired, factory resets.
• The running system does not need the Nix tooling to run or update itself, allowing for absolute minimization and hardening.

We currently focus on UEFI-based 64-bit x86 systems with TPM2 support. For ARM AArch64 and RISC-V systems, please reach out. The system we are building requires an UEFI-capable system.

An Image From Scratch

So let's build an image-based system from scratch. We will start with reviewing where we want to go.

What Are We Building?

The following table shows the partition layout of the system we are building. This layout was chosen, because it supports our goals outlined above.

Partition	Description
UEFI System Partition	Contains the boot loader and Linux kernels for each installed system version
Update Slot A	An integrity-protected read-only partition containing the /nix/store of the first installed version
Update Slot B	Initially empty, later the integrity-protected read-only /nix/store of the second installed version
Swap (optional)	The swap partition, encrypted with an ephemeral key.
User Data	The persistent root filesystem of the system.
Under the hood, the update slots consist of two partitions. One for the actual read-only /nix/store partition and another for the dm-verity integrity protection data.

Flake Setup

First, let's create a flake.nix file to define your project and pull in the required dependencies. We're also adding the boilerplate for a NixOS configuration. This NixOS configuration pulls in the image module from the Cyberus Linux Modules repository.

{
  description = "A Minimal Image-Based System";

  inputs = {
    # Pull in the latest version of Cyberus Linux.
    nixpkgs.url = "https://channels.cyberus-linux.com/channel/cyberus-linux-26.05.tar.xz";

    # Pull in Cyberus Linux modules and use the same Cyberus Linux version as above.
    cyberus-linux-modules.url = "github:cyberus-linux/cyberus-linux-modules";
    cyberus-linux-modules.inputs.nixpkgs.follows = "nixpkgs";
  };

  outputs = { self, ... }@inputs: {
    nixosConfigurations = {
      demo-image = inputs.nixpkgs.lib.nixosSystem {
        system = "x86_64-linux";
        modules = [
          # Import the Cyberus Linux module.
          inputs.cyberus-linux-modules.nixosModules.image

          # Here we will add our system configuration. This is a normal
          # NixOS configuration.
          ./configuration.nix

          # Here we configure the image itself.
          ./image.nix
        ];
      };
    };

    # A shortcut to the image build output.
    packages.x86_64-linux.default = self.nixosConfigurations.demo-image.config.system.build.image;
  };
}

For simplicity, we've added a shortcut to the attribute that builds the image as a package. This saves us from typing the long attribute path every time we build the image.

We'll look at the two additional configuration files in the next sections.

Adding a Minimal System Configuration

Now we create the NixOS configuration for our minimal system. We configure a demo user with sudo rights and auto-login enabled and leave it at that. But you are free to customize it.

{ pkgs, ... }: {

  # Ensure the demo system boots everywhere.
  hardware.enableAllHardware = true;

  # Add a simple demo with auto-login and sudo access.
  users.users.demo = {
    isNormalUser = true;
    extraGroups = [ "wheel" ];
  };

  security.sudo = {
    enable = true;
    wheelNeedsPassword = false;
  };

  users.allowNoPasswordLogin = true;
  services.getty.autologinUser = "demo";

  # Optional: Add a partitional tool to see the final partition layout.
  environment.systemPackages = with pkgs; [
    parted
  ];

  system.stateVersion = "26.05";
}

Note that we did not set up any hardware configuration. Instead we enabled hardware.enableAllHardware to make our image compatible with a wide variety of hardware. This is a good starting point, especially for using the image on a USB thumb drive or virtual machine. Later, once everything works, you can minimize your configuration by replacing enableAllHardware with the output of nixos-generate-config.

Configuring the Image Parameters

The image module comes with reasonable defaults, so there is no need to configure a lot of details. You can simply enable it and let it do its job.

The one setting we have to configure is the size of the disk partition that holds the compressed Nix store. This is the setting that influences the size of update slots and ensures that future updates up to this size can be applied without running out of space. It's good to be generous here, because this setting will be hard to change once systems are in the field.

To reduce the size of the image, we enable the image-based-appliance profile. It shrinks the image size by removing unnecessary packages and configurations, including the Nix tooling itself.

{ modulesPath, ... }: {
  # Optional: Reduces the image size by about 30%.
  imports = [
    "${modulesPath}/profiles/image-based-appliance.nix"
  ];

  # Enable the module that builds the image.
  cyberus-linux.image = {
    enable = true;

    # Set a maximum Nix store size. This is the size that will be reserved for
    # future updates as well.
    #
    # The Nix store is compressed, so this value should be plenty for embedded
    # systems.
    nixStore.sizeMiB = 8192;
  };
}

Important

Enabling the image module is mutually exclusive with enabling other boot loaders. For advice on how to refactor your existing NixOS configuration to use the image module, see below.

Running the Image

To build the image, run the following command in the flake's directory to build the default package:

$ nix build

Your initial build will take some minutes. This is a great time to fill up that caffeinated drink.

Once the build completes, you'll find the image in result/image_0.0.0.raw. The image name contains the version number. Because we didn't configure it via cyberus-linux.image.version, it stayed at the default. We support version numbers that follow the UAPI Version Format Specification.

The image can be copied to a USB drive or other storage device using dd or a similar tool:

$ sudo dd if=result/image_0.0.0.raw of=/dev/sdX bs=4M status=progress oflag=sync

You will notice that the image is pretty large, but most of it are zeroes. So if you intend to copy it over the network, consider compressing it first.

Warning

Make sure to replace /dev/sdX with the device name of your USB drive and double check the device name. Writing the image to a currently mounted device will lead to data loss.

For a permanent deployment, copy the image to the hard drive of the target system. This is out-of-scope for this guide.

Testing Inside a VM

If you do not have a physical system to test at hand, the generated image also boots in a virtual machine that supports UEFI. The image assumes that it is written to a larger storage device, so you must make additional space available.

To do this, create a writable copy of the raw disk image and extend its size. Here we do this with the venerable truncate command that is universally available:

$ cp result/image_0.0.0.raw vm-image.raw
$ chmod u+w vm-image.raw
$ truncate -s +16G vm-image.raw

The resulting vm-image.raw file is then usable in any virtualization solution that supports UEFI.

Exploring the System

When you boot our minimal image, you will be automatically logged in as demo user. You can use this account to explore the system and test your configuration.

For example, to see the partition layout, execute fdisk and print the partition table:

$ fdisk /dev/sda # or /dev/nvme0n1, if you have an NVMe disk

Refactoring an Existing NixOS Configuration

Refactoring an existing NixOS configuration is straightforward. As every NixOS configuration is different, we'll give you the high-level steps to refactor your configuration.

The main goal is to remove every configuration that conflicts with the image-based setup. There are two categories that need to be addressed. We'll cover both below.

Info

Be sure to look at your whole NixOS configuration. Boot loader and file system configurations may be configured in configuration.nix or hardware-configuration.nix depending on your setup.

Hint

Instead of removing conflicting configurations, you can also move it to a separate module mutable-nixos.nix. If you add the configuration of the image module as image-nixos.nix, you can build your NixOS configuration in either boot flavor. This is useful to spin up a mutable version of your system for testing or development.

Boot Loader Configuration

The image module replaces the boot loader configuration. As such, you need to remove any boot loader configuration from your existing NixOS configuration.

Because NixOS collects boot loader configuration neatly under the boot.loader options, you can easily identify them. It's sufficient to remove the boot.loader.YOUR-BOOT-LOADER.enable = true; options.

The most common options to look for are boot.loader.grub.enable and boot.loader.systemd-boot.enable.

File System Configuration

The image module also comes with its own file system configuration. While you can customize it, specifying file systems for the root and boot and /nix/store partitions will conflict with the image module's configuration.

In general, the image modules "owns" the block device it boots from. Your configuration must not include additional partitions on the boot device.

For a normal system without the need to mount network shares or RAID volumes for large data, you can simply remove the fileSystems option from your NixOS configuration. If you do have such a need, you can still specify network shares and RAID volumes in your NixOS configuration, but there is no need to specify any file systems for the root and boot partitions.

Typically, you can just remove the swapDevices option from your NixOS configuration. By default, the image module configures a small swap partition. This swap partition can be disabled or configured using the cyberus-linux.image.swap.enable and cyberus-linux.image.swap.sizeMiB options. Additional swap on other block devices can still be configured but is usually not needed.

Updating an Image-Based System

To update the image-based system, we first need to build a new version of the system and then place the resulting update on the system in the right location.

Building an Update

Info

The updating mechanism is currently still very basic. We are working on improving it!

After changing the NixOS configuration, you must give the system a new version number by incrementing cyberus-linux.image.version. Otherwise, the update tooling will not recognize it as a newer version that needs to be installed.

By default, the version number is set to 0.0.0. Let's add the following to our NixOS configuration to mark a new version:

cyberus-linux.image.version = "0.0.1";

We can still build a complete image using nix build, but this image cannot be used to update the already deployed system. Instead, we build and update bundle:

$ nix build .#nixosConfigurations.demo-image.config.system.build.imageUpdateBundle

This results in a few files that together form the update bundle:

ls -hl result/
total 1.7G
-r--r--r-- 3 root root  45M Jan  1  1970 kernel_0.0.0.efi
-r--r--r-- 2 root root 600M Jan  1  1970 store_data_9480f32ba76842f8baab5bd8a4e8d91f_0.0.0.zstd
-r--r--r-- 2 root root  13M Jan  1  1970 store_verity_4649363a4a668aed95097a2764a92035_0.0.0.zstd

Place these files in /var/updates on the system to be updated. If you do this for the first time, create /var/updates first.

Hint

sudo mkdir -p /var/updates && sudo chmod 755 /var/updates is a good starting point. Be sure to make the directory writable only to trusted users, such as root.

After placing the files in /var/updates, the system will automatically detect and apply the update. If a newer version was applied, the system will reboot automatically during the night.

To manually trigger an update, use the updatectl command on the target system:

$ sudo updatectl update

The system will boot into the updated version on the next reboot.

Info

Due to a known issue updatectl may say "Nothing to be done" even when it just applied an update. This issue is mostly cosmetic but means that updatectl update --reboot will not work as intended. Manually rebooting the system after updatectl update is a workaround.