A Couple Small Nix Tips

Here’s a bunch of small nix tips, for my own future reference. Everything assumes you’ve enabled nix-command and flakes experimental features.

Remote builders

Nix has a neat feature where you can add ‘remote builders’ which it can send build tasks to during the build process. This is especially useful if you’re e.g. building a Linux package from macOS. There’s two ways to do this:

nixbuild

nixbuild is a service that providers remote Linux builders as a service—you even get a quite generous 25 CPU hours per month. Setup is not too complicated.

Using a virtual machine

You can run a ‘remote’ builder VM on your computer that runs Linux. The coolest implementation I’ve seen of this is using Rosetta 2 to execute x86_64 binaries in the VM so you get two architectures (aarch64 and x86_64) for the price of one.

A way of running a Linux VM through QEMU as a builder was recently merged into Nixpkgs, this doesn’t provide the Rosetta 2 thing though because QEMU doesn’t support that.

It would be nice if someone made a nicely packaged app that would run in the background accepting SSH connections, quickly booting up a Linux VM when needed and spinning it down when idle after a while.

This could even use directory shares to share the Nix store and avoid copying the files over, though that might require some Nix changes to deal with file locking.

Deploying to a NixOS machine from macOS

There’s a bunch of different tools for deploying NixOS machines but it turns out that nixos-rebuild supports remote deployment directly.

nix run nixpkgs#nixos-rebuild -- --fast --target-host user@remote-host --build-host user@remote-host --flake .#machine --use-remote-sudo switch

Let’s go to this command step by step:

nix run nixpkgs#nixos-rebuild run the nixos-rebuild command from nixpkgs. You could also install the nixos-rebuild package into your environment.
--fast this avoids re-executing nixos-rebuild from the resulting system. This doesn’t work from macOS because it’ll try to execute a Linux bash binary on Mac…
--target-host user@remote-host --build-host user@remote-host the most important thing, the target to deploy to and where to build the system, as an SSH target. If you omit the build host it’ll build it locally, which can work if you’ve set up a Linux remote builder.
--flake .#machine which flake to configure the system with. Note that this is evaluated locally—so you don’t need to copy any nix files to the target host!
--use-remote-sudo use sudo on the remote machine to execute the commands.

I figured out these things from reading the source (always read the source!) which is a pretty straightforward bash script.

Flake pinning

What does Nix evaluate when you do nix run nixpkgs#hello? The first part of nixpkgs#hello specifies which flake to evaluate. This can be a path like ./directory, a URL or an identifier that’s in the flake ‘registry’. You can view the flake registry with: nix registry list. The flake registry is actually tiered:

global are defined at flake-registry and are the same for everyone
system is defined at /etc/nix/registry.json and shared for all the users.
user is at ~/.config/nix/registry.json.

The registry JSON format is defined here.

You can run nix flake registry pin nixpkgs to add something to the user registry.

The system registry is best managed using your nixos or nix-darwin configuration. Something cool you can do is create the following nixos module:

{ nixpkgs }: { nix.registry.nixpkgs.flake = nixpkgs; }

This pins nixpkgs to whatever nixpkgs is when you install your nixos configuration. This makes it much faster to do nix run nixpkgs#hello because the nixpkgs repository is already available and some dependencies might already be installed.

Install binaries globally for nix-darwin

If you use nix-darwin you can add things to environment.systemPackages which then makes them available at /run/current-system/sw/bin. It can be hard to configure all of your tools to take this path into consideration however, so I have the following in my nix-darwin config:

{
  system.activationScripts.postActivation.text = ''
    ln -sfn /run/current-system/sw/bin/ /usr/local/bin
    ln -sfn /run/current-system/sw/lib/ /usr/local/lib
  '';
}

This links the bin and lib directories into /usr/local, so everything that’s installed there will be available in e.g. VS Code.

Jan 2023