What if every Nix build were bit-reproducible?

The holy grail of build systems and package managers is bit-reproducible¹1It’s lame that I have to keep clarifying it as “bit” reproducible but the term is overloaded. builds. This is such a desired trait that nearly every Linux distribution has a reproducible builds project to track the progress and validity of their builds.

Why is bit-reproducibility so important? Trust. You can trust that the software you are running has not been tampered with and is the same as what the developers intended if you can verify that when built from source, the output is identical.

Making software bit-reproducible though is hard. There are a lot of ways that can introduce non-determinism into a build: timestamps, random seeds, thread scheduling, and more. Most distributions that have achieved “reproducible builds” have done so for only a subset of their packages, usually those that comprise the core of the ISO or distribution.

Nix’s default model, the extensional model (input-addressed), famously works around the need for bit-reproducibility by hashing the inputs to a build and using that hash as the store path. This lets the build be repeatable even if the output is not bit-for-bit identical. It is an incredibly pragmatic solution to making builds repeatable, but it does not solve the problem of trust. We are still trusting the outputs uploaded to cache.nixos.org have not been tampered with.

Nix also has an experimental content-addressed model (intensional model) where the /nix/store paths are based on the content of the output, but crucially, it does not guarantee that the builds are bit-reproducible.

This leaves us with an undesirable situation regardless of the model. For the extensional model, the smallest change in the inputs changes the store path which causes a cascade of rebuilds. For the intensional model, a single derivation can produce multiple different outputs.

What if every build could be bit-reproducible? What would that mean for Nix and the extensional model?

I have been working on Rewind, a deterministic virtual machine. It runs builds in a deterministic environment, which so far has mostly been used to investigate bugs. I have been thinking about what else a platform that runs builds deterministically makes possible.

I started this thought by exploring how to make profile-guided optimization reproducible but I was just scratching the surface.

§The machine is an input

When we try to make a build bit-reproducible, we usually go hunting for the sources of non-determinism: timestamps, random seeds, thread order. These are inputs the derivation does not account for, so they are not hashed into the store path.

A simpler approach is to run the whole build inside a deterministic machine.

Eelco’s thesis says outright that the extensional model assumes the builder is impure (§5.7).

Build operations in general are not pure: the contents that a builder stores in the output path can depend on impure factors such as the system time. […] Thus, while any derivation can have a potentially infinite set of output path contents that can be produced by an execution of its builder, we view the elements of that set as interchangeable.

But this is a model—an assumption about builders.

Running a build within Rewind captures all of the implicit inputs to a build and makes them explicit, which makes any build bit-reproducible. 🤯

Aside: New to Rewind? Start with a flaky build you only catch once and dafuq is a deterministic VM?

We can wrap any derivation in Rewind with a Nix function. inRewind takes the machine’s inputs and a derivation, and returns the same derivation with its builder running inside Rewind:²2Rewind is a KVM-based virtual machine, so it requires a host with KVM support. It also requires the rewind-amd or rewind-intel system feature to be advertised by the host.

let
  inherit (rewind.lib.x86_64-linux) inRewind;
  deterministic = inRewind {
    epoch = 1790985600;
    seed = 0;
    schedule = 0;
  };
in
deterministic pkgs.hello

It works on any derivation and the resulting derivation is bit-reproducible every time.³3On the same Rewind version and the same CPU vendor. Fixed-output derivations are refused, because they need the network and the VM has none.

Let’s look at a simple example of a build that is not bit-reproducible, and how when wrapped in Rewind, it becomes bit-reproducible.

Here is a simple C program that prints a greeting and depends on a library. The derivation adds a build id and a build time to the output similar to what release builds often do to “stamp” their builds. The build id and build time are two common sources of non-determinism in builds.

let
  build =
    if machine == "rewind" then
      inRewind { epoch = 1790985600; seed = 0; schedule = 0; }
    else
      (drv: drv);
  extra = lib.optionalAttrs ca { __contentAddressed = true; };
in
rec {
  greet = build (
    pkgs.runCommandCC "greet" extra ''
      cp ${./greet.h} greet.h; cp ${./greet.c} greet.c
      mkdir -p $out
      cc -c greet.c -o $out/greet.o \
        -DBUILD_ID="\"$(cat /proc/sys/kernel/random/uuid)\"" \
        -DBUILT_AT="\"$(date -u +%FT%TZ)\""
    ''
  );

  hello = build (
    pkgs.runCommandCC "hello" extra ''
      cp ${./greet.h} greet.h; cp ${./main.c} main.c
      mkdir -p $out/bin
      cc main.c ${greet}/greet.o -o $out/bin/hello
    ''
  );
}

We have wrapped the derivation in a function to control whether the build runs in Rewind or not, and whether the build is content-addressed or not.

Built on the host under the input-addressed model, greet fails --check:

$ nix-build -A greet --arg ca false --check
error: derivation '/nix/store/2hm8ram0…-greet.drv' may not be deterministic:
output "/nix/store/kjx5rzjk…-greet" differs

Built in Rewind, the same derivation passes:

$ nix-build -A greet --arg ca false --argstr machine rewind
building '/nix/store/bxh3c25a…-greet.drv'...
inRewind: run c7be39a9ea6be3de exited:0 after 2157 steps/nix/store/lrx579rx…-greet
$ nix-build -A greet --arg ca false --argstr machine rewind --check
checking outputs of '/nix/store/bxh3c25a…-greet.drv'...
inRewind: run c7be39a9ea6be3de exited:0 after 2157 steps/nix/store/lrx579rx…-greet
$ nix-build -A hello --arg ca false --argstr machine rewind
building '/nix/store/1bkqfsg3…-hello.drv'...
inRewind: run 54b543a1c6b65b46 exited:0 after 2439 steps
/nix/store/y8r98q35…-hello

$ ./result/bin/hello
hello, world
build aabe0f2b-f5d1-477f-b3b8-3e7f1755dc0f at 2026-10-03T00:00:00Z

The UUID still comes from the kernel’s random pool and the time still comes from the clock. They are the same on every build because the pool’s seed and the clock’s start are initialized from the same values.

Nice! We just got a build that is bit-reproducible even if it’s not content-addressed.

You can find more details of the library and even making Rewind a Nix external-builder on Rewind’s Nix build reference.

What does that mean for the content-addressed model?

Since the outputs are not bit-reproducible, the same derivation gets a different /nix/store path on each rebuild.⁴4Two stores are used to illustrate buids on different machines.

$ nix-build -A hello --arg ca true --store ./store-a
resolved derivation: '/nix/store/5v566cv6…-hello.drv' -> '/nix/store/83qn27rk…-hello.drv'...
/nix/store/rwaisri0…-hello
$ nix-build -A hello --arg ca true --store ./store-b
resolved derivation: '/nix/store/5v566cv6…-hello.drv' -> '/nix/store/0rjbw8s8…-hello.drv'...
/nix/store/wvy9zby9…-hello
$ nix realisation info -f . greet --arg ca true --store ./store-a
rbzv0mbw…-greet.drv^out -> bryxd8dy…-greet
$ nix realisation info -f . greet --arg ca true --store ./store-b
rbzv0mbw…-greet.drv^out -> hh2v12cs…-greet

Producing multiple variants is problematic and the thesis notes the consequence (§6.4.1): you can end up with a closure that contains more than one variant of the same derivation.

The fact that a derivation can resolve to any number of output paths due to impurity, leads to the problem that we might end up with a closure that contains more than one element from the output equivalence class of a derivation.

With inRewind, both stores build the same bytes and print the same path:⁵5Nix 2.35 changed how it hashes an output that mentions its own path, so the content address of such an output depends on the Nix version.

$ nix-build -A hello --arg ca true --argstr machine rewind --store ./store-a
inRewind: run 4d9de4235038d6d0 exited:0 after 2158 steps
inRewind: run 3a2797f47898d7ce exited:0 after 2438 steps
/nix/store/daqfk0jm…-hello
$ nix-build -A hello --arg ca true --argstr machine rewind --store ./store-b
inRewind: run 4d9de4235038d6d0 exited:0 after 2158 steps
inRewind: run 3a2797f47898d7ce exited:0 after 2438 steps
/nix/store/daqfk0jm…-hello
$ nix realisation info -f . greet --arg ca true --argstr machine rewind --store ./store-a
q6i384y2…-greet.drv^out -> kvjqc292…-greet
$ nix realisation info -f . greet --arg ca true --argstr machine rewind --store ./store-b
q6i384y2…-greet.drv^out -> kvjqc292…-greet

One derivation, one path, in every store.

Bit-reproducible outputs also mean cache hits across machines. If the bytes are the same, the content address is the same, and a path one machine built is a hit for every other: early cutoff optimization ftw!

§Both panels collapse

When we make every build bit-reproducible, the two models collapse into one. What does this mean for the extensional model?

The extensional model was a wonderful pragmatic solution to a world where determinism was not guaranteed but it came at a cost: a single change in the inputs cascades through every downstream derivation. Adoption to content-addressed derivations has been slow similarly because producing multiple outputs makes it hard to reason about the closure of a derivation. When we have guaranteed bit-reproducibility, both problems disappear.

Having a stable platform with which to build software reproducibly might completely change the toil of working to make software reproducible and change the economics of whether to leverage content-addressed derivations.

The usefulness of a reproducible environment is not well explored and I think there are other interesting use cases for it. One that I’m exploring is to use it to make the act of performing a benchmark reproducible.

Aside The cost is that the machine is now an input, Rewind included. Upgrade Rewind and every wrapped derivation rebuilds. Build on an Intel machine and you get a different, equally reproducible, output than on AMD.

A tool that was designed for bug investigation and reproducibility of builds might have a larger impact on the software ecosystem than I had originally thought. It is a tool that can be used to make software more trustworthy, and all the wonderful beefits that come with it.