One of the big ideas I think the Nix community should pursue is relocatable binaries. I put my money where my mouth is and collaborated with the VFS Linux maintainer Christian Brauner to get $ORIGIN support in PT_INTERP and the shebang via EBPF 🐝. That work is in Linux 7.3 which is about to be released.

Despite the amazing potential this unlocks, I found few to be as enthusiastic as I was. Instead of excitement, I was met with skepticism: “What’s the point beyond it being a cool technical trick?”

First off, “The pursuit of excellence does not need justification.” by Mitchell Hashimoto should be enough.

There is however a very practical reason to pursue relocatable binaries: it allows Nix to be a package set for other build systems, specifically Bazel and buck2. The two frontends can share the same store, and the machine doing the compiling does not need to have Nix installed.

§The wall

The integration of Nix and buck2 or Bazel is not novel or new. Many have tried and most have failed, or have seen mixed success.

rules_nixpkgs: Brings nixpkgs toolchains to Bazel and has been incompatible with remote execution since 2022.
buck2.nix: Brings nixpkgs toolchains to buck2 as symlinks into the local store, which works on exactly one machine.
Snowydeer: Imports buck2 outputs into Lix as content-addressed store paths, scanning the NAR for hash parts to recover the references.

The wall they all face is that a nixpkgs output is pinned to /nix/store by three absolute strings: the RUNPATH the loader reads, the PT_INTERP the kernel reads, and the symlinks that point at siblings.

A remote executor has none of those paths. you either ship the whole closure inside the executor image, mount the store over the network, or chroot every action into a fake root.

They all work but have unfortunate side effects. A new compiler version means a new executor image. An NFS mount means a shared filesystem under a system whose entire design is that it does not need one. Requiring chroot for a worker means that Nix is now a leaky abstraction.

Relocatable binaries gets rid of the wall by circumventing the problem entirely. 🤯

§A closure is a layout, not a location

Absolute paths are not necessary and we can make them relative. It requires work to find all the places a path is baked in, but this is the plight of a Nix enthusiast: we patch and fix the world.

RUNPATH: Already supports relative paths; the loader expands $ORIGIN to the directory of the executable.
PT_INTERP: Linux 7.3 supports a binfmt_misc handler written in BPF that replaces $ORIGIN in the interpreter to the directory of the executable.

When we make all absolute paths relative, a beautiful thing happens: the directory nix/store/<hash>-name stops being an address and becomes a shape.

We can put the shape under /, you have a Nix store. If we put it under buck-out/v2/gen/toolchains/…/closure/, you have a toolchain artifact. If we put it under a remote worker’s input root, you have an action that runs.

It is the same beauty of what made Nix hermetic but taken to the next maximum: the shape is now relocatable. The same bytes can be in many places at once, and the loader and kernel do not care which one they are in.

§Two frontends

Once a closure is a shape, the two build systems stop competing for the same job. 👏

Everyone loves Nix for its replicability but are furstrated by the needless rebuilds due to the input-address (extensional) model.

On the other hand, tools like buck2 and Bazel only allow content-addressed (intensional) inputs,. Coupled with a fine-grained action graph, this allows for incremental builds and remote execution, but they don’t truly model the world of packages and their dependencies, which is where Nix excels.

Each has spent years badly imitating the other. Buck2 users hand-roll toolchain downloads; Nix users wait for a whole derivation to rebuild because one file changed.

What if Nix and buck2 could natively share the same store?

Nix could provide buck2 a closure, and buck2 would treat it like any other input. On the way out, buck2 hands Nix the name of an output, and Nix fetches it like any other store path.

The “store” in the middle is not a daemon or a filesystem. It is an agreement about layout. One build, seen through that agreement:

Both ends are a store path on the host. Everything in between is a digest.

I have a proof-of-concept implementation of this idea, and it is surprisingly small. The BUCK file is the whole user-facing surface. The gcc closure is one buck2 tree artifact provided as a toolchain.

load("@rules//:closure.bzl", "nix_closure")
load("@rules//:cc.bzl", "cc_binary", "nix_cc_toolchain", "run_remote")

# The gcc closure, pinned by hash, as one tree artifact.
nix_closure(
  name = "closure",
  roots = {
    "gcc": "/nix/store/5q6bdx…-gcc-16.2.0",
    "glibc": "/nix/store/h4wfwi…-glibc-2.44-25",
    ...},
  # every store path the roots need, 17 of them
  closure = [...],
)

nix_cc_toolchain(name = "cc", closure = ":closure")

cc_binary(name = "hello", srcs = ["hello.c"])

# Run the binary on the worker and keep what it prints.
run_remote(name = "hello-report", exe = ":hello")

Every one of those paths is a projection of the closure artifact rather than an absolute /nix/store string, so buck2 can ship the closure wherever the compiler runs. This is most easily seen in the PT_INTERP and RUNPATH strings provided by the linker:

-Wl,--dynamic-linker=$ORIGIN/../../h4wfwi…-glibc-2.44-25/lib/ld-linux-x86-64.so.2
-Wl,-rpath,$ORIGIN/../../h4wfwi…-glibc-2.44-25/lib

hello is only told where to look within the closure, it has no idea it was a /nix/store to begin with.

The sample program hello.c reports its environment to drive home the point.

// Where did the kernel find me?
readlink("/proc/self/exe", exe, sizeof(exe) - 1);

// Which libc did the loader actually map?
dl_iterate_phdr(report_libc, out);

// Does the store path gcc was told about exist on this machine at all?
access(NIX_GCC, F_OK);

The worker is a NixOS VM running NativeLink on a 7.3 kernel with the EBPF binfmt_misc handler registered at boot. Its store is an erofs image of its own closure: 441 paths but crucially gcc is not present: ls /nix/store | grep gcc comes out empty.

$ buck2 run //:hello-report --show-output
Commands: 3 (cached: 0, remote: 3, local: 0)
BUILD SUCCEEDED

hello from a relocated store path
kernel:         Linux 7.3.0-rc7
hostname:       nix-origin-worker
/proc/self/exe: /var/lib/nativelink/work/8f7d…/buck-out/v2/gen/root/…/__hello__/root/nix/store/hello/bin/hello
libc:           /var/lib/nativelink/work/8f7d…/buck-out/v2/gen/root/…/__hello__/root/nix/store/hello/bin/../../h4wfwi…-glibc-2.44-25/lib/libc.so.6
/nix/store/5q6bdx…-gcc-16.2.0 on this machine: absent

§Back into the store

Getting a buck2 output into the Nix store is where the two frontends have to agree on a name.

buck2 already has one. Every artifact it produces is identified by a digest, a Merkle hash over the output tree, and the CAS is keyed by it. Nix already supports a few ways of addressing a path by its content. Add the buck2 digest to that list and the two systems name the same bytes with the same number.

The cc_binary rule already shapes its output like mkDerivation would, nix/store/<name>/bin/hello. A buck2 output’s store path is therefore /nix/store/<digest>-hello, and both sides compute <digest> without asking the other.

The rule also knows the references, because it put glibc in RUNPATH. It writes them in a small file next to the output, together with the path and the digest. That file is the narinfo. 🤯

The way back to the host store is a substitution. The CAS is configured as a substituter, the same way cache.nixos.org is:

substituters = reapi://cluster:50051?instance=main https://cache.nixos.org

On the Nix side a buck2 target is a derivation that we reference by its digest.

packages.hello = buck2.output {
  target = "//:hello";
  # the store path is /nix/store/<digest>-hello
  digest = "sha256-…";
};

Evaluation never runs buck2: nix build asks the substituter for the pinned path and the tree streams out of the CAS, references included:

$ nix build .#hello
copying path '/nix/store/<digest>-hello' from 'reapi://cluster:50051'...
$ nix path-info --json ./result | jq -r '.[].references[]'
/nix/store/h4wfwic161kxrr74jlzla5lsm28hgary-glibc-2.44-25
/nix/store/j7qx4s4mr17j1wqgvqdzj33lmrnzb387-gcc-16.2.0-lib
$ ./result/bin/hello
argv[0]:  /nix/store/<digest>-hello/bin/hello
libc:     /nix/store/<digest>-hello/bin/../../h4wfwi…-glibc-2.44-25/lib/libc.so.6

Nothing was rewritten, rebuilt or re-hashed on the way back. The shape of the closure survived because it only ever used relative paths. The bytes the worker produced are the bytes Nix serves, under the name buck2 already knew.

The two stores become one. 🫂

A cc_binary and a mkDerivation end up as the same kind of object: a tree named by its contents, with relative paths to its neighbours. Which frontend wrote it stops mattering once it is in the store.¹1Relocatable outputs also fix the hardest part of Nix’s own content-addressed derivations: self-reference, an output containing its own hash. $ORIGIN/../lib is how a relocatable output can point at itself.

Two frontends, one store.

§Patch the world

We can think big for how we want to build software. Nix affords us a world where we can make software fit beautifully together. Nix is the one ecosystem where patch the world is a reasonable plan. We can change the kernel, the loader, the compiler wrapper and the package set in a single commit.

Let’s not have static paths in the store limit our possibilities. The valhalla of incrementality awaits and we can maybe achieve it without dynamic derivations.

The yellow brick road to a two-frontend store is paved and clear:

nixpkgs: write relative RUNPATH. #534339 is the opt-in version; the default is the goal. The hash stays in the string so no cache is invalidated and the reference scanner keeps working.
nixpkgs: a location-relative cc-wrapper. The wrapper and bintools-wrapper embed absolute paths in shell. Deriving them from $(dirname "$0") turns the toolchain into something we can cp.
nixpkgs: relocatable = true, enforced. A fixup hook that fails any output claiming the flag while still containing the literal store prefix, in ELF or in text.
NixOS: the handler at boot. Once 7.3 reaches nixpkgs, a module that registers the BPF handler, gated on a PT_INTERP_NIX marker so nothing changes for binaries that did not ask.
Nix: one hash. A content-addressing method that matches an REAPI directory digest, so a buck2 output in the CAS is a /nix/store path.
Build systems: closures as artifacts. The buck2 rules in this post are a few hundred lines. The Bazel equivalent is a repository_rule away, and it would be the first rules_nixpkgs that works with remote execution.

A store path that can live anywhere is a store path everyone can use. Let’s go (re)write some strings. ✍️