Chaining Nix stores for fun
Published 2025-05-07 on Farid Zakaria's Blog
I recently realized that you can chain Nix stores 🤯 – although I’m not 100% clear on why I may want to do it.
Nevertheless, the concept is pretty cool – and I’m sure I can come up with some interesting use-cases.
What do I even mean?
Well by default nix
attempts to locate which “store” to use automatically:
- Use the local store
/nix/store
if/nix/var/nix
is writable by the current user. - If
/nix/var/nix/daemon-socket/socket
exists, connect to the Nix daemon listening on that socket. - For Linux only, use the local chroot store
~/.local/share/nix/root
, which will be created automatically if it does not exist.
You can be more explicit and tell nix
the store to use via --store
on the CLI.
There are a variety of store types: dummy, ssh, overlay-fs, s3, http and so on.
I think I can chain stores of type daemon endlessly.
+-------------------+ +-------------------+ +------------------+
| Nix Daemon 2 | ---> | Nix Daemon 1 | ---> | Local Nix Store |
| /tmp/nix_socket_2 | | /tmp/nix_socket_1 | | /nix/store |
+-------------------+ +-------------------+ +------------------+
To test this out, I have created a new nix daemon
which is listening on a new socket /tmp/nix_socket_1
.
This daemon will set it’s store to /tmp/chain-example
. When a filesystem store other than /nix/store
is used, Nix will create /nix/store
within it and chroot
so that /nix/store
appears to be the root.
If you don’t do this, then we cannot make use of all the pre-computed binaries offered by the NixOS cache. The documentation has a nice blurb on this ref.
> NIX_DAEMON_SOCKET_PATH=/tmp/nix_socket_1 nix daemon \
--debug --store /tmp/chain-example
I now create a second daemon that will listen on /tmp/nix_socket_2
and whose store is unix:///tmp/nix_socket_1
, the first daemon.
> NIX_DAEMON_SOCKET_PATH=/tmp/nix_socket_2 nix daemon \
--debug --store unix:///tmp/nix_socket_2
Now we can do our build!
We execute nix build
but execute it against the second daemon (nix_socket_2
).
> nix build nixpkgs#hello \
--store unix:///tmp/nix_socket_2 \
--print-out-paths
# bunch of debug messages
/nix/store/y1x7ng5bmc9s8lqrf98brcpk1a7lbcl5-hello-2.12.1
Okay – so we just tunneled our command through a daemon… cool?
Well we can maybe write an interceptor to log all the traffic and see what’s going on.
Here we can use socat
to pipe all the data to nix_socket_1
but also -v
will debug print everything.
> socat -v UNIX-LISTEN:/tmp/nix_socket_1,fork \
UNIX-CONNECT:/tmp/nix_socket_2
I’m wondering whether it makes sense to support multiple “read” stores and only one that gets written to.
Although at this point I’m not sure about the distinction between store and substituters…
Improve this page @ a30e17e
The content for this site is
CC-BY-SA.