caching your nix-shell

Published 2020-08-12 on Farid Zakaria's Blog

tl;dr; you can use the following invocation to cache your nix-shell

 $ nix-store --query --references $(nix-instantiate shell.nix) | \
    xargs nix-store --realise | \
    xargs nix-store --query --requisites | \
    cachix push your_cache

I have been hooked on Nix as a way to introduce reproducible development environments. However I had to introduce a shell.nix file for a project that relied on a very old version of nodejs.

Using an old-version worked – which was great considering nix is promising hermetic packages – however waiting to build this particular nodejs version on every machine was a big time sink.

No problem! Nix offers the concept of a binary cache to avoid having to rebuild packages needlessly.

A binary cache builds Nix packages and caches the result for other machines.

If you do a quick Google search for “how to cache my nix-shell” you quickly discover cachix.

The prevailing wisdom at the time, outlines the following incantation to cache.

nix-store -qR --include-outputs $(nix-instantiate shell.nix)

While it’s not technically wrong it’s caching way more than you want. It is included not only the immediate buildInputs of your mkShell but their complete build-time transitive dependencies.

Let’s break down a simple example! Here is a simple nix-shell that simply pulls in Chromium.

let nixpkgs = import <nixpkgs> {};
in
with nixpkgs;
with stdenv;
with stdenv.lib;
mkShell {
  name = "example-shell";
  buildInputs = [chromium];
}

Let’s us run the prevailing wisdom command.

$ nix-store --query --requisites --include-outputs $(nix-instantiate shell.nix) | wc -l

2102

2102 store-paths that need to be uploaded!

The problem comes with calling requisites on a derivation which is the result of nix-instantiate. We cannot call nix-build because you cannot realise mkShell!

–requisites A source deployment is obtained by distributing the closure of a store derivation.

What do we want ? We want the immediate build-time dependencies of our derivation but for each dependency, only include their run-time dependencies.

Let’s first checkout the immediate dependencies by using references.

–references Prints the set of references of the store paths paths, that is, their immediate dependencies. (For all dependencies, use –requisites.)

 nix-store --query --references $(nix-instantiate shell.nix)
/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh
/nix/store/72kcdqdnz8myr2s28phzi48cv2a8q5x3-bash-4.4-p23.drv
/nix/store/04jbnjp8522lv7bpwzp2jy8nihplj9kk-chromium-84.0.4147.105.drv
/nix/store/prnb5ax5x8xapg5xmmh8w1c7zx8f0j9c-stdenv-linux.drv

The problem is that it’s returning the .drv for chromium; if we were to call --requisites on it; we would get the huge dependency set as earlier.

The “trick” is to realise it so we get an output-path.


$ nix-store --query --references $(nix-instantiate shell.nix) | \
    xargs nix-store --realise | \
    xargs nix-store --query --requisites | \
    wc -l

221

221 store-paths that need to uploaded!

Hurray much smaller!

Tying it now with cachix we get this nice one-liner.

 $ nix-store --query --references $(nix-instantiate shell.nix) | \
    xargs nix-store --realise | \
    xargs nix-store --query --requisites | \
    cachix push your_cache