The search for a minimal nix-shell continued; mkShellMinimal

Published 2021-08-06 on Farid Zakaria's Blog

“The simplest things are often the truest.” - Richard Bach, 1936.

Earlier I wrote about trying to get a minimal nix-shell. The goal and challenge of the post was about reducing the dependency closure size of the shell.

I was asked what’s the point in trying to minimize the closure size ?

We are using nix-shell in our CI infrastructure and every CI job hydrates it’s own /nix/store from scratch. Reducing the dependency closure size would mean faster CI runs. 🏎️

The post finished with a question, “Can we do better ?”, to which I answered “No, not at this time.”.

I’d like to introduce mkShellMinimal that does better 🎊

Let’s first look at a similar basic example, a nix-shell without any user declared dependencies.

let
  nixpkgs = import <nixpkgs> { };
in
with nixpkgs;
mkShellMinimal {
  name = "my-minimal-shell";

  # No user defined dependencies
  packages = [ ];

  # You can do typical environment variable setting
  FOO = "bar";
}

If we check the closure size of our shell, we see that it’s only 1.4KiB 😮

❯ nix path-info -rSsh $(nix-build shell.nix) 
This derivation is not meant to be built, unless you want to capture the dependency closure.

/nix/store/8ka1hnlf06z3h2rpd00b4d9w5yxh0n39-setup        	 376.0 	 376.0
/nix/store/nprykggfqhdkn4r5lxxknjvlqc4qm1yl-builder.sh   	 280.0 	 280.0
/nix/store/xd8d72ccrxhaz3sxlmiqjnn1z0zwfhm8-my-minimal-shell	 744.0 	   1.4K

That’s nearly a 200_000x improvement. 😱

To facilitate a simpler way to introspect or upload the transitive closure of the shell, I’ve allowed it to be buildable.

I’ve greatly simplified the feature set of what you can do with mkShellMinimal as opposed to mkShell.

For instance the only way to declare dependencies is with the packages keyword. There is no dependency even on coreutils and one could replace it with busybox.

Thank you to jappeace for inspiring the pursuit.

Challenges & Trivia

The rest of the post will document what it took to deliver this minimal shell. It was in fact not as trivial as it might have originally seemed.

If you are only interested in having a minimal nix-shell you can stop reading here 📖.

Tight coupling with stdenv

In the pursuit of trying to remove stdenv was somewhat annoying. I discovered that the nix-shell implementation has some tight coupling(source $stdenv/setup) with it by expecting a setup file provided by stdenv; this needed to be hacked in 🐱‍💻.

std::string rc = fmt(
        R"(_nix_shell_clean_tmpdir() { rm -rf %1%; }; )"s +
        (keepTmp ?
            "trap _nix_shell_clean_tmpdir EXIT; "
            "exitHooks+=(_nix_shell_clean_tmpdir); "
            "failureHooks+=(_nix_shell_clean_tmpdir); ":
            "_nix_shell_clean_tmpdir; ") +
        (pure ? "" : "[ -n \"$PS1\" ] && [ -e ~/.bashrc ] && source ~/.bashrc;") +
        "%2%"
        "dontAddDisableDepTrack=1;\n"
        + structuredAttrsRC +
        "\n[ -e $stdenv/setup ] && source $stdenv/setup; "
        "%3%"
        "PATH=%4%:\"$PATH\"; "
        "SHELL=%5%; "

How is Bash as a dependency gone?

Well since the purpose behind nix-shell shell.nix is for a developer friendly environment (as opposed to debugging a failing deriviation), I have mkShellMinimal rely on /bin/sh which is nearly required for POSIX compliance.

In fact, even NixOS adds a symlink for /bin/sh.

This works well for this case since the builder used for this shell is exceptionally simple.

 builder = writeScript "builder.sh" ''
    #!/bin/sh
    echo
    echo "This derivation is not meant to be built, unless you want to capture the dependency closure.";
    echo
    export > $out
  '';

But my nix-shell still drops me in Bash? Huh?

This is probably one of the most bizarre aspects of nix-shell and Nix; a system which tries to have reproducibility at it’s core.

nix-shell by default, will start up Bash from whatever is referenced via your nixpkgs channel. 😳

auto expr = state->parseExprFromString("(import <nixpkgs> {}).bashInteractive", absPath("."));

This means you could, in theory, write a shellHook in your mkShell that fails on another user’s machine since their nixpkgs channel references a wildly different major version of Bash. 🤯

I would like to see nix-shell use a value specified in the derivation itself, to identify the version of Bash to use. That would make it completely hermetic.

Undocumented requirements to have nix-shell --pure work

I was struggling to get nix-shell --pure to enforce purity and not persist my $PATH. I would have expected this to a feature of the Nix CLI itself and not one of underlying derivation.

Turns out that is not the case.

nix-shell requires that the builder unconditionally clears the $PATH always at the start.