Farid Zakaria's Blog

on Farid Zakaria's Blog

Java, Nix & Reproducibility

I’ve written a lot about NixOS; specifically about some of the work I’ve done to improve the Java ecosystem.

In fact, I’ve upstreamed my notes into the Java Maven Nixpkgs documentation.

🎉 Recently, the minimal ISO for NixOS has become reproducible (r13y). 🎉

The ability to produce binary reproducible artifacts is a powerful primitive. What does it take for the JVM ecosystem to adopt reproducible builds within the Nix environment?

For rationale on the why, please see https://reproducible-builds.org/


setting up a Nix Google Cloud Storage (GCS) binary cache

A previous post documented how to setup a binary cache directly on S3. Many however are tied to a different public IaaS offering and may not be able to leverage the native S3 integration Nix offers. Luckily for those using GCP, Google’s blob storage equivalent Google Cloud Storage (GCS) has interoperability with the S3 API.

This will allow us to host our binary cache on GCS while still using the native S3 integration in Nix. Following this guide will allow Nix to leverage GCS without having to use a proxy such as nix-store-gcs-proxy.


JRuby and Sorbet

A recent tweet by the JRuby folks, let me know that the work I had done a while ago to get Sorbet working seemed to have gone under the radar.

Sorbet is a fast, powerful type checker designed for Ruby.

I wanted to reflect on it’s use at our current codebase, challenges still faced and where to go next.

If you want to cut ahead and start using Sorbet right away, I’ve contributed some documentation.


autopatchelf - what it can look like

This is a follow up post to my previous one to on-demand linked libraries for NixOS.

I previously wrote about the work into a new NixOS tool nix-ld, which will allow the possibility of hot-patching the linking of dynamic libraries.

At the moment the goal of the tool is modest in allowing modification of the library search path via a custom environment variable NIX_LD_LIBRARY_PATH.

I’m minimizing the task as there’s a ton of complexity, such as not loading in a different glibc and other subtleties – ask Mic92 about it on #nixos!

I finished with a bit of a teaser on what a full-fledged tool may look like…


On-demand linked libraries for Nix

This is a write up of some discussion ongoing with some folks on the #nix-community IRC chat primarily being driven by Mic92.

Nixpkgs maintains the highest rating on Repology for having the most packages & which are up to date. Unfortunately even with the current ecosystem of packages, there will always be gaps, and for beginners in NixOS a common question is:

“I’ve download a binary and would like to run it on NixOS”

Take a look at this graph https://repology.org/repositories/graphs

repology graph

Can we do better & streamline running non-Nix software? 🤔

This was some of the questions posed by some Nix contributors and I wanted to capture the ideas put forward for others.


Debugging a JNR-FFI bug in Nix

This is a write-up of an issue I discovered when using more advanced features of Java within a Nix environment. Please refer to the GitHub issue #103493 to see any ongoing process

I have been onboarding several engineers to doing their development workflow using Nix; promising all the benefits of hermeticity & reproducibility. The biggest challenge is making sure that onboarding continues to be seemless.

Therefore it’s 🚨 all hands on deck when someone has encountered a SIGSEGV; especially since the environment is within the JVM.

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x0000000000000680, pid=3229039, tid=0x00007ff8c5834640
#
# JRE version: OpenJDK Runtime Environment (8.0_265) (build 1.8.0_265-ga)
# Java VM: OpenJDK 64-Bit Server VM (25.265-bga mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  0x0000000000000680

Isolation using Java's ClassLoader

This is a small write-up of one I sent to my team to help disseminate some knowledge regarding Java’s ClassLoader mechanism.

A codebase I’m working on uses Java ClassLoader to help load multiple JDBC drivers with their complete dependencies.

Typically, using Maven or Gradle, a project would have to pin to common dependencies due to the diamond dependency problem at the cost of potential unforeseen bugs due to drift in versions.

Can we do better? Let’s dig into ClassLoaders.


Nix parallelism & Import From Derivation

I recently submitted a pull-request #101096 for nixpkgs to include new functionality to list all files recursively within a directory.

One of the feedback from @grahamc & @roberth was that function may cause Nix evaluation to stall.

It was not clear why and even more confusing is the name given to this gotcha; Import From Derivation.

Import From Derivation (IFD) is where during a single Nix evaluation, the Nix expression:

  1. creates a derivation which will build a Nix expression
  2. imports that expression
  3. uses the results of the evaluation of the expression.

Let’s dig into this problem and how it’s caused.


nix-copy-closure your nix-shell

This is a synopsis of https://github.com/NixOS/nix/issues/1985. I recommend reading it for additional detail. Many thanks to contributors on the issue like Infinisil for adding context.

Someone reached out to me over e-mail to discuss my previous post on caching your nix-shell.

“What I wish to do is to copy a particular development environment (nix-shell) from A to B, so that I could run nix-shell on server B. Server B is only accessible through SSH and does not have Internet access.”


VPNs continued

This is part 2 of a post regarding VPNs. Please see here for part 1.

In the previous post, we discussed the basics of setting up a virtual private network; the goal of which was bridging two distinct private subnets.

This gave us the alluring property of treating separate networks as they were colocated and can be accessed easily. However most people are first introduced into VPNs for other features such as: anonymity, secrecy & privacy.

A common example may be to tunnel traffic through a host in another country to disguise the origin. For instance, maybe you are in Canada but want to browse the American Netflix catalog.

How can we extend lametun to do this?

I’ll present a few alternatives. The main goal however will to forward all traffic, including ones destined for the Internet, through the tunnel.