Reproducible builds have the aim of generating the exact same binary files for the same input files, thus giving an actual verifiable path from source code to binary code. So this is actually a security feature.

I've set out to make our build system fully deterministic in order to enable reproducible builds. Unforunately, the expected epic journey was more of a small trip out of the door, as most stuff is already built in a deterministic way. There were only two small outliers to this.

The first one is our test suite. The "generate.py" script, which generates our test suite definitions, dumped the modules in a non-deterministic way. As such, our clar test suite was compiled with differently ordered structs and was thus not deterministic.

The second one was how we generate static libraries. The tools ar(1) and ranlib(1) both are non-deterministic by default because they do inlike information like UID, GID and timestamps into the resulting static archive. This can be turned off by enabling the deterministic mode via a simple flag. While this sounds rather simple, I don't really like the solution for the CMake build system, as there is no simple way to just pass in additional flags to these commands. Instead, we have to override the complete commands as defined by three variables. We could hide this behind a simple build-time option "DETERMINISTIC_BUILD" or similar.

All in all, this leaves us with three files which are not reproducible in the build directory (assuming the path to the build directory does not change): two of them are log files and the third is the clar cache. The first two are indeterministic by definiton and should stay so, the third is too unimportant to care. As it is a simple serialization of Python objects via pickle, there's also no easy fix here (I think, though I may be mistaken).

The script I've used to test:

#!/bin/sh

set -e

for i in 1 2
do
    rm -rf /tmp/build /tmp/build${i}.sha1sum
    mkdir -p /tmp/build
    cd /tmp/build
    cmake /home/pks/Development/libgit2
    make -j5

    find . -type f |
    sort |
    while read f
    do
        sha1sum "$f" >>/tmp/build${i}.sha1sum
    done
done

diff -u /tmp/build1.sha1sum /tmp/build2.sha1sum | grep '^+'