Running and installing Ruby in parallel using Rbenv with flock and Buildkite

I am maintaining a few Docker-images that contains all supported versions of Ruby, and they also ship with Jemalloc.

Because I use Buildkite, it is very easy for me to test multiple versions at the same time, but they all run the same setup script:

echo "--- setup rbenv"
source ~/.bash_profile

echo "--- setup ruby"
rbenv local 2.7.7 || rbenv install 2.7.7 && rbenv local 2.7.7

echo "--- bundle setup"
gem query --silent --installed --exact bundler || gem install bundler

echo "--- tests"
ruby tools/test.rb 3.1

This fails for some reason when multiple agents of Buildkite run in parallel, I don’t know if Rbenv or Rubygems gives any guarantees for this sort of usage, but nevertheless it fails.

Instead of digging into the intricacies of Rbenv and Rubygems we could make sure that the code above are only being executed by one agent at a time, we can do this with flock:

exec 10>>~/.bash_profile
flock 10

echo "--- setup rbenv"
source ~/.bash_profile

echo "--- setup ruby"
rbenv local 2.7.7 || rbenv install 2.7.7 && rbenv local 2.7.7

echo "--- bundle setup"
gem query --silent --installed --exact bundler || gem install bundler

flock -u 10

echo "--- tests"
ruby tools/test.rb 3.1

We load ~/.bash_profile into filedescriptor 10, we then lock the filedescriptor with flock 10 (all other processes will now block here, waiting for an unlock) we then execute our script, and finally we unlock/release the filedescriptor with flock -u 10.