Ruby on Guix

I’ve been working with Ruby professionally for over 3 years now and I’ve grown frustrated with two of its most popular development tools: RVM and Bundler. For those that may not know, RVM is the Ruby version manager and it allows unprivileged users to download, compile, install, and manage many versions of Ruby instead of being stuck with the one that is installed globally by your distro’s package manager. Bundler is the tool that allows developers to keep a version controlled “Gemfile” that specifies all of the project’s dependencies and provides utilities to install and update those gems. These tools are crucial because Ruby developers often work with many applications that use different versions of Ruby and/or different versions of gems such as Rails. Traditional GNU/Linux distributions install packages to the global /usr directory, limiting users to a single version of Ruby and associated gems, if they are packaged at all. Traditional package management fails to meet the needs of a lot of users, so many niche package managers have been developed to supplement them.

Taking a step back, it becomes apparent that dependency isolation is a general problem that isn’t confined to software written in Ruby: Node has npm and nvm, Python has pip and virtualenv, and so on. A big limitation of all these language-specific package managers is that they cannot control what is outside of their language domain. In order to use RVM to successfully compile a version of Ruby, you need to make sure you have the GCC toolchain, OpenSSL, readline, libffi, etc. installed using the system package manager (note: I’ve seen RVM try to build prerequisites like OpenSSL before, which I then disabled to avoid duplication and security issues and I recommend you do the same.) In order to use Bundler to install Nokogiri, you need to make sure libxml2 has been installed using the system package manager. If you work with more than a single language, the number of different package management tools needed to get work done is staggering. For web applications, it’s not uncommon to use RVM, Bundler, NPM, Bower, and the system package manager simultaneously to get all of the necessary programs and libraries. Large web applications are notoriously difficult to deploy, and companies hire a bunch of operations folk like me to try to wrangle it all.

Anyway, let’s forget about Node, Python, etc. and just focus on Ruby. Have you or someone you work with encountered hard to debug issues and Git merge conflicts due to a problem with Gemfile.lock? Bundler’s fast and loose versioning in the Gemfile (e.g. rails >= 4.0) causes headaches when different users update different gems at different times and check the resulting auto-generated Gemfile.lock into the repository. Have you ever been frustrated that it’s difficult to deduplicate gems that are shared between multiple bundled gem sets? Have you looked at the RVM home page and been frustrated that they recommend you to curl | bash to install their software? Have you been annoyed by RVM’s strange system of overriding shell built-ins in order to work its magic? I’m not sure how you feel, dear reader, but my Ruby environments feel like one giant, brittle hack, and I’m often enough involved in fixing issues with them on my own workstation, that of my colleagues, and on production servers.

So, if you’re still with me, what do we do about this? How can we work to improve upon the status quo? Just use Docker? Docker is helpful, and certainly much better than no isolation at all, but it hides the flaws of package management inside an opaque disk image and restricts the environments in which your application is built to function. The general problem of dependency isolation is orthogonal to the runtime environment, be it container, virtual machine, or “bare metal.” Enter functional package management. What does it mean for a package manager to be functional? GNU Guix, the functional package manager that I contribute to and recommend, has this to say:

GNU Guix is a functional package management tool for the GNU system. Package management consists of all activities that relate to building packages from sources, honoring their build-time and run-time dependencies, installing packages in user environments, upgrading installed packages to new versions or rolling back to a previous set, removing unused software packages, etc.

The term functional refers to a specific package management discipline. In Guix, the package build and installation process is seen as a function, in the mathematical sense. That function takes inputs, such as build scripts, a compiler, and libraries, and returns an installed package.

Guix has a rich set of features, some of which you may find in other package managers, but not all of them (unless you use another functional package manager such as Nix.) Gem/Bundler can do unprivileged gem installation, but it cannot do transactional upgrades and rollbacks or install non-Ruby dependencies. Dpkg/yum/pacman can install all build-time and runtime dependencies, but it cannot do unprivileged package installation to isolated user environments. And none of them can precisely describe the full dependency graph (all the way down to the C compiler’s compiler) but Guix can.

Guix is written in Guile, an implementation of the Scheme programming language. The upcoming release of Guix will feature a Ruby build system that captures the process of installing gems from .gem archives and a RubyGems import utility to make it easier to write Guix packages by using the metadata available on https://rubygems.org. Ruby developers interested in functional package management are encouraged to try packaging their gems (and dependencies) for Guix.

Now, how exactly can Guix replace RVM and Bundler? Guix uses an abstraction called a “profile” that represents a user-defined set of packages that should work together. Think of it as having many /usr file system trees that can be used in isolation from the others (without invoking virtualization technologies such as virtual machines or containers.) To install multiple versions of Ruby and various gems, the user need only create a separate profile for them:

guix package --profile=project-1 --install ruby-2.2 ruby-rspec-3
# Hypothetical packages:
guix package --profile=project-2 --install ruby-1.9 ruby-rspec-2

A profile is a “symlink forest” that is the union of all the packages it includes, and files are deduplicated among all of them. To actually use the profile, the relevant environment variables must be configured. Guix is aware of such variables, and can tell you what to set by running the following:

guix package --search-paths --profile=project-1

Additionally, you can also create ad-hoc development environments with the guix environment tool. This tool will spawn a sub-shell (or another program of your choice) in an environment in which a set of specified packages are available. This is my preferred method as it automagically sets all of the environment variables for me and Guix is free to garbage collect the packages when I close the sub-shell:

# Launch a Ruby REPL with ActiveSupport available.
guix environment --ad-hoc ruby ruby-activesupport -E irb

In order to make this environment reproducible for others, I recommend keeping a package.scm file in version control that describes the complete dependency graph for your project, as well as metadata such as the license, version, and description:

(use-modules (guix packages)
             (guix licenses)
             (guix build-system ruby)
             (gnu packages)
             (gnu packages version-control)
             (gnu packages ssh)
             (gnu packages ruby))

(package
  (name "cool-ruby-project")
  (version "1.0")
  (source #f) ; not needed just to create dev environment
  (build-system ruby-build-system)
  ;; These correspond roughly to "development" dependencies.
  (native-inputs
   `(("git" ,git)
     ("openssh" ,openssh)
     ("ruby-rspec" ,ruby-rspec)))
  (propagated-inputs
   `(("ruby-pg" ,ruby-pg)
     ("ruby-nokogiri" ,ruby-nokogiri)
     ("ruby-i18n" ,ruby-i18n)
     ("ruby-rails" ,ruby-rails)))
  (synopsis "A cool Ruby project")
  (description "This software does some cool stuff, trust me.")
  (home-page "https://example.com")
  (license expat))

With this package file, it is simple to an instantiate a development environment:

guix environment -l package.scm

I’m not covering it in this post, but properly filling out the blank source field above would allow for building development snapshots, including running the test suite, in an isolated build container using the guix build utility. This is very useful when composed with a continuous integration system. Guix itself uses Hydra as its CI system to perform all package builds.

As mentioned earlier, one of the big advantages of writing Guix package recipes is that the full dependency graph can be captured, including non-Ruby components. The pg gem provides a good example:

(define-public ruby-pg
  (package
    (name "ruby-pg")
    (version "0.18.2")
    (source
     (origin
       (method url-fetch)
       (uri (rubygems-uri "pg" version))
       (sha256
        (base32
         "1axxbf6ij1iqi3i1r3asvjc80b0py5bz0m2wy5kdi5xkrpr82kpf"))))
    (build-system ruby-build-system)
    (arguments
     '(#:test-target "spec"))
    ;; Native inputs are used only at build and test time.
    (native-inputs
     `(("ruby-rake-compiler" ,ruby-rake-compiler)
       ("ruby-hoe" ,ruby-hoe)
       ("ruby-rspec" ,ruby-rspec)))
    ;; Native extension links against PostgreSQL shared library.
    (inputs
     `(("postgresql" ,postgresql)))
    (synopsis "Ruby interface to PostgreSQL")
    (description "Pg is the Ruby interface to the PostgreSQL RDBMS.  It works
with PostgreSQL 8.4 and later.")
    (home-page "https://bitbucket.org/ged/ruby-pg")
    (license license:ruby)))

Note how the recipe specifies the PostgreSQL dependency. Below is the dependency graph for ruby-pg as produced by guix graph, excluding the GCC compiler toolchain and other low-level tools for brevity. Pretty neat, eh?

Given that Guix doesn’t yet have many gems packaged (help wanted), it can still be advantageous to use it for getting more up-to-date packages than many distros provide, but in conjuction with Bundler for fetching Ruby gems. This gets RVM out of your hair whilst creating a migration path away from Bundler at a later time once the required gems have been packaged:

cd my-project/
guix environment --ad-hoc ruby bundler libxml2 libxslt # etc.
# A small bash script can be used to make these gem sets.
mkdir .gems
export GEM_HOME=$PWD/.gems
export GEM_PATH=$GEM_HOME:$GEM_PATH
export PATH=$GEM_HOME/bin:$PATH
bundle install

As you’ve seen in the above package snippets, Guix package definitions are typically very short and rather easy to write yourself. The guix import gem tool was made to lower the barrier even more by generating most of the boilerplate code. For example:

guix import gem pry

Produces this Scheme code:

(package
  (name "ruby-pry")
  (version "0.10.1")
  (source
    (origin
      (method url-fetch)
      (uri (rubygems-uri "pry" version))
      (sha256
        (base32
          "1j0r5fm0wvdwzbh6d6apnp7c0n150hpm9zxpm5xvcgfqr36jaj8z"))))
  (build-system ruby-build-system)
  (propagated-inputs
    `(("ruby-coderay" ,ruby-coderay)
      ("ruby-method-source" ,ruby-method-source)
      ("ruby-slop" ,ruby-slop)))
  (synopsis
    "An IRB alternative and runtime developer console")
  (description
    "An IRB alternative and runtime developer console")
  (home-page "http://pryrepl.org")
  (license expat))

One still has to package the propagated inputs if they aren’t yet available, add the necessary inputs for building native extensions if needed, and fiddle with the native inputs needed to run the test suite, but for most pure Ruby gems this gets you close to a working package quickly.

In conclusion, while support for Ruby in Guix is still in its early days, I hope that you have seen the benefits that using a general-purpose, functional package manager can bring to your Ruby environments (and all other environments, too.) For more information about Guix concepts, installation instructions, programming interface, and tools, please refer to the official manual. Check out the help page for ways to contact the development team for help or to report bugs. If you are interested in getting your hands dirty, please contribute. Besides contributions of code, art, and docs, we also need hardware donations to grow our build farm to meet the needs of all our users. Happy hacking!