How Ruby gems work on the command line

Ruby gems can have a command line interface that allow you to perform tasks in your terminal. This can be anything from an installation script to a generator for scaffolding.

Understanding how these command line interfaces (CLI-s) work is not very difficult. It's a collaboration between Unix basics and whatever language the CLI is written in. Command line tools can be written in any language, but in the case of gems they're written in Ruby.

Unix

It begins with the Unix basics. What happens when you call rails from the command line has not a lot to do with Ruby. It works because RubyGems utilizes how executables are located and run from the command line.

Executables

There are not a lot things special about executables. Any file can be one. Some executables are plain text files while others are binaries that are compiled from languages like C and Go.

To quickly show how easy it is to make an executable from a plain text file we'll make one by writing echo hello world into the file my_executable. To make an executable from a normal file all it needs are permissions to be executed. This can be done by running the chmod command with the +x flag. This will give the file permission to be execuable. Then all you need to do is run it.

$ echo "echo hello world" > my_executable
$ chmod +x my_executable
$ ./my_executable
hello world

The examples in this post will be run from Bash.

Technical notes

PATH

The executables installed on your system can be called from anywhere, not just from the same directory or using the absolute path to the file.

When you call ruby or rails in the terminal they're not in that same directory, yet somehow the terminal knows what program you want to run.

This is where the PATH comes in. PATH is an environmental variable that's available in the session of your terminal shell. This variable contains multiple paths that are used to find executables. Every path is a directory in which executables will be searched when you type in a command in the terminal.

You can see what the PATH variable is in your terminal by running: echo $PATH. (Your PATH may differ.)

$ echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/tom/.gem/ruby/2.2.2/bin

To find the location of an executable you can search for it with which. This way you'll see where executables live.

$ which rails
/Users/tom/.gem/ruby/2.2.2/bin/rails

It's very easy to add your own executable to your PATH: just add the executable to a directory that's specified in your PATH.

You can also add your own directories to the PATH variable. On my development machine I've set up a special directory for this /Users/tom/.bin. This is where I keep executables that are meant only for my user.

Now if we move the executable to a directory that's in the PATH variable, it will be accessible from anywhere during that session.

$ mkdir $HOME/.bin
$ export PATH="$HOME/.bin:$PATH"
$ mv my_executable $HOME/.bin
$ my_executable
hello world

$ mkir foo
$ cd foo
$ my_executable
hello world

Technical notes

Ruby

Ruby uses this Unix logic to make gems work on the command line. It will add files to a directory in PATH and set the correct permissions. However, there's a bit more to it for Ruby.

Ruby environment

Any file can be an executable, so a Ruby file can be as well.

$ echo "puts 'hello ruby'" > ruby_executable.rb
$ chmod +x ruby_executable.rb
$ ./ruby_executable.rb
./ruby_executable.rb: line 1: puts: command not found

Uh-oh. Not quite what we had in mind.

The problem here is that when you run the ruby_executable file it is not interpreted as a Ruby file even though it's extension is .rb. The .rb extension doesn't do anything here, we can just as well remove it. The file is actually run through your terminal shell as a Bash or ZSH file.

To fix this we'll need to tell which environment to run this file in. We can do so by specifying it on the first line:

#!/usr/bin/env ruby

This is a "magic comment", called the Shebang, that Bash will read and then execute the rest of the file in.

Let's try it again:

$ echo "#\!/usr/bin/env ruby\nputs 'hello ruby'" > ruby_executable
$ chmod +x ruby_executable
$ ./ruby_executable
hello ruby

Now the Ruby executable finally works!

Technical notes

Normally files are interpreted in the same environment the executable is run from. This is usually Bash or ZSH depending on your personal configuration.

Using the first line of a file to specify which environment to run it in is recommended. Even for those executables that only contain Bash. This way you make absolutely sure they are run in the right environment.

gemspec

Now we know how to run Ruby files from anywhere. The only thing left to answer is: how do they become available?

Gems can configure their executables in their gemspec file. This is where all the metadata and installation configuration is set for each gem. Without a gemspec a Ruby gem can't be distributed.

Gem::Specification.new do |gem|
  gem.name        = "my_gem"
  # The executable RubyGems will load is
  # in the `bin/` directory of the gem.
  gem.executables = ["my_gem"]
end

When the gem is installed RubyGems created an executable file in the gem directory of the Ruby installation. This file is not your executable, but a tiny wrapper which calls it instead. This wrapper exist so you can call a specific version of the gem using version number as the first argument.

rails _4.1.0_ new my_project

(The underscores are required when selecting a gem version. Take a look in less $(which rails) for an example.)

Now that the executables exist they still need to be added to the PATH of your environment. Ruby version managers like RVM, rbenv and chruby add the correct directories to your PATH automatically. If you don't use a version manager, the default system install adds exectuables to the system wide /usr/bin and /usr/local/bin directories to make it work.

This means that if your environment is set up correctly you can start calling Ruby gem executables right after you run gem install gem_name.

Arguments

Now that the Ruby gem is installed you can call its executable from anywhere.

However, just printing "hello world" every time doesn't make a great program. Most executables have some dynamic element to them. Developers will want to name or select things for example.

Giving arguments to your programs makes them more dynamic. You can tell your program what to do using one or more arguments.

rails new application_name --database=postgresql

This makes setting up a new Rails application easier. In the example above Rails already knows how your application is called and what database to use. This means Rails can already do a lot of work for the user when it's generating a new application.

All arguments you give to your executable can be retrieved by using the ARGV constant in Ruby. ARGV is just an Array with all your space separated arguments as values.

# Command
rails new application_name --database=postgresql

# ARGV
["new", "application_name", "--database=postgresql"]

This is a very simplistic argument system, so you can use things like OptionParser to make it a lot easier.

For more complicated CLI there are also extensive libraries, like Thor and Rake. These make specifying and retrieving your CLIs arguments a lot easier. Thor even powers a lot of Rails' magic on the command line.

Make your own

I hope this makes a bit better to understand how the Ruby gems and their executables work. Next time you want to build your own (Ruby gem) executables you know a little bit more about how and what to look out for.