thoughtbot New Bamboo is now part of thoughtbot

Read the announcement
Makoto Inoue

"The dRuby Book: Distributed and Parallel Computing with Ruby" is finally out



In the past, I’ve written a few blog posts about Japanese Ruby community and its cultural and language barrier from the rest of the world. As the country that gave birth to Ruby, there are a lot of interesting resource about Ruby. Here are some of the interesting books written about Ruby in Japanese.

  • The world of Code = A book written by Matz about his view on programming
  • Ruby Hacker’s guide (RHG) = Page by page guide to Ruby internal source code.
  • Write your own esoteric language in Ruby
  • Garbage Collection - Algorithms and Implementations = Detailed explanation about various garbage collection techniques, coauthored by Narihiro Nakamura, the contributor of Ruby 1.9 GC

RHG is partly translated in English and available for free on internet, though other books can be read only in Japanese. It’s unfortunate that these great resources are not accessible to the rest of the world.

“The dRuby Book” is one of these books only known in Japan. It is written by Masatoshi Seki, the author of ERB, Rinda and dRuby (they are all part of Ruby standard libraries). The original book was first published in year 2001 and the second edition was published in year 2005. “The Pragmatic Bookshelf” and Seki san have been working on translating the book since 2007 and I later joined as a translator. I am glad to announce that the third edition , fully updated with 30 % of new content, is finally translated into English and available to you!!

In this blog post, I’d like to share some of the interesting topics covered in the book.

One of the first metaprogramming example.

In the foreword of the book, Matz describes dRuby as follows.

In 2004, Ruby on Rails became public. The world was surprised by its productivity and by the magic of Ruby that enabled Ruby on Rails. Many people knew Ruby before Rails, but few realized the power of the language, especially metaprogramming.

But Rails is not the first framework to realize the power of Ruby. dRuby came long before Rails. It uses metaprogramming features for distributed programming. Proxy objects “automagically” delegate method calls to remote objects. You don’t have to write interface definitions in XML or any IDL. dRuby is a good example of a very flexible system implemented by Ruby. In this sense, Rails is a follower.

The first version of dRuby was created in 1999 and posted to the Japanese Ruby user mailing list. The email is written in Japanese, but you can see some snippets of source code that look very similar to dRuby now.

# Starting drb server.
DRb.start_server('druby://hostname:port', front)
# Connecting to the remote server
ro =, 'druby://server:port')
ro.sample(1,, 3)

The original source code is about 160 lines, and about 50 lines of the core code will give you clear idea about how dRuby works internally.

class DRbObject
  def initialize(obj, uri=nil)
    @uri = uri || DRb.uri
    @ref = if obj

  def method_missing(msg_id, *a)
    succ, result =, msg_id, *a)
    raise result if ! succ

  attr :ref

DRbObject acts as a proxy object, so it doesn’t have any methods. Therefore, method_missing receives all the method calls and sends them to the DRbConn class.

class DRbConn
  include DRbProtocol

  def initialize(remote_uri)
    @host, @port = parse_uri(remote_uri)

  def send_message(ref, msg_id, *arg)
      soc =, @port)
      send_request(soc, ref, msg_id, *arg)
      soc.close if soc

DRbConn acts as a TCPSocket server. It transfers the message to the remote server—so simple. “send_message” method calls a “dump” method. The method serialises method name, arguments and return values if they can be dumped by Marshal class.

def dump(obj, soc)
    str = Marshal::dump(obj)
    ro =
    str = Marshal::dump(ro)
  soc.write(str) if soc
  return str

This may raise interesting questions. What objects can you not dump?

The ruby doc describes as follows.

Some objects cannot be dumped: if the objects to be dumped include bindings, procedure or method objects, instances of class IO, or singleton objects, a TypeError will be raised.

This means that you can not dump operations that use system resources (such as File, Dir, backtick) and block, lambda, proc, and so on. So how does dRuby work around? The answer is after the “rescue” statement where it creates new DRbObject that contains the reference information of the object you failed to dump.
When The reference object is called, it calls back to the process where the original object resides. Sounds complicated? I also made the video that demonstrates the behaviour.

At the server side, we create a “front” hash object and publish via dRuby. When the server adds a proc into a hash, it stores as a proc.

front = {}
DRb.start_service('druby://:7640', front)
server_proc ={ `pwd`}
front['server_proc'] = server_proc

However, when the client in the different process adds a proc into a a hash, it stores DRbObject instead. When the object is called, the proc is executed in the client process and returns the result back to the server.

remote = DRbObject.new_with_uri('druby://:7640')
client_proc ={`pwd`}
remote['client_proc'] = client_proc

Thanks to this behaviour (called “Pass by Reference”), you can use enumerators (each, map, join, etc) against remote objects, but you may need to take into consideration in what context the method is evaluated.

If you’re interested more in the internals of dRuby, read the rest of the original code to get a better idea about the structure of the library before jumping into reading the current version of dRuby, which is more than 1,700 lines.

Understand Concurrency and Parallel computing using dRuby.

CRuby (The official Ruby as opposed to other Ruby implementations such as JRuby and Rubinius) is known for its limitation of utilising multi core CPU due to GIL (Global Interpreter Lock), hence threading has been neglected for a long time. Now that dRuby lets multiple processes to share objects easily, you need to know in and out about threading more than ever. This book dedicates an entire chapter for threading and how to manage them easily using some of lesser know Ruby standard libraries, such as MonitorMixin, Queue, SizedQueue, and Rinda (Rinda is Ruby port of Linda. The book comes with an interesting way to parallelise your tasks by using Unix “fork” and passing results between processes using Rinda.

The section is available as an excerpt. To try it out, you can install the rinda extension with the following command.

gem install more_rinda

While I was translating the book I had many email conversations with Seki san comparing dRuby with other concurrency model, such as Erlang’s Actor model or Clojure’s STM, you can read some of his thoughts in the book as well.

dRuby for the NoSQL era.

Though “NoSQL” is often discussed in the context of scalability, another reason for choosing NoSQL over RDBMS is its rich data structures. Document, Graph, and some Key Value databases come with unique sets of data structures that are harder to express in a relational database model. Since dRuby lets you share objects remotely, you can use the rich set of data structures Ruby provides (such as Set, Queue, Matrix, RBTree) and share with others via dRuby and this book covers some of the examples. In addition, you can learn about Rinda, distributed TupleSpace which also comes under Ruby standard library, and Drip, a stream-based storage system, the author created recently.

As a teaser, I made a sample screencast of Drip.

In the screencast, 2 processes accessing Drip datasets in different ways, one process reading 4 datasets at a time in a time ascending order, while another process reading 1 dataset at a time with filter by “male” tag in a time descending order. You can use Drip as a key value store, queueing system, or messaging system.

You can play with dRuby by installing with the following command

gem install drip

dRuby provides low level parts to let you talk ruby objects in distributed environment as a natural extension of Ruby. We are looking forward to see more tools, libraries, and even your own datastore/messaging systems powered by dRuby.

Online Resources

I gave a talk called “Demystifying dRuby” at London Ruby User Group and the talk covers some of the topics I mentioned in this blog post.


It took more than a year to translate this book. The translation work was a bit of a journey. I’d like to thank Seki san for patiently answering many questions. I learnt not only about dRuby, but also about various “The Other Ruby Way” which you may never encounter while writing Ruby On Rails day to day. I hope you share the same journey like I went through by reading the book.