Comparing Ruby Mock Object Libraries

Continuing on the theme of comparing similar things that I started last week, this week I'll be taking on Mock Object libraries.

The purpose of a mock object library is to allow you to create "fake" objects that can take the place of the regular objects in your application during testing (you'll sometimes see them called "test doubles", by analogy with a stunt double, another kind of stand-in).

There are a several reasons why you might want to use a test double in your tests, the two most common are probably these:

  • To take the place of a hard to create or expensive to access object or method call, such as a web services call. Using the test double lets the test system pretend that the expensive object is there, but at a much lower time cost.
  • To isolate an object being tested from the details of the rest of the system. In Rails, for example, a controller test might create test doubles for methods in the model so that the controller test can pass or fail separately from whether the model implementation is correct.

Ruby's open object model and duck-typing makes creating test double objects relatively easy compared to stricter languages like Java. There are four major mock objects packages in Ruby:

  1. FlexMock is the original Ruby mock object package
  2. Mocha is quasi-official in that the Rails team uses it for their tests.
  3. RSpec defines its own mock package
  4. RRis the newest entry, with shorter syntax and a couple of new features

Here's a tour of what each package looks like, and when you might use each feature.

Self-promotion alert: More details about mocks, stubs, test doubles, and anything relating to Rails testing can be found at Rails Test Prescriptions, there's a free getting started tutorial, and a nearly 300 pages and counting full book for $9. Thanks.

Before we start, some important points:

  • This is in no way a comprehensive look at any of these libraries.
  • Some of these libraries have advanced or other features that don't really map to anything the other ones do (especially RR), check the full docs for each package for full info.

The first step is installing and integrating with Test::Unit. The RSpec mocks, of course, can't be integrated with Test::Unit, however, any of the other packages can be integrated with RSpec by adding the line config.mock_with :rr or :flexmock, or :mocha in the spec_helper.rb file.

Installation with Test::Unit
Flexmock sudo gem install flexmock

Then, in test_helper.rb:
require 'flexmock/test_unit'
Mocha sudo gem install mocha
Then, in test_helper.rb:
require 'mocha'
RR sudo gem install rr

Then, inside the test case declaration in test_helper.rb:
include RR::Adapters::TestUnit
RSpec N/A

The most basic thing any of these packages can do is create a single dummy object that you can specify one or more methods with return values. When those methods are called, the specified values are returned. If the methods are not called, nothing happens. In general, once these objects are created, they are treated exactly the same as pre-existing objects, and all the later filters and expectation methods apply.

Creating a simple test double with a stubbed method
mock = flexmock("name", :method => result)
Mocha object = stub(:method => result)

Methods can also be specified in a block, using the stubs as below.
RR double = stub!.method { value }
Which is a shortcut for
double = stub( { value }
RSpec double = stub("name", :method => value)

Most of the time, though, you'll want to replace methods on existing objects. In Flexmock and RR, this involves calling a special method with the object as an argument, as in Flexmock's flexmock(object), while in Mocha and RSPec, this involves calling a method on the object, as in object.stubs. In either case, further information about the method being stubbed and its return value is usually chained after the declaration of the stub.

Remember, classes are just another kind of object in Ruby, so class methods can be treated like any other method stub(User).should_receive(:find).

Creating a test double from a real object
Flexmock mock = flexmock(project)

If the object is a string or symbol, then to prevent a confusion with the simple test double, use mock = flexmock(:base, object). Also, mock = flexmock(:safe, project) does not add extra methods to the existing object -- this must be called with any expectations defined in an attached block.
To set return values:



mock = flexmock("name") { |m| m.should_receive(:method).and_return(value) }


mock = flexmock("name", :method => value)
mock = flexmock("name").should_receive(:method => value)

and_raise(exception) can be used.
Multiple values in the and_return method will be returned one at a time for each call. The last value will be repeated over and over, if needed. Also and_return can take a block, but this is not recommended.

Mocha project.stubs(:method).returns(value)
project.stubs(:method, value)
To specify errors, use raises(exception) instead of returns. Multiple values in the returns method are as flexmock, and can also be written returns(1).then.returns(2).
RR stub(project).method { value }

is equivalent to
RSpec project.stub!(method).and_return(1)
and_return can also take a block, or a list of values, which is treated as Mocha or FlexMock. Use and_raise to raise an error.

All these packages allow you to set an expectation that the method being replaced will actually be called one or more times. This is typically called a mock, as opposed to a stub. If a method is set up with an expectation that it will be called, then if that method is not called during the test, the test will fail.

The biggest difference here is that FlexMock does not have a separate syntax for creating objects with expectations, any doubled object can have an expectation added by appending a method like once to the description chain. In Mocha, RR, and RSpec, test doubles that will have expectations must be declared as such, using mock or (in Mocha) expects. In those libraries, specifying an method with mock implicitly assumes that the method will be called exactly once.

Each library offers options to change the expected number of times a method will be called.

RR has this unique feature called a proxy, where the method is actually called (as opposed to the return value being set by RR), but you can still specify an expectation on how many times the method is called.

Set an expectation for number of times called
Flexmock mock.should_recieve(:method).and_return(value).once
Other options include: zero_or_more_times, twice, never, times(n). Can also do at_least.once or at_most.twice.
Mocha Bare mock objects are just like bare stub objects, except all methods are expected to be called once
mock = mock()
Existing objects use expects
The default is exactly one call, equal to project.expects(:method).once. Other options include: twice, at_least_once, at_most_once, at_least(x), at_most(x), times(x), times(x..y), never.
RR This sets an expectation for a single call:
mock(project).method { value }
To set an expectation for no calls:
To set an expectation for a different number of calls:
mock(project).method.times(n) { value }
RR also has the concept of a proxy, which is like a mock, but it actually calls the method. You can set an expectation that the method is called, and specify a block to filter the output.

mock.proxy(project).method { |actual| "#{actual}_mocked" }
RSpec The default expectation is that the method will be called once
If the method should never be called:
Other method count expectations can be set with once, twice, exactly(n).times, at_least.once, at_least(n).times, at_most.once, at_most(n).times, any_number_of_times.

All four libraries offer similar syntax for specifying arguments that must match for the doubled method to be invoked. This allows you to specify different return values based on the arguments. For example, you could specify multiple stubs of the find method, each returning a different model object. In addition to matching based on the exact value of the arguments, each library offers some more generic matchers based on class, or matching a regular expression or whatnot.

Filtering methods by arguments
Flexmock mock.should_recieve(:method).with("a")
Also with_any_args, with_no_args. If the argument to with is a class, then any instance of the class matches. If it's a regex, then any string matching the regex matches. There is also a mechanism for more complex logic.
Mocha For a stub or a mock:

If with is passed a block, then the method matches if the block returns true. Several other matchers including instance_of, Not, any_of, and regexp_matchers.
RR Just set the arguments to the method when defined:
mock(project).method(1) { value }
There are special matchers that can be placed as an argument, including anything, is_a, numeric, boolean, duck_type. You can also put in a range or a regular expression.
RSpec project.should_recieve(:method).with(1)
Other filters include, anything, any_instance_of, hash_including,
boolean, duck_type(:message), or a regular expression.

I said a little bit ago that classes can be doubled just like any other object. Three of the libraries also have special syntax that allow you to specify stub or mock behavior for any instance of the class that is created subsequently to that declaration. (Be careful, instances of the class previously created will not have the double behavior). Typically, after the method declares that this double applies to all new instances, any other filter or expectation can be applied.

Create doubles for all instances of a class
Flexmock flexmock(Project).new_instances.should_recieve(:save => false)
After, new_instances any expectation can be written.
RR mock.instance_of(Project).save(false)
RSpec N/A? The work around is something like this:< br/>

Project.stub!(:find).and_return(mock_model(Project, :update_attributes => false))

A couple of the libraries offer special features for ActiveRecord.

ActiveRecord specific mocks
Flexmock flexmock(:model, Project)

After that, as normal, but id, to_params, new_record?, errors, is_a?, instance_of?, kind_of?, and class are already stubbed.
Mocha N/A
RSpec With the rspec-rails plugin:
mock_model(Project, :method => value)
Stubs id, to_param, new_record?, and errors.

Also, and this is a little more advanced, some of the libraries offer the ability to set a stub or mock on an entire chain of method calls in one line, without having to explicitly set the intermediate mock object.

Replacing a chain of method calls

Then as any other mock.
Mocha N/A
RR stub(project).leader.stub!.address.stub!.city { "Chicago" }


I sure hope that made sense, I find the terminology surrounding mock objects to be hopelessly and needlessly confusing. Please discuss confusion in the comments, and I'll change the text as needed.

All of these tools are very solid, flexible, and providing functionality above and beyond what you are likely to need in your testing. I think it's perfectly reasonable to make a choice based on liking a particular syntax or needing one particular feature.

I've personally used FlexMock the most, but recently switched to Mocha on the theory that it was almost a Rails default. I think RR is very cool in concept, but I've found it hard to get used to in practice, it's almost too succinct.

Related Services:
Ruby on Rails Development, Custom Software Development