It’s quite common for testing libraries (like RSpec and Bacon) to test themselves. That’s pretty cool, but it might get you into some problems. In this article I’ll try to explain one of those problems and give a suggestion that might help.
Instead of showing you a red dot and a failing test, RSpec won’t be able to run its own tests at all if you break
RSpec::Core::Example.run. That shouldn’t be anything shocking, because that method is very important to be able to actually run an example.
The problem we have here is that code you’re working on is unstable and can’t be trusted, which means you can’t really use it to test anything either. That’s testing broken code with broken code and it can get very confusing very fast.
Here’s a more simple example. Let’s say we added a method called
Object, allowing us to check the numerical value of ‘one’ by just calling
#one on any object:
class Object def one 1 end end
We also have a method to check if something’s value is 1, and we put it conveniently in the same library:
class Object def one 1 end def is_one? self == 1 end end
Now, we can write a test that makes our library test its own method:
Nice. Now the library is using its
#is_one? method to test its
#one method. There’s only one problem though. What if we dive in again and start hacking on our library? We could break
#is_one? and end up with something like this:
class Object def one 1 end def is_one? self == 2 # oops! end end
This would mean that our test for
#one will fail, while it’s not broken at all. Instead, our test is broken (which happens to be in the library we’re currently testing).
A solution would be to let a stable version of our library test the current version. Let’s try that.
In our test, we
require the library to be tested first and clone it into a
Unstable namespace. After that, we
require a stable version of our library:
# library_spec.rb require 'library' module Unstable Object = ::Object.clone end require 'stable_library'
The stable library has overwritten the
Object in the global namespace, but not the
Unstable::Object. Now we should call
#one from the
Unstable namespace instead of the global one (because
Unstable::Object is the class we’re currently testing):
Unstable::Object.one.is_one? # => 1
Unstable::Object#one returns a stable
Fixnum, our stable version of
#is_one? is used. This makes sure our test actually runs, even if the current (unstable) version of our library is broken.
Now we have a setup that uses a stable version of a library that can find bugs in an unstable version of itself. Awesome.
What do you think? Be sure to let me know if you have any ideas to improve this approach.