Typically I write code test-first, TDD style. I tend to run specs very often, before and after changes, even trivial ones. That's a lot of test runs. When working on Rails apps waiting even a second gets very annoying very fast.
Rails had some workarounds like zeus and, later, spring. They are great: they preload the Rails app and then fork the process (or do some other magic) so that your specs run immediately. They also detect file changes, and reload them automatically.
They also came with problems. I tried to like spring but quickly learned to disable it. Occasionally it would fail to pick my changes. Sometimes it would keep a CPU at 100%. Perhaps I should have tried harder.
And then I had to work with some Rails apps that in before(:suite)
performed truncation of various data stores. Time-to-first-spec was in the order of 10 seconds. Even spring could not help me with before(:suite)
(could it?).
A Demo of a Quick Hack
Let's use a demo Rails app to tinker with:
git clone https://github.com/JetBrains/sample_rails_app cd sample_rails_app bundle install RAILS_ENV=test bundle exec rails db:migrate
Now we can run specs:
$ time bundle exec rspec spec/models/user_spec.rb:12 Run options: include {:locations=>{"./spec/models/user_spec.rb"=>[12]}} . Finished in 0.02567 seconds (files took 1.39 seconds to load) 1 example, 0 failures real 0m1.771s user 0m1.283s sys 0m0.479s
Your run times will obviously be different (hopefully better!). But we're in the ballpark of 1.5 seconds to run even trivial specs, and as the app evolves this time will only increase.
Where does it spend most of the time? From my measurement require File.expand_path('../../config/environment', __FILE__)
(it loads the Rails app) in spec/rails_helper.rb
is rather expensive (almost a second). If we could load the Rails app once upfront we could shave that almost-second off of every spec run. Let's try loading Rails in IRB and then calling RSpec. We have to bypass the exe/rspec
executable to be able to pass arguments:
$ RAILS_ENV=test bundle exec irb irb> require_relative 'config/environment' => true irb> require 'rspec/core' => true irb> puts Benchmark.measure { RSpec::Core::Runner.run(['spec/models/user_spec.rb:12'], $stderr , $stdout) } Run options: include {:locations=>{"./spec/models/user_spec.rb"=>[12]}} . Finished in 0.0388 seconds (files took 4.28 seconds to load) 1 example, 0 failures 0.261190 0.055028 0.316218 ( 0.317082)
By preloading the Rails app we shrunk the rest of the run to ~0.3 seconds, great! However, the above only works once:
irb> puts Benchmark.measure { RSpec::Core::Runner.run(['spec/models/user_spec.rb:12'], $stderr , $stdout) } Run options: include {:locations=>{"./spec/models/user_spec.rb"=>[12, 12]}} .. Finished in 0.01448 seconds (files took 18.8 seconds to load) 3 examples, 0 failures 0.023071 0.001406 0.024477 ( 0.024580) => nil
Unfortunately it runs the same spec twice since it probably accumulates all arguments. Also, if you modify user_spec.rb
or user.rb
your changes will be ignored. Let's run specs in a subprocess instead:
$ RAILS_ENV=test bundle exec irb irb> require_relative 'config/environment'; require 'rspec/core' irb> puts Benchmark.measure { fork { RSpec::Core::Runner.run(['spec/models/user_spec.rb:12'], $stderr , $stdout) }; Process.wait } Run options: include {:locations=>{"./spec/models/user_spec.rb"=>[12]}} . 0.000157 0.002070 0.422430 ( 0.424204) irb> puts Benchmark.measure { fork { RSpec::Core::Runner.run(['spec/models/user_spec.rb:12'], $stderr , $stdout) }; Process.wait } Run options: include {:locations=>{"./spec/models/user_spec.rb"=>[12]}} . 0.000098 0.001635 0.413957 ( 0.420418)
Great! We have quick specs and our user_spec.rb
and user.rb
changes will be picked up.
Loading Helpers
The above is already useful. However, there are a few more things we could preload. spec_helper.rb
and rails_helper.rb
are good examples:
$ RAILS_ENV=test bundle exec irb irb> require_relative 'config/environment'; require 'rspec/core' irb> $LOAD_PATH << File.expand_path('./spec') # to make `require 'spec_helper'` work irb> require_relative 'spec/rails_helper' irb> puts Benchmark.measure { fork { RSpec::Core::Runner.run(['spec/models/user_spec.rb:12'], $stderr , $stdout) }; Process.wait } 0.000185 0.002373 0.125948 ( 0.129567)
Down to ~0.13 seconds!
Reloading Factories
We preloaded rails_helper
which made time-to-first-test even quicker. But it loaded factories, and so factory changes will not be picked up. When coding specs factories tend to change quite often and it's rather annoying to need to restart our process after each factory change. Luckily FactoryBot natively supports reloading factories.
$ RAILS_ENV=test bundle exec irb irb> $LOAD_PATH << File.expand_path('./spec'); require_relative 'config/environment'; require 'rspec/core'; require_relative 'spec/rails_helper' irb> puts Benchmark.measure { FactoryBot.reload } 0.000390 0.000209 0.000599 ( 0.000596) irb> fork { FactoryBot.reload; RSpec::Core::Runner.run(['spec/models/user_spec.rb:12'], $stderr , $stdout) }; Process.wait
We can see that factory reloading is instant, at least in this case (and in most Rails apps I've seen, too), and could be performed before each run. But we're optimizing here, are we not? ;-)
Let's track changes to factories and only reload them when necessary.
$ RAILS_ENV=test bundle exec irb irb> $LOAD_PATH << File.expand_path('./spec'); require_relative 'config/environment'; require 'rspec/core'; require_relative 'spec/rails_helper' irb> def factory_mtime irb* ( irb* ( irb* Dir[File.join(Dir.pwd, 'spec/factories/**')] + irb* Dir[File.join(Dir.pwd, 'spec/factories.rb')] irb* ).select(&File.method(:file?)).map(&File.method(:mtime)) + irb* [Time.new(2000, 1, 1, 1, 1, 1)] # default to year 2000 irb* ).max irb* end irb> last_mtime = factory_mtime irb> new_mtime = factory_mtime; if new_mtime > last_mtime; puts 'factories reloaded'; last_mtime = new_mtime; FactoryBot.reload; end; fork { RSpec::Core::Runner.run(['spec/models/user_spec.rb:12'], $stderr , $stdout) }; Process.wait Run options: ... irb> FileUtils.touch(File.join(Dir.pwd, 'spec/factories.rb')) # modify the factories irb> new_mtime = factory_mtime; if new_mtime > last_mtime; puts 'factories reloaded'; last_mtime = new_mtime; FactoryBot.reload; end; fork { RSpec::Core::Runner.run(['spec/models/user_spec.rb:12'], $stderr , $stdout) }; Process.wait factories reloaded Run options: ...
By now our one-liner is no more, and it's high time to put it in a script.
Before Suite
The last bit I'd like to preload is before(:suite)
hooks. Typically they contain DB cleanup and preparation specs which could be run once in the morning rather than before every spec run.
This is a tad more tricky; we need to dive into the guts of RSpec (and those guts do change a lot). v3.10 invokes @configuration.with_suite_hooks
which runs the before(:suite)
hooks like this. Let's run those hooks manually and then clear them so that RSpec does not run them before specs. This is RSpec 3.10 specific:
irb> hooks = RSpec.configuration.instance_variable_get('@before_suite_hooks') irb> RSpec.configuration.send(:run_suite_hooks, 'a `before(:suite)` hook', hooks) irb> hooks.clear
Conclusion
First, of all, Ruby, man!!! I am free to dive into the source of any package and call or monkeypatch anything. Of course, it's on me when it breaks later.
Second, the expressiveness of the language continues to amaze me. The factory_mtime
code approaches LISP and is way ahead of what Python or Go can achieve. Or rather, what I can do in those languages.
All in all, this is a very sharp tool for a very specific case. It will make you a 10x programmer in some situations and cut into your fingers in others.
I've been using a similar script for quite a while now. Writing this post forced me to clean it up, write specs, and make sure it works with various versions of RSpec 3.X. You can find the code here. It has more bells and whistles. I might discuss some of them in the future.