Minimum Viable Test Suite

A minimum viable test suite fails when important holes in its coverage are identified, and with Rails applications the most important holes are untested routes. Here's a quick script to ensure every route is hit during a full run of rspec:

# spec/support/minimum_viable_test_suite.rb

# singleton class that tracks hit routes
class ControllerCoverageSupport
  attr_reader :hits

  def initialize
    @hits = Set.new
  end

  def hit!(controller, action)
    @hits << "#{controller}##{action}"
  end

  def endpoints
    @endpoints ||= Rails.application.routes.routes.each_with_object(Set.new) do |route, set|
      controller = route.defaults[:controller]
      action = route.defaults[:action]
      next unless controller && action

      set << "#{controller}##{action}"
    end
  end

  def misses
    endpoints - hits
  end

  class << self
    def instance
      @instance ||= new
    end
  end
end

# listen to action controller
ActiveSupport::Notifications.subscribe(/process_action.action_controller/) do |*args|
  event = ActiveSupport::Notifications::Event.new(*args)
  controller = event.payload[:controller]
  controller = controller[0..-"Controller".length - 1].underscore
  action = event.payload[:action]

  ControllerCoverageSupport.instance.hit!(controller, action)
end

# fail the test suite if you ran the entire test suite and missed any routes
at_exit do
  coverage = ControllerCoverageSupport.instance

  # ARGV is empty when running `rspec` as opposed to `rspec ./spec/my_spec.rb`
  if ARGV.empty? && coverage.misses.any?
    Rails.logger.debug { "Only #{coverage.hits.count} of #{coverage.endpoints.count} endpoints were hit" }
    abort "  Missed controller action(s): #{ControllerCoverageSupport.instance.misses.to_a.sort.join(", ")}"
  end
end

Not the prettiest code in the world, but it gets the job done: rspec will now fail when any routes are missed. Of course, it doesn't ensure the routes are exhaustively tested. Tools like simplecov can help that but automating failure there is tricky:

Placing a minimum coverage threshold just sets a ticking timebomb waiting for one unlucky commit
Ensuring commits don't decrease the code coverage triggers when removing well-covered code

Convincing your team to enforce a minimum viable test suite should be easy: it's hard to argue testing an endpoint is unnecessary. Here are a few more rules I could see enforcing:

Every ActiveJob::Base subclass is performed
Every ActiveRecord::Base subclass is created, updated, and destroyed
Every app/views template is rendered

Let me know at dan@cunning.cc what you think of minimum viable test suites:

Are there existing solutions?
What other rules would you enforce?
Are you interested in a cleaner implementation published as a gem?

Minimum Viable Test Suite

Dan Cunning