Testing a Rails Project Structure, Dependencies and More

At Handshake we rely heavily on automated testing. This of course includes unit tests, integration tests, and UI testing which one might consider the “normal” tests for a web application. However, tests and specs are a very powerful tool that can go beyond just “normal”. Especially as an engineering team grows, writing specs for goals such as enforcing project structure, proper dependency management, and feature toggle clean up become incredibly helpful. Below is a quick outline of some of the additional behaviors we test in our automated test suite.

Project Structure

Recently while contributing to the wonderful Rubocop project, I was pleasantly surprised to run into a test failure in their “project spec” file. For their use case, the project spec made sure that the default configurations were valid and fully inclusive and also that the CHANGELOG file was properly formatted. Awesome! After quickly fixing the CHANGELOG entry, inspiration hit to add a similar spec to some of our larger services and applications.

An example project structure spec

Like Rubocop, we have now added numerous specs to enforce proper project structure. Let’s take a look at an example project structure spec:

context 'models' do   
  before(:context) do   
    model_files = Dir.glob(Rails.root.join('app/models/*'))  
    skipped_files = %w(time_slot settings application_record)  

    file_names = model_files.map do |file_name|  
      File.basename(file_name, '.*')   
    end   
    relevant_file_names = file_names.reject do |file|   
      file.in?(skipped_files)   
    end   

    @relevant_classes = relevant_file_names.map { |name| name.classify.constantize }   
  end 

  it "ensures that all models inherit from ApplicationRecord" do  
    @relevant_classes.each do |klass|   
      expect(klass.superclass).to eq(ApplicationRecord), "Expected #{klass.name} to inherit from ApplicationRecord"   
    end   
  end   
end

First, we iterate through our models directory and generate an array of relevant classes. In our case, we skip a few that are not relevant to this spec. Next, we iterate through each class and make sure that it inherits from ApplicationRecord. Although ApplicationRecord is a default in Rails 5 with its generators, in Rails 4 the generators generate models which inherit from ActiveRecord::Base rather than ApplicationRecord, so this was easily missed.

Simple as that! Some other examples of project specs are:

Ensure proper inheritance of ApplicationController, ApplicationMailer, etc.
Ensure that each model has an ActiveModelSerializer serializer
Ensure that each model spec has a unit test file and a factory
Ensure that for each controller a request spec file is defined
Ensure that each resource has a permission spec to test access controls for specific resources

Project structure specs: Just do them

Within about an hour, we had an exhaustive spec for ensuring any changes to the project follow the proper structure. One of the biggest win with this addition is for developers who choose not to use the built-in rails generators for adding new features or scaffolds. Previously, some non-critical files may have been forgotten without using generators, which is now not an issue (and engineers get to choose what works best for them!). We expect this change to also be particularly beneficial for on-boarding new engineers.

Gemfile

Rails has a vast and mature ecosystem, and with that comes bundled solutions to many common problems in the form of a gem. With such a strong emphasis on testing in the ruby and rails community, it is often safe to use gems. However, in some cases we have enforced policies regarding dependencies.

Disallowing gems

Although rare, in a couple of cases we have decided to explicitly blacklist certain gems from our Gemfile.

One of the gems we blacklisted is libxml-ruby. After a long and difficult debugging session, we found libxml-ruby and nokogiri do not play well together and cause seemingly random heap corruption. We removed our dependency on libxml-ruby and added the below spec (with verbose comments) to ensure it does not get added back:

it 'should not include libxml-ruby' do 
  libxml = Bundler.locked_gems.specs.detect { |s| s.name == 'libxml-ruby' } 
  expect(libxml).to be_nil, "libxml-ruby is incompatible with nokogiri and causes heap corruption" 
end

For more details on the libxml-ruby and nokogiri issue, check out this blog post.

Gem licenses

With the help of the LicenseFinder gem, we have a complete and versioned history of our dependency choices. The LicenseFinder gem automatically detects the License for each of your Gemfile dependencies. By running bundle exec license_finder the gem can automatically find any gems which have not been approved for usage. To approve a gem, you can either whitelist an entire class of gems (for example, all gems with MIT license) or approve specific gems for usage. Each approval documents who and why the library was approved. By adding this to our CI, we can rest assured all our gems have an acceptable license.

Bundle audit

One way that we keep up to date and informed on ruby gem vulnerabilities is through the bundler-audit gem. The bundler-audit gem will review your Gemfile.lock for gem versions w/ vulnerabilities as well as for insecure gem fetching.

Feature Toggle Cleanup

Along with automated testing and continuous delivery, we are big proponents of feature toggles. Feature toggles allow us to continuously deploy improvements in small, low risk pieces while keeping the change behind a feature toggle until it is ready.

Eventually, many (if not all) of our feature toggles get cleaned up and removed once the feature is finished. In order to make sure that feature toggles do not hang around for too long, we require all feature toggles to be defined with metadata about when the toggle is expected to no longer be needed (among other things such as the toggle name). Then, in a nightly build, we make sure that there are no expired toggles. This has helped us to stay on top of our toggles to ensure they do not linger.

Conclusion

Automated testing is a crucial and amazing tool for fast moving and correct software. Don’t limit your automated tests to just unit and integration testing — give these additional tests and specs a try on your project!