Designing & Working with a Ruby on Rails Project Audit

At Wolfpack Digital, delivering high-quality software products is our mission, and we strive towards that every day. Quality stands out when working on UI/UX designs, product development, or delivering quality client experiences, and is easy to recognize. But when we are talking about high-quality software, especially for non-technical people, quality is very hard to define.

But that doesn’t stop us! We know that having good software helps future development, squashing bugs or even straight up avoiding them. We also know that cutting corners can slowly creep on you, and the next time something needs to be done, it’ll be exponentially harder.

Defining the process

Our projects are extremely different, ranging from under a year to 6+ years and still going, from basic security needs to regularly audited, from smaller teams (single backend dev + a technical assist) to teams of 4-5 web developers. Each project has its own specific needs, and we quickly realized that we need to find a process that would:

• Easily frame the common needs of all (or as many as possible) active projects;
• Also satisfy the budget needs of each project - some may have tighter budgets or a tighter time frame, which would allow less time for reviewing, auditing & refactoring;
• Define a minimum set of requirements that would cover not only code style but also security, testing & system design;
• Be easy to use, adapt and monitor; we wanted this to become a habit in our projects’ lifecycle, so we need this framework to have a clear, versatile, and measurable output.

It’s important to note that we did not start from scratch. We had a well-designed set of guidelines that started with a Rails template (that we intend to more actively update for the latest Ruby & Rails versions) and other rules like the preferred testing and documentation libraries or database and environment setup best practices. With all these, we felt like in order to evolve, we still need to measure and improve, and we need to be a lot more organized when doing so.

The result

The final framework has two parts:

• A very detailed guideline that includes tools, code bits, and rules for architecture, file structuring, naming, a custom rubocop config, git & branching, and testing;
• A more basic process that should fit in less than 2h, every 6-12 months (depending on the project); this will be the main topic for this article.

The Basic Code Compliance Check (as we call it internally) is a set of common sense rules that should be applied to all of our projects. We decided to implement this by having a yearly or bi-yearly meeting for each project to complete this check. The meeting involves a Technical assist (which each project already has - this developer is responsible for supporting complex technical decisions, code reviews, and sometimes pair programming) and one of the main developers on the project. The output of the meeting should be an email with the exact percentage of the rules completion (how many of them have passed the review vs. how many need some work) and a list with todos, ordered by priority. A chapter will be dedicated to each of these rules, ordered by their priority. I will explain both the review process and the reasoning behind them + a potential action point.

Vulnerability checks

This is the highest priority rule by far. Although it might be self-explanatory, it doesn’t get enough attention. Making sure your gems versions aren’t open to known vulnerabilities and your code doesn’t have issues (like allowing SQL injection).

The review process for this rule goes through 4 checks:

1. Static analysis for security vulnerabilities with Brakeman;

    a. this is pretty simple; we only need the gem in the development group, and to run bundle exec brakeman -q

2. Patch-level verification for each gem inside our Bundle, with bundle-audit;

    a. also minimum level of implementation for this check: running bundle exec bundle-audit check --update -v

3. Production environment log level - this one should not be debug, we should rather use info; 

4. Access to sensitive data check - for example, access tokens and other sensitive information should be filtered inside VCR cassettes or log files; if you know you might use some type of export or log, this check is required.

Having these completed means both having brakeman & bundle-audit inside the project (Gemfile & even running them regularly inside a CI job - more details about this below), but also making sure that no warnings are thrown. Ideally, brakeman & bundle-audit are also up to date with the latest stable version available.

Action points for this part are pretty straightforward: making sure that these gems are added to the project, their audit completes without warnings, no sensitive data is easy to access or read in exported files, and the log level in production is correctly set up.

Testing

As mentioned above, this is something that we already had some guidelines on. We do enforce writing unit tests on all backend projects, and we do recommend using rspec, but we didn’t have any review process set up for coverage. So, as you probably guessed, the only check we do at this point is test coverage.

Although we strive for 100% coverage, we know that it is probably out of reach on all projects. Also, depending on the size and the timeframe of the project, test coverage might look very different.

The only 2 rules that we enforce on every project are:

• All projects must have a test suite for controllers (requests), models, and services;
• All of these should have at least 75% test coverage;
• As recommendations and general company-wide guidelines, we recommend using rspec for the suite and simplecov for coverage.

As not all projects might already have simplecov set up, we also included a short summary for that:

• Add the simplecov gem to the Gemfile - ideally only development + test (optional);
• Update the spec/rails_helper.rb file with the following snippet;

• Run the tests with the simplecov environment variable set to true: RUN_SIMPLE_COV=true rspec spec;
• A new local folder will be generated, with an /coverage/index.html file available; that file will cover the results;
• By default, there won’t be any specific group for services; in order to make that easier to read, we recommend also adding this line below the simplecov filters: add_group('services', 'app/services').

The action points for this chapter are also pretty straightforward: write some tests 😅 but unfortunately, coming back from a poorly tested project to a 75% coverage is pretty hard, especially if the project is bigger and more complex. So my recommendation would be to just take it step by step. For every new feature, make sure you write tests. If refactoring or working on a previously implemented feature, make sure to write tests. Otherwise, start with simpler controllers and models, and define a priority: first, critical & most used functionality, then validations, and ultimately everything else; also, it might make more sense to write tests for bits that are more complex or undergo regular changes, to make your life easier, rather than simple endpoints.

Code style

This is definitely the most opinionated check on this list, which also made it the trickiest. We needed to find a set of basic rules that should apply to most projects and that people can then adapt to their own preferences while also respecting the minimum standard. Also, as a short disclaimer, you probably won’t be able to implement this in the same way in your team, and that’s ok; you just need to define together a set of rules that works for everyone.

There are 4 steps that we need to go through:

• Making sure that Rubocop is set on the project and we also have a Rubocop.yml file for custom rules (our Rails template project has an example of what we usually use); especially on bigger projects, there will be some exceptions to the Rubocop rules you set up; but we should still leave the rule and mark the exception, to avoid having multiple hard to read classes/methods/etc..;
• Making sure that Overcommit is up and running, including a command to run the Rubocop check, so committing without passing the style guide will be restricted;
• Reviewing the Rails credentials implementation; we usually go through this guide, but this is a very flexible check, as you can also use environment variables; the main purpose here is to make sure that no secrets can be easily read without access to a master key or the server;
• Check for comments about deprecations and old ToDos that were long forgotten.

Most of the action points for this chapter can be completed during the meeting, like setting up Rubocop & Overcommit. The rest should just be some tasks to take care of the Rubocop warnings, old comments, or setting up a secrets implementation.

Git & CI

This chapter is also very reliant on the tools that you are working with. On most of our projects, we are using Github as our git platform of choice and CircleCI for running our CI/CD workflow. That being said, the general process for setting these 2 up will be the same, and the checks should still be there, regardless of the tool.

Our main points when reviewing this are:

• Branch naming & commit messages to follow a set of rules; as a recurring theme in this article, this is something that differs from project to project, but the most important thing is to have a guideline that everyone will follow, even if the project has the structure of git flow or trunk based development; for example, on most of our projects we do recommend prepending the Jira ticket id to the branch name & PR title;
• CI set up, that runs at least these 3 simple steps: rubocop, test suite & the vulnerability checks mentioned in the first chapter; more complex projects should also have set up a deployment process;
• Go through the git repository settings and make sure that no one can merge until a PR is open, it received at least one approval, and the CI flow has passed.

As for the action points, the git repo rules can be set up in only a few minutes, and as long as the organization has a CI account, a setup for that should also be started (we also have a template for a CircleCI config file).

Database optimizations

This chapter should probably be called n+1 queries. There are many optimizations that one can do. Still, they are very situational and usually require a great amount of time to realize if the effort is worth it (especially on smaller projects). Taking this into account, one of the few checks that should be present on all projects is making sure that no flow has an n+1 query issue.

The steps to do this check correctly is:

• Install bullet (or any other n+1 query checker) and enable it for the test environment;

• Run all the tests and, as an action point, solve the errors thrown by Bullet.

Conclusions

Hopefully, this article will help you define a guideline for yourself and your team. The main purpose of this content was to be a blueprint. For example, we use it as a bi-yearly review process that also exports a percentage of completion for monitoring project quality progress. Ultimately, you might want to review this more often, change some of the checks or even add some more.