Eric Mann reveals how to use Jenkins to automate your production test suite, helping to ensure you ship bug-free code.
Mark was excited. Just one more ticket to close and he could duck out early for a weekend on the lake. Just him and the fish: no Wi-Fi, no IDE, no client meetings. Three lines of code and the bug would be patched. A Git push and the new code would be online and the client would be happy. Simple.
Mark loaded his local dev site to confirm those three lines fixed the bug. Success! He wrote a quick commit message and pushed the code up the chain. Mark smiled as he shut the lid of his laptop, already thinking about the last-minute items he needed to pack for the trip. Next stop: freedom!
Except the code now living in production only worked on Mark's machine. A library required by the project had been accidentally gitignore-d and, while it happily ran for Mark, never made it to the server during his deployment. The site updated, crashed, and Mark's team went on red alert immediately when the client panicked. It took all weekend to track down the missing file. On Monday morning, Mark came home to an angry dev team, an angrier client, and a sudden impulse to refresh his resume.
Horror Stories
This story might seem fictional but, aside from 'Mark', it's one that I, like most developers, have experienced many times, from both sides of the team.
But solutions are lacking. Instead of real changes to our infrastructure, we try to teach good behaviours. As a policy, all code must be peer-reviewed before being checked off in the ticketing system. No deployments are ever made on a Friday. Engineers have to earn the right to work on the master branch.
These protections are a noble first step to solving the problem. Unfortunately, humans make mistakes, steps are skipped when deadlines loom, and dictated behavioural practices usually prove to be mere Band-Aids that often fail to stop the bleeding.
Continuous Integration
A better solution is to remove human error from the equation and automate these pre-deployment best practices. Through unit testing, we can be absolutely sure that any changes in our code don't break the build before it ships. Typos triggering white-pages-of-death or server-side 5XX errors become a distant memory. Stack traces printed above call-to-action forms are a thing of the past. Requiring unit tests helps ensure projects work correctly before anything ever leaves a developer's machine – most of the time.
Unit testing is great for automatically ensuring the integrity of code on one machine, but it falls short of fixing the "it worked for me" issue illustrated above. Rather, we need to distribute our testing code to the entire team while also ensuring it's actually run in multiple environments.
Enter Jenkins, an open source continuous integration server. Thanks to Jenkins, a common build server sits between the team and the production environment, automatically testing every commit before it goes live. Jenkins can also enforce generally accepted best practices in code quality.
How it Works
Code can be as broken as it needs to be locally, but once a commit is pushed up to a central repository, Jenkins runs your build processes automatically.
Jenkins runs unit tests to inspect code functionality. It can also run code linting tools to analyse code style and performance. It can even generate API documentation so the team (or external contributors) can keep up with project progress.
Determining What to Automate
Before any workflow can be automated, you have to establish a consistent checklist of actions. Not everything has to be automated right away: your checklist could be as simple as just running a code linting tool. Start with the most frequently executed tasks. They consume the most time and will be the largest early automation win.
For a PHP project, this task list probably includes:
Run all unit tests for the project
Run PHP_CodeSniffer to ensure code quality
Run phpDocumentor to generate documentation for your API
Push to a gh-pages branch on GitHub
Run a Selenium test suite to validate UX delivery
Deploy the code to the production database
If Jenkins encounters a failure in any of the steps above, it can – and should – immediately halt the build and notify the team that something is wrong.
Itinerant Jenkins
While the first step in the build process above might be testing, the first step in automation is, of course, to install Jenkins. For this, my team has put together a simple tool that allows for the quick construction and deployment of disposable Jenkins servers: Itinerant Jenkins. The tool can run either locally (powered by a Vagrant server) or on a remote machine thanks to some general-purpose provisioning scripts.
From within the newly-created folder, run vagrant up
The project will use Vagrant to download and create a new Ubuntu VM on your local machine. Then it kicks off an Ansible-powered provisioning system to install Jenkins. Jenkins can be configured to run tests for any framework in any language. Out of the box, though, the Itinerant Jenkins project gives you:
PHP 5.6 (CLI)
XDebug (xdebug.org)
Composer (getcomposer.org)
PHPUnit (gphpunit.de)
phpDocumentor (phpdoc.org)
Once the provisioner completes, you'll have a fully-functioning Jenkins server running on http://jenkins.dev:8080. Itinerant Jenkins pre-installs plugins to support PHP unit testing and code coverage reports, too!
Running remotely
Itinerant Jenkins also allows you to spin up a remote build server with ease. If you've cloned the repository locally, you've got everything you need already. Just find an Ubuntu server somewhere (Rackspace, Digital Ocean, Linode, and so on) and make sure that you have a user account with passwordless sudo capabilities.
Next, run the following command from your Itinerant Jenkins directory:
bin/deploy {user}@{host}
Itinerant Jenkins will use SSH to push the same Ansible provisioning tools used by Vagrant to the remote server and will kick off a remote build to install Jenkins and its tools for you.
Once the deployment script completes, you'll have a fully functioning Jenkins server running at http://{host}:8080.
Next Steps
Itinerant Jenkins is meant to be disposable, but Jenkins itself is indispensable. You can use the tool to quickly provision and deploy build servers anywhere, for any team, and for any project.
Once Jenkins is up and running, your next step is to gradually work through your build check list and shift the burden of each task off your team and onto your new teammate. Humans can and will forget steps – and break the build in the process.
Jenkins, though, runs every task without complaint every time without fail. Through continuous integration, you move one step closer to every developer's dream: only ever shipping bug-free code.
Eric Mann is lead web engineer at 10up. His areas of expertise include HTML, CSS, PHP, JavaScript (browser and Node.js) and WordPress. This article was originally published in issue 273 of net magazine.
