How to run your Robot Framework test suite on Heroku

This is the second part of our series of posts about writing automated tests with Robot Framework. The first one was Test automation for start-ups. 

After finishing our test automation project, we wanted deploy the tests to Heroku. Like our apps. This turned out to be a bit challenging because of the way Heroku works. I hope this post helps others who want to do the same!

We put up an example project on GitHub. Go on, take a peek.

Looks like a normal small Robot project, right?

There are two files that might not look familiar. 

requirements.txt specifies the project's dependencies for pip, which is a dependency manager for Python. Heroku uses this file for two purposes: Detecting that the app is indeed a Python app and loading the dependencies.

By the way there's a good tutorial on running Python apps on Heroku here. Read it now if you're not familiar with running Python apps on Heroku. 

How about .buildpacks then?

Not to go in too deep to Heroku's internals, but buildpacks are responsible for how apps are detected, built and deployed. At the time of writing phantomjs can't be used with the default Python buildpack. So we need to use two external build packs, which are specified in the file.

This would be good moment to try the test project for yourself. You need to have a Heroku account and the Heroku command-line tools installed first, though. Once you have those, just run the following commands.

Excellent! Now you know how to run Robot tests on Heroku. There are few additional tools that might come in handy.

The standard Heroku Scheduler is convenient for running the test cases at predefined intervals. In our case we run the test suite once a day. Just give the following commands

and add test jobs using the scheduler web app.

One of the features of Heroku is that server instances, called dynos, are transient. All local data is guaranteed to get deleted at least once a day. Since testing doesn't make much sense if you can't read the reports they need to be stored somewhere. 

Our solution is to have the test runner script copy the test log to our Amazon S3 bucket. You can enable it by setting the S3_URL env var for your app. Like this.

Happy testing!