This article is a follow-up to two earlier posts about automatic functional testing: Test automation for start-ups and How to run your Robot Framework test suite on Heroku.
In late 2014 after opening up our service to beta users we were faced with a dilemma that I suspect is pretty common to many startups. Automatic functional testing is almost a requirement to run a modern Agile software process. But building those capabilities takes time and money, both of which are in short supply at a typical software startup.
After setting up our initial functional testing system using Robot Framework we continued to build on that for a while. Writing test cases wasn't too difficult, and it was convenient to run the tests in the cloud. Our product didn't have that many features so it wasn't a big chore to keep the test cases up-to-date.
A few months ago we launched the self-service dashboard setup tools and started adding new features on a weekly basis. Tests started lagging behind because adding new cases meant more programming to our already busy schedules. We also had recurring issues with the stability of the Robot + PhantomJS combo that added lots of frustration to the process.
So we were back to square one. How to do automatic functional testing in a cost effective manner?
The promise of codeless test automation
The solution was closer than we thought.
Usetrace is Helsinki-based startup and they provide a tool that let's you create and run automatic tests on your web application – without writing any code.
Instead you teach the tool how to do the testing by recording workflows. Just check out the video below.
It's really that simple. The tool runs in the browser and the tests run in the cloud. Results are sent to your team's Slack channel.
You should aim to keep it simple too. We've found that the codeless approach works best when you select a set of the most important functions and create simple tests for those. For us it's stuff like login, sign up, setting up new dashboards; things that just have to work all the time. If you get into really complicated flows and testing edge cases, the advantages over scripting might not be that big.
The icing on the cake is how Usetrace handles flows with emails: Inviting new users, resetting passwords and so on. One video clip speaks louder than a thousand words.
How many hours of development time would it take to create that from scratch?
For bigger companies the benefits of codeless test automation are obvious. They can use their existing quality assurance staff, who are probably not programmers, to build automated test suites. So developers get to focus on building new features, but the team still gets all the goodness that test automation brings. Win-win.
For smaller companies like us the situation is different. The big thing is that creating and maintaining a baseline test suite is trivial compared to the alternative, which is scripting. For myself it's not just the time we have to spend developing test cases. Clicking away on a user interface I know by heart is so easy that I can do it even when I'd be too tired, or multi-tasking, for real programming. Actually I've created most of our Usetrace test cases late in the evening while watching football at home.
This article was written by Tuomas Tammi
Tuomas is the CTO and co-founder of Screenful, the company that turns data into stories. Screenful develops dashboards that are actionable and beautiful to look at.