Web regression testing: why move to cloud-based platforms?

My colleague once told me that regression testing is the single best reason why QA automation exists. According to the same person, doing web regression testing the right way is a major challenge in most projects.

So what is so challenging about it? And more importantly, why taking web regression testing to the cloud is a good idea?

Why do web teams have problems automating web regression testing?

Two words, “UI testing”. It’s common knowledge that the ROI of automated UI regression testing is low. I’m sure you’re familiar with the culprits of this problem:

• Even with skilled professionals onboard, most teams struggle with the maintenance of coded web regression tests.
• QA automation frameworks often lack efficient collaboration tools. In fact, few solutions will even feature shared integrated portals displaying test statuses.
• Existing testing environments require manual setup and maintenance. Specifically, frameworks often make you code the fundamentals of the framework itself before you can actually start writing your tests.

Take another look at this list, and you’ll see that most of the problems stem from the inherent shortcomings of the tools, not the process. So maybe it’s time we moved on to something different?

Web regression testing on the cloud: the case of AjaxSwing

What do most teams that struggle with UI testing automation have in common? They code tests using Selenium :-). In fact, my team had the exact same problem in the past. Trying to write Selenium tests for our product AjaxSwing was a painful experience, so we started searching for alternatives.

At some point, we found several cloud-based record-playback tools that seemed to get it right. While each platform had it’s shortcomings, they all shared the same major advantages. These cloud-based solutions ditched hand-coded UI tests. Besides, most of these tools had decent collaboration features available out of the box.

Visual tools for web regression testing

Long story short, none of the cloud-based solutions that existed at that point worked for us because of the small drawbacks each of them had. On the bright side, we got a chance to see what worked for real-life applications and what didn’t. So we picked the features we considered best and used them to build our own platform Screenster.

With Screenster, our team strived to cover the functionality we consider must-have for any testing tool that deals with web UIs. Namely, these are the features that you should expect from any cloud-based regression-testing platform:

1. Tests should cover the whole UI state, not just target separate elements

Targeting separate elements of the frequently changing UI with hundreds of coded tests is a one-way ticket to maintenance hell. Thing is, no matter how good you are at writing UI tests, these tests will never cover all the possible issues brought by CSS updates. So maybe it’d be wiser to leave this grunt work to the machine?

Instead of making you write hundreds of tests for dozens of UI components, platforms like Screenster scan the complete UI state of the page you’re on. And they can diff these ‘scans’ during the regression phase to guarantee that no issue goes unnoticed.

2. Collaboration functionality is paramount

Most projects involve teams, not just individuals. Given this fact, it’s weird that so many enterprise solutions neglect collaboration features. By definition, cloud-based solutions can store test suits on a shared portal. The exact set of collaboration tools available out of the box differs depending on a particular solution, so the best way to see which one suits you best is to play around with it.

3. No setup is the best setup

Decent cloud-based solutions do everything to minimize setup and installation pain. Ghost Inspector substitutes installation packages with a Chrome plugin paired with a web app. Besides, there are tools that offer a somewhat more sophisticated solutions by integrating a new menu into Chrome Developer Tools.

The complete functionality of Screenster is available via our web app — simple as that. In case you do need to run tests on premise, there is an option to set up a shared server. But even in that case, you’ll appreciate the simplicity of the process.

4. Web regression testing should be accessible for non-technical users

To me, the idea of hand-coding UI tests feels like hammering nails with a microscope. And having a professional programmer do this is akin to hiring a PhD to do the hammering. Most team leads will agree that it’s best to let programmers write features (and unit tests for the said features). So why don’t we all start doing just that?

With a decent cloud-based platform, manual QAs and non-technical folks can run 99% of your UI regression tests. When working on Screenster, we’ve put a strong focus on the smooth learning curve and user friendliness. Screenster doesn’t require programming knowledge, and it handles things like timeouts, locators and dynamic areas automatically.

5. Record-playback coupled with smart verification

I guess you’d pretty much expect your testing tool to do a lot more than basic screenshot comparison or basic record-playback. Differences verification is an obvious must-have, so is the ability to analyze and verify the DOM structure and the CSS of the on-page elements.

Bottom line: to code or not to code?

I have to admit it, cloud-based web testing tools like Screenster might make some QAs sceptical. After all, what’s the point in learning how to code if you can do just fine without code-based UI tests? But I’m sure that people who are able to see the bigger picture will appreciate the efficiency that cloud-based solutions offer.

Our team loves to code, but we want to code more features and less tests. The experience of using code-based tools for our product proved us that sometimes you have to look for a simpler solution. Reducing maintenance pain and allowing you to automate manual testing, Screenster offers 10x the productivity of a code-based UI testing solution. That’s the claim we make on our home page, and we sure stand by it.

So can UI testing teams ditch complex frameworks that require developers to write and maintain tests? The best way to see it works for you project is to try it yourself.