Selenium IDE alternatives for UI regression testingFebruary 3, 2017
Most of the QAs I know consider Selenium the number-one choice for UI test automation. I guess this is why Selenium IDE is the first thing that comes to mind when they start looking for a record-playback tool to automate UI regression testing. After all, if Selenium is so great for writing UI tests, it’s bound to have an equally great IDE, right?
Well, not quite…
If you have any experience with Selenium IDE, you’ll agree that it’s far from perfect. In fact, many testers will argue that Selenium IDE is doing you more harm than good. Reasons? Pick one!
- Selenium IDE doesn’t always record what you expect, and it still requires programming skills (at the very least, the knowledge of HTML and CSS).
- In a nutshell, the tests produced by Selenium IDE are poorly implemented WebDriver code which ends up being brittle and hard to maintain.
- Selenium IDE makes you handle timeouts manually which is a major inconvenience.
- Selenium IDE is no good for projects involving UI elements that change their IDs, location, and/or content dynamically. This means you can’t regression-test the UI of a web app with Selenium IDE.
- Selenium IDE only works with Firefox and may not work for sites or web apps that have slightly different DOM structure in Chrome, Safari, and IE.
In my humble opinion, Selenium IDE is something that you try while you’re starting out. You think it’s great at first, but once you’ve used it for a couple of hours, you’ll be looking for an alternative.
The good news is testing automation tools with record-replay functionality have been around since the 90s, so there actually are plenty of alternatives to Selenium IDE. But are these old-school solutions good enough?
Going old-school with UFT/QTP, Sahi or Rational Functional Tester
Or rather, “going old-school with UFT, Sahi, Rational Functional Tester, Ranorex, TestComplete, Telerik, and LeanFT”. I’m sure you can add a couple more names to this list. In fact, it’d be fun if you leave a comment with a name of an old-school platform with record-playback functionality — let’s see how many we’ll manage to collect;).
Jokes aside, here’s my question: given that there so many solutions have been available for well over a decade, why none of them presents a major competition for Selenium?
One reason is that most of these tools fail to deliver on the promise of usability and smooth learning curve. Take a look at the manuals for Sahi, Rational Functional Tester, and UFT, and you’ll see that even the most basic functionality will take weeks to master. To work with Sahi, RFT, or UFT, you’ll need a tester with solid programming knowledge who can understand and write complex code.
Besides, the only viable way to use and modify the tests recorded with Sahi, RFT, or UFT comes down to tinkering with the scripts these tools produce. These scripts tend to get messier with each new iteration of UI regression testing. Fast forward a couple of runs of your UI regression test suite, and you’ll regret not having hand-coded your test with Selenium in the first place.
Finally, RFT and UFT are expensive enterprise solutions that require local installation, certain servers, runtime licenses and lots of config. Being difficult to learn, hard to maintain, and costly, none of these solutions beats the open-source and free Selenium. End of story.
So no decent Selenium IDE alternative then? As a matter of fact, there are several
The problem with both Selenium IDE and its three old-school alternatives is that their record-playback functionality comes as just a part of the package. While speeding up the initial development of code-based tests with the recording, these tools result with low or negative ROI for code that has to be maintained by a human.
But this doesn’t mean that nobody else can get this right. In fact, there’s a whole bunch of modern cloud-based platforms built with a primary focus on record-playback functionality.
Instead of merely recording sequences of manual interactions with the UI, platforms of this kind analyze these interactions and use AI to free the tester from developing low level logic. Working with these platforms doesn’t require coding, which means manual testers or less technical staff can run and automate regression tests. And they can do this at speeds much faster than even the most productive automation testing engineer.
Let’s look at the three tools that fall into this group and see what they have to offer.
Screener is a neat, user-friendly Selenium IDE alternative that plugs directly into Chrome Developer Tools.
In contrast to the all-purpose nature of Selenium, this platform focuses on visual regression testing. Unlike Selenium IDE, it’s also accessible to non-programmers (yet it requires the knowledge of CSS selectors). Even though coding skills aren’t necessary, you can integrate scripted Selenium tests into your Screener test suits if you need to.
- What’s good: Chrome, Firefox and IE support, Ajax calls handling, smooth learning curve.
- What’s not so good: timeouts handling is cumbersome, and CSS knowledge is needed whenever you have to work with dynamic elements.
Ghost Inspector enables you to run functional and regression testing via a Chrome extension paired with a web app. Its tests take a bit longer to run compared to Screener. On the flip side, Ghost inspector offers a wider range of advanced features, and it seems to do more in terms of CI support and integration with third-party services.
- What’s good: test scheduling, Selenium support (added just a couple of days ago), video recording, flexible pricing plan, lots of advanced settings and editing features available for experienced users.
- What’s not so good: test runs are limited to 10 minutes, and the said Selenium support basically comes down to exporting tests suits in a Selenium-friendly format. Also, no visual comparison, and verifications are handled manually (which decreases speed and ROI).
While it’s easy to notice that the name Screenster is reminiscent of a previously mentioned platform, this tool offers a more user-friendly take on visual regression testing. Screenster puts greater focus on simplicity with automatic handling of timeouts, locators, and dynamic regions.
Just like Screener and Ghost Inspector, Screenster is cloud-based, but you can also run it on premise. Unlike its two competitors, Screenster doesn’t require browser extensions or plugins.
- What’s good: automatic handling of timeouts, dynamic regions, and locators; crazy high productivity compared to tools, simple CI integration; ability to run tests on the cloud and on premise.
- What’s not so good: its UX is a still work in progress; IE support requires a Windows server.
Bottom Line: what to expect from a modern regression testing platform?
Obviously, there are dozens of other Selenium IDE alternatives that you might want to try. To be suitable for real-world regression testing needs, an alternative to Selenium IDE needs to address the drawbacks of both Selenium IDE and the alternatives mentioned above:
- Ideally, a record-replay tool shouldn’t require programming knowledge, and it should decrease the testers’ dependence on developers.
- Has to be web-based. Come on, people, it’s 2017.
- It needs to work right out of the box and produce reliable tests that are easy to maintain.
- It should automatically handle dynamic UI areas and timeouts.
- Collaboration features like link sharing, commenting, dashboards.
- Visual testing that can ensure correct layouts, images and CSS.
- High ROI: if it takes more than 30 minutes to automate a test, abandon the ship!
The three cloud-based alternatives mentioned in this post cover each of these points, but they address some of them in a slightly different way.
So how do you decide on which one is optimal for your project? I guess there no better way than trying them yourself.
P.S. My personal favorite is Screenster because it’s super-easy to start with, and it’s full of magic. And who doesn’t like magic?😉