Top-5 Selenium alternatives in 2018June 6, 2018
Is there even such a thing as a real Selenium alternative? Ask any tester to recommend one, and with a 99% probability, they’ll name a tool based on Selenium WebDriver. Besides, how many companies using real Selenium WebDriver alternatives can you name? Something tells me the answer is zero.
That said, there’s whole bunch of higher-level WebDriver-based tools that do the job better than Selenium. Available as frameworks, desktop tool, and web platforms, these Selenium competitors address various issues of coding straight to the WebDriver API. And let’s face it, there are quite a few issues. In my experience, the following 5 pain points are near deal-breakers for automation with Selenium.
1. Good tests require good programminging skills.
I’ve seen many talented Selenium engineers. All of them could become awesome developers should they decide to switch some day. I’m sure everyone would benefit if skilled programmers spent time building features, not tests.
2. Test maintenance is a drag.
The more tests you write, the more time you’ll spend updating them for new features, fixing broken tests, and handling false positives. This is true for any hand-coded UI test. My problem with Selenium, though, is that it offers nothing for handling element locators, page screenshots, and error capturing. At the and of the day, it’s just too low-level.
3. Selenium test suites take too long to run.
If you aim to speed up your test runs, there are no real alternatives to Selenium Grid. The problem is very few people actually set it up and again, you need quite a bit of skill to keep that beast running.
4. Selenium IDE lacks crucial functionality.
The original Selenium IDE was too simplistic for real-life production purposes, which explains why the community nearly abandoned it. Unfortunately, the new revamped versions of Selenium IDE for Chrome and Firefox 55+ look just as limiting as the old version.
5. Low ROI.
It all comes to ROI, and Senium has never been cost-effective. Even with very capable technical test automation engineers on board, the productivity is low.
What Selenium Alternatives are there?
I know what you’re thinking. Sure, it’s easy to complain about the imperfect world of test automation. But unless you can offer something better, this is just whining. Well, I’m here to share some of the promising alternatives to Selenium that help people get to a better state.
IMHO the main problem with Selenium is that it’s just too low-level. So if you want to increase the ROI, then I think you should consider picking up a higher-level framework or tool. Based on what I’ve seen in tech startups and product companies, five options are popular in 2018. You can find a high-level overview of the five tools below or read-on for in-depth reviews.
|Protractor||E2E testing framework||Required||Advanced locators and waits for Angular, faster test authoring compared to Selenium|
|Cucumber||BDD framework||Required||BDD syntax, plain-language tests, living documentations|
|Katalon Studio||Desktop record/playback IDE||Required||Record/playback with page object models support|
|Ghost Inspector||Cloud platform for UI testing||Optional||Record/playback with screenshot comparison, click-based assertions, automatic waits, codeless test maintenance|
|Screenster||Cloud platform for UI testing||Optional||Visual testing, automatic verification, self-healing locators, automatic waits, codeless test maintenance|
Powered by WebDriverJS, Protractor primarily targets end-to-end testing of Angular apps. For instance, it offers a neat locators mechanism for Angular repeaters. In addition, Protractor has accessors for models, bindings, and ng-options. There’s also waitForAngular, a waits handling method for dealing with asynchronously loaded UI elements.
Angular stuff aside, the concise and well-structured syntax of Protractor makes for a somewhat faster alternative to Selenium. No framework can make the browser run faster, but, at least, test authoring becomes less time-consuming. Protractor has good documentation, too, and it works well with test popular runners (e.g. Karma) and unit testing frameworks (e.g. Mocha or Jasmine). Naturally, it’s perfectly usable outside of the Angular realm.
On other hand, the fundamental issues that you’d run in Selenium are still the same. A lot of setup, testers have to be capable programmers, every action emulation and verification has to be coded manually. Locators are still brittle, and there’s no CSS or visual verification of any sort. All of these decrease the ROI of UI automation.
Another open source alternative to Selenium, Cucumber has made waves in the BDD community. Used at IBM, Groupon, and other giants, it remains a mainstream automation tool even in 2018, nine years after its release.
Cucumber bridges the gap between technical and non-technical project members. In essence, that’s the key ingredient of its secret sauce. In Cucumber, acceptance tests, functional requirements, and documentation merge into a single automatically-updated source of truth for testers and stakeholders.
Technically speaking, Cucumber can both act as a Selenium alternative or work in tandem with Selenium. But in both cases, it has its shortcomings, though. The Gherkin syntax is accessible to non-techies, yet it’s a lot wordier than normal code. Programmers would also argue that it discourages code reuse.
And again, the same-old issues are still there. Cucumber requires programming, labor-intensive maintenance, and special training — even more so than Selenium. While Cucumber acceptance tests are technically UI-driven, there’s also no built-in visual testing.
3. Katalon Studio
If you think about it, Protractor and Cucumber share common pitfalls characteristic of any framework. So maybe a full-fledged automation tool will make for a better Selenium alternative?
Katalon Studio seems like a worthy contender in this department. Launched in 2016, the platform uses Selenium and Appium under the hood, combined with keyword-driven testing and basic record/playback.
Katalon Studio allows for codeless test authoring, which simplifies and speeds up automation. In addition to the full-fledged platform, Katalon offers a lightweight Selenium IDE substitute called Katalon Automation Recorder. This tools is as simplistic as Selenium IDE, but it seems more stable in both Chrome and Firefox.
While codeless test authoring sounds nice, the catch is you have to maintain auto-generated test code. According to the Katalon team, auto-generated code is fairly readable in Katalon Studio, less so in Katalon Automation Recorder. Yet in my experience, in 100% of cases of dealing with auto-generated code, you’ll regret not having handwritten the thing in the first place.
4. Ghost Inspector
Another Selenium IDE alternative that seems to get record/playback right is Ghost Inspector.
There are limitations, as well. Ghost Inspector relies on manual assertions, making you click on the elements you target. This looks like a minor nuisance, but it has an important implication: Ghost Inspector doesn’t really verify the UI. Instead, UI comparison boils down to pixel matching, which is fraught with false positives and missed bugs.
Screenster is a Selenium alternative that introduces the concept of record-playback-verification to UI testing. Screenster evolved from a visual regression testing tool that my team developed internally for our own projects. In 2016, we released it as a standalone product.
Created for less technical people, Screenster transforms the record/playback paradigm found in modern Selenium IDE alternatives. By adding intelligent verification of each on-page element, out solution overcomes the major limitations of UI automation.
UI screenshots comparison makes little sense unless you can recognize the underlying DOM. This functionality is only beginning to surface in some AI-based automation tools whereas we’ve had it for years.
When recording a UI test, Screenster analyzes the DOM and matches individual UI elements to how they render on the screen. This way, we can automatically verify every on-page element.
This also enables a ton of useful functionality. Namely, Screenster can differentiate between CSS and content-related bugs, recognize text and dynamic content, generate self-healing locators, handle timeouts, and identify dynamically-changing UI regions. More importantly, our algorithm eliminates 99% of false positives while keeping comparisons pixel-perfect.
For each click, text input, or other user action during test recording, Screenster creates a baseline, capturing the UI screenshot and the page DOM. It then compares these baselines to new versions of the UI.
In contrast to tools that pixel-match full-page screenshots, Screenster can zoom in on individual elements. When analyzing elements, it also looks at content, filters out anti-aliasing, and ignores dynamic UI areas. This method is more precise, and with fewer false positives than traditional screenshot comparison.
There is nothing for testers to install locally. I’m not just talking about desktop apps. Unlike Selenium IDE, Screenster doesn’t make you install browser plugins. Knowing that browsers sometimes drop the support for extensions formats, a full-fledged web app is a more future-proof option.
With Screenster, all testers and developers use a shared server running on premise or on the cloud. And there aren’t any manuals to read either (but we do have a detailed documentation in on our website).
Low-code automation (with support for codeless and hand-written tests)
Smart Maintenance with self-healing selectors
Screenster has self-healing selectors that prevent moved or changed elements from breaking tests. When Screenster captures a UI baseline, each UI element get a complex selector that stores its ID and other attributes, content, full XPath and CSS. So even in case of a renamed ID, Screenster will still locate the element using the stored information.
Should a test run detect expected differences, they can be approved to the baseline with one click. In case of a UI bug, there’s Jira integration for hassle-free bug reporting.
Screenster automatically detects page transitions and handles AJAX updates. No need to code timeouts and deal with complexities of asynchronous programming.
The prerequisite for using Screenster is that UI state is repeatable, meaning that repeat executions of the same tests produce the same result. The approach with codeless visual tests may not cover all use cases but it should produce a significantly better ROI for 95% of the commonly covered scenarios. At the end of the day, it’s the focus on higher ROI that makes for a good Selenium alternative.