Top-5 Selenium alternatives in 2018

Whenever someone talks about Selenium alternatives, a frequent question is whether these alternatives exist at all. Ask any tester to recommend one such alternative, and with a 99% probability, they’ll name a tool based on Selenium WebDriver. Besides, how many companies using real Selenium WebDriver alternatives can you name?

Nevertheless, there’s a whole bunch of higher-level WebDriver-based tools that do the job better than Selenium. Available as frameworks, desktop applications, and web platforms, these Selenium competitors address various issues of coding straight to the WebDriver API. And let’s face it, there are quite a few issues. In my experience, the following 5 pain points are near deal-breakers for automation with Selenium.

1. Good tests require good programming skills.

I’ve known a good dozen talented Selenium engineers. All of them could become awesome developers should they decide to switch over some day. I’m sure everyone would benefit if skilled programmers spent time building features, not tests.

2. Test maintenance is a drag.

The more tests you write, the more time you’ll spend updating them for new features, fixing broken tests, and handling false positives. This is true for any hand-coded UI test. My problem with Selenium, though, is that it offers nothing for handling element locators, page screenshots, and error capturing. At the and of the day, it’s just too low-level.

3. Selenium test suites take too long to run.

If you aim to speed up your test runs, there are no real alternatives to Selenium Grid. The problem is very few people actually set it up and again, you need quite a bit of skill to keep that beast running.

4. Selenium IDE lacks crucial functionality.

The original Selenium IDE was too simplistic for real-life production purposes, which explains why the community nearly abandoned it. Unfortunately, the new revamped versions of Selenium IDE for Chrome and Firefox 55+ look just as limiting as the old version.

5. Low ROI.

Let’s face facts, Selenium has never been cost-effective. Even with very capable technical test automation engineers on board, the productivity is low.

What Selenium Alternatives are there?

Wait. I know what you’re thinking. Sure, it’s easy to complain about our imperfect world of UI test automation. But unless you can offer something better, this is just whining. Well, I’m here to share some of the promising alternatives to Selenium that help people get to a better state.

IMHO the main problem with Selenium is that it’s way too low-level. So if you want to increase the ROI, then I think you should consider picking up a higher-level framework or tool. Based on what I’ve seen in tech startups and product companies, five options are popular in 2018. You can find a high-level overview of the five tools below or read-on for in-depth reviews.

1. Protractor

Overview: an Angular-friendly framework for end-to-end testing with JavaScript.

Pros:

• Offers utilities for Angular UIs (e.g. locators for ng-model, ng-bind, ng-repeat, etc).
• Simultaneous test runs in multiple browser instances.
• Works for Angular and non-Angular UIs.
• Better productivity compared to Selenium due to Protractor being more high-level.

Cons:

• Fundamentally the same issues as with Selenium: painstaking setup, hand-coded verifications, lack of visual testing, brittle locators.
• Protractor only supports JavaScript.

Review:

Powered by WebDriverJS, Protractor primarily targets end-to-end testing of Angular apps. For instance, it offers a neat locators mechanism for Angular repeaters, as well as models, bindings, and ng-options.

Angular stuff aside, the concise and well-structured syntax of Protractor makes for a somewhat faster alternative to Selenium. No framework can make the browser run faster, but, at least, test authoring is less time-consuming. Protractor has good documentation, too, and it works well with popular test runners (e.g. Karma) and unit testing frameworks (e.g. Mocha or Jasmine). Naturally, it’s perfectly usable outside of the Angular realm.

On the other hand, the same fundamental issues that you’d run in Selenium are still there. Tedious setup, testers have to be capable programmers, every action emulation and verification needs hand-coding. Locators are still brittle, and there’s no CSS or visual verification. All of these decrease the ROI of UI automation.

2. Cucumber

Overview: a popular Selenium alternative with a strong focus on the behavior-driven development

Pros:

• Human-readable test cases facilitate cross-team collaboration and de-silo software QA.
• Living documentations enforce best practices for requirements management.
• Good adoption and awesome community.

Cons:

• Gherkin syntax is wordier than the code of Selenium tests.
• Cucumber has a relatively steep entrance threshold even for skilled developers.
• Painstaking maintenance.
• No visual testing.

Review:

Another open source alternative to Selenium, Cucumber has made waves in the BDD community. Used at IBM, Groupon, and other giants, it remains a mainstream automation tool even in 2018, nine years after its release.

Cucumber bridges the gap between technical and non-technical project members. In essence, that’s the key ingredient of its secret sauce. In Cucumber, acceptance tests, functional requirements, and documentation merge into a single automatically-updated source of truth for testers and stakeholders.

Technically speaking, Cucumber can both act as a Selenium alternative or work in tandem with Selenium. But in both cases, it has its shortcomings. The Gherkin syntax is accessible to non-techies, yet it’s a lot wordier than normal code. Programmers would also argue that it discourages code reuse.

And again, the same old issues are still there. Cucumber requires programming, labor-intensive maintenance, and special training — even more so than Selenium. While Cucumber acceptance tests are technically UI-driven, there’s also no built-in visual testing.

3. Katalon Studio

Overview: UFT-like functionality in the form of a free open-source tool (with paid support).

Pros:

• Built-in plugin for visual testing (simple record-playback with screenshot comparison).
• Test authoring is accessible to non-programmers (which is great for team productivity).
• IDE (Katalon Automation Recorder) is more stable than the Selenium IDE.
• Built-in assertions.

Cons:

• Test-editing via auto-generated code is counterproductive due to painstaking maintenance.
• Professional use cases require paid support.
• Simplistic record-playback is a productivity killer.

Review:

If you think about it, Protractor and Cucumber share common pitfalls characteristic of any solution reliant on hand-coding. So maybe a full-fledged automation tool will make for a better Selenium alternative?

Katalon Studio seems like a worthy contender. Launched in 2016, the platform uses Selenium and Appium under the hood, combined with keyword-driven testing and basic record/playback.

Katalon Studio allows for codeless test authoring, which simplifies and speeds up automation. In addition to the full-fledged platform, Katalon offers a lightweight Selenium IDE substitute called Katalon Automation Recorder. This tool is as simplistic as Selenium IDE, but it seems more stable in both Chrome and Firefox.

While codeless test authoring sounds nice, the catch is you still have to maintain auto-generated test code. According to the Katalon team, auto-generated code is fairly readable in Katalon Studio, less so in Katalon Automation Recorder. Yet in my experience, in 100% of cases of dealing with auto-generated code, you’ll regret not having handwritten the thing in the first place.

4. Ghost Inspector

Overview: a web-based visual testing tool that’s fairly simple and fun to use.

Pros:

• Browser-based tool (works via a browser plugin).
• Basic test cases editing is codeless, which is great for productivity.
• Support for hand-coding in JavaScript for custom tests.
• Import of Selenium tests.

Cons:

• Simplistic screenshot comparison (pixel matching with thresholding).
• Ghost inspector requires manual assertions (you need to click on elements to make the tool “see” them).

Review:

Another Selenium IDE alternative that seems to get record/playback right is Ghost Inspector.

Just like Selenium IDE and Katalon, this tool requires a browser extension (Chrome plugin). Unlike Selenium IDE and Katalon, it allows for codeless test editing. You can add, remove, reorder, modularise, and reuse test steps and change actions and selectors inside of steps. All of this functionality is codeless. There’s also support for hand-written JavaScript tests, Selenium import and export, and data-driven testing.

There are limitations, as well. Ghost Inspector relies on manual assertions, making you click on the elements you target. This looks like a minor nuisance, but it has an important implication: Ghost Inspector doesn’t really verify the UI. Instead, UI comparison boils down to pixel matching, which is fraught with false positives and missed bugs.

5. Screenster

There’s a disclaimer that I need to make before we can proceed. Screenster evolved from a visual regression testing tool that my team developed internally for our own projects. In 2016, we released it as a standalone product.

Screenster is a Selenium alternative that introduces the concept of record-playback-verification to UI testing. Created for less technical people, this tool transforms the record/playback paradigm found in modern Selenium IDE alternatives. By adding intelligent verification of each on-page element, out solution overcomes the major limitations of UI automation.

Record-playback-verification

UI screenshots comparison makes little sense unless you can recognize the underlying DOM. This functionality is only beginning to surface in some AI-based automation tools whereas we’ve had it for years.

When recording a UI test, Screenster analyzes the DOM and matches individual UI elements to how they render on the screen. This way, we can automatically verify every on-page element.

UI baselines

For each click, text input, or other user action during test recording, Screenster creates a baseline, capturing the UI screenshot and the page DOM. It then compares these baselines to new versions of the UI.

In contrast to tools that pixel-match full-page screenshots, Screenster can zoom in on individual elements. When analyzing elements, it also looks at content, filters out anti-aliasing, and ignores dynamic UI areas. This method is more precise, and with fewer false positives than traditional screenshot comparison.

100% web-based

There is nothing for testers to install locally. I’m not just talking about desktop apps. Unlike Selenium IDE, Screenster doesn’t make you install browser plugins. Knowing that browsers sometimes drop the support for extensions formats, a full-fledged web app is a more future-proof option.

With Screenster, all testers and developers use a shared server running on-premise or on the cloud. And there aren’t any manuals to read either (but we do have a detailed documentation in on our website).

Low-code automation (with support for codeless and hand-written tests)

Testers can browse page elements offline and update tests through a mouse-friendly UX. For skeptics: would you deal with code if you didn’t have to? For those cases when you absolutely need to code, there a first-class support of custom-written Selenium and JavaScript code.

Smart Maintenance with self-healing selectors

Screenster has self-healing selectors that prevent moved or changed elements from breaking tests. When Screenster captures a UI baseline, each UI element gets a complex selector that stores its ID and other attributes, content, full XPath and CSS. So even in case of a renamed ID, Screenster will still locate the element using the stored information.

Should a test run detect expected differences, they can be approved to the baseline with one click. In case of a UI bug, there’s Jira integration for hassle-free bug reporting.

AJAX friendly

Screenster automatically detects page transitions and handles AJAX updates. No need to code timeouts and deal with complexities of asynchronous programming.

The prerequisite for using Screenster is that UI state is repeatable, meaning that repeat executions of the same tests produce the same result. The approach with codeless visual tests may not cover all use cases but it should produce a significantly better ROI for 95% of the commonly covered scenarios. At the end of the day, it’s the focus on higher ROI that makes for a good Selenium alternative.