Screenster vs Sauce Labs: cloud testing tools compared

In the brave new world of 2017, UI regression testing tools shouldn’t take days to set up. Elimination of setup and configurations pains makes one solid case for taking your QA infrastructure to the cloud. Another obvious case for running tests on the cloud is streamlined collaboration — after all, software testing is a team effort.

A matter of question, however, is what type of a cloud-based platform is best-suited for your project. In this post, we will consider two cloud-based tools that demonstrate radically different approaches to UI testing automation. The first one is Sauce Labs, arguably the most popular automated testing cloud for running Selenium and Appium tests. The other one is Screenster, our very own cloud-based platform that enables codeless UI testing.

Sauce Labs

Sauce Labs builds on the flexibility of Selenium and Appium tests, and it provides a readymade cloud infrastructure for running them.

The solution offered by Sauce Labs mainly caters to code-based automation needs. The platform enables QAs to run Selenium and Appium tests on remote virtual machines that mimic the environment of different OS and browser versions. All in all, there are 700+ OS and browser combinations available for the Sauce Labs users.

Aside from this, the platform offers a screen recorder, and it stores the emulated user actions as series of screenshots and videos.

Screenster

Screenster takes a radically different approach to test automation by ditching hand-coding in favor of a UI-driven codeless regression testing tool. The platform allows less-technical users to automate the routine of manual testing. In addition to offering simplicity and a smooth learning curve, this approach to testing enables the automation of real-life human interactions with the UI.

In a nutshell, Screenster builds on the simplicity of old-school IDEs (like Selenium IDE), and it expands their functionality with a smart record-playback-verification solution. Instead of merely capturing screenshots of recording videos, this tool generates visual baselines of every UI step. Complete with DOM snapshots, these baselines that remove the tester from having to create verifications manually.

So how do these two testing tools compare to each other?

When comparing the two QA automation tools that follow different regression testing paradigms, practical advantages seem like the most reliable criteria. So let’s abstract away from technical details and focus on those things that actually influence your bottom line.

In one of our previous posts, we’ve talked about the features that we consider must-have for every UI testing automation tool. In this post, we’ll tweak the list a little because some of the criteria apply to both platforms by default. Basically, we’ll focus on what each tool has to offer, rather than how it works.

Speed

People should never have to struggle with sluggish technical solutions. The speed of test creation and editing has a direct impact on the ROI of QA automation, which makes it crucial for any testing automation tool.

As far as speed is concerned, both tools seem to do a decent job. Creating a simple test with both Screenster and Sauce Labs’ recorder is a matter of minutes. Hand-coding a test with Selenium or Appium is, naturally, a lengthier process — so is the maintenance of code-based tests. On the other hand, the speed at which Sauce Labs simultaneously runs tests on 50+ VMs is impressive to say the least.

Simplicity and beginner friendliness

Simplicity is another factor that contributes to ROI. As far as user-friendliness goes, Sauce Labs facilitates Selenium and Appium testing in the way of infrastructure maintenance. This makes things fairly simple for those QAs who are comfortable with coding.

Screenster goes way beyond this, by offering a testing automation toolkit that is fully accessible to non-programmers. Using Screenster for QA automation is as simple as using Google Docs for writing — the main functionality is easy to pick up on the go.

Record-playback

While both platforms feature a record-playback solution, it is an added feature for Sauce Labs and part of the core functionality for Screenster. As a consequence, the latter offers a lot more in the way of record-playback functionality. The regression testing tool offered by Screenster enables you to create, manage, and edit UI tests, and it provides advanced features for test optimization and reuse.

When recording UI tests, Screenster identifies UI elements without manual verifications, and it enables testers to manage UI areas with a couple of mouse clicks.

Visual testing

Visual testing is becoming increasingly important due to the growing prominence of feature-rich UIs. To test what the user actually sees when interacting with the UI, you need CSS testing, which is a problem for code-based tools. Consequently, visual bugs (like broken layouts or wrong text formatting) are also a problem for the predominantly code-based Sauce Labs.

Being a UI-driven solution, Screenster excels at visual testing. The platform is able to identify on-page objects and detect content and layout shifts while filtering out differences that are incidental to rendering issues.

Test reuse

UI regression testing encompasses numerous scenarios that require test reuse, which makes this feature must-have for all testing automation tools. Reusing hand-coded Selenium and Appium tests on Sauce Labs is fairly easy, as long as your QAs can keep their tests fairly modular. In the case of Screenster, testers can reuse separate test steps thanks to a handy feature called Invoke Script.

Test parametrization

Test parametrization contributes to test reusability. Replacing hard-coded input values with parameters also makes your tests suites more comprehensive and manageable. Being code-based, the tests offered by Sauce Labs fully support parametrization. For Screenster, test parametrization is a feature that’s yet to be implemented.

Handling dynamic content

UI areas with dynamic content include dates, comment sections, content feeds, banner ads and much more. In fact, UI areas of this sort have grown omnipresent, yet only few automation tools offer functionality for handling them.

Screenster is among these few automation tools. The platform automatically detects UI elements with dynamic content during the first test run and suggests excluding them from further comparisons. Sauce Labs don’t seem to explicitly offer tools for handling dynamic UI areas, but there’s always a way to hand-code your way around these issues.

Timeouts management

Timeouts are a nuisance. No matter how skilled your testing team is, no single QA engineer will ever be happy about having to hand-code their way around timeouts.

Not having to deal with timeouts is a pleasant little feature that’s available to Screenster users. Instead of making you think about timeouts, this tool automatically determines what time is optimal for a UI element or state to load.

When it comes to working with Sauce Labs, the recommended way of dealing with timeout is explicit waits.

Collaboration

As mentioned before, collaboration support makes the life of QAs easier. Being cloud-based, both Screenster and Sauce Labs enable testers to collaborate on tests via a web interface. Besides, both regression testing tools offer neat collaboration-friendly UX.

Browser support

Cross-browser functionality is essential for any tool that used for regression testing. When it comes to the number of supported browsers, Sauce labs is a clear winner. This solution is compatible with multiple versions of every major browser, running on every major OS. Screenster, in the meantime, focuses on Chrome and Firefox, with other browsers scheduled for future releases.

CI support

Continuous integration is the de-facto standard for an overwhelming majority of software development projects. Screenster enables CI support via a proprietary plugin, and it is compatible with Jenkins, Travis, Bamboo, and TeamCity.

The range of CI solutions that the Sauce Labs platform is compatible with includes Jenkins, Bamboo, Microsoft VSTS, Team City, and Bitbucket Pipelines.

Conclusion

When comparing the tools that share so little in common, it’s impossible to tell which one is the winner. Still, comparing the two can tell you a lot about each option. So let’s take another quick look at how the two tools perform when it comes to the criteria mentioned in this post:

Speed	★★★★★	★★★★★
Simplicity and beginner friendliness	★★★★★	★★★★
Record-playback	★★★★★	★★★
Visual testing	★★★★★	★★
Test reuse	★★★★★	★★★★★
Test parametrization	★	★★★★★
Handling dynamic content	★★★★★	★
Timeouts management	★★★★★	★★
Collaboration	★★★★★	★★★★★
Browser support	★★	★★★★★

While Sauce Labs exceeds at cross-browser support, Screenster is a leader in the way of visual testing. Aside from this, Screenster seems like a better option for people looking for a simple yet efficient regression testing tool. Sauce Labs, on the other hand, caters to seasoned pros who have the time and the budget for hard coding their test suites.

So which one will you choose? Giving each platform a try is a great way of making an informed decision. And if you want to start with Screenster, all you need to do is hit the orange button below;)