UI regression testing challenges solvedAugust 20, 2016
Quality control is an important part of product development process. Some good amount of time and resources should be invested into it. New UI functionality must be covered by tests as soon as it is ready. And it is not enough to verify it once: each time an important changes are introduced, the UI regression testing phase must follow.
Manual UI regression testing tends to take more and more of the project time. A common solution for a big project is to automate this process. However, the UI regression testing automation, with all the benefits it brings, comes at a price.
Firstly, you need to invest into the automation in order to reap benefits later. And it could take a long time at the initial stage. The fact is, you need to spend resources to set up a framework such as Selenium, to build the tests (which should also be tested themselves) and to run them properly and return useful data. These resources could be spent to speed up manual visual regression testing instead.
Secondly, the completed automated visual regression tests should be supported afterwards. Their coverage may become insufficient as the product grows and evolves. Sometimes tests should be amended or replaced with new ones. Some become outdated, so a cleanup is necessary from time to time.
Summing all that up, a proper UI regression testing automation requires a proper automation testing tool.
UI Regression Testing Automation Tools
To begin with, UI changes a lot. Changes can occur due to bugs or functionality updates. ‘Unexpected’ UI changes are typically considered as bugs and need to be corrected. But with a complicated UI with many dynamic elements, you have to be a very keen observer to ‘catch them all’.
Naturally, each functional page built with HTML/CSS should be visually compared to the mockup, and functionally compared to the expected behavior. The task of manually comparing the UI shown on the screens to the expected state is not as simple as it may sound. Human eye might not catch some tiny details which however have much significance for the product. And every human develops testing “fatigue” which causes them to not notice even the most obvious problems.
Here is when the automation testing tool should jump in. However it brings some new pitfalls. Main challenges with UI regression testing automation are:
- Tests take too long to develop
- Tests require development skills
- Existing record / playback tools produce bad code and are hard to use
- Test maintenance is time consuming
A simple and effective comparison of screenshots with easily renewable ‘basic’ design images would cover up the major part of your regression testing when it comes to UI. After that your QA team could concentrate on other important parts of the project scope: databases, load handling security etc.
UI Regression Testing on the cloud with Screenster
I have been automating my UI tests with Screenster for a while and I absolutely love it! Here is how it solves the typical CSS regression testing problems.
Problem. Tests take too long to develop.
Solution. Screenster is all web based with nothing to install locally. I can record and play tests back but since it’s all UI based I don’t have to deal with the code behind it. There is a simple interface for most common tasks and I can always add Selenium as part of my test if I really feel like it.
Problem. Existing record / playback tools produce bad code and are hard to use.
Solution. During recording, Screenster captures screenshots to record the baseline (‘expected’ images), then compares actual screenshots to it at the next run – no code writing involved!
Problem. Visual UI changes often and each time automated test must be amended.
Solution. Screenster allows to turn an actual screenshot into a ‘basic’ one in just two clicks thanks to its ‘Approve to Baseline’ feature.
Problem. Some elements are designed to change all the time. So an auto-test will always show a failure where everything is in fact OK.
Solution. All you need is to exclude those area from this test’s scope. With Screenster, you can create an ignore region for an area in the page you do not want to be analyzed (e.g. an ad). Moreover, Screenster automatically ignores dates if they constantly change after a right pattern (e.g. a current date & time element).
Problem. Buttons and other controls might change their id, properties or location in DOM causing an auto-test failing to find the right element to interact with.
Solution. Screenster has smart locator management mechanism that automatically finds renamed or moved elements. For ultimate control Custom CSS options lets you edit the CSS locator of an element, so that auto-test would perform fine after the element was moved.
You can download Screenster at our Portal. Feel free to leave a comment here about your experience with it!
Want to try Screenster on the cloud?Try Online