Blog

Regression testing on the cloud

My colleague once told me that regression testing is the single best reason why QA automation exists. According to the same person, doing regression testing the right way is a major challenge for most web development projects. So what is so challenging about it? And more importantly, why taking regression testing to the cloud is a good idea? Why do web teams have problems automating regression testing? Two words, “UI testing”. It’s common knowledge that the ROI of automated UI regression testing is low. I’m sure you’re familiar with the culprits of this problem: Even with skilled professionals onboard, most teams struggle with the maintenance of coded UI regression tests. QA automation frameworks often lack efficient collaboration tools. In fact, few solutions will even feature shared integrated portals displaying test statuses. Existing testing environments require manual setup and maintenance. Specifically, frameworks often make you code the fundamentals of the framework itself before you can actually start writing your tests. Take another look at this list, and you’ll see that most of the problems stem from the inherent shortcomings of the tools, not the process. So maybe it’s time we moved on to something different? Taking regression testing to the cloud:... More

Selenium IDE alternatives for UI regression testing

Most of the QAs I know consider Selenium the number-one choice for UI test automation. I guess this is why Selenium IDE is the first thing that comes to mind when they start looking for a record-playback tool to automate UI regression testing. After all, if Selenium is so great for writing UI tests, it’s bound to have an equally great IDE, right? Well, not quite… If you have any experience with Selenium IDE, you’ll agree that it’s far from perfect. In fact, many testers will argue that Selenium IDE is doing you more harm than good. Reasons? Pick one! Selenium IDE doesn't always record what you expect, and it still requires programming skills (at the very least, the knowledge of HTML and CSS). In a nutshell, the tests produced by Selenium IDE are poorly implemented WebDriver code which ends up being brittle and hard to maintain. Selenium IDE makes you handle timeouts manually which is a major inconvenience. Selenium IDE is no good for projects involving UI elements that change their IDs, location, and/or content dynamically. This means you can’t regression-test the UI of a web app with Selenium IDE. Selenium IDE only works with Firefox and may not work... More

Why is manual regression testing still happening in 2017?

Where does manual regression testing fit in the software testing process? In fact, does it even make sense to test software manually, given the sheer number of automated regression testing tools available? I bet you’ve seen people ask similar questions on StackOverflow — as well as a good dozen of articles exploring the topic. So why would you need another post on the same topic? Well, here’s a reason to put my two cents into this same ole discussion. The choice between automation and manual regression testing looks simple on paper. Most manual regression testing tutorials will tell you that the complexity of manual testing grows exponentially as your project scales. Automated tests, on the other hand, help you move faster than manual tests, they reduce the chance of human error, and they are more cost-efficient in the long run. So, on paper, you should always strive to automate everything you can, right? After all, you can always have your developers write unit and API tests, and hire a QA team for everything else... In theory, yes, that’s how things are supposed to work. In real life, however, things get more complicated. The more so if we are talking about... More

Optimising our automated testing tool with Invoke Script

I’m sure most people reading this blog have at least some experience with automation testing tools for web UIs. And I guess most of us can agree that no matter what testing solution we use, we all follow similar algorithms when doing automated regression testing. These algorithms are, in fact, pretty simple. In most cases, they will include roughly the same series of repeated steps for different test cases. Here’s an example: Open a starting URL (in 99% of cases, this will be the homepage) Log in Navigate to a different page (e.g. a shopping cart page, or blog post) Do something on that page(e.g. click on a button or scroll) Click to check out. Looks familiar, doesn’t it? Now, the exact steps might differ, but you’ll still find yourself repeating the same routine across different test suites. So doesn’t this seem boring and time-consuming? Having dealt with all this boring stuff, nearly any person would either search for some shortcuts or drop UI testing altogether. Sadly, many choose the latter — and this is the reason why relatively few companies do regression testing automation. But there’s actually a way to optimize this process, provided that you have the right... More

CI-friendly UI testing automation tools: what are your options?

UI testing automation is all the fuss due to an advent of new solutions promising to either excel or complete the functionality of “tried-and-true” tools of the past years. But how many of these automated testing tools (both new and old) are CI-friendly? Besides, how many of them are good at handling visual testing of websites and web apps? Let’s see what the market of UI automation testing has to offer in the way of CI support. But first, let’s answer the following question. Does solid CI support really matter for UI testing automation solutions? The short answer is yes — because continuous integration solutions are the first line of defense against bugs and errors. Here’s what a typical process looks like: one job creates a build, while another launches unit tests on it, allowing early spotting of most easy-to-find problems. If the build passes this stage successfully, testers can step in with their usual routine. They’ll take a quick look at the UI and, if everything is fine, proceed with functional testing. Now, this process seems to have a lot of room for improvement, doesn’t it? In particular, it would be super awesome to automate this quick UI sanity... More

A better way to do visual regression testing

The concept of visual regression testing isn’t new, but it’s been in the spotlight lately. There are two factors causing this growth of attention. First, the advent of rich UIs and responsive design has made it next to impossible to efficiently test web applications and websites without focusing on CSS and visual layout. Second, the ever-increasing competition among internet businesses forces companies to search for faster ways to create and run automated software tests. Can manual testing do the job? Given these two trends, manual regression testing doesn’t seem like a viable option — it’s just too slow and inefficient. In fact, very few software development teams can afford to run visual regression tests manually after each UI revamp. Besides, humans aren’t that good at spotting visual differences in the first place due a thing called change blindness. Basically, if we don’t expect to see a minor change, we’re subconsciously bound to overlook it. Now, the problem here is that tweaking CSS is all about unexpected changes. Once you add a new class or CSS rule on one age, it will almost certainly override something on a completely different page. As a result, your users are more likely to spot... More

Are TDD and automated UI testing compatible?

Image credit: SoapUI Test-driven development is awesome — unless you’re dealing with UI testing automation. Sure, test-driven UI development is technically possible, but TDD isn’t exactly ideal for situations requiring full functional tests, and full functional tests are indispensable to UI testing. For this reason, using TDD for UI border on overengineering. Choose this path, and you’ll end up working with mock objects or project object models (POM), adding new layers of complexity to your project. And in many cases, you won’t be able to tell if doing this will pay off. Besides, the way UI testing automation is done in real-life projects another problem. In real life, UIs — and web UIs in particular — undergo a lot of tweaking. No matter how well-specked your UI is, once you do a UI prototype and play around with it, changes are always coming. So how do people do UI testing automation in a TDD project? The short answer is they don’t. What they do instead is clearly separate concerns between the UI and the underlying logic with MVC, MVP, MVVM, or similar architectural patterns. Basically, the main idea here is to keep the UI logic minimal so it's mostly a... More

UI automation testing with Smart Locators

When it comes to UI test automation for web projects, element locators can become one hell of an issue. But is there anything you can do about it? The thing is, poorly-written locators are just a part of the problem. Writing good locators is, in fact, not that difficult. After all, good locators are merely a matter of either accessing IDs or building robust CSS or XPath selectors. And you can always get your developers to throw in a couple of extra IDs, right? Actually, there’s a whole bunch of other factors to consider. Once these factors come into play, UI test automation turns into a challenge for any QA specialist, no matter how experienced. Let’s look at a couple of scenarios to see what factors can affect the work of your QA team. Scenario 1. Throwing in extra IDs Let’s start with something we all can agree on. An ID-based locator is the best option for 100% of UI test automation cases. But in real life, developers will not put IDs on every element upfront, and any data-driven content (e.g. a table) will have repeated elements that cannot have unique IDs anyway. In cases like this, path-dependant locators will... More

CI for CSS regression testing with Screenster

Now that the CI integration plugin has been successfully rolled out, it is time to recall how it all began... Ready for a tale about defeating a horde of CSS regression testing challenges with the help of a Jenkins server and a visual QA automation tool? Why did we need CSS regression tests on our CI Our team has realized the necessity of automating the visual regression testing process last winter, when our other product AjaxSwing gained a significant number of new users, who were actively requesting improvements. A new version was released each week, sometimes even more often. AjaxSwing generates web UI for desktop applications so you can imagine how much visual UI verification we had to accomplish each time a build was ready. It just seemed never-ending. We had unit tests in our continuous integration environment running against every new build, but they didn’t help with verifying the visual aspects of the UI such as CSS, formatting and layouts. That’s when our QA team decided it is time to try our new automation testing tool Screenster in real action, despite it being in its early Beta at the time. We upgraded our CI environment to the following state:... More

Integration with CI tools — a new plugin for Screenster

Good news! Now you can easily run Screenster visual tests in your continuous integration environment. Just visit the Downloads page on our Portal, get the CI plugin and follow our guidelines to install it and synchronize with your favorite CI tool. We have been receiving dozens of questions from our customers and guests lately about a way to closely integrate Screenster with CI platform to run CSS and web UI regression tests. This necessity is absolutely clear: it is often critically important to run quick visual tests against your web application with the help of an automation testing tool after each new fix or feature deployment. Our users have a variety of CI tools such as Jenkins, Bamboo, TeamCity, Travis and others so we designed a universal solution that works with all of them. It took a lot of efforts and hot discussions, but finally the plugin is here for you! Enjoy running visual tests on your CI server and feel free to leave a comment here about your experience! [raw] Want to try Screenster on the cloud? Try Online [/raw]
WordPress Image Lightbox Plugin