Manual testing tools in 2018

What manual testing tools is your team using in 2018? Amid the talks of AI-driven tools and the automation of everything, most of software testing is manual in 42% of companies. If near-100% automation is the goal, we’re still a long way from reaching it. This means the industry still needs manual testers and manual testers still need good software testing tools.

Luckily, there are dozens of solutions out there to aid QAs. So if you want to streamline manual testing, this post will explore the best options for test management, bug management, and low-code test automation.

Top-5 test management tools in 2018

QAs sometimes liken test management software to Swiss army knives. Generally speaking, modern test management tools consolidate numerous features that enable control over the testing process. A typical tool scope ranges from planning and test cases maintenance to running, monitoring, and analysing tests. Here a short list of the features you’d want to see in a modern test management system:

• Test cases management and test planning. Test management tools exist to facilitate test cases authoring, import, editing, scheduling, and test coverage monitoring.
• Requirements management encompasses requirements authoring, import, editing, and tracking. Another crucial feature is updating team members on requirements changes.
• Control over test execution. This includes assigning tests, test cycles management, and keeping record of manual and automated test runs. In addition, testers need to clearly see what test cases require prioritization and/or automation.
• Granular tracking. Test management tools ensure traceability between failed tests, discovered defects, and requirements.
• Reporting with reports customization, import, and export are must-have.
• Integrations with software testing frameworks and tools, continuous integration platforms, JIRA, etc.
• Intuitive dashboard with customizable fields, roles, and data visualisations.

Judging by the feature list above, test management tools act as a backbone of testing operations in a team. So which tools will prove to be the best foundation? Our top 4 picks include qTest, HP-ALM, Zephyr, and QA Complete. We’ll also consider Testlink as the best free/open-source option for test case management. Read on for brief overview of each tool.

qTest Manager

qTest is a go-to solution for 400+ teams in 20+ countries. QASymphony (the company behind qTest) promises a 40% reduction of time spent on software testing. The tool scales with your team with per-user licenses, and there’s a 30-day trial for evaluation.

What’s great:

• Fast and straightforward setup. The tool offers a neat 5-minute welcome guide and generally useful context-sensitive guides.
• Strong focus on test reusability (reusable execution suites, project cloning, etc).
• Integration with JIRA for requirements and defects management. Defects management is also available via close integration with Bugzilla, as well as a proprietary solution.
• Support for Selenium, Jenkins, Java, TestNG, JUnit, Cucumber, and 30+ testing tools.
• Out-of-the-box executive level reporting on test coverage, runs, etc.

What’s not so great:

• Being fully web-based tools, qTest can feel a little bit laggy at times. That said, its performance is mostly comparable to other feature-rich web applications.

Quality Center Enterprise (HP-ALM)

Historically, software development tools from Hewlett-Packard had a strong foothold in large enterprises. Now owned by MicroFocus, these tools hold on to the promise of increasing the speed and agility of QA in enterprise teams. Quality Center Enterprise (formerly known as HP ALM) is no exception to this rule.

What’s great:

• A risk-based approach to test prioritization. The risk-based test management feature  enables stakeholders to assign risk values to requirements for simple prioritization.
• Granular reporting for real-time decision-making.
• Customizable dashboards with configurable graphs.
• The tools integrates with an IDE that automatically links developer tasks to defects and requirements.

What’s not so great:

• Vendor lock-in alert. MicroFocus really pushes its clients to embrace its complete ecosystem.
• Quality Center Enterprise is a desktop tool with no support for handheld devices and Mac.

Zephyr for JIRA

Zephyr markets itself as the very first native test management system for JIRA. The tool mimics the look and feel of JIRA for a more seamless user experience. As of July 2018, Zephyr boasts a ginormous user base of 18K clients in 100 countries.

What’s great:

• Simple dashboards that seamlessly integrate into JIRA.
• Deep integration with Atlassian products.
• Deep integration of software testing into the project cycle helps companies de-silo QA and development.
• Linking defects, folders, and cycles to sprints for better traceability; defects interlinking.
• Good localization with a wide array of supported languages for non-global teams.
• Out-of-the-box support for Selenium and Cucumber for automation teams.

What’s not so great:

• Zephyr is useless outside of the JIRA ecosystem
• CI integration is only possible via a dedicated tool (ZAPI) that you’ll have to purchase separately.
• Customer reviews include complaints about occasional instability.

QAComplete

Built by Smartbear, QAComplete puts flexibility above everything else. This test management system can adapt to workflows that range from Agile to Waterfall. It offers ample customization options while still keeping things fairly simple. This said, it still requires training.

What’s great:

• QAComplete comes with a good bug tracker.
• Point-and-click customization of forms and fields.
• Simple integration with JIRA stories.
• Free trial and flexible pricing. In particular, there is an option to purchase a lifetime license.
• Deep interlinking of defects, requirements, and executions allows QAComplete to act as a single source of truth for both testers and developers.
• Smartbear offers some sort of free training for its clients.

What’s not so great:

• Complicated setup for projects that are already in progress.
• Browser support is weak for a commercial tool of this type.

Testlink

TestLink is arguably the best-known open-source solution for test management. This makes it a great choice for those teams that would otherwise manage test cases in spreadsheets. This fact is also the main reason why TestLink in on our top-5 list. Still, the tool offers a few neat features, in addition to being free.

What’s great:

• Customizable web-based dashboard for Chrome, Firefox, and IE 9+.
• Data visualization with charts and metrics.
• Good integration with defect management tools.
• Support for multiple projects.

What’s not so great:

• Integrations are cumbersome (mostly via custom or third-party solutions).
• The UI is straight out of the 90s.
• Being open-source, TestLink is generally more liming functionality-wise and less polished than commercial tools.

Top-5 bug tracking tools in 2018

At the end of the day, software testing boils down to tracking down and catching bugs. If you want to get good at this, it certainly helps to have the right tool at hand. So what is it that makes for a good bug management tool? Here’s a short list of the must-have functionality that you should expect to find in a bug management solution:

• Assigning. The less time and effort it takes to assign a bug, the better.
• Reporting. One can never overestimate the value of customizable reporting with graphs and charts in place.
• Bug lifecycle management.strong> This functionality includes assigning bug status (e.g. unconfirmed, new, rejected, fixed, deferred, closed, retest).
• Bug IDs, searchable history with logs and comments. Descriptions of previously found bugs should be searchable. The same holds true for all information on whether someone has fixed the issue and how they did it.

All of the 5 tools covered in this section feature all of the functionality from the list above. That said, they do it differently, and they offer different advantages on top of the core feature set. Our top-5 bug tracking tools for 2018 are BugZilla, Backlog, Lean Testing, RedMine, and JIRA.

BugZilla

Easy to use, free, and web-based — BugZilla checks the three top points on many QAs’ wishlists. The tool is open-source, with a dedicated team at Mozilla in charge of the development of the bug tracking platform. BugZilla is also integral to the software testing efforts at W3C, WebKit, Red Hat, Linux Kernel, Apache Project, and other world-known projects.

What’s great:

• Relatively simple installation and pain-free maintenance (compared to what you might expect from a free open-source tool).
• Strong focus on security with built-in bug visibility control and user grouping for projects (great for multi-project teams).
• Good filtering with ample parameters and keyword-based search.
• Support for Windows, Mac, and Linux.
• Good documentation (Wiki for QAs and developers).
• Voting system for bugs prioritization makes sense for open-source projects.

What’s not so great:

• Workflow management and customization could be more straightforward.
• Migrating to BugZilla is a lot more difficult compared to paid tools.

Backlog

Strictly speaking, bug tracking comes as a part of Backlog. The platform also offers tools for project and task management, as well as a version control solution. Nevertheless, defect tracking is what most people know Backlog for.

What’s great:

• Backlog has everything for super-detailed bug descriptions and categorization. This includes customizable fields (e.g. for specifying devices or browsers where the bug manifests itself).
• Flexible search with a dozen search criteria.
• Integration with Typetalk and Jenkins, built-in import of data from JIRA.
• Generally good mobile apps (Android and iOS) in addition to the web interface.

What’s not so great:

• Backlog pushes you to buy into its whole platform.
• Some issues (like custom fields in bug descriptions) are only available in the most expensive subscription tiers.

Lean Testing

Some would argue that Lean Testing belongs in the previous top-5, too, because it’s both a bug tracker and test-management software. However, it’s the bug-tracking utility that makes this tool stand out. Lean Testing offers an elegant, uncluttered solution that’s not devoid of unique and powerful features.

What’s great:

• Browser plugins for instant bug reporting are ingenious. The plugin functionality includes screenshot-taking, annotions, and calling a bug-reporting form — all via a single button click. This feature is available as Chrome and Safari plugins, as well as a Firefox extension.
• Android and iOS SDKs available.
• Integration with Slack and Github (and BitBucket, too).
• A strong focus on simplicity.

What’s no so great:

• Small-ish user base without mainstream brands using this tool.

RedMine

RedMine is an open-source project management software that combines a bug tracking tool with a source code management system. Written in Ruby (RoR), this tool ships as a web application with ample customization options.

What’s great:

• Support for custom multi-format fields for bugs, users, and projects. RedMine can also use custom fields as filters during search.
• Customizable bug types and statuses.
• RedMine allows to create issues via email.
• Localization with 50+ languages support for non-global teams.

What’s no so great:

• Relatively narrow functional scope, yet the tool still requires training.
• Redmine setup isn’t exactly straightforward.

JIRA

If you’ve been in tech long enough to remember the 2003’s rise of JIRA, you know it grew around issue tracking. So even though the JIRA of today is an end-to-end development platform, its bug-tracking capabilities remain the core of its feature set. Besides, let’s face it, JIRA is what your team is probably using for its current project.

What’s great:

• Jira is familiar to, literally, everyone in software development, and it set the standard for bug tracking.
• Huge ecosystem of extensions available on the Atlassian Marketplace.
• Customizable workflows and dashboards.

What’s no so great:

• The learning curve is pretty steep, especially with power-user features like JQL.
• A 7-day evaluation trial for a product this huge is a joke.

Codeless and low-code testing automation in 2018

Bug trackers and test case management systems are two types of manual testing tools that help you keep things organized. It’s easy to see how this increases the efficiency of software testing. Manual testing engineers can further improve their productivity with codeless and low-code automation tools.

The software belonging to this category encompasses automation solutions that run visual testing without hand-written code. Here’s how our own low-code automation tool Screenster does it.

From screenshot comparison to visual testing

How most WebDriver frameworks and tools automate visual testing is via screenshot comparison with pixel matching and manually set mismatch tolerance. Say, a test will pass if 95% of pixels match and fail if the percentage is 94% or lower. The problem is real bugs like broken buttons will often slip under the 5% threshold. At the same time, non-issues like a 1-pixel shift of a large element are above 5%.

It’s due to this that visual testing is mostly a manual effort. As a result, companies needs teams of manual testers to eyeball the UI before every major release. Still, no one says efficient automatic visual testing is impossible.

A smarter way to automatically run visual testing is by comparing individual UI elements. An automated test can run pixel-perfect comparison for each UI element and than check if all elements are in the right place. This is exactly what Screenster does.

UI recognition

Here’s another reason why human testers are superior to WebDriver. Human are capable of differentiating between content issues and CSS bugs. In contrast, WebDriver needs explicit instructions to check the elements’ innerHTML or content.

Unlike every other tool of framework, Screenster can distinguish content bugs from visual issues. It can recognize text and dates and won’t fail a test because of an automatically changing timestamp. It can also detect dynamic visual content like online ads or userpics and ignore them during comparison. When it comes to UI recognition, Screenster is almost as smart as a human tester.

Verification of everything on a page

Be it Selenium, Protractor, Watir, or any other framework, hand-coded tests only cover what they explicitly target. A Selenium tests can check whether a button is there and whether it triggers the right function. But if, say, some change to that same button makes it push neighbouring content off-screen, you’ll still get a green test.

Unlike hand-coded tests, manual testers see the complete user journey. This fact makes occasional exploratory testing session a must, and it also makes manual user acceptance testing necessary. But you can still automate much of that with the right tool.

Screenster uses a recorder under the hood, but it’s not the simplistic record/playback we’ve all seen. Instead, Screenster automatically verifies every on-page element during every test run. Should anything completely unexpected happen, the issue won’t go unnoticed.

Learning curve

Like many other automation tools that cater to manual QAs, Screenster doesn’t require programming skills. There’s a web-based dashboard for creating, running, and editing tests and test steps. Other features like reporting and managing roles are also 100% intuitive.

Even non-technical people will be able to use the core functionality of Screenster. That said, professional use requires the knowledge of manual UI testing best practices. There’s also full support for hand-coded Selenium and JavaScript tests for QA engineers coming from traditional automation tools.

Integration with your workflow

Screenster supports bug reporting via JIRA and email, and it’s compatible with mainstream CI tools like Jenkins, Travis, Bamboo, and TeamCity. Tailored for team collaboration, the tool is 100% web-based, with an option of running on premise. We built Screenster with Lean and Agile in mind, so it scales well to accommodate for the growth of your team.

It’s free to try

Unlike most commercial solutions, we don’t limit the evaluation trial. You can always check out Screenster in action by trying our web demo. So is you’d like to give this manual testing tool a try, hit the button below, run a few tests for your web application, and tell us what you think.