QA 101

Regression Testing

Regression testing is a critical, but often overlooked, part of any development process – agile or otherwise. Learn why.

Getting Started with Regression Testing

We’ve all been there: a new update causes a never-before-seen bug in your software. The best way to prevent a situation like this is through regression testing. Regression testing, as the name indicates, checks that software updates – upgrades, program and code updates, etc. – have not introduced errors into the existing software and features.

While most testing techniques aim to identify bugs in new features, regression testing looks to simply confirm that the software is running smoothly and is not experiencing any adverse effects from the update.

Performing Regression Testing

In an ideal world, regression testing is performed every time your code base is modified, whether through an update, a fix, or other changes. Regression testing should also be used to verify when a known issue has been fixed. In fact, some organizations perform regression testing every time a change is pushed.

Likewise, if practicing Agile development, performing regression testing at the end of every sprint, as part of your SDLC, is a great way to eliminate bugs from entering production. Given time constraints, this may not be feasible for some organizations. This leaves the other option of performing regression testing on a predetermined schedule – which could be monthly, weekly, or even daily depending on how frequently you update your software.

Considerations for Regression Testing

Regression testing requires detailed knowledge of the software and systems being tested, including how existing features might be affected by new updates or functionality. That knowledge will be crucial in regressions as you go. There are three different regression types to consider:

  • Local Regression: when a bug is found in the software component being updated.
  • Remote Regression: when a bug is found in a different area of your software than the area being updated.
  • Unmasked Regression: when a bug previously existed, but wasn’t a problem until the new software update.
Regressiontesting1
Regressiontesting2

Making Regression Testing Count

Retesting every aspect of your software for every regression type after every update is the most comprehensive way to regression test. While this level of coverage isn’t feasible for many organizations – especially without heavy test automation – the closer you can get to this, the better off your software will be.

To maximize the benefits of regression testing with a limited budget, consider the following factors when determining where to focus testing:

  • Areas known to experience frequent issues
  • Software that has undergone many code changes
  • Critical features of your software

There are services available that can help with every aspect of regression testing, from creating and managing test cases, to tracking and reporting on the tests you’re performing. Typically, the combination of test automation and manual testing offers the most comprehensive and effective regression testing solution. This allows for automated tests that produce errors to be followed up by actual testers to determine the exact problem and solution.

Concerns from Regression Testing

There are a couple concerns that invariably come up when considering a regression testing plan. In today’s fast moving, Agile world, speed is critical. Anything that slows down the software iteration process is looked upon with a critical eye.

It’s important to remember, though, that nothing slows down the software release process as much, or is quite as painful, as debugging. It’s estimated that every issue regression testing preemptively finds and helps resolve prior to release can save 20 hours of reactive and often frantic repairing. Those who have been involved in the debugging process know that 20 hours is probably a conservative estimate.

Value in Regression Testing

It’s no secret that comprehensive testing requires resources and investment. There are solutions available, however, that can help you manage the regression testing process, or even take it over altogether. While they do require a certain level of investment, they provide proven ROI – keeping bugs from escaping into production and keeping your software development process moving.

Perhaps the most important factor to consider is this: a customer that discovers a bug in your software is very likely to not use that product again. While there’s no way to catch every bug, regression testing is key to limiting the number of bugs that reach your production stage.