How to Decide on Mobile Bug Fixes

How to Decide on Mobile Bug Fixes

Any software product, be it a web, mobile or desktop application, is under suspicion unless it proves the features are working as expected. In short, software is never bug free. Depending on the software product, fixing bugs in production is not easy and, in most cases, it’s expensive.

Before the Release of an App

Before a mobile development team ships a native app to the app store, an intensive testing phase must happen to minimize the need for any hotfixes. Distributing the app to beta testers or external testing providers to gain early feedback is another approach to take. However, we all know that the real nasty bugs happen in the wild on customers’ real phones and often during scenarios that only exist in the real world.

The Mobile Bug Matrix

But how to decide which mobile bug is worth doing a hotfix? Over the past few years, I have seen many teams dealing with exactly this question. Some teams tried to fix every bug in production but ended up doing fixes for the released app and stopped doing feature development. Therefore, teams need to find the right balance between doing a hotfix and waiting until the next upcoming release.

To make the decision of doing a hotfix easier, I created the mobile bug matrix. The mobile bug matrix has 2 axes. The x-axis is the criticality/severity for the user, and the y-axis stands for the criticality/severity for the business.

Mobile bug matrix: low lower left corner, high top right: Next release for business (top left), critical hotfix not critical (top right), Upcoming sprint not critical (lower left), next release for users (lower right)


This results in three options. I’ve included an example bug along with each scenario:

  1. Hotfix: This area has the highest impact to the business and the user. If a bug was found on the live app and fits to this area, the team must fix it as soon as possible.
    Example Bug:
    The login of the app is not working anymore.
  2. Next Release: Bugs in this area either affect more the business side or the user side. The issue is not critical enough to perform a hotfix, but the team must fix it in the next release.
    Example Bug
    : UI elements of the app are in the wrong place.
  3. Upcoming Sprints: These problems need to be fixed but there is usually no pressure from either the business or user. These kinds of bugs can be added to the backlog and can be planned in upcoming sprints.
    Example Bug:
    Wrong translations or typos.

Raise Questions For Clarification

When talking to different stakeholders of the app, there will be different opinions on the criticality or severity of the found bug and in which area the bug fits. In order to prevent exhausting discussions, it might help if the team raises the following questions to make a decision easier to reach.

  1. How many users are affected by the bug?
  2. Are we losing activity on our app?
  3. Is the company losing trust?
  4. Do we have a legal issue with this bug?
  5. Are we losing money because of the bug?
  6. Is the problem reproducible on the supported devices and operating system versions?
  7. Which sections are affected?
  8. Can we switch off the feature from the backend?

Once a team has decided if the problem is worth performing a hotfix on, the existing software development approach must be used. Find the root cause for the issue, fix the issue and cover it with some automated checks. Once development is done, a software tester or beta testers must verify that the bug has been fixed. Furthermore, regression testing has to be performed to check that the fix has not introduced additional problems. Last but not least, the whole test automation suite must be executed, and the results must be green.

Find a Workflow for the Team

In summary, the mobile bug matrix is one approach to make a decision on whether or not a bug gets fixed immediately. Every team must define a workflow to handle this situation of critical or non-critical bugs in the production system. It’s important to find the right balance between a hotfix and the upcoming release because fixing a mobile bug isn’t usually fast and cheap. Next to the development and testing costs, there are also app review and submission waiting times for some app stores until the fix is live for the customers, and it might be the case that the next planned app release is just around the corner.

Whitepapers

The Essential Guide To Mobile App Testing

To effectively test mobile apps, now and into the future, enterprises must ensure a strong user experience from the beginning. Here's how to implement an effective mobile testing strategy.

Read Now
Want to see more like this?
Daniel Knott
Daniel Knott
Mobile Testing Expert
Reading time: 5 min

The Pros and Cons of a Bug Bounty Program

Bug bounties can uncover defects and vulnerabilities, but aren’t a digital quality catch-all

Why Media Companies Must Deliver Relevant Ads

Expectations have changed, and both media companies and advertisers must adapt.

The Building Blocks for a Better Personalized Experience

AI lends a big hand in helping enterprises adopt personalized experiences at scale

How Retail Banks Can Survive in the Digital Arena

Applause speaks to Credito Emiliano about how retail banks can survive in the digital arena, in which new banking business models threaten legacy players.

Finding Hidden Defects Using Investigative Testing

Take an investigative approach to overcome defect fatigue

How Usability Testing Enables Immersive Metaverse Experiences

Take a thoughtful, iterative approach to understand users’ points of friction in the metaverse.