How to Decide on Mobile Bug Fixes
Any software product, be it a web, mobile or desktop application, is under suspicion unless it proves the features are working as expected. In short, software is never bug free. Depending on the software product, fixing bugs in production is not easy and, in most cases, it’s expensive.
Before the Release of an App
Before a mobile development team ships a native app to the app store, an intensive testing phase must happen to minimize the need for any hotfixes. Distributing the app to beta testers or external testing providers to gain early feedback is another approach to take. However, we all know that the real nasty bugs happen in the wild on customers’ real phones and often during scenarios that only exist in the real world.
The Mobile Bug Matrix
But how to decide which mobile bug is worth doing a hotfix? Over the past few years, I have seen many teams dealing with exactly this question. Some teams tried to fix every bug in production but ended up doing fixes for the released app and stopped doing feature development. Therefore, teams need to find the right balance between doing a hotfix and waiting until the next upcoming release.
To make the decision of doing a hotfix easier, I created the mobile bug matrix. The mobile bug matrix has 2 axes. The x-axis is the criticality/severity for the user, and the y-axis stands for the criticality/severity for the business.
- How many users are affected by the bug?
- Are we losing activity on our app?
- Is the company losing trust?
- Do we have a legal issue with this bug?
- Are we losing money because of the bug?
- Is the problem reproducible on the supported devices and operating system versions?
- Which sections are affected?
- Can we switch off the feature from the backend?
Once a team has decided if the problem is worth performing a hotfix on, the existing software development approach must be used. Find the root cause for the issue, fix the issue and cover it with some automated checks. Once development is done, a software tester or beta testers must verify that the bug has been fixed. Furthermore, regression testing has to be performed to check that the fix has not introduced additional problems. Last but not least, the whole test automation suite must be executed, and the results must be green.
Find a Workflow for the Team
In summary, the mobile bug matrix is one approach to make a decision on whether or not a bug gets fixed immediately. Every team must define a workflow to handle this situation of critical or non-critical bugs in the production system. It’s important to find the right balance between a hotfix and the upcoming release because fixing a mobile bug isn’t usually fast and cheap. Next to the development and testing costs, there are also app review and submission waiting times for some app stores until the fix is live for the customers, and it might be the case that the next planned app release is just around the corner.
Whitepapers
The Essential Guide To Mobile App Testing
To effectively test mobile apps, now and into the future, enterprises must ensure a strong user experience from the beginning. Here’s how to implement an effective mobile testing strategy.
6 min