Maintaining High Quality While Replatforming

Replatforming, in which an organization opts to solicit services from a new vendor rather than a previous one, often makes business sense. Business needs and vendor offerings can evolve over time, necessitating a strategic change. In fact, selecting a vendor with specialized expertise in a particular technology or industry can help the business gain an edge over competitors, depending on needs.

But, whether an organization is replatforming or opting for a “buy” vs. a “build” decision, there’s no ironclad solution for all problems it might encounter. That is especially true when it comes to digital quality, which requires ongoing effort to get it right — and continue getting it right. New product features, device sprawl, market launches and changing standards or regulations are just some of the challenges organizations face in appealing to customers, once and into the future.

Let’s learn more about the challenges of maintaining digital quality while replatforming, with some challenges our clients experienced over the course of my years at Applause. We’ll start with the platform change itself.

1. New platform rollout

A launch is never as simple as flipping a switch. Rollouts often involve several risk-mitigation techniques, which is especially important with the many back-end changes that occur during a replatform. There’s no shortage of features and integrations that can break during the migration.

Canary testing and other forms of A/B rollout make sense to gauge early reaction to the launch. You might opt to route a small, random percentage of traffic to the new platform via a load balancer, then gradually siphon other traffic over as you gain confidence in the release.

During this stressful period, organizations can help ensure strong digital quality posture by bursting testing to gather additional feedback. Phased rollouts will catch some, but not all defects; the same goes with internal testing, where device labs only go so far. Institutional blindness further conceals defects. Make use of external testers and customers to validate customer flows within the replatformed app, especially to ensure API integrations and front-end services function as expected. Test and document as much as you can.

It’s also important to maintain high performance during and post-rollout. Optimize back-end services and monitor user performance to identify challenges before they adversely affect brand reputation. There’s value in replatforming, but there’s also risk, and an inattentive eye toward the user experience can have a direct effect on the bottom line.

2. Vendor management

Many organizations miss the hidden costs in replatforming. You might have to hire someone to manage the vendor relationship, upskill or train staff to handle your unique requirements or get slapped with additional charges for your individual regulatory or logical needs. When you place a significant monetary and strategic investment in a vendor, someone must keep them honest, making sure deadlines are achieved, builds retain high quality and defects are remediated.

Vendor and client priorities might not always align — heck, even the logistics might not align. If your business has a significant portion of customers in the U.K., a vendor in California that operates only on local business time could present a challenge when a significant defect occurs. A few hours with a major flaw in your customer experience is stressful at the least and detrimental to the business at the worst — and that’s even assuming the vendor is motivated to fix the problem quickly. Imagine waiting several months for a hotfix — that’s far from an ideal scenario. Many platform vendors are reputable and want to maintain strong relationships with their customers, but they also have many customers. You might just be another customer in line. What constitutes an emergency for you might not sound alarm bells for them.

For this reason, I often recommend customers construct a warranty period of sorts with their platform vendors. Negotiate no less than a 30-day window in which the vendor is responsible for fixing any defects found in your digital product. And, over the course of that 30 days, pound the stuffing out of that product, making use of internal, external and crowdtesters to find every single defect possible. A past client of ours took this approach with a platform vendor. After deploying Applause crowdtesters who attempted to validate payment flows, the client found that customers could not complete transactions over the course of two days post launch. The warranty period gave this client the leverage they needed to make those fixes and maintain brand reputation.

If your organization makes use of several platform vendors, as was the case with the aforementioned client, vendor report cards can also go a long way toward ensuring high-quality deliverables into the future. Measure whether and where some vendors succeed versus those that underwhelm. Identify metrics like mean time to resolution to grasp the effectiveness of the vendor’s escalation and response efforts. When it’s time to renew a contract, vendors with passing grades get extended, and failing vendors either get the chance to improve or they get the boot.

The biggest thing to keep in mind is this: platform vendors can alleviate a lot of problems, but they might offset it with others or — perish the thought — introduce even more challenges than you had before. Put strategies in place to wrestle back control over your products in the event something goes awry.

3. Visibility

When you contract with a platform vendor that handles most of the dev and test work, you lose some level of visibility over the quality of the app. This is a concern when so much of your brand’s reputation and profitability relies on the digital doorway you open to your customers.

Vendors develop features as required, and they typically test the product and fix bugs — eventually. But they might not ever disclose exactly what went wrong. How many severe defects did your vendor remediate, and how long did it take them? You might not ever know, even as you wonder why subscription numbers took a sudden decline.

This is why it’s important to set a UX baseline, offering a basis of comparison for the customer’s interaction with your digital products before and after launch. Subtle differences in the presentation of features can go a long way toward delighting or frustrating customers, even when the product performs well from a functional standpoint. Consider this example from a retail client that had replatformed all of its content. The company added a recommended item feature as part of its migration, a perfectly reasonable feature that had no discernable defects. However, when customers used the recommendation feature in the real world, the company discovered that the feature was offering items at price points that were inconsistent with what the buyer had added to cart. Buyers were frustrated to see, for example, a $100 suggestion for a tie after adding a $20 shirt to the cart. By setting a UX baseline and incorporating a user panel during the replatform to gain visibility into the digital experience, the company was able to adjust the recommendation engine to make it more relevant.

Remember that for some businesses that compete in highly specialized industries, unique testing needs might not be replicable with a platform vendor. Consider IoT device manufacturers who might test their products in unique acoustic, temperature or light conditions. If a vendor takes on these testing responsibilities, the organization loses immediate visibility into the products under test, which might even be provided by a different manufacturer. For example, if a smart TV manufacturer offloads dev and test responsibilities to a vendor, it would lose sight of both the software validation and quality of the hardware, which can present a number of remediation challenges for a vendor that lacks that domain knowledge.

When it comes to digital quality, knowing is half the battle. If possible, establish stipulations and practices in your vendor contract to ensure adequate testing.

4. Real-world delivery

When switching to the new platform, everything might look just fine on the back end, but that doesn’t mean it will function as intended for the end customer. How will the product work on real devices in the hands of real customers in real locations?

Consider the many different requirements a digital product might have to account for. Mobile apps must meet app store requirements. IoT devices must meet HomeKit or Google Home requirements. Streaming platforms must meet a variety of region-specific standards. Factor these into the baseline business requirements and regulatory standards of the industries in which the business competes. A lot can break when switching from one platform to another, and it’s not always easy to figure out how and where that break occurs.

Not only must the digital product function correctly, but it must also meet the user’s expectations. Take the example of a streaming media app. Yes, the subscriber must be able to access everything they want to watch, but they must also be able to use, for example, voice functionality and smart remotes.

Organizations must ensure everything works, both at a point in time and well into the future. If a platform vendor will be responsible for updates, patches and other user concerns, the organization must stay on top of all of those areas to ensure proper delivery.

5. Beyond functional testing

As mentioned, usability is key to success with a digital product. Friction in the user experience reduces customer spend, which hurts ROI and even brand reputation. A properly functioning app is not necessarily a successful one, and don’t count on the platform vendor to overturn every stone to optimize usability.

Likewise, digital accessibility presents a potential dilemma. The contract should stipulate how the vendor will approach meeting your accessibility needs on every release. Determine who is responsible for accessibility training and evaluation, and how issues will be escalated.

Ditto for localization and payment testing, both highly specialized areas that require a unique testing approach. One client we work with uses two or three localization vendors, employing one based on the location of the rollout. Translation work goes back and forth between the vendor multiple times — it’s all too easy to get this task wrong. With payment testing, make sure it’s clear how it will be conducted, by whom, and with which devices.

Vendor report cards are helpful here too, as some vendors place an emphasis on areas like accessibility and localization, while others have little to no expertise in those areas. Report cards help determine the most appropriate partners for your needs, rather than leaving your brand susceptible to reputation damage at best and litigation at worst.

Take control

Platform vendors offer a number of clear benefits, if you can stand to sacrifice some level of control. Quality will ultimately be a shared responsibility, but the burden of brand reputation is yours alone.

Turn to the flexibility and power of the crowd to cover your ongoing, ever-changing, always- important digital quality needs. Applause has a million-strong community of global digital experts, offering an unmatched ability to cover any gaps that emerge pre- or post-migration. Define technical, demographic, device or solution-specific needs, and Applause assembles a custom team to deliver documented test results and actionable insights. Where some platform vendors offer limitations, Applause offers answers.

Let’s discuss how Applause can support your digital quality needs while replatforming.