5 Most Common Payment Testing Questions Answered

In a recent webinar, Zeb Winzenried, Director of Testing Services, and Claire Dinsart, Senior Solutions Delivery Consultant, answered the five most frequently asked questions about testing payments. Explore highlights from the presentation below.

The value of payment testing

Digital payments are on the rise across all categories: online, in-app, in-store, and even peer-to-peer payments. Winzenried said, “looking at the payment testing world, you want to ask some questions about the end-to-end payment experience as a whole because every part of the experience matters.” In payment testing, organizations consider factors such as:

• Can customers successfully buy my product?

• Can my customers get a refund and credit back if they return a purchase?

• If I go into a new market, can folks in that market successfully buy my products?

• Am I up-to-date on the industry regulations and standards for card security or taxation?

• Can customers redeem points or earn points or rewards correctly?

• Does a POS terminal work correctly for certain payment types?

“You want your experience to be convenient and accessible,” Winzenreid said. “People are also looking for faster checkouts and more efficiency through the payments experience.” He and Dinsart explained how to achieve that in their responses to the five most frequent questions about payment testing.

1. Why does testing with live payment instruments matter?

“Testing with live payment instruments in real-world conditions is vital for one simple reason: It mirrors the real user experience,” Dinsart said. ”And you can only do that using actual payment instruments out there in all your markets.” She explained that testing with a dummy card or test data might not reveal all the issues that can occur in an actual payment flow.

In addition to validating that all the payment methods you accept work properly, Dinsart advised people to assess the end-to-end flow: “While proceeding to payment using live payment instruments, think about payment testing as the gate to whatever comes post-payment. There are a lot of those flows that need to be checked.” She offered examples such as ensuring that the post-purchase communications include the correct information, in the right language, including purchase amounts in the correct currency.

“Is your user automatically redirected from their banking application to your application or your website, and then able to proceed on your own tech stack with confidence? This is, to me, one of the key elements, on top of just assessing whether your payments are actually working or not,” Dinsart said.

Winzenreid reiterated that testing the happy path with a dummy card will only deliver success messages — that approach won’t provide the types of error messages that allow for effective troubleshooting. Test environments or dummy versions simply don’t exist for certain payment methods. In addition, without live payment instruments, it’s impossible to see if taxes and currency conversions are calculated correctly. Testing with live cards, bank accounts, digital wallets and other payment methods is the only way to ensure payments work properly.

2. How do you choose payment instruments and processes for different markets?

Winzenreid and Dinsart agree that a data-driven approach is best when choosing which payment methods to accept and designing checkout flows for each market.

“Here at Applause, we really focus on gathering as much data as possible about specific markets, understanding not just what’s available in the market, but what people actually use in the market,” Winzenreid said. Preferences vary from country to country, and sometimes organizations don’t realize that while a certain payment instrument may be available in a market, customers there don’t really use it.

“There are so many various payment methods out there, and so many changes that are happening daily almost, in the payments world that you really need to stock up on as much data as you can and look at the current state of the business,” Winzenreid said.

Dinsart emphasized that understanding preferred payment methods is essential in fragmented markets like Europe. “You want to provide the most convenient payment experience to your users. And that’s where doing market research prior to selection and implementation of different payment instruments is so critical. For example, Germany is still a very cash-driven country, while in Spain, France, Belgium, we use mostly credit or debit cards, bank transfers, or even mobile wallets most of the time,” she said.

“On top of the market and location itself, it’s very important to consider your audience. What’s your vertical? Are your users tech-savvy? Do they enjoy using mobile wallets or even cryptocurrency? Those are really things that need to be considered,” Dinsart said. “Scalability is also quite important. In the events where companies are looking into expanding in several markets, not just one, you really want to know how much that’s going to cost. Can you implement [a payment method] it once and release it in several markets? That’s really key.”

Dinsart added that payment processes can also be market-specific. For example, in some countries, users expect to see shipping details early on in the checkout flow — delaying that information until later in the process can create distrust or uncertainty among customers expecting to see those costs up front.

3. What’s the optimal cadence or frequency for payment testing?

Ongoing testing is the best way to make certain payments always succeed. When that isn’t possible, however, payment tests should factor in any time there’s a change, such as:

• accepting a new payment instrument

• changing payment gateways

• adjusting how loyalty points or discounts are calculated

• entering a new market

• changing subscription models

• learning about issues from customers

Winzenreid said, “Depending on your release and deployment cycles, it’s very important to test in your production environment. Before deploying any changes to your payment system, you should definitely conduct thorough testing… Or even every time you are planning a major system change that is not strictly related to your payment, do some regression tests.”

Companies also need to prepare for spikes in traffic due to seasonal changes or special events. Dinsart pointed out that for retailers, payment failures during periods like Black Friday through Cyber Monday can cause significant losses for the business, so proactive testing is critical.

Winzenreid called out an example of a customer error when changing payment gateways. The customer noticed a large decline in card acceptance in Brazil. “We actually ran some testing and found out that when they implemented a new payment gateway, they had unintentionally turned off all debit cards through that gateway,” Winzenreid said. In a cash-heavy economy, that oversight contributed to hundreds of thousands of dollars in lost revenue.

Companies that rely on technology and integrations with third parties must be able to isolate where payment problems are happening. “If your payment system relies heavily on third party services, you need to consider more frequent testing assuming those services also undergo major changes and updates,” Dinsart said. Sometimes, payment failures may be a provider’s fault, she said. “Being able to collect that data and go to your provider while being able to actually pinpoint specific failures or points in the payment flow is also very key in maintaining the partnership and driving business decisions.”

4. What are some commonly overlooked payment testing scenarios?

“Companies that don’t focus on the whole payment experience miss out on a lot of scenarios that are really key to customer retention and customer happiness,” Winzenreid said. Businesses often overlook testing returns or credits, along with post-purchase communication. Promotions and loyalty rewards are another common oversight, Winzenreid said.

For example, one company offered a promotion where customers who used a certain card to pay for travel – hotels, airfare, taxis, rideshares, etc. – should earn extra rewards. But there were various scenarios that didn’t trigger the correct increase in rewards. “Customers were losing out on their points and were quite upset,” he said.

Dinsart added that companies don’t always build in sufficient time to test partial payments and transactions using buy now, pay later. “It takes weeks and weeks before the money actually gets transferred from your user to your own bank account,” she said. “What if a partial returns happen in between?”

Testing negative payment scenarios is also important, such as expired cards or incorrect CVV codes. Understanding how your system responds to expired cards is critical for subscriptions and automatic renewals, Winzenreid said. In addition, integrations with QR codes, SMS, biometrics, and different types of authentications all call for testing as part of the payment process.

5. Are there unique payment testing considerations for different industries?

Certain industries need to consider special payment testing scenarios. Dinsart and Winzenreid highlighted some:

Retail and e-commerce companies need to make sure that their payment methods are localized and aligned with in-market user preferences. They also need to stay on top of the omnichannel experience and options like buy online, pick up in store (BOPIS) and buy online, return in store (BORIS).

Media and entertainment providers often acquire new customers through a variety of sources, including partners or intermediaries — think of a user signing up for a streaming media service through an OTT box, or getting a special introductory rate for a service bundled with another purchase or subscription. It’s critical to test payments across all possible acquisition channels and understand what happens if a subscriber decides to renew or change service tiers.

Banking and financial services organizations are in an incredibly competitive market right now with many microbanks and neobanks building simplified, streamlined applications. Winzenreid noted that consumers increasingly want a seamless banking experience where they’re able to pay their bills easily, send their friends money to split a check at dinner, and manage money through a single app.

Fintech companies face similar challenges, Winznereid said. “There’s just so many different ways that you can send money and store money nowadays: various digital wallets, crypto accounts. You need to be able to look at the industry as a whole and see where you line up, and where you need to catch up with the other people in the business.”

Betting and gambling operators also need to consider special payment testing scenarios, such as how customers can fund online accounts. They also need to thoroughly test payouts and promotional credits to ensure customers receive their funds promptly.

Though payment testing can be complex, it’s definitely worth the investment to protect your organizations’ revenue and reputation.

Webinars