Mobile Network Operators: Test Mobile User Experience for Large Events With Real People
In just over a week, the next Olympic Winter Games will be held in Beijing, making the Chinese capital the first city to host both summer and winter games (Beijing hosted the Olympic Summer Games in 2008). While general attendance will be limited due to pandemic concerns, thousands of visitors including all international athletes and their related team staff, reporters, officials and selected attendees will be accessing mobile networks on a wide variety of devices and OS versions.
All attendees expect their devices to work consistently while in China, as well as while traveling to and from the event. But how can mobile network operators (MNOs) be sure to meet those expectations?
The limits of mobile networks
Most days, mobile networks have bandwidth that is more than sufficient. Still, bandwidth is finite, and when large events (sporting, music festivals, etc.) take place, or when disaster strikes, the networks get congested from heavy traffic that leads to high volume of call failures. SMS text messages take up far less bandwidth and may work under strained circumstances, but voice and more intensive data like videos do not.
However, coverage can be boosted in a needed area when properly planned for and tested. Of course, the ongoing rollout of 5G networks brings the promise of significantly higher bandwidth, and will definitely help deliver improved mobile experience at large events like the Beijing Winter Games. Still, real-world testing with humans is the only true way to know if your mobile service will function as designed, and deliver on high customer expectations.
Exceptional roaming is no longer a luxury
The venues for the Beijing Games have been divided into three zones, or clusters: Beijing, Yanqing, and Zhangjiakou. Athletes and all visitors expect to have flawless mobile access while traveling from these venue groupings to the Olympic Village and hotels or other attractions. As videos and streaming have become the norm for recording portions of large events for attendees, mobile networks must be prepared to handle more data across the network. Of course, users want access to their favorite streamed programs via OTT apps in their free time, and expect flawless experience while doing it.
To meet these customer expectations, MNOs must test with real people using the full array of mobile devices and operating systems (OSes) that will be in use during the event. This can be difficult to do, as most firms don’t have the combined sum of possible combinations of mobile devices and OSes to make testing feasible. In addition, humans do unpredictable things; it’s key to have real testers who are adept at uncovering flaws/weak spots in your network. Crowdtesting is the only effective way to do this.
Traditional roaming testing methods used by mobile network operators fall short
To accurately assess roaming quality of service, MNOs must understand how mobile devices perform in real-world conditions.
Can subscribers easily carry out the functions most important and personal to them?
Will the subscriber receive the Welcome SMS message when they land in Beijing?
Will they be able to make calls from crowded public spaces like airports and train stations, tourist destinations and restaurants, text while in cars or on trains, and use apps and social media wherever they go?
How is the voice quality?
Will the data speed rate be good enough for important social media updates from the Olympics?
There’s a lot to sort out.
Generally, MNOs test subscriber roaming in one of two ways:
Send a tester with multiple mobile phones and the specific mobile operator SIMs to the country where the MNO wants to test outbound roaming. This diverts QA resources from other projects, and costs quickly multiply when companies must test across multiple locations.
Use a SaaS cloud platform for roaming testing with static, mobile test probes in fixed locations in major cities. MNOs can conduct manual and automated testing using SIM multiplexing technology that transfers the subscriber SIM profile virtually to any remote test probe. While static probes can validate network connectivity, testing with a black box in a fixed location doesn’t truly reflect real-world roaming behavior.
How can mobile network operators most accurately test true roaming experience?
With Applause’s Roaming Testing services, MNOs can accurately assess roaming performance at any location in over 200 countries. Using their own Android and iOS mobile devices, our curated testing teams recreate the true roaming experience to test key functionality: making calls, sending texts, posting to social media, streaming event coverage and performing other typical tasks while on the go. This approach has several advantages over traditional roaming testing:
Test under different network conditions - A static probe in optimal network conditions doesn’t give insight into real-world roaming performance. Applause uses real mobile devices to test in different network conditions.
Test in both fixed and mobile environments - Subscribers do unpredictable things. Replicate and test journeys at multiple locations to better understand the user experience at special events or tourist destinations. For example, you can test the subscriber’s entire roaming experience from arriving at the airport, receiving the Welcome SMS message, getting into a taxi, checking emails and social apps during the taxi ride, through arriving at the hotel.
Extensive device & OS coverage - Conduct tests across multiple handset vendors, with the option of requesting particular models and OS versions. Compare how different devices respond to the same test, under the same conditions.
Test more than basic data services - Test the full roaming user experience, including OTT app performance, social media, and interactive services like customer care, top-up, and SMS campaigns.
Learn more about Roaming Testing with Applause.
