San Francisco, CA
Are You Ready To Test?
An industry-leading publisher and accelerator of social games, Rocket Ninja combines a mix of proprietary technologies and top talent to help companies enhance third-party games and maximize revenues. Focusing primarily on “mid-core” gamers seeking a more sophisticated experience, the company leverages its game design expertise and 3D browser-based engine, Shr3d™, to form lasting connections between players and the games they love.
But before these games could be launched on social networking sites like Facebook, Howard Lo – the company’s Head of Quality Assurance – needed to expand their testing coverage. Specifically, he needed more real-real world testers to verify the load and performance capabilities of their gaming platform.
"We didn’t have enough testers that can be online simultaneously, and we needed many of them to test our live event feature," said Howard. "uTest (now known as Applause) was able to add a team of testers for us in a very short period of time, and the rest was history."
This case study will demonstrate how Applause was able to provide Rocket Ninja with real-world “live event” testing with select members of its global community. Additional topics will include the role of Applause's project management team, bug reporting, feedback and other aspects of the Applause customer experience.
Rocket Ninja: Testing Goals and Objectives
For this specific assignment, Howard needed to extend testing coverage for the company’s Wrestler: Unstoppable™ product – a game that lets users chop, pile-drive and pin their social media friends (all in good fun, of course). Since the game consists of several tournaments, with multiple users all participating simultaneously, Howard needed insight into the load and performance of the multi-player format while the app was under large amounts of stress.
"When the app is deployed, there are going to be thousands of users participating in tournaments at the same time, and if our engine can’t hold up, there’s going to be a lot of angry users. So we needed to know more about support for concurrent users, how the engine was responding and how well the platform recorded results. This is hard to do with a small in-house team of testers."
-- Howard Lo, Head of Quality Assurance, Rocket Ninja
To start, Howard would get set up with a select team of professional testers from within the Applause community. With the help of his dedicated Applause project manager (there’s one assigned to every customer account) Howard would upload a series of test cases to the uTest platform - tasks designed to mimic the actions of real-world users. To ensure better insight, Howard would also create a chat room to easily communicate with testers.
Test Cycle Results
After several "live load" or "live event" test cycles, Howard and his team were eventually able to complete entire projects (and receive custom bug reports and performance analysis) in less than an hour. Aside from increasing QA coverage with real-world testers, Howard would also gain a few new users in the process.
"We invited a lot of the testers back for consecutive releases, and they ended up becoming pretty avid users," said Howard. "You could tell that the uTesters (Applause testers) really enjoyed the project. They were always really excited when a new test cycle began."
Summary and Next Steps
Overall, Applause had greatly improved the productivity of the Rocket Ninja QA team – but they were not the only ones who realized this benefit.
"Before uTest (Applause), we tried to pull together as many employees as we could to test, since these were important features," said Howard. "It ended up being very difficult to schedule with the workload of a startup company, and more importantly, it took valuable time away from people with different priorities. Once we signed up with uTest (Applause), it allowed our in-house resources to do what they do best, without sacrificing the quality of the game."
Now running on a continuous testing model (i.e. unlimited test cycles per month) Howard and his team have come to view Applause as true partner in testing, running projects on a weekly basis in preparation for their launch onto several other social media networks.
"We’ll definitely continue to use uTest (Applause) in the future," said Howard. "We needed a greater number of real-world testers and uTest (Applause) delivered."