Mobile Device Manufacturer Achieves Greater Accuracy for Map Software with In-market Raters
The Company
A leading software and hardware company partnered with us to test accuracy of its new, confidential mapping software for an upcoming major release of its mobile phone operating system.
The Challenge
A large software and hardware company needed to test out map software for the next major release of its mobile phone operating system. Specifically, it needed users in multiple major cities to determine whether walking/transit directions provided by the app were accurate. The app had not yet been released and needed to be kept top-secret, so surveying actual users was not an option. Meanwhile, sending its own employees would be costly and would not provide the local expertise needed to thoroughly test the map software.
The Solution
The firm partnered with us to develop and execute a testing plan to meet its launch objectives. After consulting with the firm on the desired outcome, we developed a customized approach for the testing project. Teams of 10-15 testers per city were recruited and qualified by preparing the on-site testers with virtual onboarding materials before they showed up for day one. A temporary office space was reserved in each city where a client representative could meet with the testers to provide additional training and distribute devices. Using dedicated devices with the testing app which was preloaded with maps and directions, testers followed predetermined routes, taking notes of where the directions led them to the wrong location or where information was inaccurate. This data was then used by the client to refine its software before the release to the general public.
Since this was a top-secret project for a high-profile client, confidentiality was a priority. A code name was used for the project and the client’s identity was never revealed to the testers. During the recruitment process per our team’s recommendation, testers’ employment history was reviewed to disqualify anyone who had previously worked for the client or any of its competitors. Furthermore, testers were required by non-disclosure agreements not to share details of the project with anyone. Even within our team, knowledge of the project details and the client’s identity were kept on a need-to-know basis.
The Result
The firm successfully launched its new map application into multiple markets by partnering with us. Not only was the firm able to improve the accuracy of its software in a wide variety of locations before releasing the product, it saved money by avoiding the expense of sending its employees into the field. Significant efficiencies were gained through the local expertise of in-market testers; their knowledge of the local areas meant that testing could be completed more quickly and with more thorough feedback. In addition, the confidentiality of the project remained intact, which was of significant importance to the client.
The success of this project relied heavily on the development of a customized testing plan that was tailored to meet the client’s needs. Timelines were tight, and since client representatives traveled to different locations to meet the testers, the schedule needed to be managed very closely. Our skilled project managers stayed in close communication with the client to inform them of any schedule changes so that adjustments could be made quickly in order to meet the testing deadlines. Our close management of any attrition was also key to achieving success; if there was a cancellation from any tester, our team was able to replace them in 24 hours.