CUSTOMIZING MACHINE TRANSLATION EVALUATIONS TO IMPROVE MT ENGINE OUTPUT AND TIME TO MARKET DELIVERY
A global e-commerce retailer needs independent evaluations of machine translated (MT) content. The goal is to gather statistically meaningful results by comparing the outputs of different internal MT engines. The ultimate goal is to make products available for sale in multiple markets in the shortest time possible.
The client needs content from different sources (queries, titles descriptions and images) evaluated in multiple language combinations. A fast turnaround allows its developers to further analyze the data received in order to improve the client’s engine output.
Appen worked with the client to define the list of necessary language combinations and set up a pool of trained consultants able to deliver accurate MT evaluations on a short notice. Appen was able to confirm immediate availability for most of the language combinations required. Additional ad-hoc recruiting was performed in order to secure qualified, in-country resources.
With Appen’s expertise in MT evaluations, a new streamlined process was implemented and it helped to:
- Simplify the evaluation model to increase the number of evaluators without affecting costs
- Reduce time spent on text-heavy evaluations
- Get statistically meaningful results by increasing the number of evaluators without affecting costs
- Complete work within the time frame allocated to meet the needs of the client’s development team
Appen promptly delivered accurate MT evaluations in multiple language combinations within the timeframe expected by the client. The client was able to use the results to track, improve and train content on internal MT engines and to ultimately drive site traffic and sales worldwide.
The client indicated that the quality provided by Appen’s consultants met their internal quality standards and that it will consider Appen as the primary provider of MT evaluations going forward.
SUPPORT NETWORK OF LINGUISTS FOR NATURAL LANGUAGE UNDERSTANDING DEVELOPMENT
A leading provider of voice and language solutions was working on dozens of locales for lexicon development. The company needed a number of highly qualified, part-time native resources on an ongoing basis to meet the support needs of projects that were spread across the company’s global teams.
Instead of struggling to locate, onboard and manage linguists with specific skill sets, the client outsourced these services to Appen and its premier network of linguistic consultants. The client provided the recruiting requirements and Appen then contracted and on-boarded the consultants. Work on multiple locales was kicked-off in a matter of just a few weeks. The client representatives assign the tasks and concentrate on technical detail while the Appen Project Manager supports the operational, non-linguistic aspects of the work. Appen linguists support the client’s core in-house teams and:
- Review and verify tokens
- Review and correct pronunciations
- Lexicalize out of vocabulary (OOV) words
- Review and add spoken forms
- Provide and review spellings
- Serve as language experts for research
By gaining access to Appen’s global pool of experienced consultants, the client found qualified linguists who promptly started work in the client’s tool. The Appen Project Manager supports the communications, process development, training and management as well as administrative duties associated with each linguist’s tasks. For more than 5 years, this scalable support model has helped build strong relationships between the client, the consultants and Appen PMs enabling the client to expand its products and services in a business environment where needs and priorities shift quickly and flexibility brings a competitive edge.
Custom corpora, lexicons for multilingual entity extraction and sentiment analysis
A world’s leading provider of business software wants to expand their core information extraction and business intelligence technologies into many more languages. They needed expert resources to develop their named entity recognition and sentiment analysis components that are integrated into a number of their systems. These systems require custom lexicons as well as sophisticated and very accurate evaluation sets in order to measure and track the accuracy of the automated data analysis. The client team was looking for a partner to develop the tools and processes in order to provide the language resources needed.
Appen worked closely with the client team to gain a deep understanding of their requirements and to provide them with the different language specific resources:
- Custom lexicons (product names, company names, etc.)
- Gold corpora annotated for named entities
- Gold corpora annotated for sentiment and topics of sentiment
- Error analysis reports of automatically extracted entities, sentiment and relations
- Language identification corpora
Appen established tools, approaches and documentation to make the process easily repeatable for scaling up to many languages and provided the high-quality language resources needed.
Receiving the resources in a timely, cost-effective way enabled the client team to deliver these BI components in multiple languages in a timely, low-impact fashion. We are trusted partners in the development process because:
- At Appen, we have an extensive global network of language specialists and linguist: the language resources that we develop are not only complete and accurate, they are also done with awareness of local language preferences and trends, making them close to your customer.
- Appen project managers all have background in language technology. We speak the same language as your development team, we are comfortable navigating your internal specifications which means minimum ramp-up time and maximum commitment to deliver it on schedule and per requirements.
- At Appen, we continuously evaluate internal tools and processes to make sure we deliver the highest quality, no matter how complex a project, and stay competitive, for the benefit of our clients.