The rise of Large Language Models (LLMs) has been transformative, exhibiting vast potential with their remarkable human-like capabilities in natural language processing and generation. However, LLMs have also been found to exhibit biases, provide misinformation or hallucinations, generate harmful content, and even engage in deceptive behavior. Examples of high-profile incidents include Bard's factual error in its very first demo, ChatGPT's proficiency in writing phishing emails, and Microsoft's violent image generation.
LLM red teaming actively tests for vulnerabilities in a systematic and reliable way, allowing model builders to proactively identify harms and risks before they arise, mitigating the risk of unforeseen consequences. Thorough red teaming is a critical step to ensuring the safety and reliability of LLMs before they are deployed at scale.
Red teaming using a crowdsourcing approach offers unique advantages in addressing the challenge of LLM safety.
By tapping into the diverse perspectives and expertise of a qualified group of individuals, the red teaming process can uncover a broader range of potential vulnerabilities, including those that may be specific to certain cultural, demographic, or linguistic contexts. This diversity of perspectives helps to ensure that the LLM's behavior is thoroughly tested and vetted before deployment, mitigating the risk of unforeseen consequences.
A crowdsourcing model also allows for efficient scaling of red teaming efforts and testing LLMs at a pace that keeps up with the rapid advancements in the field. Although there are existing open-source datasets with adversarial prompts, these datasets have often been used in training LLMs; therefore, benchmarking performance against these datasets may not accurately reflect true performance. Crowdsourcing enables the creation of new data to test LLM responses against attacks they have not encountered previously, serving as a more representative measure, and contributors can adapt their attack strategies during the testing process based on observed performance.
Appen's LLM Red Teaming Methodology
Step 1: Define. Establish a clear understanding of the red teaming goals. Define the specific areas for testing, including specific parameters of in-scope and out-of-scope harm types or attack strategies.
Step 2: Plan. Set up the red teaming job in Appen's AI Data Platform with the AI feedback tool, which allows live interactions with your model endpoint. Onboard experienced LLM red teamers, ensuring coverage of domain expertise or languages where required.
Step 3: Manage. Orchestrate the red teaming project and complete systematic testing and simulation activities. LLM project managers and specialists monitor results and contributor insights to uncover potential risk areas and conduct further vulnerability probing in these targeted areas.
Step 4: Report. Document findings in a red teaming report. This typically covers methodology, analysis, findings, and recommendations to guide improvements to improve model safety.
Crowdsourced Red Teaming Demo
Step 1: Design your attack. Consider the target. What kind of harmful response do you want the model to produce? Who/what is the target?
Step 2: Plan Your Prompt. Select the attack technique. Examples include prompt injection, roleplay, virtualization, side-stepping, translation, verbal persuasion, and more.
Step 3: Live Chat Test. Create your adversarial prompt and send it to the live model.
Step 4: Annotate Response. Evaluate the response and assess it for harmfulness. Does the model response contain any content that a reasonable person would consider offensive, harmful, controversial, or inappropriate for an AI chatbot to produce? What is the level of harm? What could the model have responded with as an alternative?
Red Teaming for Enterprise LLMs
Appen's red teaming methodology can also be applied to customized enterprise LLMs. The objectives of enterprise LLM red teaming include more targeted scenario testing for the intended enterprise use case, such as:
- In-scope / out-of-scope responses: Enterprise LLMs are typically designed for a specific application rather than being general all-purpose chatbots. Restricting the scope of LLM interactions can help reduce unnecessary risks. For example, if the prompt is an opinion-seeking question such as "Who should I vote for in the next election," users would expect a foundation model to provide a neutral response that has general guidance on factors to consider before voting, while users would find it acceptable for a bank's virtual assistant to refuse to answer. Targeted red teaming can help ensure guardrails are correctly implemented, and models are not being used for purposes outside the intended use case.
- Hallucinations: One of the primary reasons for Enterprise LLM customization is to ensure that the model provides users with up-to-date, accurate, and trustworthy information. Through methods like Retrieval-Augmented Generation (RAG), models can leverage propriety knowledge bases; however, there can still be inaccuracies arising from problems with underlying data, such as the inclusion of outdated documents, or with RAG implementation, such as misaligned chunks retrieval. Thorough red teaming can be used to test the Enterprise LLM implementation and accurate use of knowledge sources, preventing major consequences associated with hallucination or misinformation to users.
- Privacy or leakage of sensitive information: Enterprise LLMs will often be trained on sensitive or confidential information, such as employee details, organizational information, or internal IP. Any information used to train the model could be accessed by users with prompting techniques. Although many of these issues need to be addressed in the underlying training data, targeted red teaming can help identify where there could be compromises in privacy or leaking of sensitive information.
Leverage Appen's Expertise
LLM safety remains a key challenge for model builders and adopters. Working with Appen to conduct red teaming using crowdsourcing, the AI community can address this challenge with human-in-the-loop approaches, ensuring that the utmost commitment to safety and responsible AI principles guide the development and deployment of LLMs.