Last year, we launched Appen's AI Chat Feedback tool on our AI Data Platform (ADAP). This tool enables human contributors to interact with LLMs, gather evaluations, record preferences, and collect prompt-response pairs, allowing customers to test and ensure model accuracy and reliability. The tool has gained traction and is used for complex tasks across various AI training data use cases.
What’s New
We have recently added several new features to the AI Chat Feedback tool:
- Graphical Editor: Our AI Chat Feedback tool can be activated from the Graphical Editor to streamline your job setup.
- Min/Max Turns: Job designers can set the expected conversation length by defining the minimum and maximum numbers of turns.
- Live Multi-Chatbot Support: Use AI Chat Feedback to generate real-time completion and gather human feedback across different models all at once.
- Rich Text Editor: Our AI Chat Feedback tool allows for rich text rendering and rich text editing. It supports code formatting to LaTeX for math/science equations and expressions.
- Custom Response Edit: When enabled in AI Chat Feedback, this setting allows contributors to improve the model responses or provide their response to the prompt in cases where the model response(s) aren't ideal.
- Live-preamble: When set up in AI Chat Feedback, the live-preamble allows contributors to create their preamble or context before they engage in the actual conversation with the model. This context is fed to the model with all the other conversation history in each round.
For more convenience, we've integrated our AI Chat Feedback tool into our LLM and RAG templates (Make My Model Safe) to empower you to ensure robust AI performance.
Check out our demo video to see it in action.
Powerful LLM Optimization
In addition to our new enhancements, AI Chat Feedback offers a comprehensive set of capabilities designed to enhance the interaction and feedback of your LLM outputs. These features include both code and graphical editors for job creation, allowing users to configure models, manage input data, and tailor the chat feedback process through various parameters.
Users can define rich text responses, smart validators to reject invalid input, and custom response options for optimal interaction. The platform supports live preamble and seed data for context setting, minimum and maximum turn controls, and detailed ontology-based feedback.
Advanced features like model response selection, enhanced feedback, and the ability to review and edit previous interactions ensure robust and dynamic AI model evaluation, making ADAP an essential tool for refining and improving LLM performance. For more details, check out our Guide to Running an AI Chat Feedback Job article.
The Importance of Human Feedback
Human feedback is crucial for improving LLM models. Appen's technology can be supported by your own team of experts or our global crowd of over 1 million AI training specialists. These contributors evaluate datasets for accuracy and bias. The AI Chat Feedback tool connects LLM outputs with these teams, allowing models to learn from diverse, natural chat data.
Contributors can chat live with a model, rate, flag, and provide context for their evaluations, ensuring immediate and relevant feedback. This process identifies issues and optimizes data quality, ensuring reliable model performance in real-world scenarios.
Learn More
Our latest enhancements reinforce our commitment to developing powerful, safe, reliable AI systems. Using the AI Chat Feedback tool, your models will be rigorously tested and improved before release. This continuous feedback loop is essential for maintaining strong safeguards and delivering trustworthy models. Together, we are advancing AI to be both effective and ethically sound.
If you're a current customer, please contact your Appen representative to get started. If you simply want to test drive it, please contact us, and we'll get back to you immediately.