Human in the loop: how HITL works in machine learning and AI agents

Just over half of organizations using artificial intelligence (AI) report at least one negative consequence from it, according to McKinsey's State of AI 2025. High performers in that study more often define when AI outputs need human validation (Singla et al.).

What is human in the loop?

Human in the loop is an approach to building AI systems in which people intervene at defined stages, such as labeling training data, judging model outputs, or approving actions, and their decisions feed back to improve the model. It combines machine throughput with human judgment where the model is uncertain, stakes are high, or ground truth is subjective.

The feedback path is what defines it. A reviewer who only fixes outputs is doing quality control. A HITL system sends those corrections back into training data, evaluation sets, or routing thresholds, so the model needs less help over time (Mosqueira-Rey et al., 2023).

Humans typically enter the loop at four stages:

• Data creation: annotators label raw data or write demonstrations that define the target behavior.
• Preference modeling: reviewers rank or rate model outputs. This is the basis of reinforcement learning from human feedback (RLHF) (Ouyang et al., 2022).
• Evaluation: domain experts score outputs against rubrics in places where automated metrics miss nuance.
• Runtime oversight: reviewers resolve low-confidence predictions or approve consequential actions before they run.

Natarajan et al. (2024) draw a useful line. In a true HITL system the AI makes the decision and a human corrects it. In "AI in the loop" the human decides and the AI assists. Which one you're building changes what you measure: model accuracy after correction, or human decision quality with AI support.

‍

How reviewer quality shapes training data is covered in more depth in human-in-the-loop AI data quality.

Human in the loop vs. human on the loop vs. active learning

These terms are often used as if they mean the same thing. They lead to different system designs, staffing models, and audit trails.

‍

Active learning is a way of choosing samples, not a form of oversight. It often runs inside a HITL loop, picking which items humans see (Settles, 2009). Most production systems mix approaches: HITL during training and for low-confidence cases, HOTL across the rest of production traffic. The monitoring side of that setup is covered under continuous HITL monitoring.

Why human in the loop AI matters: failure modes it catches

Automated pipelines usually fail without raising an error. HITL exists to catch failures that confidence scores and automated metrics miss:

• Hallucination: large language model (LLM) outputs that read well but are factually wrong pass fluency and format checks.
• Subjective ground truth: for tasks such as toxicity or helpfulness, annotators' beliefs and identities measurably shift labels (Sap et al., 2022).
• Miscalibration under distribution shift: modern neural networks are often overconfident, so confidence can stay high while accuracy drops on new inputs (Guo et al., 2017).
• Reward hacking: models that optimize a proxy metric produce outputs that score well but miss the intent. Human raters catch the gap.
• Irreversible agent actions: agents that send messages, move funds, or change code can cause harm before aggregate monitoring notices.
• Regulatory exposure: Article 14 of the EU AI Act requires high-risk systems to be designed for effective human oversight, and Article 12 requires event logging (EU AI Act, 2024). The National Institute of Standards and Technology (NIST) treats human oversight as part of its Govern and Manage functions (NIST, 2023).

Adversarial testing finds these failures before users do. See large language model red teaming and red teaming MLLMs.

How human in the loop machine learning works

Every HITL system runs the same four-part cycle: score, route, review, learn.

Score: estimate confidence and risk

The model outputs a confidence value for each prediction. That value only matters if it's calibrated, meaning a 0.9 score is right about 90% of the time. Temperature scaling is a standard post-hoc fix (Guo et al., 2017). Teams then add a risk score from business rules, such as content category, transaction value, or whether the action can be undone. Selective classification makes this formal: the model abstains below a threshold, giving up coverage in exchange for accuracy on what it keeps (Geifman and El-Yaniv, 2017).

Route: send each item to the right reviewer

Thresholds split traffic into tiers. A typical setup is auto-accept, single review, and multi-reviewer adjudication. A second layer routes by expertise: a clinical-note item goes to a credentialed reviewer, a general sentiment item to a trained generalist.

Review: apply a rubric, not a gut call

Reviewers score against written criteria with worked examples. Hidden gold items (known-answer test cases) measure each reviewer's accuracy continuously. Inter-annotator agreement then shows whether the guidelines themselves are clear. Rubric-based LLM evaluation covers rubric design in detail.

Learn: feed decisions back

Corrections go into training data. Disagreements lead to guideline revisions. Override rates trigger threshold recalibration. Preference judgments train reward models for RLHF. Where outputs can be checked automatically, such as code that compiles or math with a known answer, teams increasingly pair human feedback with RLVR, short for reinforcement learning with verifiable rewards.

At runtime: approval gates for AI agents

In agentic systems, the gate sits in front of actions rather than labels. Read-only and reversible tool calls go through automatically; irreversible ones wait for approval. Gate placement is a design decision covered in AI agentic workflow and Agentic AI.

Worked example: a moderation classifier with tiered review

Illustrative scenario. A platform screens 10,000 user posts a day. After calibrating on 2,000 held-out labeled posts, the team sets three bands:

Humans review 2,200 posts a day (22%) instead of 10,000, and the lowest-precision traffic gets the most scrutiny.

Then one post reads: "That referee should be shot for that call." Five annotators split: two choose not toxic (hyperbole), two choose toxic: violent threat, and one chooses borderline. With no majority, an adjudicator applies the guideline on violent idioms with no specific real-world target and records borderline, no action. The batch's Krippendorff's alpha comes in at 0.71, below the 0.80 target. The team rewrites the violent-idiom section with this post as a reference example and re-tests before the next batch.

Hidden gold items for catching reviewer drift are explained in ADAP quality flow and test questions.

How to implement a human in the loop framework

1. Classify decisions by risk: Score each output type on reversibility and impact. Anything in an EU AI Act Annex III category (employment, credit, essential services) needs oversight built in. Pass: every output type has an owner and a risk tier.
2. Calibrate before you set thresholds: Measure expected calibration error (ECE) on a held-out set and apply temperature scaling if needed (Guo et al., 2017). Working target: ECE below 0.05.
3. Set auto-accept thresholds from a precision target: Pick the lowest confidence threshold at which auto-accepted precision meets your target, for example 99% for safety categories and 95% for low-risk ones. Your review rate is the share of traffic below it.
4. Build a rubric and a gold set: Define each label with positive and negative examples. Seed 5-10% of each batch with gold items. Fail: take reviewers whose gold accuracy drops below your bar (85% is a common starting point) off the task.
5. Measure agreement every batch: Use Krippendorff's alpha. Pass: α ≥ 0.800. Tentative: 0.667-0.800. Fail: below 0.667, in which case revise the guidelines before continuing (Krippendorff, 2004).
6. Close the loop on a schedule: Add reviewed items to training and evaluation sets each cycle. Audit 1% of auto-accepted items. Recalibrate if the override rate in those audits goes above 2% for two weeks in a row.
7. Log every human decision: Record reviewer ID, timestamp, input, model output, decision, and rationale. This supports EU AI Act Article 12 record-keeping and NIST's Measure function.

For safety-critical categories, three-way labeling adds a useful middle option. See AI safety evaluation tricategorical reasoning.

How Appen supports human in the loop

Research and engineering teams design the loop. Appen provides the reviewers, workflows, and quality controls that let it run at production volume, and every engagement comes with measurable quality evidence.

• Routing in the AI Data Platform (ADAP): set up confidence-based routing, multi-judgment tasks, and adjudication steps, with model predictions loaded as pre-labels so reviewers correct rather than start from scratch. See the AI Data Platform.
• Quality control with test questions and golden sets: hidden known-answer items score each contributor continuously, and anyone below the accuracy threshold is removed from the task automatically.
• Contributor recruiting by region, device, and channel: reviewer pools drawn from 500+ locales, filtered by language, dialect, demographics, and device type, to reduce the single-population bias described by Sap et al. (2022).
• Subject matter expert (SME) review: credential-screened domain experts produce preference data, rubric scores, and adversarial prompts through SME RLHF / frontier alignment and red teaming.
• Production sampling: human-on-the-loop review of live model outputs through continuous HITL monitoring and AI hallucination benchmarking.
• Off-the-shelf datasets: licensed, pre-collected data to seed a model before the loop starts, available in the data catalog.

Deliveries include per-batch agreement scores, gold-set accuracy by contributor, and adjudication logs, the evidence auditors and research leads ask for. The work runs under SOC 2 and ISO 27001 controls and draws on 30 years of human-data operations.

Talk to our team about designing your human-in-the-loop workflow →

Frequently asked questions

What does HITL mean outside of machine learning?

In engineering, HITL can also mean hardware in the loop: a testing method where real hardware components run against simulated environments, common in automotive and aerospace validation. In defense and control systems, "man in the loop" describes an operator who must authorize an action. In AI, HITL means human review or feedback that shapes model behavior. The surrounding context, not the acronym, tells you which is meant.

Does the EU AI Act require human in the loop?

Article 14 requires high-risk AI systems to be designed so people can effectively oversee them, including interpreting outputs, overriding decisions, and halting the system. It doesn't require review of every decision; human-on-the-loop designs can comply if the oversight is effective. For certain remote biometric identification systems, two people must verify a result before anyone acts on it. Standalone Annex III high-risk obligations (including Article 14 human oversight) phase in from 2 December 2027 under the AI Omnibus timeline; 2 August 2026 remains a broader AI Act applicability and enforcement milestone.

How do you add human in the loop to AI agents without slowing them down?

Ask reviewers to approve an agent's plan once rather than every step. Batch approval requests so they can be handled asynchronously, and set timeouts that fall back to the safe action, such as cancel or escalate, when no one responds. Track approval latency and the share of approvals that are rubber-stamped. An approval rate near 100% usually means the gate is in the wrong place.

What is expert in the loop?

Expert in the loop is a HITL variant where reviewers are credentialed domain specialists, such as physicians, lawyers, or senior software engineers, rather than generalist annotators. It's used when correctness needs professional judgment, for example clinical summarization or security-focused code review. Expert time costs more, so teams usually reserve it for the lowest-confidence or highest-risk slice and handle the rest with generalist reviewers or automated checks.

Can human reviewers introduce bias into a HITL system?

Yes, especially on subjective tasks. Mitigations include recruiting reviewer pools that match the user population, keeping each individual label instead of collapsing to a majority vote, publishing disagreement distributions alongside final labels, and auditing for systematic gaps between reviewer subgroups. On ambiguous items, disagreement often reflects real ambiguity in the task rather than noise, and removing it can hide edge cases the model will meet in production.

Still scoping your loop? Talk to our team about designing your human-in-the-loop workflow →

References

• European Union. (2024). Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union. https://eur-lex.europa.eu/eli/reg/2024/1689/oj
• Geifman, Y., & El-Yaniv, R. (2017). Selective classification for deep neural networks. NeurIPS 2017.https://arxiv.org/abs/1705.08500
• Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On calibration of modern neural networks. ICML 2017.https://arxiv.org/abs/1706.04599
• Krippendorff, K. (2004). Content analysis: An introduction to its methodology (2nd ed.). Sage Publications.
• Mosqueira-Rey, E., Hernández-Pereira, E., Alonso-Ríos, D., Bobes-Bascarán, J., & Fernández-Leal, Á. (2023). Human-in-the-loop machine learning: A state of the art. Artificial Intelligence Review, 56, 3005-3054. https://doi.org/10.1007/s10462-022-10246-w
• Natarajan, S., Mathur, S., Sidheekh, S., Stammer, W., & Kersting, K. (2024). Human-in-the-loop or AI-in-the-loop? Automate or collaborate? arXiv; AAAI 2025.https://arxiv.org/abs/2412.14232
• NIST. (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1. https://doi.org/10.6028/NIST.AI.100-1
• Ouyang, L., et al. (2022). Training language models to follow instructions with human feedback. NeurIPS 2022.https://arxiv.org/abs/2203.02155
• Sap, M., Swayamdipta, S., Vianna, L., Zhou, X., Choi, Y., & Smith, N. A. (2022). Annotators with attitudes: How annotator beliefs and identities bias toxic language detection. NAACL 2022.https://arxiv.org/abs/2111.07997
• Settles, B. (2009). Active learning literature survey (Computer Sciences Technical Report 1648). University of Wisconsin-Madison.
• Singla, A., Sukharevsky, A., Yee, L., Chui, M., & Hall, B. (2025). The state of AI in 2025: Agents, innovation, and transformation. McKinsey & Company.