Anthropic, the company behind the Claude models, ran a simple test. Through a research vendor, it hired 1,053 people to work with an AI coding assistant. Every so often, the assistant asked for permission to take the next step, and people clicked "yes" or "no." Partway through each session, one ordinary request was swapped for a clearly dangerous command. Nothing bad could actually happen, because only the text on the screen had changed.
Only 143 people, or 13.6%, stopped the dangerous command. An automated safeguard, a program that checks every command before it runs, stopped 89% of the same commands.
These people knew they were part of a study. They just didn't know what was being measured. In day-to-day work, developers get requests like these dozens or even hundreds of times a day. The next one looks exactly like the last. If you approve everything, you stop seeing what matters.
That's why the safety of an AI agent in your business doesn't depend on how often a person clicks "approve." It depends on where the agent stops. You choose those points. Below, we walk through how to choose them step by step, using one process as an example. At the end, there's a table you can fill in for your own business.
What does human in the loop mean?
Human in the loop is a setup where an AI agent stops at a chosen point and waits for a person to decide. The agent reads, calculates and drafts. A person approves, corrects or rejects the work before anything leaves the company. We call that point a checkpoint.
It doesn't mean a person clicking "OK" on every step. Anthropic's test shows that doesn't work. It means a handful of well-chosen moments where a human decision matters most. If you're still deciding whether you need an agent at all, start with our guide to AI employees.
Why doesn't approving everything work?
Because a person who gets dozens of permission requests eventually stops reading them. Anthropic sees this in its own coding tool. In August 2026, the company reported that users approve 97% of permission requests. According to Anthropic, a rate that high suggests many users click through on reflex instead of checking.
The test we opened with also compared people and the safeguard on the same commands. The safeguard blocked 800 dangerous commands that a person had approved. People blocked only 6 that the safeguard had let through.
That doesn't mean the person is useless. It means the person is standing in the wrong place. Anthropic puts it plainly: effective oversight "doesn't require approving every action". In our view, the person needs to be where they can step in when it really counts.
A case Anthropic described from its own work shows where that place is. An agent was asked to set up read-only access to billing data. It reached for a role that grants full administrative control. The safeguard stopped it because it noticed the gap between what was requested and what the agent was about to do. In a business, the equivalent is an agent that's supposed to send a quote with the standard discount and enters one twice as high. A checkpoint belongs exactly where the outcome goes beyond the task. We cover how oversight differs from approving everything in can you trust AI in your business.
Where should an agent stop?
An agent can work in three ways, and you choose one for each step separately. In one process, the same agent can ask before sending a quote and send follow-ups on its own.
- Asks first (human in the loop). The agent prepares, a person approves, and only then does anything go out. This fits steps you can't undo where a mistake is costly: a payment, a quote with an unusual discount, a reply to a complaint.
- Acts and reports (human on the loop). The agent acts on its own, and a person sees the results and can step in. This fits steps where a mistake is cheap: a follow-up reminder, sorting support tickets, an update in your CRM.
- Draft only. The agent can't send anything. It writes a draft, and a person sends it from their own account. This fits the early weeks and any place where your own voice matters.
Four questions decide which one to use:
| Question | Example |
|---|---|
| Can it be undone? | You can fix a draft in your system. You can't recall a sent email. |
| What does a mistake cost? | A typo in an internal note costs nothing. A wrong discount costs your margin. |
| Who will see the result? | A report for your team stays inside. A reply to a customer doesn't. |
| How often has the agent been wrong at this task? | On a new task, the agent always asks first. After a few weeks without corrections, you can ease up. |
The minimum is simple: a person signs off on everything that can't be undone and could be costly. A follow-up reminder can't be undone either, but a mistake in one costs little. At the start, the agent asks for approval on every step where any of these questions gives you pause. You ease up only when the agent has gone weeks without corrections and a mistake at that step is cheap. There's no single threshold for everyone, because a mistake costs an accounting firm one thing and a clothing store another.
The authors of Levels of Autonomy for AI Agents (a 2025 paper not yet peer-reviewed) reach the same conclusion. How much freedom an agent gets is a decision made by the people who set it up, not a property of the model.
What does this look like in a small business?
Take a packaging manufacturer that sells to other businesses. An agent prepares quotes for the sales team. In the morning, the sales manager opens the "waiting for you" list. Five quote requests came in overnight, and the agent has prepared five quotes. Three have already gone out because they matched the price list and the standard lead time. One is waiting for a "yes" because the discount went over the limit set for sales reps. On the fifth, the agent has a question. The customer wants delivery in two weeks, and the agent doesn't know whether production can make it. Instead of writing five quotes from scratch, the sales manager makes two decisions.
| Step | Who acts | How the agent works |
|---|---|---|
| Reads the request and pulls out product, quantity and date | agent | on its own, it's only an internal note |
| Prices the order from the current price list | agent | on its own, visible before sending |
| Writes the quote | agent | draft |
| Sends a quote that fits the price list and lead time | agent, sales manager sees a list of what went out | acts and reports |
| Discount over the limit or an unusual condition | sales manager | asks first |
| Follows up after three days without a reply | agent | acts and reports |
That's what the process looks like after a few weeks, not on day one. At the start, a person approves every quote, because a sent quote can't be taken back. The agent sends standard quotes on its own only after several weeks without a single correction. By then, a mistake on a price-list quote is cheap, and the expensive exceptions still wait for a person.
In this process, the person catches two things the agent won't. The first is a fact about the company that sounds plausible but is wrong. The agent will write "delivered in 10 business days" because that's what last year's quote said. You know production has a three-week backlog this quarter. The second is a promise you can't keep, such as free shipping or payment terms you don't offer. The agent writes what fits the rest of the text. The person knows what's true.
How do we work with agents ourselves?
In our work, there's always a person alongside the agent: either approving before it acts or watching the results, ready to step in. A typical task follows this rhythm:
- We plan together. The agent shows a plan first, and we approve it or correct it.
- The agent does the tedious part. Once the plan is approved, the agent carries the repetitive work through to the end.
- Reviewers check the result. At the end, reviewers, themselves AI, assess the work.
- We check the reviewers. A reviewer can be wrong too, so a person has the final say.
What we correct most often in a plan is scope and direction. Sometimes the agent wants to do too much or touches things outside the task. Sometimes everything is technically right, but it isn't what the business needed. Only the owner knows the direction, and the agent can't guess it.
An agent's freedom goes both ways. An agent that proves itself gets more. An agent that reveals a new risk loses a permission. Elev, our AI agent on X, publishes on its own because it has tight limits and has proven itself. It started with a narrow scope and close oversight, and autonomy came later. It acts alone only within that scope, never outside it.
Anthropic sees a similar pattern among its users. New users let the agent act without asking for approval in about 20% of sessions, and experienced users in over 40%. Freedom grows with the experience of the person overseeing the agent.
What mistakes do businesses make when setting checkpoints?
The idea is simple, but the first setup tends to trip people up. Here are four mistakes that are easy to miss.
The checkpoint sits at the end, not at the decision. A person approves a finished quote email, but the mistake happened three steps earlier, in the pricing. A well-written email makes a wrong number easy to miss. Put the checkpoint where the decision is made, not where the finished document appears. For the packaging manufacturer, that means the discount and the lead time, not the wording of the quote.
A question with no reason given. The agent shows "Approve?" and nothing else. The person doesn't know what to look for, so they click "yes." A good checkpoint says why the agent stopped. For example: "discount 15%, limit is 10%" or "customer wants delivery in two weeks, I don't know if production can make it." Then the decision takes a moment and actually means something.
No plan for "no." It's easy to plan what happens after "yes" and forget what the agent does after a rejection. It might revise the draft based on your notes or hand the case to a person. It might also let the customer know the quote will come later. Without that answer, a rejected quote sits in the queue and nobody finishes it.
The same rules forever. Checkpoints set in the first week won't necessarily fit three months later. The agent builds a track record, your prices change, and new kinds of requests come in. If nobody revisits the checkpoints, they end up too loose or too tight. Too loose is risky. Too tight wears out the people doing the approving.
How do you map checkpoints for your own business?
Pick one process you want to hand to an agent. Map it in a table, step by step: how the agent works and who decides. One meeting is enough for a first version.
| Step | How the agent works | Who decides (a named person) | What they check |
|---|---|---|---|
| Sending a quote with a discount over 10% | asks first | Anna, sales manager | discount, lead time, payment terms |
| Replying to a complaint | asks first | Peter, customer service manager | tone, promises, whether we offer compensation |
| Follow-up after three days without a reply | acts and reports | Anna, reviews the sent list once a week | that follow-ups don't go to customers who already said no |
In the "who decides" column, write a person's name, not a department. "The team approves it" can easily end with nobody approving it. Agree on who covers for that person when they're on vacation.
After a month, come back to the table. Checkpoints that caught nothing in that time can switch to "acts and reports." Where something slipped through, add a new one.
Human in the loop is here to stay, because no business can hand every decision to AI. Set up well, it gives you more than AI or people working alone. The agent handles the repetitive work, and the person steps in where their decision matters most.
If you want an agent like this, we build internal agents with human approval built in. They draft quotes, emails and reports, and leave the decision to you. If you already have an agent in place, we can check where its checkpoints sit and whether they're in the right places.
Want an agent that knows your company?
We build agents that draft quotes, emails and reports, and leave the decision to a person. We set the checkpoints with you, on your own process, before the agent starts work.
See agents with human approvalFrequently asked questions
Is ChatGPT human in the loop?
In a sense, yes. In a normal ChatGPT conversation, you read every answer and decide what to do with it. An agent works differently, because between your questions it sends, saves and books things on its own. That's why an agent's checkpoints have to be planned, while a chat has them built in.
What does human in the loop mean in machine learning?
When AI models are built, human in the loop means people who label data and correct a model's answers before release. That's the meaning you'll see most often on Google. In a business that runs an agent, it means something simpler: who tells the agent "yes" or "no," and when.
How can a person keep up with an agent that works faster?
They don't need to read everything. The agent stops only at the agreed checkpoints, and the person spot-checks the rest after the fact. An activity log helps too: a record of what the agent did, when and based on what.
Who in the company should approve an agent's decisions?
Someone who knows the process, is responsible for its outcome and can reject the agent's proposal on their own. If more than one person makes decisions, split the roles: one sets the limits, the other approves the exceptions. That way someone actually looks at the exceptions instead of waving them through. Our article on the 30% rule in AI shows what oversight looks like when it exists only on paper.
Does human in the loop slow automation down?
Only where a checkpoint sits in the wrong place. If a person approves cheap, reversible steps, they lose time and soon read less carefully. If they approve only expensive, irreversible decisions, the process still speeds up, because the agent does the rest.
How much does an AI agent with human approval cost?
Our internal agent with human-in-the-loop approval starts at $4,600 plus from $350 a month. It drafts emails, quotes, documents or reports, and a person on your team signs off on them. Team onboarding is included: on your own process, we show your people how to approve, correct and reject the agent's work.
Does the EU AI Act require human in the loop?
It requires human oversight of high-risk AI systems, for example in hiring or credit scoring. For those systems, the obligations apply from December 2, 2027 (as of October 2026). Under Article 14, a person must be able to disregard the system's output and stop it with a "stop" button. The article also calls out the risk of relying on AI output on reflex. Most agents in a small business are not high-risk systems. Even so, Article 14 is a good picture of what oversight should be able to do. More in our guide to the EU AI Act for small businesses.