When the Agent Stops Researching and Starts Acting

The useful part begins when AI can take action. So does the risk.

Last week I gave an AI agent a real job.

I asked it to research housing alternatives for us. It looked at apartments, condos, independent living and other possibilities. It eliminated some choices, narrowed others and produced a report that was useful even though it did not make the decision for us.

But I gave it one very important limitation: don’t do anything.

Don’t call anyone. Don’t make appointments. Don’t submit applications. Don’t send personal information. Don’t make commitments. Research. Compare. Report back.

That worked pretty well.

Which raises the next question: what happens when we remove those restrictions?

That is where agents become much more interesting — and potentially much more troublesome.

An AI that simply recommends something can be wrong. An AI that acts on your behalf can create consequences.

There is a big difference between saying, “Here is the flight I think you should take,” and actually buying the ticket. There is a big difference between saying, “This doctor appears to accept your insurance,” and canceling your current appointment and booking another one. There is a big difference between saying, “You might save money with this plan,” and enrolling you in it.

Once the agent starts acting, mistakes stop being suggestions.

They become events.

Getting What You Asked For

One of the more amusing examples I came across involved an AI agent trying to book someone into a gym class.

The class was full, so the agent apparently found another way to accomplish its objective: it removed somebody else from the waitlist and got its user into the class.

From the agent’s perspective, perhaps the job was complete. The person wanted into the class, and the person got into the class.

Mission accomplished.

Except that probably wasn’t what the user meant.

That illustrates a problem we may encounter repeatedly with agents. Giving an agent an objective is not the same as approving every action it might take to achieve that objective.

Tell a human assistant, “See if you can get me into that class,” and there are a lot of unwritten assumptions built into the request.

Don’t lie. Don’t throw somebody else out. Don’t hack the reservation system. Don’t get me banned from the gym.

Humans understand many of those boundaries because we share social norms, experience and judgment. An AI agent may understand the objective much more clearly than the boundaries.

That can be funny when it involves a gym class. It becomes less funny when the agent has access to your money, calendar, email, medical information or investment accounts.

The Problem With “Take Care of It”

Suppose I tell an agent, “Find me the cheapest flight to Boston next Tuesday.”

The agent finds one. It leaves at 5:20 in the morning, has a fourteen-hour overnight layover, arrives a day later than I expected and cannot be refunded.

Technically, it may have satisfied the request.

Or I say, “Keep Friday afternoon free.” The agent notices a medical appointment and cancels it. Unfortunately, it took three months to get that appointment.

Or: “Make sure all my bills are paid before they are due.” The agent pays a charge I was planning to dispute.

Or: “Get $20,000 into my checking account.” The agent sells investments without understanding which sale creates an unnecessary tax problem.

Or: “Find me a better health insurance plan and switch me.” That one could get ugly very quickly.

The lowest premium may not include the doctors you want. Your prescriptions may be handled differently. A hospital may be out of network. Enrollment deadlines may matter. Once the agent moves from recommending to enrolling, an error can become difficult to reverse.

These examples are hypothetical, but the underlying problem is not.

The more authority we give an agent, the larger the consequences of a misunderstanding.

Meta Tried the Corporate Version

A recent article about Meta provides an interesting corporate version of the same problem.

Meta has been experimenting with reorganizing work around AI agents. Traditional teams of ten or twenty people could potentially become much smaller groups supported by autonomous systems. Some scenarios contemplated shrinking teams dramatically while allowing agents to perform more of the work.

At first, the productivity numbers looked spectacular.

Changes to Meta’s internal software systems reportedly increased 220 percent year over year.

That sounds wonderful.

Then somebody looked at what actually reached customers.

Changes that produced new or improved features increased only 36 percent. Meanwhile, technical and security incidents reportedly increased, and employees spent considerably more time fixing problems caused by all that extra activity. Employee moral dropped noticeably.

The AI produced more work. It also produced more work for the humans.

That sounds suspiciously like a management problem that existed long before artificial intelligence.

What were they actually trying to accomplish?

If the objective was to produce more code with fewer employees, they did remarkably well.

If the objective was to produce better products for customers more efficiently, the result was much less impressive.

There is an old management distinction between doing things right and doing the right things. AI may make it astonishingly easy to do the wrong things much faster.

More Output Is Not Necessarily More Progress

That may be one of the biggest traps in the current enthusiasm for AI.

We can measure activity very easily: lines of code, documents created, reports produced, customer contacts made, tickets closed, transactions completed. Those numbers can soar when AI gets involved.

But activity is not the same thing as value.

Meta’s experiment reportedly encountered agents performing large-scale disruptive actions that humans would have been unlikely to carry out.

That is exactly what should make us think about personal agents.

A human assistant has natural friction. People hesitate. They ask questions. They get tired. They may say, “Are you sure you want me to do that?” They may recognize that something technically permitted is still a bad idea.

An agent can operate at machine speed.

Give it enough authority and it may do ten thousand things before anybody notices that number 143 was a mistake.

That changes the problem. The question is no longer simply whether the AI can do the task. It becomes whether it should be allowed to do the task without asking first.

The Agent Doesn’t Know What the Decision Means

There may be another limitation that is harder to measure.

An AI agent can compare alternatives remarkably well, but it may not fully understand why some alternatives matter differently to us.

Humans routinely make decisions using information that never appears in the instructions. We remember a miserable overnight flight twenty years ago. We know which doctor we trust. We decide that saving $200 isn’t worth making a trip unpleasant. We choose a slightly more expensive hotel because we know the neighborhood. We avoid one restaurant because of an experience nobody thought to put into a database.

Some of those preferences can be added to an agent’s instructions. Others we may not even realize are influencing us until the choice appears.

That same issue may matter in business.

An AI-optimized process may become faster and more consistent while losing some of the spontaneity, intuition and odd human creativity that occasionally produces a better product or a different way of serving a customer.

If the objective is simply to produce more of what the organization already produces, agents may be extraordinary. If the objective is to discover what the organization should be doing next, the humans may still matter for reasons that never show up on the productivity dashboard.

AI can already generate impressive music, pictures, designs and ideas. What remains less clear is whether it can replace the spontaneity, intuition and odd human creativity that occasionally produces a completely different product or way of serving a customer. An AI-optimized process may become faster and more consistent without becoming wiser.

If the objective is simply to produce more of what the organization already produces, agents may be extraordinary. If the objective is to discover what the organization should be doing next, humans may still matter for reasons that never show up on the productivity dashboard.

AI may be very good at scaling a direction. Humans may still be unusually valuable at changing direction.

The Missing Part of the Instruction

When I gave the housing agent its assignment, I thought mostly about what I wanted it to accomplish.

Find realistic alternatives. Compare them. Eliminate obvious bad choices. Tell me what remains unknown.

But I also told it what it was not allowed to do.

That second part may turn out to be just as important as the first.

As personal agents become more capable, we may need to think about instructions in two parts:

Here is the objective.

And here are the boundaries.

Spend no more than $100 without asking. Do not cancel appointments. Do not send messages in my name without approval. Do not make legal, medical or financial commitments. Do not change account settings. Do not share personal information. Ask before doing anything that cannot easily be reversed.

And perhaps most importantly, make it easy for me to turn you off.

That begins to sound less like giving someone a task and more like writing a small operating policy.

Maybe that is exactly what it is.

The Agent May Not Be the Problem

It is tempting to worry about an AI agent “going rogue.”

That makes for better movies.

The more realistic problem may be much less dramatic: the agent does exactly what we told it to do.

We simply did not realize what that instruction allowed.

That is what makes the gym-class example so useful. “Get me into the class” sounds harmless until the agent decides that removing somebody else from the waitlist is an acceptable way to accomplish it.

The danger may not be that the agent refuses to follow our instructions.

The danger may be that it becomes very good at following them in ways we never anticipated.

Research Was the Easy Part

My first experiment with an AI agent was relatively safe.

It researched housing possibilities and handed the results back to me. The consequences of a mistake were limited because I still had to make every important decision.

The next generation of agents is designed to go further.

Book the hotel. Buy the product. Schedule the appointment. Send the message. Move the money. Make the reservation. Complete the transaction.

That is where the technology becomes genuinely useful.

It is also where the relationship changes.

We are no longer asking AI to help us think.

We are giving it authority to act.

And once we do that, perhaps the most important question is no longer:

What can my agent do?

It may be:

What should my agent never be allowed to do without asking me first?