Who Teaches the Engineer to Manage the AI?

I recently attended a discussion at the University of Nevada, Reno with Bill Dally, NVIDIA’s chief scientist and senior vice president of research.

There was plenty to consider about artificial intelligence, computing, and the changes ahead. But the point that stayed with me involved something considerably older than AI.

Management.

As I understood his discussion, software engineers will increasingly need to direct AI systems, monitor their work, and evaluate the results. They will also need substantial knowledge of the field in which that software will operate.

That sounded reasonable. It also raised a question.

After the session, I asked him how the current generation of software engineers would learn those management skills.

I came away with the impression that this was a challenge even within NVIDIA. When I asked an associate dean how business learning might be incorporated into computer science education, he did not have an answer ready.

That doesn’t mean there is no work underway. But it left me thinking that we may be moving rapidly toward a new description of a software engineer’s job without being quite as clear about the preparation it requires.

Someone Still Has to Define the Assignment

During my working years, I encountered plenty of technically capable people. Being good at the technical work did not automatically make someone good at organizing and assigning work to others.

A project begins with a picture of what we want to accomplish and why. Then someone has to divide it into workable pieces, assign those pieces to people capable of doing them, and explain how their contributions fit together.

Delegation also requires context, limits on authority, and a clear understanding of what constitutes an acceptable result.

“Build me a system that handles customer orders” seems straightforward until someone asks what happens when an item is unavailable, a payment fails, or a customer changes an order after it has shipped.

Those questions help define the overall assignment. We might then divide the work into inventory, payments, shipping, and customer notifications. Each piece needs its own objectives and checks, while remaining consistent with the whole.

A payment function can work perfectly by itself and still create a business problem if it charges for an unavailable item without explaining what happens next.

That is why monitoring individual assignments is only part of management. Someone must also check whether the pieces work together to achieve the intended result.

I see no reason that responsibility disappears when AI performs the detailed work.

Activity Is Easier to Count Than Accomplishment

The objectives also need measures that tell us something useful.

Can the system handle an ordinary order correctly? Does it update inventory and initiate shipping correctly? Does it respond appropriately when payment fails? Does it prevent duplicate charges? Can customers understand the messages they receive?

Those are tests of results.

The number of AI requests made, lines of code generated, or tokens consumed tells us about activity and resource use. Tokens—the units AI systems use to process information—can help track costs. Their consumption doesn’t establish that the project is succeeding.

I have written before about organizations confusing measurable activity with useful accomplishment. AI gives us new ways to repeat that old mistake, potentially at impressive speed.

A vague objective, broad authority, and little monitoring strike me as a poor combination. Faster execution could simply get us further down the wrong road before someone notices.

The Familiar Art of Delegation

After the discussion, I also asked Dally whether he was familiar with Peter Drucker. He was. Drucker has long been one of my reference points for thinking about management, and these questions sounded familiar: What result are we trying to achieve? What responsibilities are being assigned? How will performance be evaluated?

Those questions existed long before anyone could ask a computer to write software.

Of course, managing an AI system differs from managing a person. The system doesn’t have career ambitions, family obligations, or an opinion about the office coffee. Nor can we assume it will recognize every occasion when it should stop and ask for clarification.

The person directing it still needs to make judgments about scope, authority, progress, and results.

That responsibility could reach people much earlier in their careers. An engineer who once would have spent years doing individual assignments may now be coordinating several streams of AI-generated work.

We should think carefully about what prepares someone for that responsibility.

Testing the Work Is Part of the Job

The companies building AI are developing ways to evaluate its work. Anthropic, which develops Claude, has published research on how people supervise agents and a technical explanation of how to test their performance.

I read these with interest, although I will admit that parts of the technical explanation required more concentration than I could comfortably supply.

One useful distinction did emerge: an agent saying it completed an assignment is different from evidence that it actually did. If it says it booked a flight, does the reservation exist?

That is an important test. But management has another level of responsibility: deciding whether the assignment serves the larger objective.

Returning to our customer-order example, we need to know whether payments, inventory, shipping, and notifications work correctly. We also need to know whether the complete system helps the business fulfill orders accurately, avoid unnecessary costs, and serve its customers.

The technical tests can help establish the first. Someone still has to define and measure the second.

Anthropic’s work does not claim to be a complete management framework. It addresses important parts of evaluation and oversight. My point is that those practices need to fit within a larger process connecting individual assignments to the result the organization wants.

Giving an AI an assignment is the beginning of supervision. Completing that assignment successfully is evidence of progress—provided it was the right assignment in the first place.

Where Does the Experience Come From?

The other point that stayed with me was the need for domain knowledge.

That is an impressive-sounding term for knowing the field in which you are working. Software for a hospital requires understanding something about healthcare. Software for a delivery company requires understanding how deliveries actually happen.

I have worked around both. The details matter.

But how does a beginning engineer acquire that understanding?

Traditionally, some of it came through doing smaller assignments, encountering mistakes, asking experienced colleagues questions, and gradually learning why apparently simple requests were complicated.

If AI takes over much of that early work, we need to consider how people will gain the experience required to supervise it.

When I was bringing new software people into an organization, I would assign them to work in the operating departments they would eventually develop tools for.

Actually seeing how the work was done added context quickly. It also exposed them to problems and opportunities for improvement that might never appear in a written request for software.

They could ask why something was done a particular way—and hear the answer from the person who had to do it every day. Sometimes there was a good reason. Sometimes the answer was simply that nobody had found a better approach.

That experience helped them understand what they were being asked to improve.

Universities and employers could build similar experience into the preparation of people who will direct AI. Real projects, time in operating departments, and responsibility for checking results would give management principles something concrete to attach to.

AI may help with that learning too. But expecting deep knowledge to appear simply because it has become more valuable seems an inadequate plan.

This Reaches Beyond Software Engineers

I have written about drones, robots, and AI agents over the last several months. The same questions keep returning: Who decides what is to be done and why? Who directs the work, and who decides whether it was done properly?

A small-business owner assigning customer correspondence to AI faces a version of this. So does someone using an agent to compare insurance or research housing choices.

I face it when I use AI to help develop these articles. I need to explain what I am trying to say, question the response, check important claims, and recognize when a polished paragraph has wandered away from my point. The tool can assist with the work. I still have responsibilities.

The encouraging possibility is that more people will be able to accomplish things that previously required resources they didn’t have. But making good use of that opportunity requires judgment.

I left the UNR discussion thinking that management education may deserve a larger place in our conversation about AI.

We are spending considerable effort teaching machines to do more. We also need to prepare people to direct that work intelligently.

If you have supervised people or projects, what lesson would you want someone to learn before putting them in charge of AI?