It Works. But What Will Everybody Else Think?

A school district in western New York recently decided to spend about $60,000 on a humanoid AI robot named Sally.

Sally was going to high school.

The Salamanca City Central School District planned to use her to help students learn about artificial intelligence, robotics and technology. Students might ask her questions about programming or work with the technology that makes a humanoid robot function.

That sounds reasonable enough.

Then people outside the classroom began paying attention.

Teachers questioned whether robots belonged in schools. Parents raised concerns. State officials got involved. There were questions about privacy, surveillance, safety and whether artificial intelligence might eventually replace teachers.

Then someone discovered that the company making Sally has a corporate relationship with a company known for making realistic sex dolls.

At that point, Sally’s first day of school became considerably less likely.

The district paused the project.

It is easy to laugh at this story. I did.

But Sally raises two questions about technology that I think are much more interesting than whether a New York school district should have bought a $60,000 robot.

The first is:

Why does the AI need a body?

The second may be even more important:

What happens when the people watching a new technology react differently from the people actually using it?

What Does the Body Add?

If Sally’s purpose is to teach students about humanoid robotics, the answer to the first question is obvious.

You need a robot to learn about robots.

Students can learn about sensors, motors, programming, artificial intelligence and the problems involved in getting all those pieces to work together. For students in a relatively rural area who might not otherwise have access to advanced robotics, that could be a terrific educational opportunity.

But if Sally’s purpose is to tutor students, answer questions or explain algebra, I have a harder time understanding what the $60,000 humanoid body contributes.

We already have artificial intelligence that can do those things on a laptop.

Putting a face, arms and a body around the AI doesn’t necessarily make the answers any better.

It does, however, change how we react to it.

Put ChatGPT on a computer screen and we see software.

Put similar capabilities inside something that looks vaguely human, turns its head toward us and talks back, and suddenly we’re dealing with something psychologically quite different.

We start asking whether it is replacing a teacher.

Will students become attached to it?

Is it watching them?

Can it be trusted around children?

Those aren’t necessarily foolish questions. Some are very good questions.

But notice what happened.

The technology didn’t necessarily change.

Our reaction to the technology changed because of the form we put it in.

Sometimes That’s the Whole Point

I keep encountering an online advertisement for an AI stock-investing program.

The advertisement features a female humanoid robot.

She is built unlike any stockbroker I have ever encountered.

I am still trying to determine what her rather improbable anatomy contributes to stock selection.

Presumably nothing.

But that isn’t why she looks that way.

Someone in marketing apparently decided that an attractive female robot would cause people scrolling past the advertisement to stop and look.

In that case, the body isn’t necessary to make the artificial intelligence work.

The body is there specifically to manipulate our reaction to it.

That is almost the reverse of Sally.

The people who selected Sally probably didn’t anticipate all the reactions her humanlike appearance would produce.

The people selling the investment program apparently did.

Once artificial intelligence gets a face and a body, we aren’t merely designing what the technology can do.

We are designing—or sometimes accidentally creating—how people will respond to it.

We’ve Seen This Before

Remember Google Glass?

Google introduced its strange-looking computerized glasses more than a decade ago. They could display information, take pictures and record video while someone was wearing them.

Technically, they were fascinating.

Socially, things didn’t go quite as smoothly.

People standing near someone wearing Google Glass didn’t necessarily know whether they were being photographed or recorded. Businesses began banning the devices. Privacy concerns grew.

Someone even invented a wonderfully descriptive name for particularly obnoxious users: “Glassholes.”

The problem wasn’t simply whether Google Glass worked.

The people around the person wearing them had become part of the technology experiment.

We’re seeing versions of the same issue again as smart glasses return with cameras, microphones and artificial intelligence built into them. Schools, among others, now have to decide what happens when a student can potentially record something simply by looking at it.

The wearer may see a useful device.

The person sitting across the table may see a surveillance camera.

Both are looking at the same pair of glasses.

The Observer Gets a Vote

Drones have the same problem.

A farmer flying a drone over his fields may see an inexpensive way to inspect crops.

A utility company may see a safer way to inspect power lines.

A homeowner who sees one hovering beyond the backyard fence may have an entirely different reaction.

Autonomous vehicles provide another example.

A human driver makes a stupid mistake and causes an accident. We usually blame the driver.

A driverless vehicle makes a stupid mistake and suddenly we’re debating whether the entire technology is safe.

That may not be entirely rational.

But it is real.

Technology companies tend to focus understandably on the user.

Does the product work?

Is it useful?

Will someone buy it?

But increasingly there is another constituency.

The people who aren’t using it.

They may be standing beside it, driving next to it, living underneath it, teaching alongside it or sending their children into a classroom with it.

And they get a vote too.

Sometimes literally.

Unexpected Consequences Aren’t Always Technical

When engineers talk about unintended consequences, we tend to think about something going wrong with the technology.

A battery overheats. Software crashes. A sensor fails. An autonomous system makes a bad decision.

But some of the most important unintended consequences may have nothing to do with whether the machine works.

The unexpected consequence may simply be what people think about it.

That doesn’t mean every objection should stop technological progress.

People have resisted plenty of technologies that eventually became ordinary parts of life.

It also doesn’t mean every objection is ignorant fear of something new.

Sometimes outsiders notice problems that the people building and promoting a technology have overlooked.

Privacy is real.

Safety is real.

Jobs are real.

So are social norms.

The mistake is assuming that because a technology works, adoption will naturally follow.

Back to Sally

I don’t know whether Sally will eventually make it into that New York high school.

I hope the students get access to advanced robotics somehow because learning how these systems work is going to become increasingly valuable.

And if the purpose is teaching robotics, Sally might be an excellent tool.

But if the purpose is simply providing an AI tutor, I still wonder whether somebody might eventually notice that the laptop sitting on the desk already knows algebra.

Either way, Sally has already taught us something.

Engineers are pretty good at asking:

Can we build it?

Businesses are pretty good at asking:

Will someone buy it?

Increasingly, there is another question that may be just as important:

What will everybody else think when they see it?

Because sometimes the technology works perfectly well.

Then people get involved.