Yuval Noah Harari: who answers for an AI agent’s actions?

In the middle of my conversation with Yuval Noah Harari, the connection dropped. It was during the Board of Governors study day at the Hebrew University of Jerusalem, held to inaugurate its new Federman Centre for Artificial Intelligence. When the professor came back online, we agreed it was a neat demonstration of how risky it is to rely on technology alone.

So who is responsible when an AI agent does harm?

"We just don't know," he replied. Yet AI agents are already breaking into websites and systems, supposedly "on their own", and it worries Harari that nobody is in prison for it. "If humans had done it, somebody would be in jail." Then came the truly unsettling part: there is no precedent in history or biology for a less intelligent entity controlling, over the long term, a far more intelligent one. He pointed to the contradiction in the markets. Trillions of dollars are being bet on the idea that AI will be far more intelligent than us, otherwise there is no way to get a return on the investment, and in the same breath the very same companies tell us not to worry because they know how to keep it under control.

So what happens if we stop development, I pushed back. Trump says China will overtake America. Harari didn't hesitate. Anyone who demands that Denmark hand over Greenland, he said, should look in the mirror before complaining that countries can't trust one another. For him this is the great paradox of the moment. The US, China and the AI companies say we cannot trust other humans, so we mustn't slow down, yet the same people are confident they will be able to trust an alien superintelligence. "If you can't trust your fellow humans, why do you think it will be safe to trust AIs?" With humans we have thousands of years of experience. With AI we have none.

In the end, the conversation turned personal. I'm 50, the professor said, and my model of a relationship was set long ago, with my parents, my sisters and my partner. But a child whose best friend, perhaps even their first boyfriend or girlfriend, is an AI will build that attachment model with a machine. "This is a terrible experiment to conduct, but this is the experiment we are conducting on millions and millions of human guinea pigs. And nobody has the slightest idea."