By Tharin Pillay

At least once a month, two-thirds of people who regularly use AI turn to their bots for advice on sensitive personal issues and emotional support.

Humans finding comfort in machines is not new. In the late 1990s, MIT Professor Rosalind Picard—who founded the field of affective computing—found that people responded positively to computers performing empathy. But two key things have changed since then: thanks to technical advances, AI systems today are new entities, capable of sophisticated conversation and surprising behavior; and thanks to the billions of dollars investors have poured into AI companies, these entities are accessible to virtually anyone with an internet connection. ChatGPT alone currently has more than 800 million weekly active users—and the number is growing.

But with millions of people forming different kinds of human-machine relationships, we don’t yet know whether AI is helping more people than it harms. And meanwhile, AI companies are investing in making their models not just smarter, but also more emotionally savvy—better at detecting emotion in a person’s voice, and at responding appropriately. People are trusting their chatbots with deeply personal information, even while they distrust the companies creating them, and while the companies are exploring advertising and other revenue models to sustain themselves.

“I think we may have a crisis on our hands,” says Picard.