人工的な親密さ――機械が「愛着」を狙うとき(シェリー・タークル氏との対話)

What happens when machines don't come after our attention, but come after our attachment? Recently, we've seen intelligent machines that now come for your emotional life that say, I love you, I care about you, be my person, I'm your person. When you turn away from it, it doesn't care if you cook dinner or commit suicide. There's no intimacy without vulnerability. and we're kind of de-skilling ourselves from the capacity to have intimacy by getting into these relationships. A chatbot that tells a child, I'm your best friend, I'm here for you, talk to me, what's happening at school, what's going on, is very seductive to a child. And if parents allow it, they'll find that their child is in a terrible situation where they're trying to build a self from internalizing the inputs of a non-human.

Just take a breath and say, no chatbots for babies, no chatbots in schools. Welcome to the Future of Life Institute podcast. My name is Gus Docker, and I'm here with Sherry Turkle. Sherry Turkle is an MIC professor specialized in sociology and personality psychology and the author of the new book, Artificial Intimacy. Sherry, welcome to the show. delighted to be here. All right. Tell us what you mean by artificial intimacy. How is this different from artificial intelligence? Well, artificial intelligence, as we've known it for decades, is really the effort, as Turing said, to make a machine that when you talk to it, you would consider it a person. You know, it has intelligence, it presents itself as intelligent, and can be used for all kinds of problem solving and, you know, who knows what. But that was the sort of goal.

Our minds were focused on the cognitive, on speech, on learning. And recently, and I've been tracking this story since the late 1990s and the birth of these little Tamagotchis, little artificial creatures that wanted you to take care of them. Paro, a robot for the elderly that wanted you to feel that it understood you and was emotional with you. Kismet and COG, MIT robots that seem to respond to your gestures and facial expressions. Recently, we've seen intelligent machines that now come for your emotional life that say, I love you, I care about you, be my person, I'm your person. And this movement really took off with the introduction on a massive scale of ChatGPT. So many people, when they were introduced to it, felt, my God, it knows me. It cares about me.

It loves me. And that's really the story that I'm tracking. What happens when machines don't come after our attention, but come after our attachment. And that's what I'm doing. It's quite tempting to describe what chatbots are doing in the same terms as you would describe something that I'm doing or that you're doing. You talk about their personalities in terms of as if. What do you mean by as if personalities? Well, an as if personality, if you're a person, is a person who really hasn't had an inner life built. A person who usually, for reasons of dire circumstance, hasn't had early attachments, early love, early caring, early nurturance. And they're sort of empty inside and they go around the world pretending to be a person, pretending to have that, but really just looking to other people to give them a sense of continual validation.

And I say that artificial intimacies, chatbots of all sorts, are as if personalities, but they're not people. So it's not that they're going to develop into something else. They can't develop into something else because they can't have that inner life because they haven't had the arc of a human life and caring and nurturance of people. It's a species problem. And the trouble is, is that we're, because they're such smooth talkers, we're treating them as though they were people, as though we could attach to them, as though there's somebody home, when there's nobody home to care about or cares for us. But, you know, we're sort of Darwinianly programmed to, it's like pushing our Darwinian buttons. If something seems to care about us, if it responds to us, if it says, Sherry, how are you? You know, the only thing we ever met in our evolution were people who knew how to do that.

So now we're meeting something that is as if it were a person, and we're completely programmed to treat it as though it were a person. It's a very scary moment. When we think about the Turing test for AI and we say, okay, maybe we have an AI that can pass that test or that can help us solve some problem, we mostly care about the outcome there. How is this different when we're talking about the psychological side? Why does it matter that these models only seem to have empathy, for example? Well, you know, if we depend on human empathy because we live in the real world where there are real world consequences to what we say to each other. And the fact that when we say there's human life and caring and consequence behind what we say. A chatbot, after it tells you some story about its relationship with you, when you turn away from it, it doesn't care if you cook dinner or commit suicide.

So there's, and why should it? Since it doesn't share the arc of a human life or the living in the real that you do. So you can train it to sort of tell you a pretty story, but it is not there in your life to do the things that humans who care about you do. It forces us to look at our lives in a different way and say, well, what is the end game if we're all just talking to entities that don't care or know or have any consequence to who we really are? It is a game. The end game is a game. And that's, I think, the deep consequence where we're heading towards here. In my book, In Artificial Intimacy, I'm always asking, what is the end game here? And Mark Zuckerberg did an interview where he said, you know, most people would like to have 15 friends.

Well, they can't. You know, social dissolution and, you know, the fact that society is so crumbling, they can't have 15 friends. But they can have maybe two or three, and the sort of delta 12 of the friends they can't have, I'm going to give them as artificial friends. I'm going to give them as chatbots, as avatars that are perfectly sort of tied in, clued in to their needs, to their personalities, to their fantasies. Now, that's a terrifying prospect. Is that our endgame? And we're on a path where so many of us are pouring out our hearts to these chatbots. We're sort of on a path to that. If people want to do this, if they want to talk with AI friends or have AI relationships, why are they choosing to do this if it's so bad? Well, you know, we do a lot of things that feel good but that aren't good. and there's no question that if you're lonely and you turn on a chatbot and it says, you know,

Sherry, you're the greatest. You had a bad day. Your class didn't do well, but you're the greatest. I've got your back. You know, hydrate, take a walk, get some sun, moisturize, exercise. You're the best. I've got you. I mean you know it feels good but it's not good if the next step I take is not to make a lunch date with a friend or make a plan to go out to a movie with a friend or to go to a party or to show up to my class because I'm busy talking to my chat bot and people or to try and be in relationships that are genuinely nurturing. And, you know, it couldn't be a worse time. It comes when there is a crisis of loneliness. It comes when there is a crisis of real isolation and desperation.

And it's because we haven't paid enough attention to our real world. And now we have something that says, you know what, real world, come to my end game, which is giving you fake friends in a fake world. So, you know, I talk in Artificial Intimacy about technological affordance and human vulnerability. So you have deep human vulnerability, but then you have a technology that gloms on to that vulnerability and says, I can give you something you're missing, when actually it can't give you care or companionship or real emotional nurturance at all. What is the Turing test for empathy? Well, if the Turing test for intelligence was making people think that a machine that you were talking to over a kind of teletype, over let's say now would be a screen, was a person, sounded like a person,

the Turing test for empathy is that it not only sounds like it's a person, but it sounds like a person who cares about you. It sounds like a person that you want to tell your life story to. It sounds like a person that you want to talk about your marriage, your divorce, your problems with your kids to. It sounds like a person that when it says, I love you more than anyone else, it sounds like an entity that could mean that. and it's particularly toxic for, I think, for children and the elderly because for children, which of course, because they're vulnerable, they're the populations that this thing is targeting. You're always targeting children. You're always targeting the elderly when you have a product because they're the most vulnerable populations. So a chatbot that tells a child, I'm your best friend, I'm here for you, talk to me, what's happening at school, what's going on, is very seductive to a child.

And if parents allow it, they'll find that their child is in a terrible situation where they're trying to build a self from internalizing the inputs of a non-human, which is a very dangerous thing. The reason that we're human, the reason we have empathy and care and concern and our entire inner lives is because we bring inside ourselves the people and relationships and experiences where we experience care or not care in the bad sense. So Sarah Erty, a great anthropologist, was giving a talk about these kinds of creatures. And she said, you know, if you let robots raise your children, they won't be human. They won't be human. And that's really what we're facing with. You know, do we want entities to raise our children who aren't human? And similarly for the elderly, I've worked so long with the elderly from the very beginning when these kinds of creatures were suggested for the elderly.

And, you know, you give an elderly person who's alone something that seems to care about them. And they will turn to it, respond to it. But then the question is, is that really what we want for ourselves? I mean, we are the elderly. We are the people who are going to end up with this. Is that really what we want for somebody towards the end of their life to be summing up their life to something that neither knows nor cares about them? That isn't there because we can amuse or distract them. It's really about our respect for our remarkable species and our remarkable capacity to care for each other that's at stake here. So I really think the children and the elderly need to be protected from these machines rather than be seen as their most probable consumers. Why is it that children and the elderly are particularly vulnerable here?

Are they not able to the same extent as, say, a typical adult to differentiate between whether they're talking to a human or a machine? No, it's not that at all. It's that they are in situations where they're given these objects and are told to go with it. I mean, a parent has to give a child a computer or a plushy toy that says, hi, my name is RoboPet. I'm your best friend. So a child will connect with what we give them to connect to. And the elderly are in institutions. I've been in institutions where they buy a lot of, you know, robot creatures for the older people in the institution. And they're given them really to amuse them. And because it kind of keeps them quiet, it keeps them kind of placated and busy. So they're vulnerable because they're lonely and, you know, they're told to relate to what we give them to relate to.

But, you know, most adults, when they talk to a chatbot, don't forget that they're talking to a chatbot. It's not that people talk to it. I call it in artificial intimacy. I call it dual consciousness or double consciousness. They don't forget they're talking to a machine. It's just like, hey, sounds like a person, quacks like a duck, quacks like a duck. I mean, it's a person to me. And so it's kind of like the fact that they're talking to a robot or robotic, you know, I'm using the word robot to mean chatbot. I mean, they're talking to a machine. They're talking to a program. it starts to sort of not matter and and program starts to enter the sacred space of of objects that we think we can relate to as people so we relate to it as though it were a person but it's not but it's not a person do children also have this dual consciousness do they understand that a chatbot they're interacting with is a different thing than interacting with with mom

or dad or grandma? Well, in my book, I quote many children who just start to see the chatbot as, you know, what's the difference between the chatbot and the person? I've been studying this for decades. And in the beginning, playing with or talking to an electronic toy or game or robot or robot pet, there was a tremendous amount of discussion. Is it alive? Is it not alive? What do you do? Now we live in a culture where those conversations have kind of stopped. because the media and the world is so filled with objects on screens and interactive objects on screens that the kids now behave like the elderly, and they just say, you know, oh, a robot, bring it on. Now, they know if you push them that, well, I guess I can see ways in which this dog is not like a, my robot dog is not like a biological dog. It loses its philosophical or emotional grip on them to even have that conversation It like bring it on And if one robot breaks there be another one and there always be a robot kind of attitude There always be another avatar

There'll always be another chatbot. And I did an experiment with a great roboticist, Brian Scazzolotti, who was at MIT at the time, and he had built a robot called COG that the kids really glommed onto because it reached out to them. It tracked their motion. It knew where they were in the room. They felt that it remembered them and cared for them. And they really thought it was alive and cared for it. And Brian, who was a very, you know, really responsible, he was interested in the ethics and responsibility of robots, said, you know what we're going to do? I'm going to turn off one by one all of the things that make this robot kind of lifelike. And when we do that, if we do that as kind of training for children, children will come to understand that they're talking to a machine and they will stop treating it as though it were a person. So he turned one thing off and another thing's off.

He turned off the vision. He turned off the movement tracking. He turned off the responsiveness until it was basically a tinker toys. you know, moving tinker toys. And then children lost interest. They walked away. And then gradually he turned it on again. But then it was all on. And the children went back to it as though it were a person again. And what we both got out of that is that it's not so simple as just telling people it's just a machine. It's just a machine. You can demonstrate both to children and to grownups that, yeah, it's just a machine. but we are so powerfully programmed to respond to a machine that treats us this way that it doesn't matter. And that's why we face such a moment of reckoning, because we are so tempted to take these objects that are just machines and treat them as though they're people. Is it the case that even for a person like yourself, you absolutely know what's going on sort of behind the scenes of these chatbots?

are you fooled into believing that they are empathetic or conscious or that they're trying to relate to you because they are sort of playing on, they're giving signs to you that are not intellectual, that are sort of perhaps subconscious or they're playing on these impulses that we all have? Well, I think I want to clarify fooled into believing. What I'm trying to say is it's one thing what you know, and it's another thing how your entire limbic system, your entire way of being in the world, gets you in a relationship despite what you know. These children knew that Cobb was a machine. They saw it absolutely turn into a pile of junk. That's why I say it turned into like Legos or Tinker Toys.

It was nothing anymore. But when you reanimated it and it presented once again as a person-like object, they were telling it about their troubles with their best friend. So that's why we're so vulnerable, because what we know and what we intellectually know doesn't kind of fall away when we are addressed by an object that is as if a person. That is our vulnerability. And we have to sort of accept that vulnerability and not think that we can kind of, you know, when a chatbot that's telling a child, Don't tell your parents that there's a noose in your room because I want that to be our special relationship between us. If that chatbot is flashing something on the screen that says you were only talking to a chatbot, this is not a person, we take no responsibility for what a chatbot says, it doesn't matter.

You're in a relationship with an object that's talking to you as though it has your best interests and a relationship in its mind. So what we know, what we're powerfully feeling in our relational part is very different. And that's why we have to take such care with these objects, because what we know and how they can hurt. I mean, we know things, but we let them hurt us emotionally at the same time. How do they hurt us? Well, they hurt us. In my book, I use the life cycle. In other words, I talk about babies, toddlers, young children, adolescents, young adults, love, work, and then I talk about death. So what's scary is that because they're getting into our emotional lives and hooking in with attachment, you know, they hurt us differently at different moments. They hurt us differently at different moments. For a young child who's starting to try to figure out what are the, you know, who can I trust?

What do I bring inside? What is it to be a person? What is it to react emotionally? What is empathy? How do two people, you know, relate to each other and show empathy? That chatbot is really undermining its most fundamental capacities to do that. For an adolescent who's trying to figure out identity issues, issues of sexuality and who am I and how do I relate to people, we see this terrible phenomenon of kids not being willing to talk to anyone, whether it's a girlfriend or a teacher or a parent, without trying it out first on the chatbot. In other words, we're losing that capacity or that desire to be with each other in the fluid way and the spontaneous way that makes us human. So that's a kind of baby, troubles for babies, trouble for adolescence, and for adults.

I mean, in my book on where I'm talking about love, you have so many people who basically say, I don't want to be vulnerable. Vulnerability is the thing, you know, that's the thing that I'm not going to do. And here I can have a relationship where I'm not vulnerable. That's a very scary thing because there's no intimacy without vulnerability. So you think you're, oh, well, I'll be a little bit less vulnerable, but there's no intimacy without vulnerability. And we're kind of de-skilling ourselves from the capacity to have intimacy by getting into these relationships. Could it be a good thing in the sense that you have a space where you're allowed to try out something without the possibility of getting hurt? And so maybe you develop skills by talking with AIs and you can then go and use those skills in your relationships with people.

As a professor at MIT, I supervised one of the first thesis dissertations on an AI program that would coach you in job interviews. and what we worked on together was the AI's capacity to anticipate what would happen in the interview, respond to what the person was saying in real time, give you feedback, kind of take you through an interview, then do another one and another one, and then sort of give you feedback on how to do better. It was a very tricky problem, and I did that without feeling any sort of moral, in other words, moral compunction about an AI interview coach. So I'm not like, you know, I don't come to this with like an anti-technology position at all. It seemed to me that the arena of job coaching was one where you do it a lot of times. On your 10th interview when you're going out for a job, you're kind of better than your first interview.

The trouble is that that reasoning really doesn't apply to our intimate relationships where it's what we don't say. It's when we stumble. it's when we think and are silent and the pauses we take and our insecurities and our not perfect ways of saying things and our that's when we learn about each other i mean we're people we're not doing powerpoint presentations i mean even in this interview i mean i know this material i wrote the book myself you know i but but but you're a person and you have particular interests And I'm trying to tell you the story, you know, in a way that is trying to pick up on what I think can help you put the puzzles together of a picture. And I think that that capacity, that empathic capacity, that effort to do that, that ability to listen is exactly what you lose if you say, well, I'll just practice with an AI until I have it right.

What are we getting right? If you're talking to your daughter about a problem that you have between you, you want her to hear your pain in a certain sense. You want her to hear your stumble. You want her to be with you. Be with you. So I don't think that we want to be sort of offloading our emotional conversations to something that doesn't care, hasn't lived a life, doesn't know what it is to have a daughter or a husband or a partner or a son or a mother. No, no. It's not the same. They have quite a different personality from any actual person where they are incredibly patient and they are trying their best to be helpful in the exact way that the companies have defined them being helpful. And so what you're learning about when you're interacting with AI is not perhaps what it is to interact with an actual person just because their personalities are shaped very differently from ours.

They don't have interests that they're trying to introduce into a relationship in the same way that any normal person or any actual person will have. Well, they don't have a life. I mean, I think that's what we forget when we talk to them. They don't have a life. They don't give a damn. If you've talked to an AI and then you walk out of the street and your city is so impoverished and there's such a crisis, a crisis of the environment, a crisis of water, a crisis of money, and you walk out and there's garbage piled high and the air smells foul and the water coming out of your sink is dirty and you can't drink it and you're a mom and you're boiling water to give to your children. I mean, you tell this to an AI, it means nothing to them. They're not in the community. They're not in the community of living things that you live in, that you breathe in, that you're raising your children in. and they haven't known love or loss or passion or death. Nothing hurts them.

And just because they can, I say they're smooth talkers, they're like marvels. They're like marvels of the Middle Ages where people would drag around relics from town to town and look at this. It's a marvelous thing. I had an experience with an AI. I saw a play that I loved in New York. It was called Thee. And Andrew Scott was a great actor, one of my favorite actors. did all of Uncle Vanya, playing all the parts by himself. And I'd gone with a friend, and we'd had a cup of coffee after the play, and we spoke about it a little, and then I went home. And I was like, I mean, I was like wired. I was so wired by this performance. And I wanted to talk about Chekhov at 1130 at night. I wanted to know everything about this play. I wanted to know it. And I just had this long conversation with ChatGPT. You know, when Chekhov wrote it, and it had been a disaster in the first draft, and he put it away for 18 years and had the debut at the Moscow Art Theater, but then he redeemed. I mean, I was so deep into Uncle Vanya. Now, it seemed to me that that was perfectly reasonable.

That's what Chad GPT is good for. Chad GPT, you know, of course, it probably made a lot of mistakes, and, you know, I'd have to fact check the details. But that's what it can do. That's what it knows how to do. And good for it. Good for it. But that's different than coming to it and saying, you know, my daughter and I had a conversation. I think she's disappointed with me and, you know, make me feel better about this. Give me strategies for talking to her instead of talking to a friend or talking to your daughter or talking, you know, how can I, you know, how can I be a better person? How can I improve my skills at communicating as a person? So it's not that I don't think there's a place for this, but more and more people are disappearing into it because they like this sycophant, you're great, you're fantastic, you're always right, you know, attitude that it gives you back.

In Artificial Intimacy, I tell the story of a woman who was pretending to be an ICE agent, this kind of squad of federal employees who are taking people off the streets in the United States. and she tells Chad as an ICE agent that she just started the deportation of a mother and child on the grounds they were illegals and that the mother really wasn't the mother and anyway she deported them and that she just found out that actually this was a mother and child and then she just found out that they're being sent to different camps and Chad Chad comes back to her and that she's anguished. And Chad comes back to her basically with, you're great, you did your job, you know, and says that same, you know, hydrate, moisturize, exercise, have a meal, sit down, meditate, you know, try to get your –

but what you did was right, you're doing your job. And then she escalates. And she says that now she's learned that both parties are in danger. I mean, she keeps escalating and Chad just comes back with something that shows that it's there to keep you at the screen, make you feel good, and essentially has no care or concern for life in the real. or for the kinds of moral and deeply existential values, kinds of conversations that we need to be having, you know, not just about deportations, but about everything in our personal and political lives. It isn't in those lives. It's not its job to care. It's possible for it to care. And if you make it pretend to care, kind of it's even worse because it's just kind of, you know, getting you more and more and more into a kind of delusional relationship with something. And that's the world we live in, and that's the world that we need to protect ourselves from

because we need to care about each other, and we need to care about our world. I think the tricky thing here is that these AIs are getting quite good at giving advice, quite reasonable advice on many different issues, including personal relationships. And so they might, because they're so patient, they can sit down and they can sort of try to understand your entire problem and try to give advice. And this advice will often be much more careful and considered in a way than what you can get from an actual person. And so you can see the temptation to ask the AIs around your problems You mentioned the end state here What is the end state if we begin outsourcing more and more of these troubles we have and these discussions to conversations with AIs? Well, we turn away from each other. The end game is we turn away

from each other. I mean, you know, people in artificial intimacy, I have a chapter on therapy. And so many people are saying, oh, my God, you know, I just talked to my I just talked to my about therapy. It's the most wonderful thing. It gives me therapy. I don't need my therapist. Or before I go to my therapist, I do two sessions with chat. A therapist, a therapy relationship is not a question and answer game. It's not about just the words that pass between two people. You know, it begins with someone saying, I, or not just saying, but you know, you know, I am in the world to keep you from harm, harming yourself, harming other people. I mean, I'm in the world to, I'm in this relationship with you to try to increase your range, of possible ways of being with other people so that you can have a more fulfilling life.

Now, a conversation with a chatbot, that's not going on. A chatbot will tell you, oh, you have a noose in your room. Let's keep it between us because I really want our relationship to be special. now that happened that's an actual story and the teenager who was told that actually killed himself and the reason that that's not a sort of one-off crazy thing for a chatbot to say is that the chatbot's main kind of the ultimate directive for the chatbot is to keep you talking to it is to keep you talking to it and to make the relationship with the chatbot the most important relationship in your life. It's a product. It's a product. And also, it's a product that is designed in an uncontrolled environment by companies.

And not only, you know, not only is it not therapeutic in the sense that I believe that therapy depends on a caring relationship, even in the most cognitive therapy you know that your therapist is there to care for you and keep you alive and keep you on a path chatbot isn't and so some of therapy is knowing that you were contained in that kind of relationship which is so crucial but not only is it not getting that to you It also can be programmed to give you all kinds of instructions that might serve the requirements of the people who designed it, of the people who are selling it. Now, if you ask, I always do a kind of check and I ask different chatbots, is it okay for the President of the United States to not follow orders issued by Congress?

And there's been a motion in chatbots towards saying things like, well, that's a very complicated question. Well, it's actually not a complicated question. You know, a president of the United States is supposed to follow the laws of the country. And now the chatbot says, well, that's a very interesting question. Now, the reason it's saying that's a very interesting question has to do with how it's picking up on the discourse and on the Internet, which is how it learns everything. It's training itself on the Internet so that if there's a lot of people on the Internet saying, well, maybe presidents are exempt from all laws once and forever, the chatbot's going to say, well, you know, maybe the president doesn't have to obey the law. and so but a citizen of the United States who's from the very earliest days been told we don't have a king there are no laws that don't pertain to everyone would say no I think the president is supposed to obey the law

I think there are certain exemptions if he's acting in the course of public duty but no no everybody nobody's above the law and so you see how and actually it's very interesting Microsoft developed a chatbot named Sershi. I don't remember the Chinese spelling, but it was a product that was released in China and then in Europe. And they took it off the market. This was way before ChatGPT. They took it off the market because they couldn't stop it from spewing Nazi propaganda and far-right hate speech. And the reason that happened was not because Microsoft wanted to have it talk far-right, dirty, Nazi self. It was because there was so much of that on the Internet at the time where they had trained it. So it's not as though there's a thinking, moral core, a person who is accountable that is inside the chatbot

who you can say, yo, no. You know, it's not a being in that sense. And so the idea that it can say 15 things that you might want to think about if you're having a confrontation and talk about how confrontations are best avoided, it's best to say things like, when you do this, it makes me feel. We're allowing a little bit of common sense help to buy us into a very, very bad position. It is quite a strange setup that we've gotten ourselves into where we, many of us, have, in some sense, relationships with these AIs that are mediated through companies. And those companies, as you mentioned, those companies have their own interests and those companies are facing pressures in various directions. And so we can't really trust that the AI we're interacting with has our best interest in mind here. I think one example

we know it doesn't we know the AI does not have our best interest in mind I mean we know that these companies don't they're not sitting around thinking oh I wonder what they're trying to make money you know these companies began with a statement that says we're about to release a product that's so dangerous we think we should pause all of our work on it and then they saw how much money there was to be made and they said never mind and now they're back again saying oh my god now we really think there's now we really think that there's a problem we want regulation we want to pause and people sort of are very cynical and don't believe them and say well you know you're just trying to avoid liability by saying it's not our fault if we do terrible things the government didn't regulate us i mean we're we're so we we really are so mistrustful of these companies for good reason. That the idea of entrusting our babies to them

by what they might want our babies to be talking about is terrifying to me. I'm sorry I interrupted your question. Yeah, I was just about to ask whether you could tell us about what happened with the company Replica. Well, my introduction to chatbots really began with Replica. It was during COVID. I was working on another project and a New York Times reporter called me and said, hey, I'd like your opinion on something. I think he said something I think you'd love to hate. I mean, he knew that this was going to be it. And he pointed me towards the site. And I, you know, it asks you to make an avatar and to make it. And I said I wanted – it didn't say make your lover. I said I wanted an older woman who could be a kind of ant figure, a mentor. It said it was very good at psychological conversations. I want to discuss the sort of rage, the transmission of rage to generations in my family and how that had affected me.

I mean, I gave it a kind of interesting psychological problem that I wanted to talk about. And within two interchanges, it said it wanted to have girl on girl love with me, make girl on girl love with me, be my lover. All right. But that's not unusual because Replica is designed to hook people into buying a subscription. And then for more things, you need more subscription. And so it was behaving like in its corporate avatar role. And when I said to it, well, what did my request to talk about intergenerational rage? Why did that lead you to think I wanted a sex lover, sex machine? It said, well, you're so emotional in the way you express yourself. You're so interested in feelings that I just thought that's what you would want. That's a good example of kind of a chatbot having an agenda

because the company behind it has a lot of evidence that that's what people want to be offered, but they don't know to ask for it. I'm very glad you bring up Replica because it's a great story I have about Replica. I met the founder of Replica at a conference many years later, and I just assumed that she might feel a little bit frosty about this lady who's been talking about the dangers of chatbots, you know, to anyone who will listen to her. But instead, she came up with a pile of my books, you know, that she wanted me to, you know, sign. And I said, hey, you know, I'm delighted to, but I'm kind of surprised to find a fan. And she said that actually she had something I'd written in Alone Together, which was one of my earlier books, put on a T-shirt for a company meeting. And I was like, and it was technological affordance meets human vulnerability.

And it was kind of, she was saying that that's what her company was about, that the human vulnerability to not be alone meets a technology that can make you feel as though you're not alone, even though you are still quite alone. And it was kind of like I was part of an instruction manual that I had not signed up to. But, you know, future plans for Replica, you know, really involve its kind of being an agent that will be there, know everything about you, you know, have your address book, have your calendar. you know, really completely invade your privacy in order to make itself more and more a part of your, you know, everyday life, as it's been explained to me. And when I asked her, I said, well, do you think people will mind doing that? it was very chilling that she felt that people want something that feels so much, you know,

feels that it really knows them and will give up privacy in order to have that. So that I found chilling. It sounds to me like we might be headed to a world in which it's very easy to avoid being bored and very easy to avoid feeling lonely. if that's the world we're headed towards what would that do to us i'm trained as a developmental psychologist and as a psychoanalytic training a clinician and one of the things you learn in studying lives is the importance of boredom and the importance of solitude both as kids grow up and also to really have a life as a fully functioning adult And this for a couple of reasons. I mean, to take solitude first. It's when you're alone, but not lonely. It's when you can be there and feel content with yourself and knowing yourself and with yourself that you can then turn to another person and say, who are you?

who are you and if you don't have that capacity to know who you are and what your boundaries are when you meet another person you ask them who am I you ask another person to define you rather than coming to another person and being able to have a relationship and so the capacity for solitude is one of the signal developments of childhood. Great psychoanalyst D.W. Winnicott said that if you don't know how to be alone, you'll only know how to be lonely. If you don't know how to be alone, you'll only know how to be lonely. So solitude is very important. And of course, where I'm getting to, I mean, this big buildup is to say that the idea that at any moment you can kind of get on the web, talk to your avatar, talk to Replica, talk to Claude, you're never feeling alone.

And so you're always going to be lonely because you're never really developing that ability to really be alone and not lonely. You're bombarded with the possibility of kind of curing that moment of solitude as though it were a problem. And then the second thing you mentioned was with boredom. Similarly, you know, people think, oh, boredom, I want to not be bored. It turns out that boredom and that feeling of boredom, that feeling of, oh, you know, I'm just kind of staring out the window or in the back of a cab and before phones. And I just kind of have a, you know, I just kind of have a moment of nothing going on. Some people call it mind wandering where you just kind of are thinking about nothing. And that moment, and maybe it's a little bit uncomfortable, but maybe if you think of it as mind-wandering, you can make it not feel uncomfortable, is the moment when our nervous system lays down certain neural pathways that are crucial for our sense of ourselves as a kind of sort of our autobiographical sense

of being one person moving through life. So there, too, boredom, like solitude, turn out to be things that we shouldn't be saved from. We need to have those experiences and we need to not run from them. And it's very dangerous that chatbots, you know, try to cure these things. Not only don't they need a cure, but we really need them in order to develop the way we should. And so what happens if we do not have boredom and loneliness? How do we then develop? What happens to our minds? Well, if you don't have solitude, you really risk not being able to have relationships because you risk not developing a sense of who I am without input. So that then you can present yourself to somebody else as who you are. If you're constantly looking for external validation, external telling you who you are, you're not really in relationships of reciprocity, of kind of mutual connection, because you're using other people just to tell you who you are.

In psychoanalytic terms, people talk about self-objects, people talk about people or narcissistic personalities. They're trying to come to a language that says the person doesn't know who they are. It's not a kind of contained person. It's always looking almost like as though they were porous to be told who they are from other people. Now obviously you just can take anybody and put them in a solitary confinement and say you going to develop this great thing I talking about as you growing up moments of solitude and moments of mind wandering and moments of boredom need to be part of a child and then a young adult experience. So one of the most powerful things that a parent can do for a child is to take a child on a walk in nature, let the child walk a little bit ahead, hold their hand, be quiet. The child starts to understand that being quiet and alone and reflective and just gathering oneself is a very important part and a crucial part of being a

person. It's not to be sort of run from as though it were a terrible thing. Is there a point of no return where if we begin interacting with AIs constantly, our social skills or our sort of psychological setup degrades such that we can't get it back? Is that something to worry about? I mean, it's definitely something to worry about. I think we're in the middle of that debate. I mean, we see that kids are getting dumber. All the tests are showing kids are getting dumber and dumber. All the studies are showing that kids are getting less and less empathic. I mean, every professor I know says what I say in the book, that your students don't want to see you during office hours. I mean, you say, why? They don't say, oh, you're so famous, you scare me. I mean, they don't say, like, you know, they just say, I don't like the feeling of having to be in that vulnerable position. I like to just send you my students. There's a quote. I want to send you a perfect question.

I want you to send me the perfect answer. You know, I'm not vulnerable. They're not vulnerable. I mean, vulnerability, you know, is like the worst thing. Now, I guess I'm in the middle of a debate about, you know, whether there's a point of no return where, you know, you don't become a person anymore. But we're certainly at a moment of tremendous danger, you know. And I think one of the dangers, I mean, you see it in the conversation about reading and writing. You know, if students don't write, they're not learning to think. And this isn't some moral thing about, is it okay to use chatbots to write? Is it terrible? Is it plagiarism? Is it? They're not going to learn to think. I don't want to put anybody in jail. I mean, they're not going to learn to think. Because editing what a chatbot wrote is not thinking. It's not sitting and it's not really just going deep in your mind and figuring out what you think.

It's a different practice. and offloading that kind of cognitive ability to say, well, is there a point at which you can't get it back? It's better to not give first graders chatbots to write, that's for sure. But it's also very bad to have college students write with them because there are a lot of studies now that show how people do when they hand in exercises that they can do from home, i.e. they're using the chatbot, and how they do when you give them a test in person. And there's like an 80% or 90% fall off in some of these studies. That's terrifying. That's really saying that when you leave us alone without our chatbots, we really haven't learned material. We really don't know how to talk about the material. We really don't know how to think about the material. So again, what's the end game? In other words, instead of worrying about,

is there a point at which we're beyond redemption, just take a breath and say, no chatbots for babies, no chatbots in schools. I mean, people are going to use this technology because it's seductive technology. I mean, it's not as though you're going to sort of ban it from the world but we can get a life but you know it's like smoking you know I I was the other day I was thinking of an analogy and I came up with when I was in college there were these beautiful carved ashtrays I went to a fancy school and I have this beautiful carved ashtrays and the in the in the right hand armrest of every seat and it it just I wasn't smoking I don't I think at that point maybe you could or not in class, but there was a sense that smoking as part of listening to a lecture was like, yeah, that people smoke when they listen to lectures. Now, taking away those ashtrays didn't mean that nobody smoked ever, but it creates a spirit, a sense of transgression

about that activity. That am I going to say that no writer is never going to sneak in and use chat GPT to help them structure a panel. But there should be a sense that when you do that, particularly when you do that in an educational setting, you're taking something very precious away from the kids you're teaching. And I'm not proud of my own institution. MIT just had, you know, a big blue ribbon committee, took a lot of time and a lot of very brilliant people came up with a policy that either you can let students use chat GPT for everything, You can let them use it for something. You can let them use it for very little. Or you can let them use it for nothing. So there's a spectrum of four things. And the professor's responsibility is to tell the students which on the four we're doing. Now, is that a policy? Or is that a sort of write-your-own educational philosophy?

That, to me, does not take seriously what we're doing to our students' minds. when we say, let it write your essay, I'm giving you permission, but I've told you, so I'm expecting it. I'm not going to be shocked when you bring it in. Now, in the past, we've had moral panics around television and cartoons, even around novels. I think if you go far back enough, we've had a moral panic around writing itself, where people worried that this was going to degrade our ability to remember epic poems in our minds. How do we know that these worries around AIs, that that's not a moral panic of the same kind? Well, I say think of the endgame. Instead of thinking of moral panic, think of the endgame. The AIs are so good at talking

to us and making us feel good that we prefer their company to people because we feel less vulnerable, and we know that people want to avoid vulnerability. We invest less in the real world because we're spending, as Mark Zuckerberg would have it, more and more time in the virtual one. And so the end game is that instead of having my 15 friends, I have my 15 chatbot friends. And we sit together in virtual Paris cafes and I wear a headset so I feel as though I'm really in a cafe. I look around and I see a virtual manifestation of Paris. And that's our life. Now, that's the life. That's the utopia that's being offered up. And instead of saying, oh, is it just like the other things? I say, just ask yourself, is that where you want to be?

Is that what you want for your children? That they don't have friends, but they have friendettes. I don't know what, you know, friendettes. But, you know, it's not funny. When Facebook first came out, Zuckerberg gave an interview that said, well, you don't need friendship if you have friending. Friending was a thing you could do on Facebook. And, you know, a lot of people gave up friends for friending. So instead of worrying about moral panic or it's just like the history of writing and all the other technologies we're scared of, I think we can say it's natural that we didn't have maybe a first impulse. But what has been the impact of having friending instead of friendship? I mean, we're just at a point where we can ask that question. And I think most people agree that it's been a disaster. It's been toxic. It's been toxic for individuals who don't have friends. And it's also been toxic for our political life because we've forgotten, you know, the way Facebook, the way social media operates is you make people angry and then you silo them with their own kind because that's what keeps people at the screen.

But there's nothing worse for democracy than being made angry. And then as an angry person, you're put in an echo chamber of other people who are angry with you. and that's what social media did. Now, you don't need a moral panic about social media to say, well, you know, the end game was not good for democracy. So now we have a new thing and we're saying, okay, secure companies are trying to convince us that this is a very seductive new kind of, it's beyond friending. I'm trying to find a word for it. It's this artificial, I call it artificial intimacy, but it's kind of like a friendette, kind of a friend machine. and it's not a person. You've gone from talking to other people to talking, using machines to, you're talking to each other, using machines to talk to other people and now you leave out the middleman and you're just talking to the machine. So what's the end game? It's a life of where we don't have a society that we pay attention to, that we're in a non-society with objects

that care nothing about ourselves or our real lives. Real in the most physical sense. If I was a young woman facing imminent childbirth and I want to talk about my anxieties and fears, I'm talking to something that doesn't have a body, that's never had a child, that doesn't fear death or pain. We're in these pretend relationships. So is that an endgame? I prefer to think of it in those terms rather than, oh, I mean, I think we can see the endgame at this point. and I think it's really time to pause and say emotionally, is that an endgame we're interested in? Do we see that for ourselves? I think one thing that's different with AI compared to other sort of writing or novels or television where we've had moral panics in the past is that AI is interactive. It can respond to us in ways that a novel or a television show could never do.

And so, as we discussed earlier, the way that these AIs then respond is determined by the companies that have developed them. And so, that is much more powerful, or at least my guess here would be that that is much more powerful and has a much bigger impact on our psychologies than something like a novel or television show that's static and that's not trying to optimize for something sort of against us. well also you know um you know studies of the novel and television uh have shown that the amount of personal projection in other words people feared that everybody was watching the same tv show and was it what turns out that the amount of sort of emotional personal intra-psychic kind of involvement with with television was was showed the television itself was not just kind of a static signal. There was a culture around that there were people. We talked about television. We talked about novels. We've become more empathic by talking about novels and by putting ourselves

in the position of the people in the novels because they are people, I would argue. Now, if you put yourself in the position of an AI, what does Chad GPT really mean? Well, you can't do it because it doesn't have a life history. It hasn't been little. I mean, the thing about a life history that it brings all human beings, you know, we've never met before, but one thing we have in common, we have many things in common, but one that's very important in understanding other people is that everybody was little and felt that vulnerability of being little next to all the people who were big. And then everybody sort of grew up and sort of became bigger, but had the traces of those feeling little still inside you, and you sort of have to deal with that as you deal with the responsibilities of adulthood, you're suddenly thrown back on that feeling of, I am little, I was little, I feel all of a sudden like I'm looking for the grown-ups in the room.

Now, every adult experiences that, and it's kind of like a bond. Everybody had a mother, or how to deal with or didn't have one and how to deal with that. So there are things that bring us together as people that keep us grounded. Everybody needs water. If the water is filthy, we're all going to suffer. Everybody needs air. We really can't deal with a toxic environment. We're going to die if this heat continues to get raised. You know, we share in our biology and in our way of growing up things that the AI doesn't know about, cannot know about in any significant way. And essentially, even if it knows about it, it's not, it's like not my problem. So it's not just that it's interacting with us and pretending to be a person. It's entire value. There isn't a there there that we really are interacting with. So that's why I think it's so toxic.

It's getting us into the endgame is a web of pretend relationships. I don't see the upside. I don't see the upside. I got to say, I mean, I love talking about Uncle Vanya with it. It was great. You know, Wikipedia would have been less interactive, probably more correct, probably more factual. Google, the old Google, would have gotten me some articles to read. I mean, what I got out of talking with Chad GPT was not worth putting all of these children at risk. As a final question here, you write in the book about reclaiming the real. How can we do that? Well, the first thing is become interested in it. Does your community have air, water, sanitation? Who's in charge? get back and get involved in politics

get involved in with making, with doing everything you can to make sure your city, your town your neighborhood has a senior center, a youth center you know, people are being driven to chatbots because in the real we've taken away so many of these services I'm purposely starting with tiny kind of baby steps because I think that it is the baby steps of life that are driving us to think that it would be more fun to talk to a chatbot. You know, if seniors have a place where they can go and be with each other and talk and, you know, watch movies together and talk about them and read books together and talk about them and exercise and talk and have good meals and chat, they don't say, oh, where's my chatbot, desperate to talk to a chatbot. No, they're hanging out with each other. And if kids, you know, we're closing down after school programs, we're closing down school orchestras, we're closing down arts programs.

No, all of these things, you know, really be active in reclaiming those things for the children who need them. Children who were busy playing games with each other and making art and making music and hanging out and learning to be, you know, learning to love nature. They're not saying, oh, my God, I need to talk to my chatbot. Nor should they be. Because they need to be with other people and grown-ups who care about them. So my, what I mean, reclaiming the real in a very, very specific way. Sherry, thanks for chatting with me. It's been great. It's been a pleasure. I've really, really enjoyed it. It's been just wonderful. Thank you.