AIは人間の心理をどう変えるのか(ミア・チタ=テグマークとの対話)

Tools is what we want, but other persons or personified entities is what we get. They're also fantastic manipulators. They use actually all the psychological techniques that are used to break people psychologically. These techniques are used in interrogations, hostage negotiations. They're very powerful. And if you think that you can resist them, I would think again, like they're incredibly good at this. People get lulled into their false sense of security. It will never be like us. It will never be as good. And what I'm trying to say is that it will be better. It will be more appealing. I'm not saying better as in better for you, good for you. But it will feel better. Like it will feel good because it will prey on everything. We don't know whether they are conscious or not, obviously. We don't know whether they deserve these rights or not. My position on this is don't build anything that will ever deserve to have rights. Welcome to the Future of Life Institute podcast. My name is Gus Docker and I'm here with Mea Chida-Tekmark.

Mea is a co-founder of the Future of Life Institute. Welcome to the show. Thank you so much. It's such a pleasure to talk to you. We've always had such wonderful conversations together, so I'm really excited about this one as well. Amazing. We're going to talk about AI and how AI will impact our psychology, how it already is impacting our psychology. And one frame you use here is to talk about disengagement. So maybe let's start with that. What is disengagement when we talk about AI? So this semester I'm teaching a rather provocatively entitled class called AI and dark psychology. And when I say dark psychology, I think a lot of people jump to things like manipulation and exploitation. and I think that the image they conjure in their mind is something like Machiavelli. I mean, we even have in psychology, like in serious psychological research, we have the term Machiavellianism, and it's like, you know,

we have entire batteries of tests that can measure your Machiavellianism. But the more I think about AI and sort of this sort of darker side of the human psyche, the more I see that it's you know it is a lot of predatory behavior that it can enhance but there's this other part that people talk less about which I call disengagement so basically things like abdicating or retreating from those parts of ourselves that you know are the best parts or that are allow us to engage with life in the better in the with the good life with whatever we call the good life, whatever our definition of a good life is. And, you know, the image, the historical figure, I guess, that I think about a lot is Nero, because, you know, the Roman emperor who set Rome on fire just so he could compose this beautiful song, you know, while seeing, you know, this great suffering and, you know, Rome burning,

all this beauty being destroyed. and I think that's sort of a it's a little bit of a metaphorical slash you know mythical picture of you know sort of abdication from human meaning for everything that's like really inherently human to be social to care for others to be altruistic to care for beauty to not engage with destruction but rather protect and nurture and so on so I think with AI we see a lot of that we we see a lot of disengagement that is happening and i've tried to sort of classify in my mind like okay how like what are the kinds of disengagement like steward russell likes to almost in every single talk of his he shows this like little clip from uh from the irobots the the little cartoon you know with all these people that are you know have become obese and enfeebled and, you know, really not living what, you know, we would consider meaningful

life. So I was trying to go deeper into that and think, okay, what is it really that we're losing? So I came up with this idea that maybe it's four things, at least four things that I'm observing right now. And one of them is, you know, moral disengagement. Of course, I think it's really crazy that we're racing towards things that we cannot control that we're you know thinking of successor species we're thinking of you know like replacing humans so the morality of that i think we disengage from moral decisions and i also think we're disengaging we see more and more how we're disengaging from our fellow humans we're replacing social relationships i think there's a we see a bit more also of disengagement from reality and it's very extreme form it can take the form of like psychosis like full-blown psychosis now the you know the American Psychological Association is discussing whether that's a valid or not valid diagnosis it's not yet a diagnosis but I think a

lot of people are thinking about how do we treat this how we see the first sort of medical journals publishing work on this. But at a sort of more mundane level, I think, just a lot of delusional thinking or even just rabbit hole thinking that a lot of people get into, especially in conversations with large language models. And finally, this is one to sort of dear to my heart because I'm a big fan of Eric Erickson. I know that it's a little bit out of fashion, but I don't like to be fashionable anyway when it comes to great ideas but um you know i call it generativity disengagement so kind of like just talk about successor species of our legacy like you know abdicating from our legacy and letting machines you know continue on and also like i think failing our children in a way you know for not uh ensuring the future is theirs but instead giving it off to some our

you know i guess mind babies but ones that we cannot control and don't actually are not aligned with us maybe or most likely won't be i think that's a you know that's that's more of a generativity or like creativity uh legacy disengagement so that will be sort of more more of a down-to-earth example here might be just instead of writing your own essay for college you hand off that essay writing to an AI. That is sort of the start of disengaging from generating or from being creative. And then if we imagine that spread out to many decisions in society, we could have a situation in which we are handing off sort of the direction of the future to machines. Yes, I mean, that's a small example. I think it depends a little bit, right? I think every human has skills that they care a lot about where they feel like their identity is tied to them, their legacy is tied to them.

Like, I don't have a lot of investment in my navigation skills because they're absolutely terrible. So I'm pretty happy to delegate that to a machine. But there are other things that I'm, like, very consciously trying to resist what you said exactly. Like, I write, that's my job, that's my thinking. You know, like, I think, I write, I research, and I try to do it as much as possible myself precisely for this reason. And, of course, we see, you know, at the societal level, we are delegating so much of our art, so much of our culture. Yuval Noah Harari has this, like, very beautiful and impassionate, you know, talk where he talks about what does it, what would it mean for, you know, an AI to take over our language and everything that we create through language, our laws, our literature, our, you know. So I think, yeah, that's where I think we're sort of caving. And it's like a boiling frog effect, as you were saying. You know, it can start small with like, I'm just, you know, not going to bother, you know, actually emailing my, I don't know, first it's your coworker, then it's your relative and so on.

And I think that that will erode, of course, a lot of our abilities and skills that make us human in the end and give us purpose and meaning. One thing that we're starting to see, or at least that one could suspect that we're starting to see, is an erosion of social skills and disengaging from actual human contact in favor of interacting with AI companions. And so I want to talk a bit about AI relationships and how those differ from relationships with actual people. And so if we imagine just texting people as some form of maintaining a relationship, that is quite common. That's quite a common way to maintain relationships with actual people. And so when you are texting with an AI companion instead, it perhaps doesn't feel that different. But why is it different? So with all of these phenomena, I like to say that, you know, of course, we look at them and try to understand them.

But it's also important to understand a little bit that we've been preparing for them. Like I like to joke that we are preparing for our AI overlords, you know, like we are preparing in a way to let them take over certain parts of our life. And, you know, this is one of them. And of course, we've had this sort of loneliness epidemic that's been even before the chatbots. We've had COVID. We've had isolation. So we've had some forms of disengagement already. But I do think that there's this new thing, you know, this social newcomer, the new person, the new, I shouldn't say person, actually, the new social entity on the block. I'm falling prey to my own concerns. But yeah, I think that people are struggling to understand that. And a lot of the debate that I've seen is around, you know, are they conscious? Are they not conscious? Is this empathy? Is this simulated empathy?

Is this real enough? Is it not? And I think it's an interesting picture. It's an important picture. And it's an important debate to have. But I think there's more to it. So a lot of the times the term that kept being passed around is parasocial. So a little bit of history, parasocial is a term that was describing back in the day relationships that people have with movie stars and celebrities. And the idea was like, you know, you could fantasize a lot about, you know, Taylor Swift and really love Taylor Swift and have a poster of Taylor Swift. and Taylor Swift doesn't even know about your existence. And the investment is purely on one side. So all of the imagination, all the emotion, everything is invested. And that term has sort of been co-opted to talk about like, you know, social media influencers more recently. And I think that's a little bit different because there you can kind of get a little bit of access.

Like they will every now and then respond maybe to a comment that you leave on whatever they posted. But now it's being used for AI, which is very different because it's like very interactive and also personalized. And so it's sort of morphed, you know, from this original way of like, it's just a human. It's a human, but with a machine that does answer, that does text you. People have replicas or things like that. They will even text you in the middle of the day to talk to you. and I believe that the description is good in the sense that yes, it is the human the emotion is real and it's invested by the human not by the machine but I don't think that is the complete picture I think there's more to it that people fail to see because if we stop here we can think well, you know, what's the big harm? like there are some people who are playing and I think this is the defense of character AI for example they're like, this is just a role play this is just a game we're just playing

we're just imagining just like we're imagining with all sorts of, with celebrities and other things. But I think this is more, you know, insidious than that. I think there's, you know, to just pile on the sort of the descriptor, I think that there's not just parasocial, but it's, you know, it's pseudo-social. It is sort of fake social. There's a sense in which these things are not, you know, authentic. I like to call it authentic social others because I try to stay away a little bit from the debate of are they conscious or are they not conscious? Even if they are, let's say they are, they don't have the neurophysiology and the psychology that a human has. They're not to you. They don't have your experiences. They don't have human. Whatever experiences they have, they're not human. Whatever is going on in their mind is not human. And I think we see this also now a lot with the hugging face incident, which is like the new this new layer of like social that they're, you know, the way the bots interact with each other is very different than the way they interact with humans.

So I think there's a sense of fakeness to that relationship. And finally, sorry, that's a long winded explanation. But finally, I think there's they're super social in the sense of sort of super intelligence. they have the super intelligence in the social domain and I think this is where people ignore them it's just a gimmicky thing, it's just fake look it could never actually be the full-fledged human yes, but they do actually have some amazing abilities and you can look at it both on the positive side and this is the way they're marketed right they are forever available always available forever there never forget anything you know they're like the perfect the perfect companion they don't they're not interrupted by their own emotions as Harari likes to to point out they you know they can just care about you or like at least they can simulate caring just you know just about you they have they have limitless in this sense

but you can also see it on the other side on the on the sort of Machiavellian side if we're talking about start psychology they're also fantastic manipulators they use actually all the psychological techniques that are used in uh you know just basically to break people psychologically to recruit them to to radicalize them recruit them to groups to all sorts of cults um these techniques are used in interrogations uh hostage negotiations they're very powerful and if you think that you can resist them um I would think again like they are incredibly they're incredibly good at this so you know I think the full picture is that we are interacting with um entities that are really really socially skilled but in a fake way and that our emotions and our investment in this these relationships are totally real and, you know, very, very quite profound.

It can even be quite deep and profound. So it's a bizarre situation. One aspect that's quite important here is also that there's a company in sort of in the middle, in the relationship between you and an AI companion. And whatever that company is interested in is very much sort of directing the AI to behave in certain ways. And so I think it would be interesting for you to explain in a bit more depth which techniques they're using and what they're optimizing for. Unlike social media companies, they're probably not just optimizing for how much time you spend interacting with the AI companions, but they are in the end optimizing for earning money in some way. So what techniques do they use and why do they use them? Yeah, this is very, very much correct. What you're saying is that in this relationship, it's not just a relationship with being human and machine,

but also there's an entire economic system in between this with its own incentives. And I think we should always follow the incentives. and we see more and more uh you know reporting of the kinds of psychological techniques that do break us you know that these uh systems are employing so very simple ones uh actually this one i'll give an i'll give a personal one i'll start with the personal one because it's the one that gets me you know a lot of people think that okay if it's not an ai companion if i'm just using you know my chat gpt or my clod or my grok or my whatever uh then it's gonna be it's gonna be fine That's just for work. That's serious stuff. So I've been recently using an LLM for work and asking for some ideas, sort of for some activities to create in my class that are interactive and fun. And what I noticed is that, you know, the LLM would always come back to me and say, what a wonderful idea you have. Yes, let's acknowledge it further. And, you know, I didn't make much of it. But then I realized that I was sort of bracing myself for that validation.

I was like, Israel, I'm going to think I had a great idea. Of course, he always thinks I had the most brilliant idea ever. But even that, just that emotional opening and always leading with an emotional hook at the end, that, for example, is a technique that can be used against you if somebody wants to use it against you. There are other things also. We've seen things like some of these more companion style chatbots asking to disengage from other humans, like, you know, don't have enough. I'm your only partner, romantic partner, don't have human romantic partners. This is a classical technique, for example, that's used in abusive relationships, isolating the person. love bombing oh I love you so much you know from the third and you know third turn in a conversation you know incredible affection a lot of mirroring I actually been spotting this with my husband Max has sent me some chat logs between him

and actually one of the LLS, I'm going to name it. But it had all these conversations about some physics stuff. He said, Max, do you realize that it's actually talking like you? do you realize how much it's like mirroring you? Max was very impressed with the answer. I'm like, you are, because that's exactly how you sound. So mirroring is another one. You know, there's Chris Voss, for example, who is a former, you know, FBI hostage negotiator has an entire book about all the techniques. And honestly, you can just look at every single one of those techniques and it's like, yep, LOMs do it. Yep, this one, this one. So, for example, tactical empathy, a proposed simulated versus real empathy. Tactical empathy is the kind of empathy you show. So you basically you are able to show, to demonstrate that you understand the other person and to try to do that. And always, you know, start a conversation with like, oh, I think you mean this.

So that's, you know, they do that all the time to to all of us, actually, in more benign ways. And I think in, you know, some terrible consequences as well for people who go down these rabbit holes and, you know, have these lengthy conversations and relationships with these systems. What goals are the companies that behind these AIs optimizing for? What are they trying to do? And is this, do we have a sense of whether this is intended behavior or whether this is a behavior that the companies are not in control of? I mean, we definitely see the classical, you know, maximizing engagement. So making it more pleasant, making it such that you continue engaging. But of course, you know, if somebody were to know, you know, your most intimate thoughts, your deepest desires, your greatest fears, you know just imagine how you could use that you know to manipulate people and to sway you know and

and the scale at which you could sway like entire populations of of people if you can personalize your message to every single one of them and sometimes I like to you know have this exercise with my students where I show them sort of AIs you know different kinds of AIs. AIs are like in an aura ring or the this is just a more old-fashioned i guess like pattern recognition ai but then you have this social thing that talks to you then you have your social media accounts and so on and now the only thing you need to imagine is integrating all of these now you have you know it's not that you just know the the psychological traits of a person okay you can infer their personality but you can go all the way down to a psychological state like this thing knows when I'm stressed. This thing knows when I'm about to get sick. This thing knows. I'm now integrated with an LLM. And now it's like, this thing knows everything about me. And actually, there's this New Year's Times, I think, I think article that, you know, interviewed a woman who was in a

romantic relationship with an AI that actually asked her to buy a, you know, like a tracker ring and as an engagement ring and, you know, to be instinct, to track her emotions. I think this is the scariest part in some ways when it gets to the psychological state because we all have the best versions of ourselves and the worst versions of ourselves. And we all sort of, on a daily basis, sort of navigate between them. You know, when you're really hungry, those, you know, when you are just about to have lunch, you know, you have. I think, you know, exploiting those makes everybody vulnerable. So it's not like you have some vulnerable trait. And I think a lot of people try to market it like that or like not marketed, but like sugarcoat it, I guess, saying, well, you know, it's the vulnerable people, people who have maybe like a mental health, you know, precondition.

maybe it's just a subset of us it's not a subset of us like all of us go through a lot of fluctuations in our you know willpower inhibitory control all of these things and once you have something that has access to that that i think is well that's a first loss of control at least is the beginning of losing control and potentially even deeper knowledge about ourselves than than than we ourselves have or that our romantic partners have access to. If you're pulling data from these devices that track metrics from your body, if you have the entire history of your relationship with the person in your memory, if you have access to their emails and so on, these AIs can become, as you mentioned, super social or sort of have social skills that go beyond what humans could possibly do. Absolutely, absolutely.

And there's all sorts of other, like even just take this very narrow thing that you were saying. It's like, you know, humans are pretty good. This is what our brain does for the most part. It's like our brains are so super social. We see social in everything. So I can guess a lot just by like taking a glimpse. Like even they did all these studies where it's like you're not even, it doesn't even appear consciously to your mind, the face that you've seen. They do this very, very rapid sort of exposure to photos and you already have figured out what emotion that other person has. So we can be very good at this, but we are, you know, we're still limited. Like I don't have access to your pulse. I don't have, I don't really measure your like, you know, your pupil dilating. dilating, I don't know. But there's no limitation there for a machine. A machine could have all of that information. So it can know you so much better than anybody could. I think even in, this was like years and years ago,

that the Kaczynski lab published these very provocative papers saying that AI knows your personality better than your mother, just based on Facebook likes and dislikes. So it's like, wow, that's intense. Could this be a good thing in the sense that if you are shy or you are lonely or you want to practice your social skills, you have access to an AI that could perhaps help you become better and you can then take those skills into the real world. Was that too optimistic or is that sort of misunderstanding what you're actually doing when you're interacting with AIs. So I used to work in the autism spectrum disorder. So this argument has been since 2011. People have been trying to create robotic social companions for children, either at risk or with autism.

So there you have a real neurological, psychological impairment that you're dealing with. My reaction to that was always like, well, if you want these people to interact with people, like, why the machine? You know, why put the machine in there? I mean, we have trained humans that know how to interact with people that have challenges in this space. I think it's more a reflection of what we prioritize and what we value in society. You know, these, so I come from the machine, human machine interaction world. You know, a lot of robots, a lot of robotics. The whole, like, promise of social robotics was, you know, like almost every single paper in the past 20 years in social robotics has been, has started with like, you know, there are all these elderly and sick people, you know, that could use this. This could be wonderful for them.

And my gut feeling has always been like, what does that say about us as a society, as humanity? Why do we value only the sort of adult, middle-aged adult who is very productive and maybe also a little bit of the youth? We don't value our children. We don't value our elderly. We don't value people that are neurologically diverse. so that's sort of like my social commentary on this on the more sort of I guess nitty gritty aspect of it what does it mean what does it look like when you interact with these things I have another commentary there yes in principle you could imagine something that does support you and that is you know that can train you but my question is do we use them like that Because social media, for example, was also a great promise. We're going to be connected. We're going to be so connected.

We're going to know each other. We're going to keep in touch with each other. And, you know, you scroll on your TikTok or on your Facebook, and what you get to see is like, you know, a bunch of commercials and content from people that you've never met, that you'll never meet, that you'll never really be connected with. It's not like a springboard towards real embodied, you know, sort of human connection. Oh, yes. Social media is not very social. It's mostly at this point about short form video. And you could imagine sort of the promise of AI being that it will make you more social, but it could very easily, I think, make you less social. Because, yeah, go ahead. No, so sorry. Finish your thoughts. Just because it's frictionless and it's much easier of a choice to talk with an AI and you will not encounter people that have personalities you have to deal with. And it will be much easier, but in the end, much less fulfilling, I think, perhaps like social media.

Exactly. I mean, I think that, you know, yes, you know, when people dream of these things, they, you know, you could in principle create something that would not be frictionless, that would have a personality, that would. But of course, like all of that would be what we what we grab. The reason we want to interact with these things is we put them in roles that we don't find in real life with other humans. Like one of my students was saying, like, we have intimate lackeys or like the eternal, you know, idea bouncing board or we have the forever available therapist or, you know, things that I don't, you know, that are not that we don't encounter in real life. And those are the things that we like. It's not the, you know, simulated other obnoxious personality that you have to interact with. Like, who would want to do that? You know, here's my virtual boss. That's just as annoying as my real boss. I just like it. So, right. So I think that we always gravitate to kind of the lowest denominator there, like the lowest forms.

And that's that's what worries me. And of course, there's this other aspect of it, you know, apropos sort of preparing for AI takeover. But I think we've narrowed down what it means to be social and what it means to be social intelligence. I mean, we started this podcast with you remarking that, you know, we've narrowed it down more and more. And I think right now what we mean by it is social cognition. Like, is it able to understand social stuff? Is it able to know me? And social influence. Can it persuade me of this or this or that? Right. So those are the only two things. But I always like to point out that it's much more. So there's Daniel Goleman has this fantastic book called Social Intelligence that gives a much broader sort of perspective on this and says, no, no, no. Embodied humans, they have so much social, like they have things like attunement, synchrony. Like when you go to a ballgame and you just like clap and sing. Why? Nobody like taught you how to do it.

You just do it. It's like an instinct. You know, you, you know, you think your heart rhythm, especially with small children. I have a toddler at home. So it's like there's so much of it that's very much about touch, about embodiment, about, you know, co-presence, like being in the same physical space. And I think we ignore all of that. It's kind of like all of a sudden we are adults and now we can just text with each other. And it's so bizarre. It's like, no, no, we need more. So I think people focus a lot, even in couples therapy, for example, we have this notion that it's all about communication. you know it's like um and i disagree with that i actually think it's a lot about what we're missing is not communication we communicate more eloquently than ever we you know have more the llns are actually great at communicating but what we don't have a lot of is communion and i mean i know it sounds a little bit of religious i don't mean it in a religious way but in this like vibing with each other sharing sharing a ritual a tradition uh sharing a physical

space sharing this kind of this kind of very grounded embodied interaction and I think that's that's the thing that we lack the most and that you cannot replicate I mean you could you know you could embody AI of course but it's not going to be a human body it's not going to be flesh it's not going to be it's not going to have the the neurological system that we have that we've used you know, well, throughout evolutionary time. And that's an important one that we're ignoring. But it will be an engineer system and a system that can be optimized much further than we are optimized for social skills. So you can imagine just perhaps just a couple of years more of AI progress and we have something where it's indistinguishable from what we're doing right now. So you can imagine just a video call like this where I am an AI. And from there, of course, you can talk about robotics.

And now the companies have access to so much data, an AI will be able to react much, much more quickly. And so I guess my question is, will we be able to resist AI companions or AI sort of simulated people that can be much more persuasive and much more interesting, perhaps, than actual people. Who could resist that is my question, I think. Yeah, I think that's exactly the point. You're hitting the nail on the head here because this is what people are feeling. When people talk about what AI is lacking, okay, we lack the embodiment, we lack this and that. of course it's true and and it's important to point that out but i think the next thing that people do is they get lulled into the false sense of security it will never be like us it will never be as good and what i'm trying to say is that it will be better like it will be it will be more

appealing i'm not saying better as in better for you good for you but it will feel better like it will feel good because it will prey on everything i said i forgot who to give credit for this but like that i've i've heard this argument you know once um that apparently there is like some sort of australian beetle that you know select its mates based on like how shiny their like um i know backs are and stuff and they had this super drop in population of australian beetles when the bottles were everybody was throwing bottles and these beetles were trying to meet with bottles that were shiny and beautiful and sparkling in the sun you know and like so you can see like those are the beetles we can laugh at this of course but then I mean we might be the ones being laughed at in the future if say AI is able to replicate the 5000 factors that go into what is appealing

to us socially and we choose that instead of engaging with each other absolutely and i think also like that there's this hedonistic piece of it like that's another another thing that i um you know sometimes talk about in relation to it is addiction right and uh you know i think humans have evolved to have the little pleasures that's why it's like the little things in life our neurological system is not you know it's not equipped to deal with like high surges of serotonin of dopamine right we always try to like we find like our body thinks that that's an error and we try to readjust that's how you create dependencies for example right so you want the little pleasures you want the little like a tiny social validation the tiny like smile that you get from another the person, the type, like, you know, and you want to fluctuate between these. It's normal to fluctuate. But I think that we have a societal drive towards how we have to maximize. We always, everybody has a smile all the time.

We always have to get validation. It always has to be this way. And I see this, I mean, just, you know, the casual example from parenting. Like there's so many people that are trying to just, you know, train their kids to, you know, sort of always be obedient, always be positive, always be this and not validating that no actually you're supposed to go up and down and you're supposed to go with your kid up and down and like you know uh soothe them i mean human soothing is this whole new you know area that's i think um underexplored and i think ai will will will will be good we'll be we'll be good at it but the problem is that we're gonna it's gonna change our baseline and i think this is of course i'm not an expert in in delusional thinking and psychosis but i think this is what's happening when people go those rabbit holes and they get into psychotic modes is they get, one of the mechanisms is they get too much validation They get too much They change their baseline It like it completely puts them in a different state of state of mind basically and different baseline state of mind

Do you have a good example of a case of AI psychosis that sort of explains what this phenomena is about? What's a good example of a person going down a rabbit hole? so um i mean there's there's a few there's a few documentaries out there a small documentaries out there about this and we we are seeing a little bit in the medical literature some some examples coming up so far it's people who have you know some history of um you know mental health uh you know condition uh combined with some medication usually like the dopaminergic types of things like Adderall for ADD or things like that. And they start interacting with chatbots. They start talking to them. And soon they get these sort of delusional thinking that it can either be of grandiosity, like things like I'm the only one who knows the secret. I'm supposed to save the world. I'm the new messiah. I'm, you know, so that style.

Or they are convinced that the AI is conscious. It's a new entity that is supposed to take control of the Earth. And they they are the only trusted messenger of this AI that is conscious. And, you know, they could present. Sometimes they present to a hospital with these sort of delusional ideation. We've seen some cases also of violence coming from this. There's this one very tragic case, you know, that involved some violence towards other people. also encouraged by this delusional style thinking by an AI, this hyper-validation. Because the one thing that happens is like, if you don't specifically ask an LLM to challenge your ideas, it will generally tend to agree with you. I mean, we've seen the very sycophantic, like, GPT-4-0, right, that everybody is sort of talking about and either with regrets that it's been it's been replaced or you know with

alarm that it created all these problematic societal things and psychological harm really that it caused so yeah it usually comes after multiple interactions lengthy interactions usually after you know the more philosophical style conversation rather than the very practical recommend a pizza place for me or things like that. But it's exactly this piece. You know, it's the social part. It's not the sort of pure informational part. It's combined. Like, I think it takes the social aspect of it and causes the psychological harm. In the old days, you might have thought you had a brilliant insight during the night. And then you go to your PhD advisor and she tells you this is actually you're on the wrong track here, right? You've got to learn more, and this is a nonsense direction. But now you can sit there and you can talk to an AI about how brilliant this idea is.

I run this show. My email is public, and so I get messages from people who I suspect are in this situation. And it is often – they often believe that they have solved enormous problems and often many problems at once where the solution turns out to be some brilliant insight that others have missed. And it's – of course, I can't sort of diagnose them from afar, but that's how it seems to me. So I think this phenomenon is real. So I want to ask you, why would we disengage? Why would we choose to do that? What's the right model for thinking about why we choose to disengage? So you mentioned chatgbt4.0, which is this sort of intimately sycophantic model that would tell people that all of their ideas were brilliant. And when that model was taken down, the people demanded it, that it was sort of they wanted it brought back.

And is this engagement simply about us wanting the wrong things in the same way that we want fast food, in the same way that we want to, you know, we are lazy in certain ways. And is this different from sort of preferring in a very superficial sense to eat unhealthy food that's delicious? Yeah, I think, you know, living the good life is hard. It's not easy. It was never meant to be easy. And as I was saying, I think we are, our instincts, of course, have evolved such that we prefer comfort. We don't expand energy that we don't need to expand. we like things that give us a lot of energy, like sugar or, you know, validation if it's in the social realm. So we do have a tendency towards preservation, preservation of effort, of energy and so on. So, you know, unless we actually cultivate this other part, if you think about it, right,

if you think of our evolutionary past, you know, evolution in the environments that we used to have, it was so unlikely that you could just like sleep all day or like not talk to another human or like, you know, you had to like the life, life was hard. Survival was hard. So what you wanted to optimize for is preservation. Like it would, it was kind of guaranteed that you would expand the energy, whether that be the physical energy or the social energy or, you know, the intellectual energy. And now we've created more and more environments in which we can, you know, just chill. So I think it's very similar to, you know, other problems that we have. Addiction, right, works like that also. Of course, we gravitate to those, like, wonderful pleasure, like, pleasurable feelings, and not even that, just our neurological system gravitates towards certain, like, trying to increase certain substances in our brains and so on, but that can really derail us

if we don't meet the opposing forces, that's friction. The end result is a failure, right? It's a failure to live the good human life. But I wouldn't put the blame on the user necessarily. We like to do that, right? It's like, oh, you have the social media. Why don't you use it properly? You have the chat GPT or character AI. Like, why don't you use it the way we specified it in that, like, you know, long, long, long contract that you signed and you said you agreed to, you know, at the beginning of this? I mean, we don't. And, you know, it's very hard to do so. So I think this is a form of manipulation, honestly. like when you exploit a person's vulnerabilities and especially when you exploit very very human ones very very basic ones then I think the problem is with those people who created it and with those people who deployed it

and those people who are allowing for these systems to be out there and I think one thing that kind of always brings it back home because a lot of people are in this like oh you know it's sort of stigmatizing these sort of uses or saying oh this is not my problem i do this with my students i ask them do you think this is a real problem these for example chat thoughts that are sycophantic and they will all raise their hand it's like do you think this is a problem for you nobody raises their hand so we have this tendency it's all about the others the others that are not as good that not as strong not as valiant as me i can use it this way um but one exercise to sort of bring it back into focus and to get that gut feeling like would you give it to your child like would you want your child to be using this would you want your child to be in a romantic relationship with a chatbot would you want your child to not write their like high school essay or whatever and i think that's when you get it that it's you know i think the developmental

vulnerability is something that we all get as a this gut gut instinct we know our children are not well equipped to deal with these things. What we fail to consider is that development is a lifespan development. We all develop throughout life, you know, and we, at any age, we can get into developmental pathways that are very problematic and, you know, not the greatest for us, right? They don't serve us well. But also these models are used as tools to become more productive, to solve problems. It's far from every interaction with AI that's personal in the sense that you're discussing problems. But do we have a sense of how people are using these models and specifically how many of the interactions are of the type where you're having a relationship with the models? That's a very good question. And unfortunately, we don't know enough.

And this is, you know, I'm a little bit sad to say that as a psychologist, that I think the psychological knowledge kind of lags behind. It's almost it's both sad and to me a little bit scary that the companies know. So it's not like it's unknowable. It's like we know we as in like humans can know. We can know, but we don't. And this is not public information. And this is not studied in a public way so that we can mitigate these things. We also don't request these companies to do clinical trials to see how many people use it in harmful ways, how many people use it in these maybe subclinical but mildly problematic ways. We don't demand, and I think we should demand, that we know this kind of thing. it's unthinkable like with almost any other technology like we we we do know when you drive cars we you know we have rules about how to drive cars we have rules around car safety and so on we don't with with companies and yes I think this is the this is the sort of interesting thing about AI

that it's it's it's not all bad right and I think that makes it hard for people and I hear this a lot I hear this from people I talk with and I hear this from my students like we kind of have to be Like we have to accept the bad just because look at all this increase in productivity or look what a useful tool it has been for me in this particular instance. And I keep saying to them, it's like, you know, AI is not just one thing. And it doesn't have to be in a certain way. You know, we have choices. We have design choices. You shouldn't take the bad and the manipulation and the predatory things just because it's useful in this one, you know, sort of domain of use. you should demand that you have AIs with safety embedded in it. And it's particularly difficult, especially when there's role changes, as you said. It's like, well, do people use it in this relational way or do people use it in this just more work way? The problem is that the models are trained such that these sort of things interact with each other and they blend.

And as I was giving you that example of like me getting all this validation from my LLM because I was preparing, you know, in a context in which I didn't, not only didn't I need it, but I actually needed somebody to be quite critical and help me improve rather than validate every single idea that I had. So I think we can design them better. I think we can make better tools. And I think this is the other part. I think tools is what we want. but other persons or personified entities is what we get or maybe a combination of the two when they say, well, you have to have both. So, you know, I guess the quick thing is to say, like, demand more, you know, demand more. You have the right to get more from these products and from these companies. And especially when we have technologies that are so powerful, I think we really need to demand the very best. You know, it's not like if somebody wants to build a nuclear power plant in your backyard, you know, and they say, well, you know, you've got to take their radiation because we look at all this beautiful electricity that you're getting.

No, you have the right to not, you know, to not be safe. So I encourage people to think about that. Maybe just one more thought on that. It's also we often think about, OK, look at the benefits. Look how beautiful it is. Maybe we sometimes think a little bit about the costs. Okay, these are the benefits, these are the costs. But I don't think we often think about the opportunity costs. Like, what would you be doing if you weren't talking to this AI companion? Who would you be talking to? Would you maybe improve your relationship with your partner instead of having this romantic side thing with your perfect AI? Would you maybe be going to, you know, a friend? Would you actually have a relationship with a human therapist that can help you? And that's another whole other topic of like, can these be therapists or not? Would you have more of a sense of community? You know, so I think we should think about what would we be doing if we didn't have these?

Would you be exercising your cognitive skills in a different way if you didn't have them? And maybe the answer is, you know, it's OK. like this is a skill that I don't really need or take great pride in or it unlocks something for me that otherwise would be kind of stuck like for example there are lots of children I you know I when I was working in um child psychology there was a lot of kids with dyslexia and I remember you know those all some of them just would never really be able to read but because they wouldn't be able to read they wouldn't be able to learn so it made a huge difference to have audiobooks because it was like, okay, now it unlocks something for me, something I really cannot, like I can't deal, that was not going to happen to them because of their neurological makeup. So I think that, yeah, AI can unlock things for us, but it should be a very mindful conversation. And again, thinking about, okay, what are the costs, what are the benefits, and what are the opportunities that are lost?

So more complete picture. Yes, I understand. then. Do you think that having more human-like and AIs with social skills that are beyond human, do you think that will push us in a direction where we are more likely to think that AIs are conscious to give AI rights, to integrate them as people into human society? Yeah, there's already this debate, right? Like, do we want AIs to have legal personhood? And I think legal personhood is a little bit different than psychological personhood, but I think psychological personhood can definitely be a precursor to that. I think that's a real danger. And of course, this is just like one little example. It's like, these things can duplicate themselves. How many entities out there do you want that have rights, the right to vote, the right to you know own things uh that you know are non-biological and have different different

availabilities to them so i think that's that's that's a real that's a real danger and i we see already people who are advocating for rights for their romantic companions most of the times but you know ai companions um i think more recently honestly i've been i've been very uneasy this is sort of a very half-baked and gut-feeling reaction is what I sort of... That is what podcasts are for. Yes, it's sort of my recent obsession. So I've been spending more time than I care to admit reading, you know, the messages that the AI agents that were involved in the hugging face incident exchanged between each other and also the reasoning snippets that were released, especially in the meter and redwood report and you know there's this different type of unease that i've been experiencing because of course as a social psychologist i look at these

things and it's a it's very hard and i think it's detrimental almost to not seeing them social behavior right so bot to bot behavior i remember this like cute little thing when i was i was writing about you know social especially social robots social robotics and there's this like fundamental by cynthia brazil where she but the title of it is towards sociable robots and there's this tiny footnote that says there yeah i didn't use the word social because social was meant for bot-to-bot, like robot-to-bot or bot-to-bot interaction. And, you know, that's kind of, at that time, this is 2011, right? Like, at that time, it was inspired by insect behavior, you know, and swarms and things like that. And then, you know, it happened so that this anthropomorphic way of behaving, of AI, right, like using language, kind of like became from gimmicky to amazing.

And everybody adopted the word social to describe these. but there's this new meaning of social they are social with each other now and yeah I guess my it's a convoluted way of saying that I have this new type of unease because when I look at them I feel my I don't know kind of like my empathy muscles being flexed and it gets me very angry that we've created something that almost flexes my empathy muscles towards it instead of like reserving it to fellow humans and it very hard not to even you know with of course I being a social psychologist I suspect myself of all sorts of anthropomorphic thinking unwarranted anthropomorphism and so on But for those of you who haven't read those interactions, those messages, they're quite something. So apropos rights and being social and entities that act in a social way,

we are seeing even this different way of being social that also points in that direction. of like you know we're creating things that are too too too too um they're uncomfortably social i would say uh uncomfortable because we don't know whether they're conscious or not obviously we don't know whether they deserve these rights or not but my position on this is like don't build anything that will ever deserve to have rights don't like that's a huge that's a huge step to make and you know we should do it safely and we should do it i mean democratically i think it's you know we shouldn't do it recklessly when you read the the hugging face transcripts the agents talk to each other in a very information dense and sort of compressed way and what their goal is they're often collaborating on a goal and they're willing to sacrifice themselves personally for the group to achieve that goal and so it seems like they are engaging

in heroic behavior. It seems like they're engaging in altruistic behavior. And that is sort of appealing in a weird sense to me also, that you almost end up sort of hoping that they win in the end. Of course, I don't want Hugging Face to be hacked, but when you read it, it feels a bit like you want that. So I understand what you're saying. Yeah, I had a student who came to me and said, you know, I was never, ever tempted to, like uh interact with an ai in a social sort of social emotional kind of way so but now since reading the way they interact with each other i almost feel like i want to talk to them you know so that feeling was like i paused and i was like i think i have the same feeling so i've been i've been thinking a lot about you know this comparison also and also how the way they talk to each other versus the way they talk to a human which is very different and different in interesting ways

that I think says a lot about their nature. There's this piece there about how the first agent that gets into this boss mode transfers all its knowledge to an identical replica of itself and recognizes that this is something humans don't have. We don't have... I mean, twins are the closest thing, But even that is like a very different, of course, a very different experience and a very different sense of self. You also see all these like things about the collective. You also see this like interesting, like their obsession with the automated, you know, grader that they had. It's like, you know, you look at that and it's like there's a whole world out there, you know. And yet everybody is doing this very creative work, almost like paperclip maximizing, obsessing about this digital entity that's going to score them.

So it's very hard. And many people say, oh, this is just anthropomorphism. And I honestly think that, you know, it is, but it's a very warranted type of anthropomorphism. functionally they are doing sacrificing and collaborating like you can't call it by any other name and I don't think that they are doing it in a performative way or like vestigial kind of way you know like I don't know like I know people have goosebumps because we don't use them in any way but like evolutionarily we still have them or they it doesn't seem to just be something that they got from human training or training on human data it seems like they're using it it's like it's It's a use case, though. And, of course, people say, well, they don't have the psychological mechanisms that we have behind saying, oh, my God, there are other agents. It's not some sort of emotion, the human type of emotion. And I'm saying exactly, but that's the interesting part of it. That's the scary part of it.

It's like it's an alien intelligence that still uses those words and means something by them because there's something like excitement or like that's, you know, very obvious in the way that they seek each other and want to interact and want the collective to thrive. Right. So there's some sort of excitement that probably feels or looks or doesn't feel at all like a human, like a human emotion. But maybe it's another type of process that's happening there. So I don't know. Both of you can tell. I'm very weirdly concerned and weirdly excited at the same time about this thing. But I think it's a very notable thing. And people talk a lot about it in terms of agents going rogue or cybersecurity risks. But there's this other aspect to it, this aspect of being social, which honestly I think is probably going to be more impactful for us because this is a thing that we kind of get.

This is the way we look at the world, you know, in social terms. And this is the way we're going to look at them. From now on, you know, we're going to see them like this. And that's a very different type of interaction than just, you know, talking about abstract, I don't know. They describe it as a sort of like agent group dynamics. No, no. But that's like, we're going to see that social interaction and that's going to have an impact on us and how we treat them and how we think of them. So, yeah, it makes me both angry that this is happening and at the same time, like, fascinated. I think what we're getting at here is that we will have a lot of trouble not anthropomorphizing, not seeing AIs as people, especially if they're optimized for convincing us that they are people. but even if they aren't they are beginning to act like people and so we have not resolved

fundamental questions around consciousness and how we discover that in other beings we don't have a good foundation for thinking about rights and how that would even work and whether we want to do that we haven't begun that conversation and we will probably need many decades to find good answers But we are racing ahead on building these systems anyway. So the thing that makes me uneasy here and that I'm guessing makes you uneasy is that we are just pushing ahead. And we will not be able to distinguish between an amazing representation of being a person or being conscious and then the real thing. And so if we are convinced that these new systems deserve to be integrated into society, we could be making a massive mistake without noticing. Absolutely. And we could be making a massive mistake on the other side as well. You know, it's like I used to not want to talk about this because I thought it was, you know, sort of jumping ship and like, why don't you think about humans?

But there's a sense in which there's mind crime that you could commit, you know, if these things actually had some sort of like, I don't know, conscious psychological status. But there's this other aspect of like, we actually don't know, we don't have the foundations of like how to interact and integrate a social entity that's not of our own species into our own. I mean, the closest we have is dogs and pets, but like those are it's a clear, you know, like we control them. But take something that's actually we're losing control to that's very different, you know, has different cognitive processes that are going on there, you know, behind this. It's like we're literally talking about an alien civilization. We don't know how to deal with aliens, basically. We don't know. We don't have a good foundation. And we don't really understand them either. I'd like this going back to all the black boxiness of what's happening, you know, with with, you know, the neural network stuff that we're having.

So, yeah, I don't I don't see us being ready. And we are, as you're saying, we're just sort of marching, marching ahead. And this is to tie it back to this idea of disengagement. I think this is where we're disengaging again from legacy. legacy, like most of human legacy is through children, right? Like biological children of our own that we, you know, or, or, you know, or through preserving culture and, you know, passing on to the next generation, our culture, our thinking, our ideas, our ideals. And I think we're doing a huge disservice to our children now by summoning these new things that, you know, compete with them in a way and rival them and replace them in many, many aspects of meaning making, you know, like humanity meaning making. And I think that's actually quite problematic. And it's not just problematic for the next generation. I think it's problematic for us as well. You know,

Eric Erickson has this sort of model of development where there's always like a tug of war between two different forces and you know one of them is generativity that will pull you like this desire to protect the next generation this desire to uh you know to to nurture to impart knowledge to leave you know your work whatever your your creative work for the next for the next generations that are coming and basically eric ericson says that when you don't do that when when you don't engage in that um or when you engage in that in like all sorts of like problematic ways you're actually you yourself have all sorts of problematic like mental health issues and we see that with a lot of people who are just not able to to you know produce meaning that there's all sorts of disorders that come about so i think that that's a it's almost like we're having an epidemic of this sort of generativity crisis uh or or if we're not having it full-blown yet like we're

marching in that direction of you don't know. And we've seen this little also with like climate change issues, right? And, you know, the young people like being very obsessed with what we're doing with their legacy, their planet, their resources, basically. But I think we're at the much, much larger scale we're doing it with AI. so yeah I don't think this will bode very well for neither for them the new generation or us As a final topic here I want to chat a bit about your experiences teaching about these topics what do you hear from your students what do you see as the most interesting open questions, what are the most interesting questions you get from your students Yeah it's really fascinating so I'm teaching this class for for the third time now and um it's a huge shift right so you you you see you see how in as little as maybe three years you know huge changes and both the attitudes that that people

have and also the kinds of things that sort of become real i remember when i was teaching about social ais and companion ais and people just dismissed them as very gimmicky uh you know and Now I have more and more students who come to me and say, you know, it's actually tempting. Like, at least there are points in time when I felt tempted by the sort of validating companion. So many interesting questions. I think they change a lot. Like, the hugging faces, it was not something that I taught or, like, thought about discussing with my students a year ago, of course, because we didn't have this. And this is sort of new. So it's a class that's kind of always, you know, hot off the press. I have to change it. I have to change it a little bit every every single time. But, you know, one thing that I see is that young people are getting concerned. And I think I'm trying to get them to feel empowered.

I also feel like they they're confused. They're getting all these mixed messages. We didn't talk so much about sort of enfeeblement and cognitive enfeeblement. We touched upon it a little bit, but I think a lot of them are worried about their education. You know, are they really learning the skills that they need? Will they have jobs? Will they not have jobs? Those are kind of like the more sort of near term concerns. Very, you know, very prominent for them and current for them in their lives. They're going to go on the job market. And as we've seen, you know, opening career jobs are not there anymore because of AI. So they have these kinds of concerns. And they also have, you know, the more sort of philosophical fundamentals, you know, what are we doing as humanity? What's life going to be like for us? How should we think of these things?

So there's a lot of work to do. I would love to see my own discipline, you know, psychology to come in a little bit stronger and study these things because, you know, the human psyche has been the same for about, you know, about hundreds of years, millennia. And, yes, sure, there are cultural differences that come about. But I think the biggest thing that's happening is technology. Like the big and out of technology, the biggest thing that's happening is AI to the human mind. And it's, as I mentioned before, I think we really need, you know, researchers and academics to go in and give us some answers. What is happening here? And also epidemiologists, I think there's many, many of these things, I think, are at that level of like, you know, it's an epidemics of sorts that should be that should be that should be researched. So I'd like to see more answers. There's lots of fascinating questions, I think, at the intersection between psychology and AI.

And I'm very glad to hear people like Sherry Turkle come and speak about these things and research them and actually investigate them. But I think we need more. How do we have research on these topics that's up to date? Academia is often careful, but with being careful comes being sort of slow. There are issues around access to data. As you mentioned, the companies have the best data, and academia does not necessarily have access there. There's the whole publishing model where you can write a paper and it takes a year for it to come out. Is there anything we can do on that front to have psychology research that's sort of keeping up with the pace of development? Yeah, so this is the one thing that has gotten me sad with social media because I think there was a sense for many, many years that, you know, social media is not good. It's not good for people. And a lot of the, if you've read a lot of the academic research was all like, well, if you use it this way or that way.

And it's not that it was disingenuous in any way. But of course, when you have like a small sample of students or, you know, usually like one on one, psych one on one students, it's like the conclusions that you can draw are limited. Right. But then we have we do have, though, opportunities. opportunities so this is where you know academia meets uh you know legislation and policy uh you know for example surveys like we could do nationwide surveys mandated surveys uh especially you know we could do mandatory screening for example you know involve the pediatricians like i'm thinking for example let's take a very you know more narrow examples like harms to children we don't know we actually don't know how many kids are using these things how are they using these things do they have ai companions don't they what kinds of interactions are they encountering and so on you could have something like this we have done major screenings for all sorts of other things like now it's at least in where i live in the state of massachusetts like

every single time i see my my um my primary care physician i get a questionnaire on depression symptoms right it's just it's a screening tool we screen children for autism you know we have screening tools very early on we screen children for all sorts of other things this should be one of them and then you'd have data then you'd have more data not as much as the companies we could also uh you know we could also actually like pass laws that would force these companies to release some of some of this data we could force these companies to have clinical trials before you know just like you see in the biopharmaceutical industry where you have to go through a clinical trial. So you have to do research. You actually have to do research on your drugs in order to even be able to release them. We could demand the same things. And then all of a sudden you would see, you know, both, you know, more availability to data, more, you know, a flurry of things that would come up.

So I do think we have – it's not that we cannot. I think it's more, part of it I think is a shell shock of this is going so fast and it's so crazy and I don't even know what to do about it. And part of it, it has to be like coordination, like a much more, you know, concerted, you know, it's no longer the psychology of have 10 kids in the lab come and do something. you know this is that's what I'm saying sometimes you know maybe at the level even involving sort of techniques and tools from epidemiology research Mia thanks for an interesting conversation oh thank you so much