機械が「主体性」を持つために必要なこと(マイケル・トマセロ)

What It Would Actually Take for a Machine to Have Agency (Michael Tomasello)

Machine Learning Street Talk (MLST)1時間6分
機械が「主体性」を持つために必要なこと(マイケル・トマセロ)

機械が「主体性」を持つために必要なこと(マイケル・トマセロ)

Machine Learning Street Talk (MLST)

0:001:06:33

要約

発達心理学者マイケル・トマセロが、著書『The Evolution of Agency』を手がかりに、主体性(agency)とは何かを語る回。環境が予測不能なときに自然が「意思決定者」を作るという視点から、爬虫類・リス・大型類人猿へと続く進化段階、そしてヒト固有の共有された主体性を説明する。LLMは言語という終着点から出発し、慣習を生み出す土台を飛ばしているのではないかと問題提起し、ボトムアップで機械を作る道を提案する。

この回のポイント

  • ●環境が予測可能なら行動は生得的に組み込まれ、予測不能なときに意思決定する主体が必要になるとトマセロは述べる。他の主体(捕食者・競争相手・協力者)が最も予測しにくい。
  • ●主体性の最小モデルはサーモスタットのような制御系だが、トマセロ自身は単純な制御系を「エージェント」とは呼ばないと明言している。
  • ●個体の主体性は、基本的制御系(爬虫類)、実行機能を持つ意図的主体(哺乳類)、メタ認知を持つ合理的主体(大型類人猿)と段階的に進む。
  • ●ヒトの違いは共同目標・共同注意・役割と規範を持つ「共有された主体性」にあり、そこから視点の概念や言語的慣習が生まれると論じる。
  • ●LLMは言語という頂点から出発し、慣習の創出や談話を欠いていると指摘。機械を主体的にするには、協調や視点取得からボトムアップで築くべきだと提案する。
  • ●嘘が成り立つのは人間が協力的で騙されやすいからであり、集団間の対立は協力の暗黒面だと述べる。
  • ●「知能」という語は学校型の言語的知能に偏るため避け、認知能力という語を好む。

章立て

  1. 著者紹介と研究の背景

    発達心理学者として類人猿と幼児を同一実験で比較してきた経歴と、本書執筆の経緯が語られる。

  2. 予測不能性と主体性の定義

    予測可能な環境では行動が組み込まれ、予測不能な環境で意思決定者が必要になるという主張と、制御系・意志・因果効力という定義の議論。

  3. 信号機とサーモスタットの例

    単純な機構を賢くするには知覚・目標・意思決定が要るという例と、制御系をモデルにする理由。

  4. 機械は人間以上の主体性を持てるか

    実行機能とメタ認知の階層、新しい目標を作れるか、学習偏重のAI研究への批判、LLMが終着点から始めている問題。

  5. 主体性の四段階

    爬虫類、哺乳類(リスの例)、大型類人猿、そしてヒト固有の共有された主体性という分類。

  6. 共有された主体性と慣習・視点

    共同注意から視点が生まれること、スタッグハントと慣習の調整機能、文法の視点性、指差しの例。

  7. 文化・虚構・協力の暗黒面

    地位機能やハラリの虚構、嘘が成立する理由、集団間対立について。

  8. 資本は主体か/チューリングテスト

    創発現象と主体の区別、資本を主体とみなすことへの慎重さ、志向的スタンスとチューリングテストへの見解。

解説記事

発達心理学者のマイケル・トマセロ(デューク大学)が、著書『The Evolution of Agency』をもとに「生き物や機械が自らの理由で行動するとはどういうことか」を語った回である。マックス・プランク研究所で20年にわたり幼児と大型類人猿を同じ実験に置いて比較してきた研究者の視点から、AIへの示唆も交えて議論が進む。

主体性は「予測不能性」から生まれる

トマセロの出発点は、進化的に重要でも環境が予測可能なら、呼吸や嚥下のように行動が生得的に組み込まれるという考えだ。意思決定する主体が必要になるのは、環境が予測不能なときだと述べる。特に予測しにくいのは、獲物や捕食者、同種の競争相手や協力者といった他の主体だという。

主体性の最小モデルとして彼が好むのは、サーモスタットのような制御系である。信号機を賢くする例では、車を知覚し、目標を持ち、それに基づいて判断する仕組みが必要になると説明する。ただし単純な制御系自体は「エージェントとは呼ばない」とも明言している。目的は、ホムンクルス(頭の中の小人)のような神秘的な説明に頼らず、行為・知覚・認知・目標の組織化として主体性を捉えることにある。

四つの段階と、学習偏重への疑問

現代の爬虫類をモデル種として、ヒトにつながる系統を整理する。爬虫類は単純な決定を行う基本的な制御系を持つ。哺乳類は実行機能の層を持ち、想像上の状態を表象して計画や仮説検証ができる。例として、枝に跳び移る前に失敗を想像して別の枝に回るリスが挙がる。大型類人猿はさらにメタ認知を持ち、情報が足りなければ探し、難問では選択肢から降りる戦略も取る。

この観点から、彼はAI研究が学習に偏りすぎていると見る。学習は主体が適応的な決定をするための下位プロセスであり、強化学習も主体的な性質は少しあるが学習中心だと不満を述べる。また生物学の生気論の例を引き、鍵は特別な物質ではなく新しい種類の組織化だと主張する。

共有された主体性とLLMへの問い

ヒトを大型類人猿から分けるのは、共同目標のもとで役割を分担し、互いの役割の基準を知り、守られなければ互いに正す「共有された主体性」だとする。共同注意から「同じ対象を別の視点で見る」という視点の概念が生まれ、無人島で育った子には得にくいだろうと語る。実験では、子どもは先に報酬を得ても相手が得るまで協力を続けるが、チンパンジーは報酬を得たら終わりだという。

ここからLLMへの批判が出る。ビデオ通話の音声を切られた子どもたちは新しいジェスチャーを作って慣習化するが、LLMは慣習を作らず談話も持たないと述べる。スタッグハント(二人で協力しないと獲物を狩れない協調問題)や、慣習を協調の装置と見るデヴィッド・ルイスの議論を引き、言語はこうした土台の上に成り立つと考える。文法も同様で、cleft構文(It was Fred that…)は相手の誤解を正す場面に由来する視点的な構造だと説明する。一方で、LLMが想像以上に健闘するのは文章内の論理的・談話的つながりのおかげだろうとも認める。言語の土台を飛ばせない保証はないとしつつ、主体的な機械を目指すなら協調や視点取得から作るボトムアップの研究も必要だと提案する。

文化、嘘、集団の暗黒面

文化は大統領や貨幣のような、合意によって存在する地位機能を生み、個人を制約すると同時に可能性も広げる。嘘が成り立つのも、人間が基本的に協力的で相手を信じるからだと述べる。協力は「われわれ」対「彼ら」の対立という暗黒面も持ち、現代の相互依存する世界に共有志向性が適応できるかが課題だとする。

市場や資本を一つのエージェントとみなす見方には慎重で、渋滞やインフレ、株価暴落のように個々の意図的行為から意図せず生じる創発現象と捉える。政策担当者は、個人の意思決定と結果の関係を理解してルールを設計すべきだという立場である。

まとめ

本回の要点は、主体性を知能の高さではなく「予測不能な状況で自ら制御し決定する組織化」として捉え直す点にある。エージェント型AIを開発する日本の読者にとっては、学習中心の設計に加え、実行機能・メタ認知・新しい目標の生成、さらに他者との協調や慣習形成といった観点が示唆になるだろう。トマセロ自身はAIが世界を支配するとは懸念せず、有用な道具として押し進めてよいとの立場だ。チューリングテストは十分条件に「近くもない」とし、身体性が重要かという問いにも結論は出していない。

文字起こし(英語・自動生成)

When you need a decision-making agent is when the environment is unpredictable. The living things are made of the same basic elements as non-living things. They're just organized in a different way. What makes humans different from other great apes is that we can form shared agents. Our shared agency and shared intentionality can adapt to this new world before we blow ourselves up. By the way, a Turing test is not even close to sufficient, not even sort of close. Hi, I'm Michael Tomasello So I've just read your book, The Evolution of Agency Let me just show that to the camera I must admit, my head is exploding Because I have so many things in my mind So many things that I want to discuss with you And I know we're probably not going to do it justice So I do recommend that folks read this book It's actually quite a short read And it's just so information dense But Professor Tomasello, would you mind just giving us a bit of an introduction? I mean, how did you come to write this book? I'm by training a developmental psychologist, but I've always been interested in evolution.

And my first job was at Emory University, and they had the York East Primate Center. And I went out and started watching the chimpanzees and said, my goodness, they're often like two-year-olds. In a lot of ways, they're like young kids, and in a lot of ways, they're different. And that was fascinating of both similarities and differences. So I ended up zeroing in on the question of what makes humans different from other primates, what makes their cognitive and social capacities different from others. So I ended up 20 years at Max Planck in Germany where we had our own grade 8 facility and did 20 years' worth of comparative studies. We kind of pioneered studies where you put children and grade apes in the same experimental situation and see what the subtle differences are. And the differences are often quite subtle, and you need experimental methods to see them. And then they had mandatory retirement in most of Europe for academics, and so I came back to my alma mater, Duke University, 10 years ago.

Wonderful. So I think the TLDR of this book is that agency is an extremely important lens to understand adaptive behavior, not only in humans, but just across a spectra of life. And the way you describe it, roughly speaking, is it's the ability of a living thing to deal with ambiguity and to handle novel situations. Can you put a bit more flavor on that? When something is evolutionarily very important and the environment is predictable, nature tends to build things in. So think breathing and swallowing, sexual behavior, whatever. These are things that are evolutionarily critical and the environment is more or less predictable. And then there doesn't need to be a decision-making agent When you need a decision-making agent is when the environment is unpredictable. And so what Mother Nature builds is a mechanism, is an architecture enabling the individual to assess the situation

and make a context-sensitive decision with flexible behavior. So people think, oh, gosh, you know, all situations are at least at some level predictable. So for me, the most unpredictable are other agents. So prey, predators, but also conspecific competitors, conspecific cooperators. So when you have two agents, they really can't predict one another. So in any case, the idea is it's for unpredictable situations. There's a mechanism to have somebody on the ground, so to speak, to make the decisions. because Mother Nature can't predict every possible contingency ahead of time. And I suppose we should talk about how to define agency. Now, this is a really vexed issue. We have many arguments in our community about it, and God knows we've spent a lot of time on the Stanford Encyclopedia of Philosophy looking at different definitions and so on. So when we interviewed Carl Friston, he was kind of talking about it in terms of

a similar kind of thing to where you're coming from, So some kind of a control loop, some kind of a boundary, some kind of active sense making. But roughly speaking, there seems to be a couple of dimensions. So there's the dimension of the will. So, you know, am I the cause of my own actions? Am I doing things for reasons? Am I am I pursuing goals or intentions? Am I acquiring goals? Maybe at the strongest, do I have free will? So I'm completely almost doing things appropriate to the situation, but not caused by it. And then there's this causal efficacy dimension, which is quite interesting. So the ability to control the future to kind of realize my goals. And you see the deflationary definition is very much like, oh, I've just got this outer boundary. And it's a thing that sort of senses and acts and does something. And then one step up, I have a target like a thermostat. And I'm trying to make the environment like something. And you see there's a kind of hierarchy to this agency. I mean, does that gel with how you think about it? Yes, in general. But I use a control system like a thermostat as a concrete model because I know a lot of the discussions of free will and all that kind of stuff, they get a little far away from psychology.

And the book on agency, I actually wrote it for animal behavior people, and the AI people have picked up on it. But a lot of them still kind of operate implicitly with a stimulus response, which is the classic, and you still hear them talking. They would say, oh, I'm not a behaviorist, but they still talk about stimuli and responses. No, you have an active organism with goals that they're actively pursuing, and when they aren't reaching their goals, they try something different. They have something that you didn't mention. They have executive function, executive processes, and metacognitive processes from animals, which I consider part of executive functioning. All of these are different depending on different animals. That's part of what the book is about. From mammals on, you have executive function. And so they are second-guessing themselves and inhibiting things that they were intending to do and deciding if they need more information.

And so agents are in control. That's the point. They're in control of what they're doing, of the causal effects they're having in the environment. I mean, obviously they're not totally free. I say in like the second paragraph, we're not talking about free will untethered from biology. We're talking about within the context of the organism's biological adaptations, there's a space. There's space in your, for example, pursuit of predators. There's space for making certain decisions. I mean, chasing prey. There's space for making certain decisions. So, yes, the way you described it is fine. But I just think that if we the reason I'm so attached to these control systems, cybernetics mechanisms is because it's a concrete manifestation of of agency in its simplest form and with its necessary elements. This episode is supported by CyberFund.

If you're building at the frontier of AI, they want to hear from you. CyberFund believes the future belongs to AI natives who want to achieve the impossible. And that's why they're introducing the Monastery for AI native founders. It's an environment of pure focus and rapid execution for founders operating at AI native speed. and they're offering teams $2 million each to participate. Apply now at cyber.fund. And one of the things I like to do with students is to just take some ordinary dumb mechanism and say, how would you make it smart? And my favorite example is a traffic light, which just works by a timer and it's not sensitive to the fact that there are no cars coming the other way, but you still have to stop. How would you make it smart? And, of course, there's only one solution. First of all, it has to have a way of perceiving the cars.

It has to have a goal about what should the result be that, for example, equal numbers of cars at each road to the intersection or something like that. So it has to have a goal. It has to have perception. It has to have a decision-making mechanism that given what I'm perceiving and given my goal, here's what I have to do to make the world conform to my goal. So you can have that kind of organization that is completely different from stimulus response reacting to prepotent stimuli. And even a thermostat or something captures the basics of that, not the entire thing. And in fact, the substance of my book is about different levels of this type of agency. yes and there's a beautiful bit in your book where you said you can have an ant colony and it can still have incredibly sophisticated behavior but the bright line between our kind of agency is that we actually are in control of what we're doing yes that's exactly right um i think i say right in the

beginning a lot of people a lot of ai people actually was in an ai symposium where they were talking about being smarter uh it's got to be good so there has to be selection for being smarter But actually, there are a lot of dumb creatures out there that do quite well evolutionarily because they're adapted for their environment. So it's really not about complexity, like building a spider web is quite complex, but it's about control and being able to control your flexible actions on the basis of your perception of the immediate situation. yeah I mean one thing that concerns me a little bit because um yeah so control is important and also you've said doing things for reasons are important so you sketched out a couple of examples so there's the thermostat which has a target temperature and it can kind of you know modulate the furnace and there was another example you gave of a robotic hoover system I think which which would kind of move around and it had a camera and I guess my one concern is that I don't know the extent to which you're kind of human chauvinistic about agency but it kind of feels

that, yes, that has this control feature that you're talking about, but it doesn't, I wouldn't say that it had agency. I mean, to me, something is missing there. Okay, I'm perfectly happy with that. For me, I was coming from a slightly different direction, which is that in the cognitive science community, AI is the subgroup of that, but in the cognitive science community in general, often when you talk about agency, they start worrying about a homunculus. you know you have some other person in your brain and they don't like homunculating so I'm trying to say look all I'm talking about is a kind of organization whose basics can be seen in a thermostat or an automatic Hoover machine and that's just the basic organization no I would not call them agents I would not but it's just that it's not about a miraculous homunculus or immaculous mental substance or anything the old philosophers used to think about it's about an organization between action and perception and cognition

and goals and desires but do you think the strongest form of agency could in principle be replicated in machines i mean just as a tangential thing i read an interesting book the other day called the irrational decision and this was a professor at berkeley and he spent his entire career doing control theory and he was kind of saying that we've gone a bit too far with over optimization so after the second world war you know we developed algorithms like the simplex algorithm and powerful computers and um immediately we tried to solve the optimization problem of what is the optimal diet you know so can we sort of minimize um the costs and and get the appropriate calories and get the right types of foods and whatnot and what we did was you know there's this whole kind of field of rationalism mathematical rationalism which is spearheaded by people like Steven Pinker and you know is there a similar thing here because the thought occurs it begs the question that well if if we are making decisions surely we could make a computer program that makes even better decisions than we do we could make a computer program which has more agency than we do

I mean do you subscribe to that uh I'm I have to say that um I equivocate on that to some degree um in the old days the idea of artificial intelligence was um one of the main criticisms was it's like a brain in a vat because the machine itself doesn't have a goal. It only, you know, and it doesn't perceive the world. And they were trying to just model just the computational parts, assuming perceptions and goals and actions and whatever. But now we're in a new world where we have self-driving cars and drones and things like that, which are doing something very, very similar. And I would say structurally very similar. And so obviously the question then becomes what's missing. And I'm not sure. I actually think that one of the keys is, which I mentioned before, is this executive processes where you know what you're doing.

And I think the machines have not really embodied that in the right way. It's not just a separate little skill. It's a whole other level of organization. I call it a tier of organization. and the executive tier observes perceives pays attention to if you will what's going on at the attention and action level and monitors it and can proactively avoid mistakes by anticipating them by planning and all of that and you need an executive tier for things like planning hypothesis testing it requires imaginative representations as i call them that you have to imagine the way a world could be. I think animals like reptiles, snakes and turtles and things, I think they don't have those. I don't think they can imaginatively imagine what might happen and plan a strategy and stuff like that. So now you're getting a little bit more, even with the animals, I call that

the beginning of consciousness where I'm conscious of what I'm doing. I'm perceiving and attending to what I'm doing. And then another tier that I think comes with great apes and humans, humans being a type of great ape, is metacognitive, which is another form of executive functioning, where I reflect on my executive planning and all of that. And I say, is that a good plan? Could I have a different plan? Do I need more information to have a good plan? Am I putting my resources in the right place or am I wasting effort? So these kind of metacognitive things is another tier. It's not a separate little skill over off to the side. It's another tier of functioning. So if you could have, you know, drones or self-driving cars that have these executive levels of functioning, perceive the world, have goals. If the goals are built in by humans, that doesn't bother me totally because, of course, most human goals are pretty much given

by Mother Nature, at least the most basic ones of, you know, eating and procreating and stuff. They're already built in. You'd really want them to be able to create novel goals in situations where they don't know what to do to think, well, okay, here's a novel goal I've never had before that might help get me to this novel situation. So I just think, I guess I'm not afraid that AI is going to take over the world. I've been here since the 70s, and they've been claiming that it's going to take over the world since the 70s. I think of it as a really important tool, and I don't see any reason why we shouldn't just keep pushing the envelope and trying to make them more agentive, and that's my suggestion, is look at natural agents and how they work and use that as a model and see if that will help us get further in that direction. I suppose what's interesting about AI agency is it's something that wasn't – We didn explicitly train them to be agentic yet it seems to be some kind of an emergent thing And actually I think the first quotation in your book was from darwin and um he was saying in the distant future psychology will be based on new future um mental powers that are acquired by gradation and isn it weird because i don know

whether i'm missing maybe i'm misinterpreting you but you seem to be saying that there are there's almost like geological strata you see these kind of cognitive capabilities that one builds on top of the other and certainly with ai and maybe in other places as well what we see when when we kind of zoom out and we're being externalists rather than internalists and we just see this huge collection of kind of cognitive atoms that weren't evolved towards a particular capability but the capability just suddenly appeared in that milieu i mean could that be something that's happening in I'm not technically sophisticated enough to have an informed opinion about that, but that's what it would need to do. It would need to be able to, as I say, formulate new goals, formulate new strategies. In evolution, what happens all the time is what they call pre-adaptation. So the textbook example is the opposable thumb was for climbing, and then we come down onto the savanna, and it's useful for picking up things and whatever.

So you have something adapted for one thing, and then now you're in a new context, and the organism, the agent, figures out creatively how to apply it in a new way. Yeah, I know that a lot of AI mechanisms do some surprising things, but again, I'm just not technically knowledgeable enough to know exactly what's possible there. I will say that I do think that a lot of the AI models focus almost exclusively on learning. Everything's learning, learning, learning, learning. But learning only exists to help the agent make adaptive decisions. So learning is a sub-process. It's important. Obviously, it's really important. But you could, in theory, have an organism like a thermostat that just behaves in a situation without learning. That turns out that pretty much all organisms we know save their experience from when they have some, you know, they do something, they save it for future reference.

But learning is not the be-all and end-all. And that's one of my things like reinforcement learning, that's one of my sort of gripes about it is it does have a little agentic quality, but it's mainly focused on learning. And I don't know if you're planning to come to this topic at some point, but large language models have a different problem. And the large language models have the problem that they're starting at the end point. They're starting at the pinnacle of human achievement and thinking we can build artificial general intelligence from that. And it's theoretically possible. I wouldn't rule it out, I guess. But for me, language is built on all of these other more basic cognitive capacities. And can you just skip them and skip to the final? I'm not so sure. But, yeah, so I guess the one thing I would, there's an analogy that I have in the book about back at the beginning of the 20th century, there was an argument in biology about vitalism.

And vitalism, you know, what distinguishes living things from non-living things? And there was a school of thought that there was some substance, some special elan vital that was a special substance that made living things different. and through painstaking research we find that it's the living things are made of the same basic elements as non-living things they're just organized in a different way and so i'm kind of drawing a general analogy to um uh say here and that's why i focus on the control systems analogy is the key is a new kind of organization um and so uh and and again the book lays out all the details for uh this sequence of growing complexity to humans um to try to um say some of the kind of explicate specify uh some of the um details of that story yeah and that really stuck with me in your book when you were saying that you don't actually need to have learning per se like a lot of behavioral adaptation can be explained just just through this agentic lens

that was very fascinating but i'm kind of interested um i mean there's so many directions to go from here like for example um individual agency is something that is valorized particularly in in american culture and um you know you gave the example of richard dawkins who was talking about the selfish gene and what he was basically saying is that there's some kind of a super agent and actually post-modernists make a similar argument there are machinations of power and they parasitize us and and they kind of there was actually a really funny example i was watching an interview with Noam Chomsky and Andrew Maher. He's a British journalist. How can you know that I'm self-censory? I don't say self-censory. I'm sure you believe everything you're saying. But what I'm saying is if you believe something different, you wouldn't be sitting where you're sitting. Do you see what I'm saying? So, you know, where do these goals come from? So, there have been two kinds of determinism. If you want to think classically, historically, Darwin introduced this idea that you're naturally selected that sort of Nature is in control, at least some people can construe it like that.

And then Karl Marx is the one who said the culture. All right, no, no, no. You're born into a social class. You have an ideology. There are issues of power. And so he was saying that it was the culture. And I believe that the notion of agency is that we should, of individual agency, is we should think of those as powerful forces that influence the way we decide things, but they don't determine it. And I mean, like Tubian Cosmetes, classic evolutionary psychologist, they always give the example of like, you know, we're attracted to sugary foods, and they give an example of why we're attracted to sweet foods and stuff. But it's not absolutely determined that I'm going to eat ice cream tonight. I'm on a diet, and I think I don't want to have the ice cream. So I'm attracted to it. I don't control that attraction. That's a biological given. Or I'm attracted to some young, cute woman, but I'm married, and so I'm not going to do that.

And we have competing, sometimes we have competing interests. So in the more social realm, I made a promise to somebody, but now my daughter was in a car accident. So I break my promise. I have to decide whether to keep my promise or go help my daughter. So I just don't, one of the main arguments in the book is against this tendency to deterministically. Think in terms of influences, factors that are, nature puts her finger on the scales in various ways of your decision making. And your sociology puts its finger on the scale in various ways as well. But you, the agent, have to resolve moral dilemmas where, as I say, you made a promise, but now something more important comes up or you'd like to eat the sugary food, but you're on a diet. We need an agent to adjudicate between those different competing influences. Yes, yes. And I suppose it's interesting that you said we should think more deterministically, but still in the, you know, in the realm of ambiguity and chaos that you can still end up doing very, very different things in different situations.

But it's really cool that you brought up Tubi and Cosmedes because I took a note that you quoted that they said that the brain is not a general learning or computational device. So you were talking about, you know, it being very specialized and akin to a Swiss Army knife. Could you explain that? So they just focused on the fact that we have a number of things where there's a very strong biological signal. They call them innate modules in their classic thing. They've softened up a little bit. And they're independent of one another. That's why they call them modules. So there's a computational module for mate selection. And for males, males are attracted by signs of youth and fertility in females, and females are attracted to this in males where. But it's a little module. It has nothing to do with foraging for food. Foraging for food is a completely different problem, and it has different heuristics.

So you're looking for the most amount and the most nutrition and whatever it is. And so we have these little modules. And I don't have any real objection to the existence of those. I think they do exist, but they have a hypothesis of what they call massive modularity, that everything is modular. And that's where I differ is because when you have an agent, the agent has to use general processes that cut across all domains. Things like perception, attention, learning, memory. All these things are not specific. They're not modules. They're not specific to specific domains. They are used by the general decision-making process in the moment. And as I say, what do you do if you have two competing modules at the same moment? You have an attractive sexual partner and attractive food, and both modules are going.

What am I going to do? So I don't object to the existence of little special-purpose modules. and the animal world is full of them, where you have little special purpose, you know, squirrels cashing nuts and et cetera, et cetera. There are all kinds of special purpose modules, so I'm happy with that. But I think with creatures, at least from mammals on in the human line, I'll leave the birds out of it. The birds are off on a different track and they have a different explanation. But along the line to humans, at least the mammals are clearly agent of making decisions influenced by emotions. What are emotions? In an evolutionary sense, the emotions are nature putting its finger on the scale of your decision making. So nature makes you incredibly afraid of snakes. And you see a snake and you drop everything else. You drop all your other goals and leave. but I would say if somebody had a gun to your head

and say don't move you might let the snake crawl over your foot who knows so you have these emotions that attract you to other people make you afraid of certain things make you happy in certain situations and make you angry when certain things happen and that is nature's way of influencing your decision, but it's still you making the decision. Yeah, that's so fascinating. I interviewed Mark Solms. I don't know if you're familiar with him, but he said that affect was the felt sense of unpredictability. So feeling enables the system to navigate unpredicted environments. In other words, to manage uncertainty. Okay, that's good. Yeah, that's quite an interesting theory. I wanted to talk about the distinction between agency and intelligence now in your book whenever you wrote the word intelligence you put it in air quotes and i'm just trying to understand because a lot of folks certainly in the ai world

they talk about intelligence in similar terms so it's it's basically about um the adaptation to deal with novelty and so there seems to be a relationship between the two but but you were a little bit scathing about it kind of saying that no it's actually far more specialized and it's misunderstood and so on how do you distinguish them uh i don't mean to get philosophical on you here but the uh the word intelligence uh the concept the concept of intelligence was created by um frant uh by alfred bernet bernet in a french laboratory where the french government gave him the task of of figuring out who's ready to go to university and who should go off in trade school. They do this in high school level. And so he's basically studying school type intelligence. Verbal, abstract thinking. And then people started giving IQ tests and they reified IQ into intelligence. So I have a kind of a knee-jerk reaction

to the notion of general intelligence. And the reason I put it in square codes is because the reason what people, when they use the word intelligence, They often think about general intelligence. But, you know, squirrels can remember more hiding locations than we can. And a lot of birds can probably discriminate more different sounds than we can. And, you know, chimps use tools better than children do anyway when children are outstripping them and other things. So there is a lot of specialization. And so I'm just not, we acquire knowledge about the world. We apply it in specialized ways, et cetera. I just think it's to aim that verbal intelligence and of a type that's tied to school and that kind of thing. And I think if you're thinking more in terms of the natural world and other animals and stuff, general intelligence is not the best construct. Yes, I suppose that's true. I mean, I guess I think about it in terms of it's the ability to crystallize skills.

So, for example, you learn to drive, and I don't know how to drive, and then over time I crystallize it into a skill, and now I can do it automatically. The only reason I bring this up is could there be a similar charge of agency here that we're also reafflying agency and we're kind of reducing it in an oversimplified way? I guess that's possible. but again, that's why I have this sort of control system thermostat kind of model because then it gets real. You can touch it. It's a physical mechanism. And it's that organization. And I think that's the starting point. And then I don't think there's an issue of reification. Whereas intelligence, there's nothing like that that you can point to. and this, I'm fine about being adaptable and flexible and all that, but I just prefer, because of the baggage that the term intelligence carries of the type that I was just talking about, I prefer talking about cognitive capacities,

knowledge gained from experiencing the world and learning. So knowledge, cognition, planning, foresight, hypothesis testing, all the language of cognitive science, I prefer that to the language of intelligence just because of its historical associations. And can we talk through the core taxonomy you have? So you're speaking about the evolution of agency and you have four clear levels. Can you explain that? So I'm only talking about the lines to humans. Okay, again, the birds are off to the side. And I think with the first land vertebrates, which is where we have good data, And let me say the method is a method used in comparative biology in general of having model species. That is, we don't know who these first land vertebrates were, but we have current day reptiles, some of whom are posited by biologists to be very similar to the first reptiles.

And we can do experiments with them. So it's not like we can't go back at all. Yes, we have to be careful. The model species are not exactly like the original, but we can do that. And so experiments with reptiles used as models for the first land vertebrates, they are organized like a control system. They are making decisions, but only very simple decisions. And so I think of them as basic control systems. Then with mammals you get an executive layer of control and everything is focused on the decision process And to make the kind of decisions that they make a squirrel in a tree you know starts you can just see them. Should I jump to that branch or not? I saw one while I was writing the book. I think I might have given an example. They look like they're going to jump, and then he crawls down and crawls up the other one because he can imagine himself failing. So they are, in their planning and hypothesis testing, they have to have imaginative, what I call imaginative representations,

representing states of the world that don't currently exist, and they have to have executive level to do the planning and the hypothesis generation and so forth. Their learning is Bayesian, because now you can have a hypothesis generation and probabilities and whatnot. So that's the second level is, I guess I call the first level sort of basic agency and then the goal-directed agency and then this level I call intentional agency because when you form intentions and plans, you can abort them before you actually enact them. I think Daniel Dennett said at one time that with intelligence or something like that comes that my hypothesis dies instead of myself, right? So I see, whoops, that's not going to work. And so I saved myself the problem of failure. So I think mammals are acting in that way. You can look at your pet dog acting like that all the time.

You can look at squirrels. If you want to see some hilarious examples, look on the YouTube under squirrels outsmarting humans with bird feeders in their backyard. They try to protect their bird feeders. And clearly these are novel, but the squirrels are inventing novel strategies for that. And some people have built obstacle courses for squirrels based on this kind of thing. So they're flexible, creating novel behavior, hypothesis testing, planning, all of that stuff needs an executive tier of organization. and then the third level of individual agency and the highest level of individual agency is metacognitive or rational agency which I see for sure in great apes we don't have enough data on other primates but for sure in great apes and they do things like they make a decision but the result doesn't happen right away and then they get more information

and they change their they look for more information so they can make a better decision. And they have various strategies for when the problem is very difficult and they know they're going to make a mistake. They can opt out and do something similar, more high probability. So this metacognitive functioning is another level of flexibility. And in each case, I propose, like reptiles are mainly solitary, So most of their unpredictability comes from their prey. They mostly prey on insects and other little creatures. Mammals all of a sudden have to compete with group mates because they're group living. And so they want to get food. They have to compete with. So they have to make a faster, better decision than their group mates or they're going to die. And grade eights have a lot of contest competition where they have to outsmart the other one. Most mammals mainly do, well, they do a lot of scramble competition where they just get their first.

Whoever gets their first gets it. But then the gradients have a lot of contest competition where they're having to outsmart one another. And then you were saying four levels. It's slightly off to the side, but the humans, and this is what most of my work for most of my career has focused on. And the agency stuff is more recent. But what I focus on for most of my career is that what makes humans different from other great apes is that we can form shared agents. And so I know that there are various kinds of AI collective agents. What's the word they use for? Oh, sort of like AI societies of agents or something like that. But they mostly react to one another. And what a shared agency is, is we put our heads together. We have a shared goal. And you have your role and I have my role. And we both know that. And the role has certain standards. So if we're going to be successful, you have to do your role this way.

I have to do my role this way. And we both know that. And if you don't do your role the right way, I say, hey, what are you doing? Come on. Your bad behavior over there is going to make us fail. And so there's a kind of a self-regulatory mechanism that in this case is social. It's a shared commitment that we have. There's a postdoc in our department, Xiaojie Cheng, who will be coming to England soon, who is trying to model this kind of shared agency with a joint commitment. And what that does is, cognitively, is we have joint attention. So we're trying to pursue some goal together cooperatively as a joint agent. And if we're going to do that effectively, we have to both jointly attend to possible obstacles or opportunities as we're going along. And we have to jointly attend. We have to both know we see them so that we can have a unified strategy as we go.

And so that joint attention, which I've been studying for a very long time in young children, creates the notion of perspective. That we're both seeing the same thing, but I see it this way and you see it that way. and this is one of the most powerful parts of uniquely human cognition is the idea that the same object can be viewed from different angles the same object can be a dog or an animal or a pet we take that for granted well of course it can we say it just depends on how you look at it well guess what other creatures don't have this it just depends on how you look at it they see it as whatever they see it as but they can't just flexibly switch perspectives and I come from a long tradition of people like George Herbert Mead and the pragmatist, Donald Davidson's philosopher, people who say that this is all social, that this is the notion of perspective is only, if you were a child raised on a desert island, which is one of my favorite examples I've been using for over 20 years,

wouldn't get the notion of perspective because they'd be going around looking at the world but there wouldn't be a contrasting way of looking at it that is right there with them at the same time. So humans have a unique form of sociality. At its basic is this joint agency. But then ultimately what really, you know, the booster rocket is we have these collective agencies known as cultures. And cultures give you, you're born into a culture, and it gives you conventions like linguistic conventions and norms about ways you ought to behave or the way you ought to think about things in education. That's what you're taught when you were saying that Noam Chomsky was saying, you know, it starts in school. They're telling you how to think. That's what they're teaching you in school, how you ought to think. This is a better way of thinking. We're telling you about it. And so we're born into cultures, which are kind of collective agencies with conventions, norms, and institutions. And they are both constraining, as Chomsky was focusing on, but they're also empowering.

because when you get a language and you get social norms and you get conventions that you yourself didn't have to invent, those are given to you by your culture, assuming you have the ability to learn them, then you can write poetry with the language. You can do new things with it. So this final type of agency is a little bit off to the side. It's not just another step up. It's a new version, is the shared agency. And I personally believe it's unique to humans. Yes, there are other creatures that cooperate with one another, but we've done experiments, many experiments with chimp pansies compared to human children. And you can see that children share the spoils of a collaboration differently. The chimps don't correct one another when one of them doesn't play their role the right way. When kids are collaborating and one of them gets their reward first, they keep collaborating until the other one gets theirs. The chimp, as soon as they get their reward, they're done. So there are demonstrable empirical observations that one can make that I would argue show that the human shared agency,

including both collaboration among individuals and culture among humans as members of cultural groups, is what makes humans different. And if I can say, this is one of my issues with the large language models, which I use every day, incredible tool, you know. Humans not only use language, they created languages. They created them. So they create, we have studies with children showing that when we don't let them talk to one another, we have them going through video monitors. This was back at the time where it was Skype and not Zoom. and the kids knew about talking to their grandparents on Skype, and we have them, and then we say, oh, the sound went off, so you've got to gesture to one another. And they make up novel gestures, and they conventionalize them. The ones using one another start using the same one, and they start adapting one another. So we create linguistic conventions. There's 6,000 different languages in the world.

We create linguistic conventions, and maybe you don't need to have that capacity, but large language models, to my knowledge, don't create linguistic conventions. They don't have discourse around topics and so forth. So I'm waiting for that. But I think this notion of shared agency and shared intentionality are the sort of infrastructure of the creation and use of linguistic conventions. And as I say, it's possible we can skip that step. But I would say I think it's more likely that we're going to get to more useful and more human-like machines if we start with those steps and have language be the sort of ultimate outcome or the sort of pinnacle. Yeah, that was absolutely fascinating. And first of all, Leonardo da Vinci said that from constraints comes freedom, which I thought was quite nice. But yeah, I mean, just to backtrack a little bit,

so you were talking about this cognitive evolution, and a few things you said really jumped out at me. So, you know, like the ability to imagine and simulate and delete strategies and do epistemic foraging, you know, all of this is fascinating. But certainly the thing that interests me the most, though, is the final step you spoke about when we talk about having shared agencies and, you know, where did the gods come from and all of that kind of stuff. now um isn't it interesting that you spoke about ai and at the moment they don't do this kind of um constructive um you know like coming up with conventions and it's a linguistic agency basically is the technical term for it but maybe they will in the future because you know in the world of evolution we have this kind of um physical evolution and we have you know sort of ontogenic accumulation of of of um learning and then we have this um cultural accumulation of learning and the speed goes up as you as you kind of go up the hierarchy the speed goes up and maybe ai agents will be able to do that when they can talk with each other and create conventions but do you think that that's a reasonable way to think about it do you think ai could be kind of um socially

constructive in the same way we are um that's why i say the uh postdoc xia jicheng is um working on something like that and so in my theoretical books before this one on agency which were on shared agency, that evolutionarily, probably the originating evolutionary ecological challenge was coordination problems. I don't know if you're familiar with game theory, but Thomas Schelling coordination problems, because coordination problems require a certain kind of logic. It's not just being smart, okay? It's taking the perspective of someone else, taking your perspective while you're taking their perspective. and the sag hunt is a classic coordination problem where we each have our own sort of low-level resources that we're foraging and then something much bigger and better comes along but I don't know if you're going to go for it I only want to go we have it takes both of us to capture it

and so I only want to go if you go because otherwise it's futile and you only want to go if I go and so it's like I only want to go if you go you only want to go if I go it's like Abbott and Costello walking through the door after you, no, after you, no, after you so so I do think that evolutionarily the originating parts are not linguistic they are about putting your mind together to solve these coordination problems where we each have to do our part in common knowledge with the other one and then David Lewis, the philosopher David Lewis has a book called Convention and his argument is that convention is a coordination device. That a convention, it was created, linguistic conventions were created by humans to help coordinate their activities together. So again, I'm not, I would never make an in-principle argument that we could skip that kind of thinking and go straight to language. Maybe we can.

But I would say a reasonable, I would think somebody ought to at the same time that other people are just working on mature adult language some other people ought to be working from the bottom up and saying let's see what it's like to put your head together and what it's like to create a convention as a coordination device and then we'll have that infrastructure on which we can start putting the linguistic skills yeah and can we explore a little bit more on this perspectivalism so what you've described here is a picture where essentially complex cognitive capabilities can be factorized in a collective. And we've spoken about agency, but I also love this idea that even just the epistemic, the representations, the knowledge part require factorization, because I have an intuition that it's quite path dependent. So we have different perspectives of the elephants, and it seems like there's something strictly better about forming a collective of different perspectives than having one sort of very intelligent agent doing it all on their own um the philosopher donald davidson was basically

he says something like uh you know this kind of engagement with another human being is what raises the possibility that you could be wrong because uh it happens all the time in science it happens all the time i think i'm right and somebody else shows me something else and i go whoops all right and so uh and if it's just you yourself yes you can learn from your mistakes but But learning from mistakes is not quite the same as seeing things in a different way as you can sort of take the perspective of another person. So, yes, I think that's underappreciated because if I talked long ago to artificial intelligence people, before the modern wave of stuff, they'd say, oh, perspectives like that, that's simple. We build one node for animal, one node for pet, one node for dog, and it's no problem. But in the natural world, you have a single perception.

You have this thing in front of me, and I can see it as a dog or an animal or a pet. And I would say that it's the social. In language, why do I call it? I can call it a dog, an animal, a pet, or I can call it it or she. When do I call it those things? it's because I'm adapting for my listener and a context where in certain contexts the only thing that's important is that animals are not allowed in here. And in another case, it's I want a pet. What kind should I get You should get a dog And another kind it like should I have a companion with me or not Should I get a pet So depending on your common ground and your perspectives of the person you talking to we have to construe things in a different way. And I think that's an underappreciated aspect of linguistic communication is that it is, in its core, perspectival. When people talk about grammar, computationalists talk about grammar, and I know grammar is a topic that bores most people so I'll be brief

but when you talk about grammar the computationists want to talk about combinatorial possibilities and combining and all that but if you look at the fanciest grammatical constructions take what in English is called a cleft construction it was Fred that broke the window think about that it's a very strange instruction it was Fred that broke the window why didn't I just say Fred broke the window The reason is because you think somebody else broke the window, and I'm trying to correct you. So I say, it was Fred. It was not Mary. It was Fred that broke the window. So the origin of that construction is in communicative situations where you have a mistake. I believe you have a mistaken impression, and I'm trying to correct it. So it's a perspectival problem. Yeah, that's fascinating. And that's just, that dimension of grammar is not appreciated. So it's not just words as conventions. Grammatical constructions are conventions that we have created to view things a certain way.

The dog is chasing the cat. The cat is being chased by the dog, the passive construction. Actives and passives come from the perspective of the dog or from the perspective of the cat. We find that trivially easy. but again, it's all about taking the perspective of the other person so all I'm trying to emphasize here is that language as we use it depends on these more fundamental abilities and I guess one of the surprising things about the harsh language models is they seem to do that a lot better than I would have thought they possibly could and there's something in the textual material the connection between sentences is the logical connections, the changes of perspective within the text. And I actually asked in a recent AI workshop kind of thing, I asked, what if you took all of the stuff, everything on the Internet, and considered each sentence individually, and you randomized the order,

so you no longer have any texts with coherence, no causal or logical or discourse coherence, but you just have a group of random sentences. What would the LLM look like if it used that material? And, of course, nobody knows, but the consensus was it would be pretty bad, that you need those. One of the reasons they were able to do all these intelligent things is the inherent logic in a discourse, which goes from one sentence to the next one that follows it, et cetera. and so um uh so all of that is is is language building on this ability to uh have a common perspective with others and have different perspectives ironically on that one this is something that chomsky says quite a lot that um the reasons that the reason he doesn't like llms is that they can learn impossible languages so so you could just randomize all of the words and the llms would learn it just fine obviously it would just be um garbage to us but but one thing

LLNs can't do is, you know, a really good writer, they use agentic storytelling and they write in this perspectival way that you just articulated. So, you know, in terms of me, my experience, my goals, conflicts, things that stop me from getting to my goals and so on, LLNs don't really do that. Yeah, I think that's right. And I think, again, that's the infrastructure. I've used the word infrastructure. I have a book on the evolution of language and I use that as the infrastructure. And, for example, I don't know whether this is fair, LLMs or not, but what children learn to do before they learn to talk is they learn to use the pointing gesture. So when I point for you to something, I mean, I have practically a whole book on the pointing gesture. So I point over there, and the way I just did it, you would say, huh, what? But in the right context, my point can mean anything. There's the thing we need. There's your sister.

The library is open. I mean, my pointing finger can mean anything because we are in a shared context where we both know what the other one is thinking and we can take the other one's perspective. And basically the pointing gesture, one way to gloss it is, look over there and you'll know what I mean. Well, how do you know what I mean? Well, because we are in some kind of communal thinking about things and taking one another's perspective. and as I'm saying it's not exactly fair to say an LLM can't interpret a pointing gesture but I believe that that's part of the infrastructure leading to language is the ability to sort of know what somebody means just by the fact that they're pointing something out to you I hold something up and show it to you for a reason and again maybe that infrastructure can somehow be skipped and it certainly can be skipped if what you want is a useful tool like LLMs are right now. But if you want them to be agentic, then I think you're going to need that infrastructure.

And the methodological prescription then is try to build something that starts from the bottom up, that starts from animals coordinating and taking perspectives and maybe communicating in some simpler ways before they build conventions. Fascinating stuff. Now, we were speaking before about this cognitive evolution. And one way of couching it is that it's about increasing virtualization away from the physical so you know at the beginning we can we can imagine and simulate and then um at some point you're saying we actually have this kind of social agency formation where we can basically coordinate and we can come up with goals and the more you virtualize because if you kind of think about it as increasingly kind of propagating storing information because culture is this magical hard drive in the sky so i could die tomorrow and whatever organized you know let's say my podcast might survive after i die because someone else would take it over but are there any pitfalls or degeneracies with this kind of virtualization so i know you've done a lot of research around things like collusion and bad

things that happen in in these collective um agencies um i i just before i get to the bad thing i just want to when you say there's a virtualization there's another element to it Have you read Harari's book on sapiens? Oh, yeah, many years ago, yes. Yes. So what he wants to point about in the virtualization is he says we live in an imaginative world where he's talking about things like dollars or pounds, presidents, prime ministers. Those are all things that only exist because we agree they exist. And if all of a sudden, you know, World War II happens, or I went to East Germany, and East German marks were money, and then one day they weren't money anymore. They were pieces of paper. So this virtualization in culture comes from us creating what John Searle, the philosopher, calls status functions. That is, we make a person into a president. We make a piece of paper into money. And the most powerful forces on the world, Harari argues, is like corporations.

You know, what is the Apple Corporation and what is Meta? These are what he calls fictions. They're made up. Corporations are fictional individuals that are legally individuals, but, you know, obviously not in a direct sense. So this virtualization is one of the things that culture does best and most powerfully is it creates new entities. Searle is quite strong on this. It is a fact that, unfortunate fact, that Donald Trump is president of the United States, right? That's a fact about the world. Now, it's a human relative fact. That is, it only exists because humans have got this concept, created this concept of president and so forth. But it's still a fact about the world. And the corporations may be fictions, but they're the most powerful things happening on the earth today. Okay, so now your question is, can this fictionalization, can this sort of, you know, have the bad side to it?

And I think George Orwell, my daughter was just reading George Orwell for her high school, and I was trying to remember parts of it. And Donald Trump is a master at creating this alternative reality by playing on certain tricks, repeating things often enough. He always says, you know, they say I'm hearing that, which cues into the natural thing is that we want to know that everybody else is thinking about things a certain way, and then we think, oh, everybody in our culture is thinking a certain way. That's probably a good way to think. So he tries to make it look like everybody thinks like this, and again, repeating it over and over again is the same thing. And so there are these sort of ways of kind of co-opting the natural system. system. I mean, lying. Lying only works because we're cooperative. Chimpanzees don't lie to one another because it would never occur to them that you would tell the truth if it didn't benefit. So, the point of lying, I mean, the reason lying works is that we're all gullible. We all

expect everybody to be cooperative unless they have some ulterior motive. If I tell you it's raining outside, it would never occur to you that I'm lying about that. I don't have any ulterior motive, nothing. So, lying works because of our general cooperative nature. And these things that politicians pull off work because we generally expect people to be cooperative. And, yes, it can be used for nefarious ends. I mean, a lot of times people say to me about, you know, you're talking about how humans are so cooperative, forming shared and collective agencies, but what about all the strife in the world? Well, almost all of it is they are attacking us. It's we versus them. And so the human version of the collective intentionality, collective agency and intentionality is our group. And now in the modern world, we've got to find a way to cope with the fact that all the different groups are interdependent with one another and live in the same place and everything like that.

But that's part of the dark side of cooperation is that our evolutionary history is about people that we live in a group with people that look like us, talk like us, eat the same food, dress the same way. That's our adaptation is that's the we that we live in. And the other we's over there on the other side of the river, you know, they look funny, they talk funny, they eat disgusting things. It's not that I hate them. It's that I'm just not sure about them. and so but in the modern world modern world meaning since agriculture and cities and and lots of migration and stuff we live in a different world and that's what we hopefully our shared agency and shared intentionality can adapt to this new world before we blow ourselves up and to what extent are these collective agencies a convergent or natural phenomenon a lot of people We'll talk about the market as if it is a sort of super organism. Yeah, I'm not a big fan of talking about inanimate things as agents. I mean, you could talk about capital as, you know, capital tends to flow toward this or that.

But I would say it's the capitalists who make the decisions about where it flows and stuff. And there are these sociological level phenomena like the invisible hand and also collective action problems are of this same type. Collective action problems, nobody intends the bad outcome, but everybody has intentions. Everybody's trying to get home as fast as they can, and it creates a traffic jam. Nobody intended the traffic jam. It's an emergent phenomenon, but it emerges from the intentional actions of individuals. And the invisible hand is supposedly, you know, that prices get fixed at their maximal level because individual agents are making decisions to pay the lowest price for the best value, etc. and the sellers are doing the opposite. So those emergent phenomena, sometimes called phenomena of the third kind, are kind of where sociology begins. These are forces that are not due to individual agents making decisions, but they are things that emerge. Inflation. Nobody intends inflation, but it's a product of people's intentional decision.

Nobody intends a stock market crash, but it's an outcome of people's intentional decision-making. So these kind of negative things are the things that, you know, policymakers and politicians and stuff need to be aware of and need to be aware of the connection between individual decision making and the outcome. So stock market crashes, for example, one could have policies around risk taking and so forth that would try to mitigate that. You know, and if you had, you know, stricter traffic laws, you might not get, I don't know what they would be. But they're the kinds of things that politicians and policymakers need to see these patterns and see the connection to individual decision making and make some kinds of rules or laws that regulate individual decision making in a way that these unfortunate things don't happen. Yeah, I think about this a lot because you were saying that in the market, it's supervenient on. It's like a super organism, supervenient on rational, intentional agents. and I mean personally my intuition is the best lens of analysis is to think of this thing as being

an agent rather than a collection of individual rational agents and yeah I mean what's your what's your argument to to sort of go one level down rather than one level up um you were saying it's a useful way to talk about it and that's fine okay if you're saying it is an agent I start to get a little nervous. But yes, I think it's fine if you're sort of aware that, but does capital have a goal? Does capital make adjustments if it's not reaching its goal? I'm not so sure. Maybe this is just I'm a psychologist and that's the sociological level of analysis and it's different for me. Yeah, it's a useful way to talk, absolutely, fine. but I'm sure I like really thinking of it you know in reality as an agent wonderful and final question is it possible for a system to act as if it has agency but not actually have agency

that's Daniel Dennett what he calls the intentional stance yes yeah yeah boy I don't know now we're getting metaphysical I don't know I mean, I would say if it walks like a duck and it quacks like a duck, it's a duck. And so I think if we built something that really had all these qualities that – and by the way, a Turing test is not even close to sufficient, not even sort of close. But if we had a machine that really could mimic all these kinds of things, then, yeah, I guess I would have to feel like it was some kind of an agent. You could say, well, it's not a human agent. I don't know. I mean, I saw a talk by the philosopher Ned Block, and I think the title was, Does Meat Matter? Meat meaning, you know, does the flesh, the brain, the hormones, and all that, does that really matter? Because they're never going to have the, well, at least the way we're going now,

they're not going to have emotions and hormones and neurons of the type we have. And does it matter? and I'll just say there was no consensus in the room and I don't have any brilliant insights into that either. Amazing. Professor Michael Thomas-Allowe, it's been such an honor and a pleasure having you on. Thank you so much. Thank you, Tim.

番組の概要欄(原文)

<p><br></p><p>Michael Tomasello is a developmental psychologist who spent twenty years at the Max Planck Institute in Germany putting children and great apes into the same experiments, and is now back at Duke University. His book The Evolution of Agency asks what it takes for a living thing, or a machine, to act for its own reasons.</p><p><br></p><p>SPONSOR:</p><p>---</p><p>Cyber Fund built the Monastery to help founders ship products that were impossible a year ago.</p><p>Apply now: https://cyber.fund</p><p>---</p><p><br></p><p>His answer starts with unpredictability. Where the environment is predictable, nature builds the behaviour in; where it isn&#39;t, it builds a decision maker. Tim and Tomasello test that on a dumb traffic light, a thermostat and a robot vacuum, then follow the line to humans: reptiles running simple control loops, a squirrel that imagines itself falling and takes another branch, and great apes that go looking for more information before they commit.</p><p><br></p><p>The second half is about shared agency. Humans form joint goals, attend to the same things together, and learn that one object can be seen from different perspectives. Children who lose the sound on a video call invent gestures and turn them into conventions. Large language models, Tomasello argues, started at the endpoint and skipped that infrastructure. On the way: the stag hunt, why lying only works among cooperators, whether capital is an agent, why a Turing test isn&#39;t close to sufficient, and Dennett&#39;s intentional stance.</p><p><br></p><p>TABLE OF CONTENTS:</p><p>0:00 Cold open: agency begins with unpredictability</p><p>0:29 Michael Tomasello and The Evolution of Agency</p><p>0:59 From chimpanzees to two-year-olds</p><p>2:05 Agents exist because the world is unpredictable</p><p>3:56 Defining agency: control loops, will and causal power</p><p>7:19 Sponsor: Cyber Fund</p><p>7:55 How would you make a traffic light smart?</p><p>9:57 Thermostats, robot vacuums and what is missing</p><p>11:32 Could a machine have more agency than we do?</p><p>16:26 Darwin, emergence and the limits of learning</p><p>21:04 Selfish genes, Chomsky and where goals come from</p><p>24:17 Modules, emotions and nature&#39;s finger on the scale</p><p>28:25 Why Tomasello avoids the word intelligence</p><p>32:06 Four levels of agency: reptiles, squirrels, great apes</p><p>37:04 Shared agency, joint attention and perspective</p><p>42:25 Why language models do not create conventions</p><p>44:08 The stag hunt and conventions as coordination devices</p><p>47:41 Perspective, grammar and why discourse matters</p><p>53:12 Pointing and building agents from the bottom up</p><p>55:44 Money, presidents and status functions</p><p>59:26 Lying, gullibility and the dark side of cooperation</p><p>1:01:27 Is capital an agent? Emergence and the invisible hand</p><p>1:04:48 Acting like an agent: the intentional stance</p><p>1:06:16 Closing</p><p><br></p><p>BOOK:</p><p>Michael Tomasello, The Evolution of Agency: Behavioral Organization from Lizards to Humans (MIT Press, 2022)</p><p>https://mitpress.mit.edu/9780262047005/the-evolution-of-agency/</p><p><br></p><p>KEY REFERENCES:</p><p>[11:41] Benjamin Recht, The Irrational Decision (Princeton UP, 2026) https://doi.org/10.1515/9780691272467</p><p>[28:07] Mark Solms, The Hidden Spring (Profile Books, 2021) https://profilebooks.com/work/the-hidden-spring/</p><p>[37:42] Michael Tomasello, The role of roles in uniquely human cognition and sociality (2020) https://doi.org/10.1111/jtsb.12223</p><p>[38:15] Cheng, Gao, Tomasello &amp; Kushnir, The Agency Gap (CogSci 2026) https://escholarship.org/uc/item/8mv6v0s1</p><p>[38:59] Tomasello, Melis, Tennie, Wyman &amp; Herrmann, Two Key Steps in the Evolution of Human Cooperation (2012) https://doi.org/10.1086/668207</p><p>[40:28] Michael Tomasello, The Normative Turn in Early Moral Development (2018) https://doi.org/10.1159/000492802</p><p>&lt;truncated, see rescript/YT&gt;</p><p><br></p><p>SUPPORT MLST:</p><p>Patreon: https://www.patreon.com/mlst</p><p>Discord: https://discord.gg/HNnAwSduud</p>

出典・この記事について

番組
Machine Learning Street Talk (MLST)(Machine Learning Street Talk (MLST))
元の回
What It Would Actually Take for a Machine to Have Agency (Michael Tomasello)
配信日
要約作成日
公式サイト
podcasters.spotify.com/pod/show/machinelearningstreettalk

この要約・解説記事は、元の音声を文字起こしした内容をもとに AI(Claude)が日本語で作成したものです。 固有名詞や数値に誤りを含む場合があるため、正確な内容は元の音声でご確認ください。制作方法について

X でシェアSpotify で聴くApple Podcasts で聴く

関連エピソード