Karl Friston - Why Intelligence Can't Get Too Large (Goldilocks principle)

This episode is brought to you by Diet Coke. You know that moment when you just need to hit pause and refresh? An ice-cold Diet Coke isn't just a break. It's your chance to catch your breath and savor a moment that's all about you. Always refreshing, still the same great taste. Diet Coke. Make time for you time. The Subaru Share the Love event is on from November 20th to January 2nd. During this event, Subaru donates to charities like the National Park Foundation, helping protect over 400 national parks. When you purchase or lease a new vehicle during the 2025 Subaru Share the Love event, Subaru and its retailers will make a minimum $300 donation to charity. Visit Subaru.com slash share to learn more. Keith has been talking about this moment where we had a glass of sherry with Professor Friston for about the last four years. Right. Our dream has been realized today. Absolutely. So pleasure to meet you. It's absolutely been a pleasure. Cheers. Cheers. You mentioned consciousness, which you shouldn't really do with me. But if we stay here for long enough, for more than five minutes, we will ultimately become completely entangled. We will not become one. We'll still have a, we'll still have, well, actually, Chris Fields thinks he would become one. But is evolution an intelligent process? It's certainly a free energy minimizing process. It is just basic model selection. Well, it's like, is the planet intelligent? I don't see the weather planning. I don't see evolution planning. It doesn't think about its future. Life is the intensive property of matter and intelligence is the extensive property. It's not an ensemble of universes. Of course, for the purpose of this argument, there's only one state of the universe at any one time. I'm not sure this is correct. I don't want to disappoint you. Don't worry, it's well beyond my lifetime when that will happen. If you're going to make a difference, it's really understanding how to use all the marvellous engineering that we've witnessed in terms of generative AI and large language models and the like. In the service of understanding the principles of natural intelligence and deploying the understanding either to make life better, fun, or to underwrite sustainability in a slightly more sort of political way. or to deploy it, which is why I got into this game, in the context of mental well-being and computational psychiatry. To quote Feynman, that which I cannot create, I don't understand. That means we have to be able to create things that suffer, which means that we need to understand the principles of natural intelligence. I don't know if you saw, I interviewed Yoshio Bengio. All right. And I used the term epistemic foraging a few times, and he said, I love that term. I love that term. Where did it come from? Professor Friston. Absolutely. Epistemic foraging. It's what we do. It's epistemic foraging. It's epistemic foraging. There are certain causal forces in our universe that are at a distance. Those usually mediated by electromagnetic radiation or magnetic fields in this particular instance. And that really sort of complicates the way in which we construe all the cause-effect structures that we have to model in our brain. Because if I was a simple little virus or a little bacterium, I wouldn't have to worry about electric fields and seeing things or hearing anything. My world would just be that world that I could touch, literally. It would just be my next-door neighbours, which leads to a very particular kind of Markov blanket structure and ecosystem of Markov blankets that precludes... Oh, it certainly does not license some of the deep structures that we were talking about earlier on in terms of strange things. So strangeness of a beautiful sort, beautiful loops to use one of my colleagues' notion, rests upon action at a distance of a very non-spooky sort that is nicely exemplified by this compass. So at the moment, I don't know of any good maths that allows you to work out what is the best thing to do to disambiguate between this structure and that structure. So if you do that, that'll be good. Structure learning. Structure learning. To structure learning. Structure learning. Somebody out there, help us out. Yes. human data is shaping the direction of frontier AI yet there's little visibility about how teams are actually using it our sponsor prolific are putting together their first report on human data in AI and they need volunteers it just takes a few minutes to fill out and you'll also get early access to their findings so you can see how you compare this podcast is supported by google Hey folks, Taylor here, creator of Gemini CLI. We designed Gemini CLI to be your collaborative coding partner at the command line. Think about those tedious tasks like fixing a tricky bug, adding documentation, or even writing tests. We built Gemini CLI to handle all that and more. It iterates with you to tackle your most challenging problems. Check out Gemini CLI on GitHub to get started. MLST is supported by Cyberfund. Professor Friston it is absolutely amazing to have you back on MLST and as you know Keith and I are incredibly fond of you you've been a big hero of ours for many years. I think this is the fourth time that you've been on show. Welcome Professor Friston. I've lost count but it's lovely to see you both in person and congratulations on your recent marriage. Thank you very much. Thank you very much. Yeah, so, you know, we were just thinking about this. We've done these interviews. The free energy principle originated around about 2005, right? And I just kind of wanted to get a bit of a retrospective from you on, let's say, like, what have been the successes so far of the free energy principle? How's it been going? What could have gone better over the last, you know, 20 odd years? You know, really just What's your take on the progress, I guess, of the free energy principle? It's a difficult question to answer in the sense that if you asked me if I'd made a meal, and you've asked me, how's that meal going, and did you really enjoy it? When you're doing it, you're sort of in the middle of it, and it's very difficult to assess. So it's been a journey. I do catch myself from time to time thinking this is the right way to think about things. Whenever you come across a phenomena or a problem or an application that seems to just slip in quite neatly to the overall theoretical framework, every time that happens I get a little buzz of dopamine that consolidates and comes, Yeah, I'm on the right track. I haven't wasted so much time. There have been times in my life when I realize I'm on the wrong track. So I'm used to that. But for the particular application of the free energy principle, I have yet to have that. Oh, well, that doesn't work then. Think about it. Do something else. What could have gone better? I'm not, I'm ambivalent about this, but I am sometimes, well, I am ambivalent when I read the free energy principle is notoriously difficult to understand. Now, half of me thinks, good, because it's always important to have a slight degree of magic and mysticism to engage people. If people don't think there's a challenge, that they have accomplished something by understanding it, then they're not going to be motivated to inquire and to think about it or indeed debate it. On the other hand, the free energy principle is not meant to be complicated or difficult to understand. It's actually almost tautologically simple. So being able to communicate the free energy principle in a way that people find a useful and obvious sort of tool or method to apply could have, I think, gone better. and that may be because I'm not very good at keeping things simple or intuitive. We can attest to that. One thing that's interesting is I feel the same way about probability theory. Conditional probability theory is very easy to write down. You end up with the two fundamental rules, the sum rule and the product rule, but the consequences of it are very profound and probability is notoriously misunderstood, misapplied. There's many errors of reasoning that happen when people try to reason through statistical correlations or whatever. So I think there are things like that in life that are very simple and yet very hard to kind of understand the full extent of their impact, right? Yes. I mean, it's interesting you picked up on conditional probabilities. I mean, that is the heart of the free energy principle at many, many different levels. So the classical formulation, the free energy principle, just starts off with a partition of different states of being. And that partition suddenly means you've got the probability distribution or density of one set of states of being that now can be conditioned upon another. So the whole free energy principle is just basically a principle of least action pertaining to density dynamics, the dynamics or the evolution of not densities, but conditional densities. That's just it. This is before thermodynamics. It's before quantum mechanics. It's just about conditional probability distributions. So it's interesting you picked up on that as something that is so simple, yet so absolutely powerful. So, Professor Preston, we were reading your paper earlier. It's from 2023. Path integrals, particular kinds and strange things. and it introduced a categorization of particles. So, for example, inert to active and ordinary to strange. The strange categorization was particularly interesting because you were giving an account of how these particular particles could give rise to phenomenal, you know, conscious states or even agentic states. And we felt that this was almost an account of panagentialism. So, you know, historically, we've done interviews about the free energy principle and we've spoken about self-organisation and emergence and the agency and the phenomenal states are something which emerge when you have this sort of temporal and counterfactual depth. And maybe we're misunderstanding, but it feels like this is an account of low level agentic potential and phenomenal potential. So to set the scene, just to come back to that partition that we were just talking about, there are many ways of arranging the conditional independences to disconnect various partitions, external states, internal states, my sensory states, my active states. and the combinations of disallowed influences give rise to an ontology of different kinds of things and you could call them natural kinds. I've been told by Maxwell I shouldn't use that but I like natural kinds. The things in nature that just prescribe themselves just by being special instances of a lack of causal influence. So if we start off with the simplest case where there are no internal states and there are no active states. And what are we talking about? We're talking about some kind of causal black hole, something that's quintessentially inert. And you can never see it because you can only see the active states that act back on the environment in which it is embedded. But then we get to more interesting things that have a full complement of internal states and external states and a bidirectional coupling between the two, which of course is mediated by the sensory states and the active states. And if you disallow action at a distance, by which I mean I have to be in some metric sense next to you in order for me, my active states, to become your sensory states and your sensory states to be my active states, then you have a very simple kind of Markov blanket which would be fine for describing active matter, for example, or any medium where I have to touch, to feel, to influence or to sense. And this kind of particle, interestingly, has its... or this kind of system, partition, has its active states on the inside, which is, you know, I had to think about. And it can be no other way if you try to arrange all the dependencies. So what would that look like? It would look like something like a cell that had on the outside its sensory states that was supported by a layer of active states that surrounded the internal state. So you're complying with the laws of the Markov blanket of thingness. You need to have that conditional independence to separate the thing from everything else or the self from the non-self. and everything's fine because the active states are hiding behind the sensory states so the external states can't influence the active states so that's tick one that's you know what we need but we also have to ensure that the internal states symmetrically cannot act influence the sensory states and that's fine because they're hiding behind the active states so you've got you know a nice mathematical image of you know a cell if you like but that doesn't work for things like you and me. Things like you and me have a hierarchical structure. And what that basically means is that the active states, which were hitherto in very simple organisms, immediately juxtaposed to the internal states, now become sequestered. So now, effectively, the active states are no longer seen by the internal states. And then something quite remarkable happens mathematically. It's really simple, and it's just another aspect of conditional probability distributions. Because you can't see your active states, so you can now only see your sensory states, it looks as if, from the point of view of the internal states, that the active states now become causes of sensory states. So now your world is caused not just by the external world, by the environmental states, by my heat bath, by my external milieu, but also my own actions. So now I'm inferring the causes of my sensorium where I'm actually a cause. So I'm inferring myself. So there's this beautiful recursion which licenses this sort of strange loop analogy, with a bit of poetic license. Another way of putting that is that for these simple structures, for these natural kinds that would be, say, single-celled organisms, the internal states can be read as modelling the causes of their sensations that just are the external states. So they have direct access to the active state, so there's no conditional independence that licenses the notion of description in terms of inference or sense-making. Whereas now the Bayesian mechanics that attends the internal activity, internal machinations, now covers both my action and the things that I'm acting upon. And of course this is a nice metaphor for planning as inference that I'm now thinking about and trying to infer what am I actually doing. And that, to my mind, produces a very unique and special kind of thing. You know, things like you and me, basically, which are pretty unique. We look at all the different kinds of things that could be around. This episode is brought to you by State Farm. Listening to this podcast? Smart move. Being financially savvy? Smart move. Another smart move? Having State Farm help you create a competitive price when you choose to bundle home and auto. Bundling. Just another way to save with a personal price plan. Like a good neighbor, State Farm is there. Prices are based on rating plans that vary by state. Coverage options are selected by the customer. Availability, amount of discounts and savings, and eligibility vary by state. Well, it's another example of how as soon as you had this recursion, right, as soon as you had these strange loops, especially for systems that are computational, you know, and I forget who it was that said life is the computational phase of matter. But, I mean, when you're at this level of complexity where you're having some kind of information processing and you introduce this strange loop, right, that unlocks a completely different, like, category of behavior, you know, tier of computation, right? Absolutely. It was David Krenkowen. He said life is the intensive property of matter and intelligence is the extensive property. No, this is a different one, though. It may have been Wolfram who said life is the computational. I don't remember. I'd have to look it up. But yeah, so it unlocks a completely different category of behavior. Well, you alluded to that earlier on in terms of phenomenology. And you mentioned consciousness, which you shouldn't really do with me. But we'll read that as an elementary kind of sentience. So I think that's really important, that point that you do unlock or you do manifest. or now at least you have a mathematical calculus that allows you to talk about things that model themselves. And as soon as you're modeling yourself, you become an agent or you have an authentic kind of agency because now, in order to model myself or to model me as a cause of my environment, I have to infer what I am doing, and particularly the consequences of what I'm doing. And because the consequences have not yet appeared, this has a quintessentially future-pointing aspect. So we're now moving from a thermostat or a virus or a single-celled organism to things that actually have a future on the inside, their private future that just exists for them. And, of course, if you can get suitably far into the future, where you have a divergence of particularly paths into the future, just as a nod to the pathological formulation, then you have to select one. So now you've really unlocked, again, a calculus of selection of your paths into the future, which I think is not a definition of agency and certainly not consciousness, but certainly would be, I would imagine, arguably necessary to support something that had true agency and possibly even consciousness of an elemental kind. Could we press on the consciousness a bit? Because the account that you just gave is a beautiful account of agency, and it's very plausible. So as you increase this reflexive layering, this recursion, you get these multiple paths into the future, and you become the cause of your own action. So you become more of an agent. So it's almost an account of strengths of agency with more and more layers. But in the abstract of this paper we were quite struck with the language that convolves consciousness with agency And intuitively I feel that that phenomenal experience is quite orthogonal to agency So why were they kind of convolved together? I'm desperately trying to remember how I referred to consciousness. You've got to be very careful when using that word, depending on who you're talking to. The C word. The C word. Oh, naughty. so yeah and I think people like Neil Seth make a very similar point quite earnestly that intelligence and consciousness are completely orthogonal, that you're making agency and consciousness are completely and I would agree entirely so consciousness is not implied by being agentic having agency you would first of all well there are many stories you can tell here and you know my mind goes to the people might be watching this to make sure I don't offend anybody by not mentioning it so yeah from from the point of view of three energy principle the way that you'd look at consciousness is um in terms of a dual aspect monism which um you get for free from the treatment of the density dynamics in exactly the way we were talking about before but you know so now my internal say brain states um have a thermodynamics they have that can be written in terms of an information geometry that itself is just predicated on conditional probability distributions. But they also have an information geometry that inherits from the fact that my internal brain states represent the outside world, including my own actions. So there's both an information geometry and by implication a thermodynamics of my brain activity which supervenes on exactly the same substrate as does the information geometry which is representational which is the inference the basium mechanics side of things so that gives you license to talk about a sort of dual aspect monism but what is it what does it really mean to be conscious some people might believe that it's just having a posterior belief having a basium belief which is literally just a conditional probability distribution that is encoded or parameterised by some physical state of being, and in this instance, under the free energy principle, it's the internal states of being. Other people say, well, no, just being able to read a thermostat, for example, as having post-trail beliefs about the temperature of its external world does not licence you to ascribe consciousness to this thing. so the next step would be oh it has to be ignited in some way it has to be realised, it has to be emergent and I'm sort of paraphrasing Jakob Howey here and to a certain extent Mark Soames, so they emphasise now it is when these beliefs become sufficiently precise in a dynamical way that they are realised and influence other aspects of belief updating in a hierarchical or distributed system, so Mark Soames would call this felt uncertainty, so he would emphasise that the feeling part of consciousness just is, or is mediated by, the representations not of the content, but of the precision or uncertainty or the confidence with which these beliefs are currently in operation. and he would bring to the table all sorts of very compelling neurobiological evidence as to why this is, how you can switch off consciousness literally by, I think he says it's a three millimetre cube part of the brainstem that is the cells of origin, these ascending systems that encode and represent, I repeat, not the content but the confidence or the uncertainty or the precision of these conditional probability distributions. Jakob would, I think, say, well, to be aware is to equip all those messages that are providing evidence for your current explanation with precision, which has these physiological and bioelectric mechanisms behind it. Other people go further. people in the world of phenomenology of the kind that Thomas Metzinger pursues, would say that it would be necessary to have control over the precision. And furthermore, to be aware that you are rendering that which was once transparent, opaque, you actually have to recognise that you're attending. So, again, not only do you have this very simple recursion of self-modelling in terms of planning as improvements, but now you've got this hierarchical recursion where you're now recognising that you're attending to something. And then if you're somebody like Lars Sundered Smith, you'd say, well, to actually be self-conscious, I now have to recognise that I was recognising that I was attending to something. And you get layer upon layer upon layer. And to join the dots with Chris Fields, who has brought to the table the inner screen hypothesis. Have you come across this yet? No, tell us about that. Oh, well, strictly speaking, you should get Chris to tell you about that. But I'll give you a quick preamble. So Chris Fields is the theoretician who has formulated the quantum, quantum information theoretic version of the free energy principle. So for him, the Markov blanket that separates self from non-self becomes a holographic screen upon which classical information to which it is written and from which it is read by some internal bulk and some external bulk. And in the sense that strange loops and strange recursions are only allowable when you have this hierarchical structure on the inside, you've now got lots of inner holographic screens, inner Markov blankets. I mean, Markov blankets, in their pragmatic sense, just define things like hierarchies, for example, and they define the architecture of any network, factor graph or computer. And when you've got lots of them, you've effectively got lots of inner screens. So the idea, well, my reading of that idea is that there is one irreducible inner screen. So there's one inner screen that within it has no other screens. And this is an interesting screen. Markov blanket from the classical perspective because the only way that the internal states of this irreducible Markov blanket can know themselves is by acting on the exterior, which, of course, the exterior is the rest of your brain. So you've got this metacognitive, self-recursive aspect to this inner screen hypothesis. And that has, I think, a lot of mileage. that particular notion has a lot of mileage in relation to other people's theories of consciousness. So we're talking about higher order thought theory. So the very notion of higher order thought and its implicit appeal to metacognition and metametacognition is again this notion of looking at oneself and inferring oneself but on the inside. So looking at the looking at the looking, which is of course a natural consequence of having this hierarchical structure. You could also argue it's completely compatible with global neuronal workspace theories that there is this sort of precision-dependent ignition of certain sources or messages, sufficient statistics of conditional probability distributions, that gain access to, you know, across all of these inner screens in a dynamic way because you're controlling the access by attending, by encoding and representing and optimising the precision or the uncertainty or the confidence. So you've got this notion of ignition and penetration through to the global workspace, which, you know, you could read as this sort of irreducible inner screen, the core, you know, the deepest part of your sense-making, your brain. So I think this notion, not only is it consistent with the conditional independences that are implicit in the free energy principle as applied to strange things that have this hierarchical and recursive aspect, but it also, I think, relates very comfortably with extant theories, all coming from different perspectives, in different parts of the elephant, as it were. But it is, I think, a comfortable accommodation of things that we presume or things that people are brought to the table in order to explain consciousness from their direction of theorizing. It seems that if we kind of split, let's say, the camps or the thought, the groups of thought about this, there's almost all of what you just talked about now are there is a minimal complexity required for there to be consciousness. And then we can talk about what that level is. Well, it requires this or this or this. But then there's also people who say, no, there's no minimal complexity. It's like electrons have some kind of small amount of consciousness. And I fall much more into the category of people who say there is a minimum. Like you have to reach a certain something. I don't really know where that boundary is. Maybe it's one of these that we've mentioned. Maybe we'll figure out a more elegant kind of categorization. But it's just like you need a certain number of pixels in the game of life before you can have a glider. You can't do it with three. Like you need – there's some minimum number. And so there's some minimum amount of brain material. You know, three cubic millimeters is still a lot of neurons. I don't know how many are in there, but, you know, hundreds of thousands or whatever it is. You need some level of complexity before you have consciousness or before you have, let's say, recursive agency or intelligence or whatever. Would you agree with that? Like there is some minimum. We don't know where it is yet. It's hard to place that. But there is a minimum or not. No, I would agree entirely. Okay. So there is a minimum. And not only that, it has to have a certain causal structure, right? And we can kind of debate that. And that's really in line, too, with Searle in the Chinese Room argument, which is like, look, a dictionary doesn't have understanding because it doesn't have the right causal structure. You have to have a certain causal structure or a certain minimum complexity, and then you reach this whatever it is, whether it's consciousness we're talking about, understanding, agency, all of these things, right? And so I guess my question to you is, will we be able to build machines based on our current computer architectures someday, whether it's 100 years from now, 200 years, doesn't matter. But in principle, can we build machines that have understanding, that have consciousness, that have all these capabilities? This episode is brought to you by Indeed. You're ready to move your business forward. But first, you need to find the right team. Start your search with Indeed Sponsored Jobs. It can help you reach qualified candidates fast, ensuring your listing is the first one they see. According to Indeed data, sponsored jobs are 90% more likely to report a hire than non-sponsored jobs. See the results for yourself. Get a $75 sponsored job credit at Indeed.com slash podcast. Terms and conditions apply. We've got gifting all wrapped up at Sephora. Gift more and spend less with our value sets. packed with the best makeup, skincare, fragrances, and haircare they'll love. This year's Showstopper gift sets are bursting with beauty products from Rare Beauty, Summer Fridays, Glossier, Amica, and so much more. Shop holiday gifts at Sephora.com and give something beautiful. Yes, that's a good question. And the answer, I think, in principle, yes. Can I just come back and qualify that answer by reference to that wonderful example of what I would read as vagueness in a technical sense. So vagueness, how many grains of sand constitute a pile? So it's not well defined. It is in philosophy, but not in mathematics. I think that's absolutely the right way to think about the bright lines. And if I had to commit to the dimension, the number of grains of sand that you get before you have consciousness or there is a pile, I would say it's something you actually referred to earlier. I think it's the depth of your future or your future in your head. So if you're talking now about an algorithm or some artificial intelligence that is equipped with a generative or a world model of the consequences of its action, there will be a time horizon associated with that component of its generative model. And I think it's the depth of that time, what Neil Seth would refer to, not as counterfactual breadth, which is the number of divergent paths one could take into the future, the options that you select among, but the counterfactual depth, the temporal depth. And so that means that you can be panpsychic. You can say that a thermostat has a notion of the future in the sense that it operates of pathological control and differential equations. As soon as you put a differential equation in play, you've got an instantaneous future because you've got some gradient with respect to time. that's not though the kind of depth that you and I enjoy so I would imagine it is really just the depth you are very very reflexive and I think people like Maxwell Ramsted talk about this as merely reflexive active inference with very myopic, very short term self models right through to fully well to be conscious in the way that we've been talking about I think you need to have a long depth. So what does that mean for building AGI or conscious artifacts or machine consciousness? It means, first of all, they have to be agentic, because we've just said that having a world model of the consequences of your action would be necessary to be an agent. But more than that, you'd have to look quite a long way into the future. So your generative model, your world model, would have to go quite a long way into the future. And I repeat, have that both counterfactual breadth and counterfactual depth at hand. Would that be sufficient? And I'm now remembering that I forgot to mention a Neil Seth in the list of people not to upset when reviewing theories of consciousness. So this is an opportunity just to say that there are people out there who would say, well, you know, OK, you can write down the maths of all this and you can write down inner screen hypotheses or indeed simulate global neural workspace theories and try to produce things that look as if they have consciousness. That's not going to work unless you actually embody it, unless you are, in Anil's words, a beast machine. And I think that that sort of coheres with the argument for mortal computation. So when you may ask the question, can't we build it on our computer architectures? I would have to ask you, do you mean of a Newman architecture? Or do you mean a processing in memory or in memory processing architecture? Which I would take as synonymous with a neuromorphic architecture. not spiking neural networks. You don't need those. But you do need the processing in memory to be mortal. You need that substrate dependence, read in terms of Geoffrey Hinton's definition of mortal computation and Alex's subsequent elaborations of that. So if you've got, I think, I would subscribe to that. I think largely to keep Neil happy, but I would subscribe I don't think you can do this on a von Neumann architecture because the Markov blankets of a von Neumann architecture where you're reading and writing from memory make it very difficult for the memory to self-organise I see and people have written about this philosophically, I think Vanya Weiss has written a paper about this and I think Anil Seth speaks to this argument in a recent behavioural brain science paper it may not be possible and certainly from the point of view of efficiency it's highly unlikely that von Neumann architectures are the kind of things that would conform to let me reverse to be is to pursue a path of least action in accordance with the free energy principle to be conscious does not excuse you from that So if you want a conscious machine, you have to have a machine that pursues a path of least action. Via the Janinski equality, that has to be true both thermodynamically and informationally in terms of the conditional probability distributions. If you don't pursue that path of least action, you can't be and you can't be conscious for a non-trivial amount of time. And I can see you want to ask a question, so I'll interject. Well, only to say Maxwell pointed me to a Nils where he defends biological naturalism. And of course, even Searle said that the biological substrate is an existence proof. He's not saying it has to be biology. But Keith and I certainly agree that there's something about the substrate which is very important. I wanted to talk a little bit about viruses. As you know, I'm a bit of an externalist. I've never been able to completely pin you down, Professor Friston, because there have been so many interpretations of the free energy principle that lean internalist and externalist and even the hybrid version, which Maxwell also wrote a paper about. But I'm fascinated in this idea of diverse intelligences. And, for example, could a virus be intelligence? And I spoke with David Krakauer and he said intelligent things do inference and they have representations and they're adaptable. And we are a little bit, you know, we're a bit chauvinistic about our brains, aren't we? Because our brains seem to have a privileged status. But what say you of viruses, or actually, we read your paper with Calvo, predicting green, really radical plant predictive processing. and in that paper you gave a beautiful account of how even plants could be doing inferencing and and that to me seems it seems incredible because i'm amenable to the idea but it seems to me intuitively that plants are not as sophisticated as as we are so it comes back to this line that keith was talking about before that if you go too far down the stack it's an account of we let's call it pan intelligence where you have a tiny little bit of intelligence even if you go all the way down the stack have you spoke to mike levin yes relatively recently about a year ago ago Right I mean you know it would be nice to revisit him I mean, that is exactly his big question at the moment. And, of course, he has as an intellectual accomplice, Chris Fields, with him as well. so that the two of them are really pursuing this notion of basal cognition, that there is intelligence everywhere. And it's just a question of how we conceive of it and how we test of it. And indeed, his argument, Mike's argument, is that you have to design the right experiments. It's an empirical question. Is this virus intelligent or not? Well, you have to design the right experiments to disclose or evince its intelligent behaviour. And Mike would claim that by many metrics, many yardsticks that we use to measure adaptive intelligent behaviour, yeah, slime moulds and viruses and xenobots are incredibly intelligent. And I think he's fighting against the chauvinism that you mentioned, that intelligent things are just properties of things like you and me, creatures like you, me, and our brains. And that kind of cognitive capacity and competence can be found everywhere. So I'm very sympathetic to that. But it does tread on the toes of the vague argument. You know, I certainly don't think that viruses have the same expressive kind of agency that we were talking about and indeed you know the counter argument to the vague notion of intelligence and or consciousness is of course hitting you in the face when you know when you read that strange things paper you know I'm talking about categorically different natural kinds that do and do not have for example a causal power or influence of active states on internal states. You're either one of these or you're one of these. I'm not so sympathetic to the view that, you know, a virus is intelligent, right? Because I think this falls in the category of things like where you're in a biology class and you learn how to define life. And then somebody's like, what about fire? Then like, it seems to meet all these criteria here, you know, it grows, it expands resource. You know, so for example, when a virus, quote, mutates, the virus doesn't mutate itself. It is mutated by a gamma ray hitting it or a, you know, transcription error or whatever. And, and oil, you know, vinegar and baking soda undergoing a reaction is not intelligent, right? So there's, I think there's some level of complexity and, you know, whether it's recursion or these other types of causal structures that have to be there before I'm even interested in talking about intelligence or entertaining intelligence, you know, other things are just kind of dynamics that unfold in certain ways, chemical reactions, you know, that sort of thing. And I think that at that point is a nice opportunity to introduce the notion of scale freeness, or scale invariance. Because I can see you could also take this towards collective intelligence and federated learning and federated inference. So it's not the single cell. It's a single cell with its neighbours and its neighbours and neighbours and neighbours and neighbours. And so what one's looking for is some conservation of intelligent dynamics that is preserved over different scales. So, you know, the single virus is probably not interesting. It's probably the colony, which is interesting. So I thought that was a nice point to introduce the notion of that kind of scale invariance. And of course, as a mathematician, you would be looking at the renormalization group to see how that unpacked mathematically. And interestingly, it also speaks to this notion of a mutation. Is evolution an intelligent process? You know, it's certainly adaptive. it certainly has the level of complexity that you would require in order to pass those vague thresholds but is evolution in and of itself an intelligent process? It's certainly a free energy minimising process it is just based in model selection natural selection just is or selecting those things that have the highest model evidence a marginal likelihood of being that phenotype in this kind of environment well it's like is is the planet intelligent you know it certainly contains eight billion or so of us so does that count i mean yeah why not well i mean that's where well i would say like uh and we talked about this way back when we talked about um you know is a flotilla um an agent or is it the pilot who's you know kind of kind of controlling the convoy or whatever. I would say like if there's a more minimal boundary that contains the intelligent, then the larger one is not. Like I'm talking about, so the planet, for example, I would say since I can draw a smaller boundary, which is an individual person, the individual person is intelligent. Anything that contains that person is just more stuff. Keith, what is your, the issue with viruses, is it because the individual virus is a nut, whereas a cell for example you can partition down to the individual cells and the cell still has some degree of agentic property as as as professor friston was describing is is it that you see them as inert and being carried by something else well i think it's that too much of their their causal structure and machinery is basically outsourced to other things it's like they you know they're kind of just these little particles that go around and if they happen to stick on a cell they're like a mousetrap that activates and just injects some DNA. But, you know, the cell does all the rest of the work, right? It does the transcription. It has all that machinery. By themselves, a virus doesn't change over the course of its lifetime. It doesn't really, other than this simple mousetrap kind of activation, it does nothing. You know, it doesn't have any machinery to learn and adapt by itself, like over the course of the lifetime of a virus. You know, so I might be more amenable to like if somebody says, well, like an ecosystem of, you know, viruses or something has some degree of, you know, intelligence, but it still doesn't have the type of processing and causal structure and minimal complexity that at least for me is the useful concept of intelligence. So I think we're coming back now to the number of inner screens and the counterfactual depth and the definition of agency in the sense of strange things that we're talking about for this particular scale. So if we just read the scale as, if you like, the size of the Markov blanket. If within, at this scale, for this kind of thing, there is a sufficient degree of complexity, read specifically, though, in this instance, as the counterfactual depth and breadth of your world model about the consequences of your action, that particular part of your implicit generative model. And that would happily accommodate the fact that when you get something of the size or the scale of a virus, there just isn't the machinery or the space to entertain that in any non-trivial way. One could also argue, and I got a sense that you were arguing yourselves towards this. If you go too big, you also lose that. So coming back to this wonderful question, we have 8 billion intelligent, really intelligent, sounded like Trump there, didn't I? Entities constituting our biosphere, but is the biosphere, from the point of view saying that the guy hypothesis, is the biosphere of itself intelligent. And I would say no. Not because, it's simply because at that scale, the elemental, all the complexity of the constituent elements disappears at that scale. So, you know, it'd be a little bit like saying, well, let's just take it to the limit. Let's just take it to the astronomical limit of the motion of heavenly bodies. Now, the motion of heavenly bodies is completely described by the position of the planets and the moon, right, and the position of the Earth. So at that scale, you've averaged away the fact that the Earth contains a biosphere, and the biosphere contains human beings, and human beings contain cells, and the cells may or may not contain viruses, depending upon who you've been exposed to. So there will be no intelligence at that level. So it's perfectly possible to have intelligence at a particular scale that disappears when you get too big and when you get too small. Another way of looking at that is going right back to things like brigagine and dissipative structures. You know, you're talking about complexity. if you just think about how have people tried to understand complex systems and self-organising systems that are open. So we're not talking about 20th century physics and equilibrium physics. We're talking about the physics of non-equilibria and things that are in open exchange with each other. And of course, you get to the notion of dissipative structures. What does that tell you mathematically? Well, what's not dissipative? That's a schoolboy question. Can you remember? You've probably forgotten. So what is not dissipative is conservative. Oh, that's it. Sorry, I'm teasing. So another gift of Helmholtz, of course, is that any dynamics can be partitioned into a dissipative part and a conservative part. and the dissipative part is that which rests upon very fast, random, complicated fluctuations of the kind you might find in, say, quantum mechanics or thermodynamics, whereas the conservative part does not rely upon that and just goes round in circles, basically. Literally, it's called solenoidal flow or conservative flow. So that basically means that dissipative structures have to have an admixture of this circular aspect, this conservative classical life cycles, reproduction, oscillations, plus the random dissipative part that you'll find in things like thermodynamics and quantum mechanics, and that is definitional of dissipative structures. As you go too small, then everything becomes quantum and random. It all becomes probabilistic. There is no conservative stuff other than a scroting of potential, at which point I think you'd find it very difficult to find something that was intelligent in the sense we're talking about because we have to have this recurrence, this solenoidal aspect in order to revisit the states. So you know a virus is a virus in order for there to be a non-equilibrium steady state distribution as a solution to that. But as you get bigger and bigger and bigger, you get to viruses and they're still not quite sort of complex enough to be intelligent in the way that we mean. And then you get to our size, and that's perfect, because we both have both this dissipative... We deal with a random world, with an itinerant world, and yet we keep revisiting states of being. So we have this sort of conservative, biomimetic kind of self-organisation. But then you get bigger and bigger and bigger. You get to the level of the biosphere or the size of the moon or the sun. And of course, by averaging, all the random fluctuations go away. So you're just left with the solenoidal part. You're just left with the solenoidal motion, the Newtonian motion of classical mechanics. So one thing that really fascinates me about this is, however, it is possible for us, and we do, construct large-scale things that are intelligent, like a corporation. You know, a corporation is seen as like a super intelligence, but we're only able to do that by putting in structure, right, that maintains this balance of the dissipative and conservative flow, right? Like we have to put in organizational structure so that it continue to function at these larger scales, right? Like isn't that pretty interesting? I mean, it's again this yin and yang thing that we run into all the time, right, where it's the balance between two forces, like either between, say, complete order and complete noise or between dissipation and conservation. And you have to be almost on the edge of chaos. And it has to have a certain causal structure in order for it to be intelligent. No, no, I agree entirely. and that sort of golden locks regime where you are on the edge of chaos I think is quite specific to a particular scale certainly you could invoke a sort of strong anthropomorphic principle here and say that the kind of intelligence that we will recognise has to be at our scale but I think there's something more fundamental than that I think the very existence if you subscribe as an externalist to quantum physics and Newtonian physics or Lagrangian classical mechanics, I think that there is a golden locks regime, which means that we can only exist at this scale with, as you say, this sort of yin-yang, this admixture of dissipative dynamics and conservative dynamics. Just to reinforce this, because conservative dynamics is absolutely essential because it is that which causes this Poincaré recurrence. It defines these strange attractors, which is the other way of licensing the notion of strange things. You always come back to somewhere near where you started, like Red Queen Dynamics in theoretical biology. So you have to have the conservative, the circular motion, just to have a routine, have biorhythms, replicate, reproduce, many, many different levels at many different scales. But it's always remarkable in the face of a dissipative itinerant world. everything is changing all the time um and yet we somehow sort of resist that change by being at the edge of chaos just oh no you're you're more anxious than i do to say something carry on please go on professor president no no well i mean i'm just so fascinated by by your observation that with gaia theory for example when we zoom out the the the apparent phenomena seems less intelligent and as you said maybe intelligence is just what we recognize and um when we spoke with wolf from that time he was talking about how we are computationally bounded as observers and we we do this thing called abstraction where we ignore details or even idealization where we deliberately distort the truth and um david krakauer said that intelligence is doing more with less and emergence is more is difference and i mean the earth is not doing it's not doing less we just don't see what it's doing but if it's if emergence is more is different than that it licenses a fundamental reorganization of the underlying substrate which means it isn't actually doing the thing at the lower level anymore it's it's a fundamental course graining and it's actually changing at the higher level so which is it well um i didn't realize there's a choice there because i agree with everything you said but i like the notion of course grading because of course that is exactly what you get from the renormalization group treatment of these things and you can simulate this you can simulate um free energy minimizing processes that are running at different scales and then you have to ask the deep questions of how you couple between the scales and and there you get to supervenience and emergence, mathematically so defined. But it all boils down to exactly where you started and where you ended up, which is a coarse graining in the right kind of way. So to actually write down a renormalization group, you have to have an RG operator. What is an RG operator? Well, it has two parts to it. It has a sort of dimension reduction, the R part, if you like, and a grouping operator. And they basically do the right kind of coarse graining. They reduce dimensionality and they group together in the right way those states of being at the higher scale and so on ad infinitum. So I think the notion of coarse graining is absolutely essential here. And on that level, you could argue both ways. I'm getting a sense that your question is, is there a true emergentism here or not? Again, I don't have a philosophical training, so I'm not sure I can really answer that. But certainly, the intelligent dynamics read as self-evidencing perspective on a free energy minimising process are recapitulated at a completely different scale in a different way at each scale through the recursive application of RG operators. so yes there is something brand new going on and yet in the spirit of say Haken's Synergetics these RG operators are just taking functions of stuff that's happening at the finer scale so you're not inventing anything new it's just predicated or emerging or supervening on finer scale stuff but what emerges at the high scale has all the attributes of self-evidencing, and you could argue consciousness or intelligence at some level. But coming back to the point about building bigger organisations, if the argument that as you get bigger and bigger and bigger means that you are effectively to exist, you have to be effectively conservative. So a company, for example, what is its metric of goodness is how long it survives you know can you get from series eight series whatever in some recognizable form so as you get bigger and bigger and bigger I think there's less opportunity for this complex scale free within it's a not the hierarchical recursive structure within the scale so I do I don't see the moon thinking or planning I don't see the weather planning. I don't see evolution planning. It doesn't think about its future. It's too big. And I would imagine that you could apply the same arguments to globalization and institutions that get too big for their own good because they can't plan anymore. So we're coming back to the pilot who's really the intelligent person, not the flotilla. Right. Yeah, in a way, I guess maybe it's slightly depressing to me because it means we can't build an intelligent intergalactic civilization. It's almost like beyond a certain scale, we just have to leave it up to kind of distributed, emergent processes that may or may not end up doing something intelligent as a whole. because at some point you got the speed of light and that basically going to be a barrier to organization across light years So maybe there is an ultimate Goldilocks limit to intelligence It can't get too big. That's certainly what the arguments from the physics part. So there is a Goldilocks of scale or zone in scale space. I'm not sure this is correct, and I don't want to disappoint you. Don't worry. It's well beyond my lifetime when that will happen. There's a certain beauty, though, in federating and distributing and thinking in terms of ecosystems of things at a particular scale. Evolution is a beautiful thing. Yeah, yeah. But it doesn't have to be intelligent to be beautiful. Correct, correct. Well, good point. So it could be beautiful but not intelligent. We could have a beautiful distributed civilization. That's fine. I'm happy with that. On the internalism and externalism debate, I'm a huge fan of Andy Clark, as you know, and he famously argued in his, was it 1999 paper with David Chalmers, that the phone extends your mind. And even when we were talking about the plant example before, when we were talking about plants doing modelling um there's a there's an inactive interpretation there and keith and i were arguing about this he doesn't agree with me but um is the model the the physical morphology of the plant you know it fits the environment like a key in the lock or is the model the software is it the dna of of the plant and and indeed it's just fascinating isn't it that that you can think of cognition not being entirely inside our heads, but it's just this unfurling causal bi-directional process of so many things around us. So how do we actually draw boundaries around things where we say, okay, well, here's the principle modeling and cognition is in here. This is where bona fide beliefs are happening. This is where bona fide thinking is happening. Is it even possible to make that distinction? um i think i think it is um i'm answering in an almost trivial way that if you want to talk about something you have to be able to define its markov blanket so yes you have to be able to draw lines bounds and lines around things to talk about them i think that your question though speaks to um something that we were that we've just been um covering which is the um the separation of scales and that everything, everything literally from the point of view of the free energy principle that is equipped from the Markov blanket is contextualised by a scale above and the same rules apply to the scale above as to the thing itself. So you have to have a context in which everything is operating. So the virus may not be intelligent but it certainly has to be living in a world which is conducive to its existence, and that world and the states of that particular world at the scale above, for example, the host cell and the host organism, has to comply and be intelligent in some sense in order for the virus to be there, even if the virus itself is not intelligent. So applying that to the plant, but we should come back to the extended cognition and Andy Clark but to the plant thing you made an interesting point is it in the DNA or is it in the morphology and the phototaxics everything else that plants possess and we infer goes on on the inside first of all I think you're absolutely right to say yes it is in the morphology And in a sense, that's what I was getting at when talking about the importance of mortal computation for machine consciousness. That you have to realize a Bayesian mechanics or self-evidencing, you physically have to parameterize your conditional probability distributions, your Bayesian beliefs about the environment to which you are coupled. So, the substrate is the parameterization, so by definition it's substrate-dependent computation, which means the structural form, the morphology, is the structure of the generative model in the spirit of structural learning. So, to be part, very much in the spirit of the good regulator theorem, to be in my world, my world, I have to physically encode and embody a model of the structure of my world, which means that I have to have, if you like, if you think your world is scale-free, that means that my brain must have some scale-free hierarchical aspect, and indeed it does. You can actually write that down and model it with a renormalization group. And that's just a reflection of the fact that I'm immersed in a world that has some scale invariance in it. So that is certainly true for the plant. And just one little aside here, that paper was written before David Attenborough's series on the life of plants, where he was able to speed up by a factor of 10 or 100. And of course, if you look at plants doing stuff, they're conspecifics by sending their little roots off in a particular direction or eating insects or whatever. If you speed it up, these things are very animalistic and they look you'd be hard pressed to say that they weren't intelligent, irrespective of whether they're conscious or not, but they certainly start to look much more like you and me when you speed things up. So yeah, their morphology matters. Matters more than one could possibly imagine because this is how you become a good regulator. You model your environment. You are, you install that cause-effect structure into your computer architecture, which is, again, the argument against von Neumann architectures, which is, you know, which is why one might, I think, look to all the processing in memory, neuromorphic photonics, possibly quantum computation, but I think that's gone off the boil recently. But all that processing in memory stuff, memoristers, for example, I think that's where the answer will be. Is it in the DNA? So does the DNA cause a structure? so your comment on the expression make that point yeah so i think um i think some of the hang-up or crux may actually be like what we mean by model and i'm sure this is a whole philosophical debate that i'm probably not qualified to you know uh decide but i think for example um you know if you're flying an airplane like the airplane internally has a computational model of the airplane and you're interacting with this computer and through controls that then are enacted through all kinds of gears and cables and levers and things like that. But I think of the model as being the thing that's in the computer and then it's modeling the airplane, right? And so in that sense, you know, the DNA of a plant species must incorporate the code for the model, right? Because that's what unfurls is the plant. It's the same DNA in every cell that's enacting a program that's essentially happening through gene expression, you know, and so a particular cell, maybe some salt content is in there, which causes it to release a hormone that binds to all the other cells in the plant, which activates certain genetic pathways. So I kind of think about in that same analogy, you know, the core of the model is the code, you know, that's running and operating and controlling these processes. And then I think of the embodiment of the plant as the stuff that the model is modeling, right? But I guess, you know, there's some vagueness there too. You could, if you wanted to simulate these kinds of things, you could read the DNA as the code that specifies, the priors that specify the structure. And, you know, I think that if you think of DNA as prescribing the structure of your generative model or your world model or your factograph that will be fit for purpose and is learnable. So you start off with, say, DNA. DNA does not tell you what particular kind of plant you're going to be. It's not going to tell you how to engage in your phototaxis and point towards the sun or compete with other plants that are trying to deny you sunlight. That is something that you have to update and learn during your particular lifetime. But you're equipped with the basic structure, the prior on the structure of your generative model. And, of course, that is most graciously accommodated, I think, again, with respect to the renormalisation group. We're just talking about two scales. So you've got a slow scale, where you've got the slow... You mentioned before viral mutation. So, the viral DNA, should it have RNA or DNA, is changing very, very slowly on a time scale that is greater than or equivalent to the lifespan of any given virus. And that's, if you like, specifying the initial conditions, the structure for the specification of a particular instance of a virus that lives. Well, maybe just jump in for a minute because I think actually it's not even specifying the structure. I think, and this is what you pointed out before about the difference between, say, active inference versus modeling, is actually the DNA is specifying a policy. Yes. It's effectively just specifying a policy that every single cell in the organism follows. And so this policy is not instructions for a structure, but it's just instructions on what to do in response to a certain set of sensory states, right? So it's almost, it's the policy that the DNA specifies, and this policy applies to every cell. And then somehow, miraculously, it works out to create a structure that is fit for purpose in its particular environment. Yes, yeah. And of course, to work, that specification has to change at the same rate at which the environment changes, which is very, very slowly. I think that's absolutely right. I mean, just practically, that hits you in the face when it comes to thinking about how you commit to one policy or another policy. So in my world, that would be the expected free energy. The expected free energy comes in two parts. It has a sort of expected information gain and the expected cost or constraints or utility. And that is at the point you need your DNA. You need to know what it is to be the kind of thing that I am. What do I not do and what do I do? One of the fascinating things in organisms is this is different even on a cell-by-cell level, right? Because morphologically, the same code evolves into all the different organs and parts and roots and, you know, bark and whatever else, which is pretty fascinating. And, you know, there's exactly the kind of, you know, fascinating issue that preoccupies Mike Levin. You know, and how do you? I spoke with our mutual friend Maxwell Ramstead the other day, and he actually gave a wonderful description of the free energy principle. because you know what you want in a good pitch is for it to be replicable, easily understandable. And I'll try and recapitulate it to you just to test how replicable it was. He's, you know, second law of thermodynamics, closed systems. What if we have open systems with boundaries? And he was saying when things can't merge together, they instead share information with each other. So there's this informational synchrony. and that was his way of describing the free energy principle but first of all is is that a good description but number two the boundaries where do the boundaries come from because we want to have practical implementations of the the free energy principle and of course we can we can implement it with computer games or contrived environments where the boundaries are clear but what if i want to build a robot and i have a camera and i see all of these pixels on the screen and I need to divvy that scene up into objects so I can start doing the free energy principle. How do we do that? Right. Five minutes, five questions. It's got to be practical. That's why we're only giving five minutes. Right. First of all, the notion of being separate but part of a universe through synchronization I think is absolutely correct. and could be unpacked in two ways. One, you could say that to find a free energy minimising solution to your dynamics, namely the path of least action, just is to evince a generalised synchrony between the inside and the outside. Generalised synchrony is also known as synchronisation of chaos. So it's basically the two systems that are loosely coupled in the sense of dynamical systems will ultimately converge on a synchronization manifold and they will show chaotic dynamics on that manifold. That manifold is the synchronization manifold and it is being on that manifold that is the minimization of free energy. So you can talk about the free energy principle without mentioning self-evidence, without mentioning Bayes, without mentioning predictive processing, or even extended cognition. And you can just talk about it as a variational, specifying a variational bound on the Lagrangian that specifies generalized synchronization or synchronization of chaos. And what you're talking about is exactly this sort of separate but the same. I think quite nicely, although I don't fully understand it, recapitulated in Chris Field's quantum treatment. So what he would talk about there is not the synchronisation between the inside and the outside, or me and you and everything else like me and you, but entanglement. So the principle of unitarity is this basic... Well, you can read the free energy principle as just the principle of unitarity, which just means if we stay here for long enough, for more than five minutes, we will ultimately become completely entangled, which means classically we will be engaged in a generalized synchrony. There will literally be a synchronization of our itinerant dynamics. We will not become one. We'll still have a... Well, actually, Chris Fields thinks he would become one, but we will be indistinguishable because we're just playing out on exactly the same synchronization manifold. So I think that's absolutely right. Your last question, and you had about three in between, was how would you practically use the free energy principle or the notion of thingness that inherits from Markov blankets practically if you're building robots? I think you're talking here about physics discovery or Markov discovery algorithms, which of course Maxwell and Jeff Becker are furiously working away on to try and... Assuming you didn't know anything about that stuff, I mean, how would you approach that problem? From first principles, if you had a robot and a camera, how would you partition the scene into boundaries? You would be appealing to the notion of Markov boundaries. And, you know, I mean, you can turn it on your head and look at image segmentation as just, you know, what has emerged in computer vision as a way of identifying things. so what you're doing is you're committing to um an implicit world model or generative model for your robot you're assuming that it is going to be operating as a good regulator as a good model of its environment in a in an environment that is composed of things yes but i think um my intuition is we've been talking a lot about understanding and creativity recently and i i intuitively feel that understanding is about knowing the history of something it's about knowing how you got there It's not about knowing the state. So it's a little bit different from segmenting an image. To understand the history and the dynamics of a system seems to be much more powerful. Absolutely, yeah. Yeah, absolutely. Which is why, you know, Markov blanket discovery is not image segmentation because to do Markov blanket discovery, you have to look at the dynamics and the history. I mean, there's quite a fundamental point to be made here. that we're talking about, we have been talking about from the beginning right through to the end about conditional probability distributions. Probability distributions over what? Because there's only one state of the universe at any one time. There are not many worlds. There's not an ensemble of universes. Of course, for the purpose of this argument, there's only one state of the universe at any one time. So you can't have a probability distribution unless you invoke some sort of ensemble, chronical ensemble assumption, as they do in thermodynamics, where there's some exchangeability you can swap around universes and have a distribution. You can have an epistemological probability distribution, right? I would argue, though, that what you're implicitly doing is actually doing that over time. There's a history, a past and a future. That epistemological notion really has to refer to what is the background over which these multiple realisations occur from which you can select to build a probability distribution. And, of course, from the point of view of the Fientist principle on also building robots, this has to be time. So by definition, it has to be in the dynamics and the history and not just the short-term dynamics, but this recurrence, the characteristic states that I keep returning to as a good robot. So yeah, absolutely. So segmentation algorithms are a good start, but they're going to be completely useless when it comes to autonomous vehicles, for example, unless you've built in the fact that there is conservation of this particular Markov blanket over time, which of course they do. but it's probably better to start with the generative model that this two-dimensional RGB feed has been generated by Markov blankets that by definition persist over time because that is that time is the support of the conditional distributions that define the existence of the Markov blanket and then it's a question of how many what kind of prize do you put on these things are they things, are they stuff you know, what's the Markov blanket of water, or perhaps a fog, you know, if we're doing autonomous... And do they... To what extent would I now apply my priors to these things? And you can go right through to sort of object-centric priors, you know, of the kind, you know, sort of physics engine-based stuff for the kind that, say, Josh Tenenbaum pursues, assuming that you've got some Newtonian behavior, or you could be much more relaxed. I would relax, but not beyond the renormalization group. Professor Fristin, it's been an absolute honor. Thank you so much for joining us today. It's been amazing. I really enjoyed it. It's lovely to speak to you both again. Yeah, and it's really been an honor for me because this is the first time I've met you in person. And, you know, I've been thinking about you for many years, so it's been really a pleasure to meet you. And thank you for coming to the studio today. Thank you for coming to England. Absolutely, anytime.