BI 249 マイケル・コール:脳のフローこそが認知の単位である

Regions that are very, very consistently more active when there's something challenging, some challenging goal you're pursuing or it's novel, that takes up about 40% of the brain. One kind of unspoken assumption in a lot of cognitive neuroscience is that if you see an activity in a brain region or a neural population go up, say for a given test condition, there's some unknown within region process. Like if it's the fusiform face area, there's face processing going on. You can generate face selectivity in the fusiform face area from other regions just based on the flows to the face area. So the computation isn't inside. That's just indicative of like one One of the most well-localized functions in the human brain, just over two decades of work showing us, is actually a distributed process.

Current AI is kind of untethered relative to how our brains work in the sense that they don't have a proper global flow hierarchy. Like our global flow hierarchy is grounded at the bottom by perception action cycles were embodied, and that's important for preventing what's called hallucinations. And then on the other side, at the top of the global flow hierarchy, we have core values that shape our goal pursuits. So they don't have either. They might follow the sides of the pole. Exactly. They're just like the middle part that's like. Just bringing it back. That is not good. This is Brain Inspired. powered by The Transmitter. Hello, I am Paul Middlebrooks. Michael Cole is a professor at the Center for Molecular and Behavioral Neuroscience at Rutgers University. After 10 years running the Cole Neurocognition Lab and reflecting on the work that they have done and reflecting on the field more broadly,

Michael decided to write a book, Brain Flows, How Network Dynamics Compose the Human Mind. In the book, Michael argues that brain flows are the fundamental units of brain function and therefore cognition. We discuss what a brain flow is, what a brain flow is not, and how viewing cognition through the brain flow lens differs from other approaches, like viewing populations of neuron activity as fundamental units of cognition. In fact, Michael has found that most of the signal processing going on is between areas of networks, not within the areas of the networks. The first part of our discussion covers those kinds of basics. Then we move on to discuss how different organizational features of brain flows, like compositional hierarchies, might explain, quote-unquote, higher cognitive functions, like value-based goals, emotions, creativity, consciousness, intelligence, and mental disorders.

In the show notes, I link to the book and to Michael's website. Those are braininspired.co slash podcast slash 249, where you can also decide to support Brain Inspired on Patreon if you're feeling grateful and generous. There are some bells and whistles that come with that. Thank you to my Patreon supporters and to the transmitter for their support. Okay, here's Michael. Michael, the book is called Brain Flows, How Network Dynamics Compose the Human Mind. We were just discussing it's been four years since you were on this podcast last. And a lot of what we talked about on that last episode is in this book, actually, which I found interesting. And so we might cover some of the same territory. I mean, I would assume that we will, but that's okay. It's been four years. So welcome back and congrats on the book. Thank you. And congrats on having an approach that seems so fruitful and will just generate lots of productivity for you for years to come.

So that must feel pretty good as a working scientist. Yeah, definitely. I'm hopeful that's the case. That's the ambition, I guess. But I think it's possible. You know, I thought it through and spelled out the vision in the book. Yeah. Well, and there's a lot of vision to be a lot of fruit to bear. Okay, so I'm going to kind of zoom out and start with very broad, and then we can kind of get down into the details of brain flows and their promise, et cetera. And I'm going to start just by reading a quote, I think, from one of the early chapters in the book. The theory, so talking about brain flow theory, the theory proposes that brain flows themselves are the fundamental functional units of brain function rather than brain regions or their localized activity. So that's a big statement, right?

Coming from, you know, the neuron doctrine. And these days it's like population doctrine. So can you just talk about how brain flows, why you've said that, why you feel that way, and contrast it with other fundamental functional approaches? Yeah, sure. So the idea here is to not look just at activity or even activity patterns, manifolds, on one side. And the other side is maybe we should just look at connectivity and get really deep into that. The idea is that the two have to interact to actually generate functionality. And the interaction is of activations or activity levels and connections are what brain flows are. And it's not anything that mysterious. It's really a perspective shift because how are brain flows implemented? their active potentials, and that's, you know, neuroscience 101.

But what we've actually been doing mostly in neuroscience is recording activity and trying to make sense out of the activity, and then separately looking at the connectivity, and not really looking at these flows that are the basis for generating different functionality, like, say, transformations of representations between brain regions. That's really about the interactions between the brain regions in a dynamic sense. And that's implemented by these brain flows. Yeah, so the word flow actually appears in the book over 2,000 times in one form or another. Yeah, because I did a couple controls. That's good. It's the title, right? But, you know, it's kind of a stupid question, So the word flow itself, like it tends to invoke, you know, it tends to make someone think, okay, that's cool. It's like dynamics. And, you know, it sort of goes against the grain of a sort of static computation picture where, you know, like, you know, in that quote I said,

rather than, you know, brain areas being the fundamental functional units, right? So we had this idea that V1, visual area 1, does some computations and then passes it on to V2 and it does some computations. But this flow idea is that the computations, if you want to call it that, or the information transformation, the operations, whatever words that we use, are actually in the transit between areas. Is that accurate? Yeah, that's accurate. There's also room for local computations because of recurrent dynamics inside of regions. But empirically, when we've looked, starting with fMRI and then moving on to EEG, and we have one non-human primate study that's multi-region, or it's multi-unit, multi-region. So when we don't model the within-region dynamics, the predictions are still quite good.

suggesting most of the variance is really between regions. The local dynamics seem to be more about maintaining the state of the representations. So they're still important, but we've been surprised empirically that most of the variance seems to be between regions. And so that's why I went stronger on that hypothesis, right? We actually are starting to have evidence for that. One of the things, so you get the reader, or at least I did, and by the way, this book is targeted for scientists, right? I mean, I thought that was an interesting move as well. Talk about that just for a second before we get back to flows. So in terms of what audience I was thinking about, I was really thinking about first-year grad students, maybe advanced undergrads. but really not holding back from really getting into complex topics.

I figured people can skip that part too much. But at the same time, I actually read about some work. I started worrying about whether people actually read it. Like once you get the book, are you actually going to read it? So I started putting some stories in just to kind of sweeten the pot, you know, make it lazier to read. Is that where Dad Day came in? Because that's a recurrent theme. You tell a story about going to your daughter's school on Dad Day and building a unicorn out of some sparse materials, et cetera, and then you return to that theme throughout the book. So was that one of the stories that you decided to frame it? Yeah, and I literally had those thoughts right after it happened. I was like, this is a great example. This is something that's really profound in my five-year-old. daughter was doing, did it preschool where she even just, a lot of the lesson is just not realizing how difficult it is just in terms of say engineering or just, just the

basic science of the mechanisms to do something like know what a unicorn is. It's a very abstract thing, but then also even have the idea that I can make a figurine that represents this abstract thing. Like even just that concept. abstract way too it wasn't a realistic looking unicorn that you guys were that's a unicorn, it has a horn and it kind of has a horse flake head that's a unicorn and it's like oh yeah okay but yeah it's amazing that yeah even just having the idea in your mind but then implementing it too, there's all these this kind of like inverse problem how do you move your limbs to be able to, like, even take those pieces and put them all together. Yeah, so it's a great example in a way that's kind of surprising, right, because you think, oh, a five-year-old could do it, that must not be that big a deal, but it is a big deal. That's, you know, the human brain is an amazing thing,

and it's a great way to illustrate that. Yeah, I mean, it's a big deal when you consider just how many things have to go right in your brain and interact and flow. And because it's multimodal, there's abstraction, there's implementation, like you were just saying. And you come back to this story so frequently when you're talking about the different sort of levels of brain flows and how they're organized hierarchically and compositionally. So it was a really nice example that maybe we'll come back to also during our conversation. But so maybe two things. I'll ask you the mind question in a little bit. So as the reader, as I was saying before, my experience reading the book was you can sort of envision these flows kind of at different levels, interacting, and then there's like these higher-level flows, control.

A lot of it is about cognitive control and controlling lower-level flows, et cetera. But then I also started wondering, like, okay, what delimits a flow? Like, is a single action potential along an axon? Does that count as a flow? Like, what isn't a flow maybe is a question. Yes, there's the practical question of how we measure them. I think that could kind of delimit somewhat. So with fMRI, right, it's pretty indirect, but the idea is you have some activity that stimulus evoked in, say, one set of areas, and you want to basically create a small model of how the activity gets generated in a held-out region. And then based on the connectivity from the source regions to the target,

you can just multiply the activity level by the connectivity and then sum them at the target. And the more recent methods have nonlinearities in as well. So it's inspired by how artificial neural networks function, but these are all empirical estimates. And then with more invasive methods, you can do basically the same thing. You have activity levels. You have connectivity. It's just harder to get the coverage you need to do it well. But that's where the field's going, is having more invasive coverage. So we've worked with epilepsy patient data where you have a good amount of coverage, but better ground truth data. Do the same procedure, just much higher temporal resolution, and make inferences about how these flows go over the connections to generate activity distally.

And so I could imagine we have used spiking data, but to get a single spike flowing along an axon, it would be pretty tough, and it's very sparse, so there's a lot of challenges there. But in theory, yeah, you could go all the way down. That's the basis of it. So in theory. But in terms of like what's a, you know, you wouldn't consider a single potential traveling down an axon. That wouldn't count as a fundamental functional unit. So I guess that's what I'm wondering is like where is that fundamental size? Because you're talking to someone who is a neurophysiologist, right? So I'm one of those people who count the spikes in an area and then count the spikes in a different area and do some population analyses on these things. So I think very much old school, quote-unquote, in terms of action potentials. And here you come along with your fancy human methods, you know, that are getting better all the time.

And then in a sense, what you can measure is sort of how you think about what you do, right? So I think about single and populations of spiking neurons. So where does the fundamental start? I do think it's action potentials among axons, but a lot of those don't elicit a neurotransmitter release on the other side, so I wouldn't. I emphasize functional connectivity partly for that reason. It's more than the structural connectivity, more than the action potential flowing along. Does it actually have a causal impact in the distal region? And so causality is at the core of this, which, you know, helps and doesn't help because causal inferences are challenging, but I really think that's where you have to go to have a good explanation. Can you just unpack, I'm going to have to ask you to, like, do some of these things, like what functional connectivity is, you know, just a broad picture for people.

Functional connectivity is the statistical association between pairs of neural populations. Um, we have a paper, um, it's in Nature Neuroscience Read It All 2019 that makes, digs into the question of what functional connectivity is. And it's a bit controversial, but our group thinks of it as a causal inference problem. Um, when people say they're looking at statistical associations, they tend to use causal language in the end. like saying just I want to remove the compound of motion, say the animal or the human is moving, and that's causing the statistical association. They're moving into causal language, basically. There's some other cause they want to rule out. And so I think that reveals that functional connectivity is really what is also called effective connectivity, which is trying to make causal inferences from observation of activity of neural populations.

Yeah, and you go through in the book, you detail how hard it is to infer causality in those types of situations. And that read at all paper that you're pointing to, is that the one where you used time series to help get around that? Yeah, I mean, it's a perspective paper, and it goes over different methods that can be used. But, yeah, usually or always use time series for functional or effective connectivity. Yeah, when we say connectivity, though, you know, we automatically think of, like, architecture, like physical connections. And so to be functionally connected, do you have to be physically connected? that's another controversial perspective we have the answer we would have in my lab is yes they need to be physically connected as well there's this notion of indirect connections which I was on board with for a while so basically if region A goes to B goes to C

then functionally A and C are connected indirectly you'll see that in the correlation matrices most people use correlation to estimate function of connectivity. The problem is that there's another network motif where B causes A and C that would show the exact same correlation pattern. And A and C could have no way of communicating at all. And that's a problem. That's a problem for all sorts of inferences you want to make, especially flow inferences, right? So there's no flow between A and C, but it looks like it is if you think of it as a regular connection. So we're going to have to go through B, you mean? Because the flow is between A and B. I'm saying B goes to A and C. There's no, if you think about it, right? And there's no other route between A and C. That's a false inference if you just use the correlations. So we use partial correlation with regularization is our main approach now.

Regularization is important. That's just a way to simplify the model. I don't want to get too into the weeds but yeah it's basically a way of dealing with the noise and helping with the fit so that they generalize and they're repeat reliable yeah yeah so I want to get into like what that I'm going to ask you to unpack what activity flow modeling is but you talk in the book about how you sort of there are a few key ingredients that led you to end up feeling like this is the right way. This is the approach that you want to take and will be valuable. And you just mentioned like network models, right? Like artificial neural networks was one of those key ingredients. What were some of the other things that sort of came together where you had this, I don't know if it was an aha moment or an epiphany or if it was a more prolonged labored thing, which is the usual story, you know.

But what were some of the other key ingredients that led to you doing what you're doing now? There were many inspirations for the activity flow modeling approach. The main one you kind of mentioned, computational modeling, was very much in the mix. I've done a fair bit of modeling, though I've been mostly an empirical guy, really collecting data and analyzing data. And it was actually the kind of disconnect between the two that was bothering me. Every time I'd try to get into a modeling project, I'd usually be dissatisfied with the lack of constraints on the inferences I could make. I basically wanted to keep myself more honest by getting more and more empirical data into the mix. And so it was about the end of 2013, 2014, I was working on a project, and I just thought, you know, I'm going to take the empirical data, the connectivity activity I'm working with, and just feed it into a basic artificial neural network algorithm.

And then I was actually trying to make an inference about what task functional connectivity was doing relative to resting state functional connectivity. So that's just the correlations when you're performing a task. And you can see that there's some small differences from when you're just sitting at rest. And it was a small there. That was a surprise to you at the time that there were such small differences, right? Yeah. Yeah, there was a study in 2014 that was the focus of that study. And I was just seeing that they're smaller than I expected. And so I thought, well, do they even matter? Is it just kind of a blip? There's no effect on functionality. So kind of in the middle of that kind of question, that kind of like confusion, I started thinking of ways to measure functionality. How do we even define functionality? And I realized that task-evoked activity is much clearer, actually,

than functional connectivity or connectivity in general because there are things like if I have activity in my motor cortex, I actually move my hand. If I have activity in the visual cortex, I actually see something. These are amplitude changes. But those amplitude changes come from somewhere. That's what the flows are. And so that's what I started simulating. And I found actually some inferential issues when looking at the task function connectivity, but I realized it worked with resting states. So that's what I ran with. And in 2016, we had a paper out that just focused on, you know, intrinsic resting state function connectivity and how we could do these activity flow models with that. And then later we made the inference after doing some background work to fix the issues. There's some confounds that we corrected. And we were able to show that those small changes in task function and connectivity actually do impact task-informed activity.

So you're guessing they're functionally important. So then very broadly, because the activity flow modeling is at the heart of everything you talk about in the book, right? So you go on to talk about how activity flows are relevant for cognitive control, which is a huge part of what our brain does, and I agree with you on that. I mean, it would be crazy kind of to disagree with you on that at this point, and some other cognitive domains. But then, you know, in the second part of the book, you go on to talk about how these flows might relate to or generate things like consciousness, creativity, free will you get into, and, you know, and disorders of the mind and stuff. So can you broadly unpack just how activity flow modeling works? Yeah so it just a matter of taking I should back up actually because a lot of the insights in this book didn just come That more of a bottom kind of constraint that is kind of emphasized in the title

But there's a lot of top-down kind of from psychology and the cognitive work that we do in the lab that came in and kind of shaped where the book went. So you mentioned cognitive control. I think of that as a very high-level cognitive thing, and it framed things from the top saying, okay, what are the flows for, kind of? Like, what did evolution select them for? And the argument is that it's for goal pursuit, and that's really at the core of what cognitive control is. It's about having something you value, a whole set of things you value, and shaping perception action cycles and actually frame it, the brand organization, in terms of a global flow hierarchy, which means that there's perception action cycles at the bottom. You can think of that in terms of flows, right? Flows between perception and action.

And there's kind of feedback with the environment, right, where you change the environment and then you perceive that change. So very dynamic and a lot of feedback. And then at the top are values, and goal pursuit is really what bridges the core dynamics of the brain. And it's all implemented by flows, as you say. But it goes beyond the particular methodology of activity flow modeling. That's what I'm saying is a good way, a way that has been working to get at this. but there's a larger conceptualization where we, you know, saying that there's some value in thinking about these feedback loops and large-scale dynamics and also thinking in terms of Mars levels is another way to think of it, like the brain flows or the implementation, and then there are these algorithms like at the next level, certain motifs and network organization, like flow collider is a concept in there

that gets you from, you know, there are flows, how are they organized to actually implement functionality. That just means, you know, two or more flows are coming in, colliding in a given brain region or neuropopulation, and that can generate new functionality. It's related to the classic computational modeling, artificial neural network view of how logic gets implemented and actually how computers work. But that's just one simple example, and, of course, it gets much more complex in brains. Yeah, well, in the top level is the... Sorry, just to talk about the collider more, right? So in a sense, you can think of, like, two areas, right, that then have their own flows converging onto the same area, and they collide. Those nodes are like artificial units, But standing in for that is larger populations within an area, right?

Right. And this is where the interesting transformation happens, just on a larger scale than two single connection weights going into one unit, which is like a classic. Yeah, thousands, millions. Yeah, extremely complex. And I try to get the minimal kind of circuit examples in there. but yeah I want to mention that that's just so we can understand the basis of it but in reality it's much more complex and then that's kind of borne out by thinking about the top level the computational level as Mark called it I think of that as just just goal pursuit towards core values that we have at the top of our global flow hierarchy and this the global flow hierarchy actually can see it in the functional connectivity patterns. It's the prominent... So if you take a functional connectivity matrix between all the brain regions and human cortex and you just run PCA on it,

that first principal component is the global flow hierarchy. You can see it starting in sensory motor cortex and then going up into kind of control regions and the default mode network at the top. And that's where, at the top, you get goal representations and value representations. So it's all consistent just in the actual empirical data with this really abstract notion. But it really aligns with a lot of things, right? Like evolution, if you think about it, right, like the goal of survival, all these survival goals we have is totally consistent with what likely was selected for in evolution. and so it makes sense that the brain would be organized that way. I mean, it begs the question, right? We can go on this infinite regress of like, okay, so it's the values and the goals that are shaping these top-level brain flows. And sure, we got those through evolution, but like how does it get set?

What's controlling the controller, you know? This is an impossible question. Somehow they're there. Yeah, somehow. I mean, and then there's individual differences in them, so that's interesting. There are a lot of commonalities, right? I just mentioned survival, right? People try to survive, but there's all these other, especially once you in society don't have to worry about food and shelter. There's all sorts of things that open up, but lots of interesting questions along those lines. Yeah, I'm glad you reoriented to talk about sort of the top-down way that you've approached this because I was asking you, like, all right, so how does the activity flow modeling work? I didn't get into that, but just, yeah, why not? Yeah, yeah. No, that's great. That's great to reorient us because, I mean, as you were talking about that,

You talk about the flows as sort of Mars implementation level. And, you know, a popular thing to do these days is use deep neural networks as models for brain processing. Right. And so that's sort of an implementation level approach from the neural AI, quote, unquote, approach. Right. where you're using deep nets to either as models for brain areas or to constrain the modeling approach or simply to, you know, match the data, et cetera. So maybe if you describe a little bit how the actual brain flow activity modeling, activity flow modeling, sorry, works, I wonder if it would be useful to contrast it with some other approaches as well. Yeah, sure. So you could think of it as, or I do think of it as the deep neural network approach, but just constrained by empirical connectivity and activity.

But here the units aren't supposed to correspond to individual neurons, right, like they are in classic deep neural networks. It's a graph neural network approach, but where, again, the nodes are like populations. Yeah, so the typical thing is we'll have one node per brain region and then have functional connections. It also works with structural connections, as in the predictions still come out above chance. One thing that's really useful, Red, is we have with something like fMRI or if we just source localize DEG, we have the actual answer to the question of what the activity is in all these distal regions, and we just hold them out and take the source activity, multiply it by the connectivity, and then do a summation to make a prediction at each target.

You can just go around. It's based on how machine learning works with, you know, classifiers where you hold out some data and then do a prediction and then see how accurate you were. So, yeah, it's kind of predicting the activity levels, right? You're not predicting some output. You're not predicting next word token, you know. You're not predicting the label or the image. Oh, yeah, it's activity levels, but embedded in activity levels, you can then – use all these sophisticated pattern recognition kind of methods like decoding. So you could look at representations. We have a paper coming out in Nature Neuroscience any day now that does this for across, what is it? I want to say it's like 12 different tasks in the same individuals. and we look at the flow of representations throughout all the cortex,

and we're really looking for generalities and the representations in these transformations between brain regions in terms of activity flows. And, yeah, it's highly related to deep neural networks, just more constrained, and that constraint actually helps us make inferences so that, you know, So like with typical deep neural networks, you actually do have flows in them, right? Because you do the same, you know, there's a source activity, multiply them by the connectivity, and then that generates something in the next layer. The issue is if you look at them, I don't think, I guess this is an empirical question, but I don't think those are going to correspond to the actual brain flows because they're under constraint, that there's like a degeneracy, right? There's so many possible connectivity patterns that could generate the same functionality that it's unlikely that those connectivity patterns correspond.

Plus, each node in a deep neural network doesn't correspond to a single, typically, doesn't correspond to a single node in the actual brain. So it's more grounded, I guess, in the empirical data. and that has a lot of utility. Yeah, for sure. I mean, yeah, by constraining it to the empirical data, you are grounding it. But one of the things that was surprising, you know, modeling all the way from, like, the early visual deep neural network modeling is that, in fact, when you do look at the activity of many of the units, they do seem to correspond to different brain areas along the visual hierarchy, right? And so that was kind of a surprising finding. But you're totally right that it is degenerate, and it doesn't have to be that way. And so what you're ensuring with this approach is that it is that way. Yeah, and then you can start asking questions about functionality by manipulating the model

and looking very carefully at extra generation of functionality. And I should mention we've also gone all the way to motor cortex, and it can actually generate behavior where we decode motor responses just like we do in actual brain data and just apply it to the model, and then you can see it can perform tasks, pretty simple tasks at this point. But in theory, yeah, we can do more complex tasks in the future. One of the other things that you talk about that's important, at least from your perspective, is to have a generative understanding. And this approach sort of is maybe unique is a strong word, but one of its strengths is that it provides a generative understanding. So what do you mean by that? Yeah, so it's really a lot of neuroscience is looking at activity patterns

in very sophisticated ways, and it's very useful because you characterize the functionality. But in the end, even if you knew all of the activity dynamics and even all the connectivity, you wouldn't know how the functionality that you characterize in the activity dynamics actually came about, like what generated it. And so the advantage of looking at brain activity flows is that you can start to actually generate the functionality. And you can ask questions like about causal sufficiency. Is my explanation sufficient to explain the generation of the functionality? Or is it just correlative or something like this? So it gets kind of deeper into understanding. I have an example in the book of, like, if you're trying to understand how a car works, and I use this relatively simple example on purpose in the engineering system,

because if it was some kind of alien artifact, right, you would come up to it and you'd say, like okay I think this wheel turns the car and I think this pedal makes it go and this is like some understanding it's important and I know the pedal connects to this thing called the engine and that's important for making it go So, but it's not until you actually have causal flows, quote unquote, between these elements that you're going to really understand how functionality is generated. Like going 60 miles an hour in a controlled manner on a highway or something, you really want that generative understanding to really understand it so you can fix it if something goes wrong, for instance.

So that's the same for the brain. Yeah. So you said fix it there. So maybe the main thing that's important for a generative understanding is for treatments of disorders, et cetera, and diseases. Yeah, I think it's more fundamental than that. But, yeah, that's the practical. That's very motivating, right? That if you have that kind of understanding, you can have more of a handle on how you develop treatments. And there's a very practical thought, too, related to that, which is if you took this activity flow modeling to an extreme, you could have a model. I guess these days some people are calling it like a digital twin brain kind of approach where you would reconstitute the key details in an activity flow model that are in an actual brain. and then you can test novel treatments that way or discover them maybe.

Because you have the generative process in a model, you don't have to go and poke and pry the actual system with a thousand possible interventions. You can at least narrow down the possibilities in a model. What would you call what neuro AI calls digital twins? Would you use the same term? Yeah, it says the same thing, yeah, just in terms of this activity flow approach. I mentioned before that you apply the brain flow theory ideas to lots of domains of our cognition throughout the book. So I'm not sure which one of those we want to tackle first. Maybe one way in is to ask you, like, which chapter or which cognitive domain you found most difficult to write about or think about in terms of how a flow-based approach can account for it explanatorily? Yeah, so, yeah, it sounded like the cognitive control stuff, like, is, like, you have a deep history in, right?

Because you came up studying working memory and cognitive control under some big names throughout your graduate school career and beyond. So I sort of imagine you easily writing through that because it's part of what you've already studied, right? So a lot of, you know, the first part of the book is a lot about what you've already done. And then the second part of the book is getting into a little bit more to speculation about how these ideas might be applied to some of the bigger questions, like consciousness and free will and creativity, et cetera. So, I mean, were those? Those were definitely challenging. I think the chapter right after the cognitive control chapters, that's chapter three, was probably the hardest. It was kind of like I started writing it, writing the book before, at a sabbatical for a semester to help me have time to write the book. And it wasn't until I was writing chapter three,

I got at the beginning of the sabbatical, and then I'm like, here I am, deep in it. This is my main focus for my life for a living. And then chapter four hit, and that was like really about, think about compositionality and about novelty. Yeah. And now that was challenging. I had to like expand my thinking for that one. Oh, okay. It's at the stage for the second half. Ah, okay. So maybe I should start about compositionality. So the chapter three is about pursuing goals and the importance of pursuing goals, which is what you alluded to earlier, that sort of top-level thing that's, like, orchestrating from top down the higher-level brain flows that end up compositionally or hierarchically constraining and controlling, like, the lower-level brain flows. Right. So that's what you do. Yeah, I've been thinking about that for, like, two decades or something. So it was like, there's a lot in there. And I think there's, you know, there are some new ideas. But, yeah, it wasn't until I got to the next part, I was just like, hold on.

I got to really think this through because the other theme that's been running through my work is novelty. Because the main thing I did in my dissertation was taking cognitive control and really looking at how we learn new tasks from instructions. and so I thought a lot about novelty and how essential that is to goal pursuit kind of like the lack of novelty is the opposite of goal pursuit in some sense you know automaticity so when we've we have a habit or something like this we've done things a thousand times that's a very different brain flow process than if it's something novel and you need a lot more uh complex functionality for that. But then there's a deeper thing there, right? When like the fact that each moment in our lives is, is novel technically, like even if you're doing something repetitively, the fact that your brain is changing as you go, right, is, is making it novel. And, and the fact that just things in the world are changing. Um, and so we always need some kind

of learning transferred into every, every new situation. And then the more distinct each situation is from before, the more cognitive load it takes in terms of control and goal pursuit. But how do we even do that? And the answer I've been working on and that I've dug into in chapter four is that compositionality is the key. So that means we have these representations of the world around us that are in terms of components, right? So things like the concept of it, and they're kind of abstract, right? Like the concept of an apple or a tree or, you know, a face or something like this. And each kind of scene or situation is a conjunction among those components. And so I started thinking pretty deeply about, in Chapter 4, about that and really realizing it's, our environment needs to be a certain way for that to be useful, right?

There needs to actually be stability in terms of components and the kind of combinatorics of them that we can comprehend. And so I dug into that, like thinking beyond the brain and about how you can think of it in terms of a nested hierarchy, right? So you have something like a tree that's really composed out of atoms and molecules and leaves and a trunk, and we're able to understand things that way. And that's really important for us to understand a new kind of tree, for instance, or a new kind of – and that same process applies quite generally. and so out of that kind of conceptualization um that led into chapter five that was really how cognitive levels of organization get built in a similar way out of components and it's really about our cognitive control ability to make comparisons and work in memory that can help us with that so

we have abstract concepts that we can build on and generalize in all the situations. And then that set the stage for the second half of the book, where we could basically reuse those kind of concepts. It's kind of a meta thing, right? Compositionally reused. The concepts from the first half of the book to kind of explain the second half, where you have these high-level things like consciousness, intelligence, mental health, free will, creativity. So, yeah, we can dig into that. We'll dig into a little bit more about how novelty arises. So it's a bit of a paradox, right, because something – okay, so novelty is defined in different ways, right? But it's something new. But you have to have these sort of subroutines that are old, right, and you have to have a bunch of them. And is it just recombining the old automatic flows and that's how you get something new?

How does novelty work? Yeah, so I have a bunch of examples in the book. One that's maybe more intuitive because we're doing it right now is we're making sentences that are novel. And we have these words that are well-practiced. And they're automatic, right? And that's indicative of the kind of flow processes that I talk about in the book where we need automaticity, we need extensive practice in order to build these components, which then we can reuse to make novel things like use sentences and understand new sentences. And if we didn't have that kind of like automatic process, then we wouldn't. We'd be thinking about each phoneme, right? instead of the whole word or how they all fit together. We're able to function at this high level because we have these automatic flow routes that we're just really rapidly without conscious effort.

It generates the meaning. It's interesting that we don't consciously think about the phonemes. We consciously think about words or phrases or sentences. It's an interesting problem to think about where mind is, right? So what we can pay attention to, and in some sense this is about the affordances, you know, available to us and at what level we need to act on for things to be useful for that survival and the values and goals, et cetera. I don't even know where I was going with that. If you have a comment on that, you can make a comment because I realize I didn't ask a question. yeah um we could get into one of the topics on the last half of the book i mentioned consciousness i guess consciousness kind of goes down with practice which is interesting to think about you can see it in the brain activity too it'll reduce actually the so when something's novel much more of the brain is involved and a lot of cognitive control regions actually so that something I didn mention is that Yeah yeah So why are cognitive control regions so involved Why do they take up so much space in the brain

and why are they more active when you're aware of something? I mean, there's a lot of mystery to that, to be honest, but that's just an observation. So we see in terms of the resting state functional connectivity networks that the regions that are very, very consistently more active when there's something challenging, some challenging goal you're pursuing, or it's novel, that takes up about 40% of the brain. 40% of the cortex, right? Not of just the brain, of large. There is, we included subcortex, but it's a corticocentric view. we had this network definition that we defined in cortex, and then we extended it to subcortex, kind of forcing each part of subcortex to be part of one of the cortical networks. So it's definitely biased in some extent. But if you had to say, you'd say.

So for instance, one thing that was surprising, if you do that, most of the potamen and most of the caudate and the basal ganglia are assigned to counter-control networks. And I think that makes sense. I mean, there's a lot of evidence that those subcortical regions are very involved in goal pursuit, and they're not just motor only. That small portion was mostly assigned to the motor network. But I think there's some really deep stuff to be discovered about the kind of control networks that we just have no idea. So why so much? I don't really know. If you assume more cortical space means more importance or more complex computations, it's an interesting wrinkle in the mystery. But it does make sense in some ways why it would be much more complex, right? because you don't have these well-worn pathways to work with, these flow routes,

when you have a novel situation or a novel task. And so you can think of it like you need a kind of workspace of possible combinations. There's vast combinatorics to work with with novelty. That's kind of how it's defined. That's one way to think of it is like there are these components in your environment. And when it's novel, really it's some recombination of components. And then so, yeah, working memory is highly flexible. It's limited to around four items. You can use some tricks to get up to like seven or a little more, but it's around four items. And one new concept that I introduced in the book was the zone of proximal compositionality, which is the idea of given what you know and what you've kind of had these automatic, well-learned concepts, representations that are compositional,

what's the set of concepts that you can recombine them to form within working memory. And so that's important for being able to understand something, right? If it's beyond, if it took like seven or let's say ten things all combined together, you're just not going to be able to understand that immediately. Not everyone fails to hoop, but you could train yourself over time, right? You've got to combine these and then combine these. Yeah. Yeah. Yeah, that's interesting because it's related to, like, that's exactly what I was thinking about when you were just talking about novelty, that in some sense we are limited by what we have already stored, our routines, those flows that are already well stored. So, you know, this is related to the idea of, like, the Stuart Kaufman's adjacent possible, right? It's like the next novel thing that you can even generate is limited and constrained by the tools that you have at your disposal at the time, right?

So it takes time to – that's an interesting limitation almost to novelty. Yeah. Yeah, it's limited and, you know, it's getting into speculation a little bit. But computationally, it's extremely challenging to think of, to be able to represent. So think about the concepts we do have well-learned. Say there's 10,000 of them. It's probably the scale that we're at, something like that. And then if you do the commentatorics on it, it's billions of possibilities. And so you can imagine you need this kind of vast workspace to be able to even just move through a day with, you know, if you're, just imagine walking down a city street and all the novel things, technically novel things you're seeing, like just a person coming by. You've never seen them before. And you have to like represent this novel face and they're doing something you've never seen before.

And the whole combination of the sounds and everything. that's a huge challenge and to really understand it you need this really big workspace just to have this really big zone of compositionality so you can have like even with it only being around four items at a time it's still really big because you have a big set of possible items that could all be combined into sets of four Right. Okay so why Michael, why am I so much more creative than 99.99% of other humans? Why might that be flow-wise? Let's see, Paul. No, so I don't know if you want to talk a little bit about creativity, or we could go down the intelligence and consciousness road, but I just figured it's a segwaying from thinking about novelty and the target-cultural aspect of it. So it really is an easy transition from this zone of proximal compositionality to creativity.

I thought, so again, everything is based in flows, but I also have these high-level constraints of goal pursuit and counter control in the mix. and the idea here was to think in terms of the fact that most of the flows in our brains at any given moment are spontaneous. Most of it is random for the most part but shaped by these kind of flow pathways that are already present. So we can see that with the functional connectivity matrices. We just have someone rest in the scanner. but it could even be of light anesthesia, and you could see the same thing. There's just this spontaneous activity, and you get these resting state networks, as they're called. But what are the spontaneous activity? What's it doing? And one utility of it is for things like creativity or just generating at the base,

just like deciding how to act. And there's, is it kind of degeneracy even at how to move your hands to go pick up something. And it's kind of like pushing you through that safe space. A lot of it is the spontaneous activity to just be generative, to produce something. And so that's the idea for creativity. And the last chapter of the book is that you have this kind of random variation, kind of inspired by evolution, right? You have random variation, and then the values you have, the goal you're pursuing, that is the selection on top of it. And so you can think of it like you have this zone of proximal compositionality with your working memory. You have all these different combinations of things you know or actions you can take, and then you have this kind of pseudo-random search through that space from the spontaneous flows, and then you have top-down flows from your value representations, the top of the global flow hierarchy

that is going to constrain which ones you choose. And you may even, like how you do brainstorming or something like that, you're just thinking of different possibilities. It can be iterative, right? Like your kind of idea will pop up and it could be a little off and you get feedback from whatever your goal is. Is that a good idea or not? And so there's an operational definition of the book of what creativity is, and it's really a new idea that's useful. Utility here is defined by the goal you're pursuing or the values you have. Right. I think useful is an important thing there because I was going to ask you, Like what is the just right amount of spontaneous activity that you want, right? And I think that depends because you can imagine with no top-down goal pursuing constraints,

someone who could be highly, quote-unquote, creative, and that they could be coming up with all sorts of ideas, none of which are useful, right? So that's where the usefulness comes in. So there's probably some sweet spot of amount of spontaneous activity that you're peppering in with these top-down constraints. Is that the idea? Yeah, totally. Yeah, and there's also this, it feeds into compositionality. So these concepts that are reused tend to be, they basically have to be the components that you're recombining, right? So you want to have, yeah, that could be worked backwards, right, from where a good creative idea came from. You could say that they had the right components. They put them together in the right way. Beforehand, it's hard to know what the right components are, but it is useful, I think, to – the framework gives a good setup for starting to understand creativity

where it's compositional representations. It's about having this zone of proximal composition. where you have it's limited but vast still, and then there's these random pseudorandom flows that are going to be kind of bubbling up, and you get new ideas from there, and then you evaluate them based on your goal or your values, and you just can keep going and try new ones. And, yeah. Okay, let me ask you this. So this is a favorite example of mine of how creativity works in one domain, because this has happened to me occasionally. I'm not a professional musician by any means, but, you know, when I'm practicing more on the guitar, let's say, occasionally something feels like it just drops out of the ether and, you know,

this is the solution to the musical songwriting problem that I'm having, right? And that's like, that's such a common thing that you hear in musicians. But so where they're working, working, working so hard, they have all these well-honed skills. And then suddenly while they're noodling or whatever, it just seems to drop into existence. Right. And so, but an important component of that is these well-honed skills, you know, hours and hours of practice. It doesn't just come out of nowhere. where somehow you have to have, have to is a strong phrase here, but it's highly, highly desirable or better to have these hard-earned skills that have come from lots and lots of practice and hard work, and then it just drops out and comes from nowhere. So how do you account for that in the flow-based story? I talk about that some, but it's one of those things where it's still kind of a mystery.

I've had experiences, actually thinking a lot about that, I've actually tried a couple times, or periodically, if there's a really hard problem, while writing the book, I did some of this, where I'd be like, I can't figure out this one thing, how do I say this, whatever, and then I'll, but I guess it comes back to something you said earlier, like what's the right amount of spontaneous activity? and what I realized is if I wake up a little early in the morning and I'm thinking about the problem, sometimes my mind will just snap into place. But it required me to practice the ideas. I have the components all there and I have the goal and I feel like I've practiced configuring the brain full of patterns aligned to this goal enough, but I'm just stuck.

And you think of it like, I don't know, in terms of a state space, I'm in a local minimum, I guess. But there's a global minimum that I'm searching for and something about more spontaneous activity, maybe just this excitatory and inhibitory balance or something when I'm just about to wake up. Sometimes it'll just like, oh, there it is. So, yeah, I don't know. I think there's some deep questions about that. But it's hard. Like, how do you even study that, right? I don't know. I guess thinking about it, when people are just about to fall asleep or about to wake up, maybe you could look at the spontaneous dynamics in the states. Maybe there's something to it. But there's preparation, like you mentioned, for sure, where I think it's about the zone of proximal proposition has to be, like, not too far away. But you relax some of the constraints, I guess.

Like, you're stuck because you're thinking about things a certain way. I wonder why it's hard, though. I mean, there is like the, you know, there's the classic going for a walk, you know, or it came to me in the shower. but it's so hard when you're working so deeply on a problem it's really hard to say what I need to do is go on a walk because you're so frustrated going for runs sometimes going for a run and then having your mind not continue to think about the problem back off and let it work underneath Yeah, I wonder about that. It would be really cool to study that somehow if we could get people into that kind of state. And it really is counterintuitive to where I want to try harder. I mean, we could speculate about neurotransmitters too, right? Like norepinephrine is supposed to kind of sharpen your perception and your thinking.

It does something, like it makes the non-linearities more steep, I think, is what some studies have shown. So you can imagine that that could be part of it, right? So you're in a really focused state. You can think about, like, the flows are kind of just, like, more rigid and not shifting around very much. And so that could keep you stuck in a local minimum. and then once you relax, it kind of moves around. You can see that in resting state activity with like fMRI or EEG, for instance, that the dynamics get more fluid kind of at rest compared to when you're focused on a task. Yeah. Yeah, right. But by the way, I didn't mention this before, but you have a really nice overall summary figure in your book that connects all of what you write about in the book. and your conception of how they connect. By the way, how long did that, was that an easy figure to make,

did it just come to you? Yeah, I actually have a story about that figure where I sent the book to our mutual friend Patrick Laurent, and he had some criticisms, and part of it was that he wasn't really following the overall picture, and then that directly, to make that figure, toward the end of writing the book, where I was just like, okay, I'm going to just go for it. I'm just going to try to figure it. And some of those things where, yeah, actually, yeah, it's kind of interesting. It's too detailed to have kind of conceptualized all at once because it's beyond our working memory capacity. So I just kind of like, it felt like, I don't know, like a trust fall or something. I'm just going to start with brain flows at the bottom, that word, and then I'm just going to put little arrows and then it just kind of filled out. But, yeah, I mean, it was challenging, but somehow because I'd already written everything, it wasn't too bad.

Yeah, yeah. But it also sort of crystallizes like, okay, now you have to put your picture where your mouth is or whatever. You're like, okay, now this is my position. Right, yeah. You can't back out of it. Yeah. Yeah, that was kind of, yeah, I've caught myself, I think we all do this, we're like, yeah, I don't agree with that. And you have a strong opinion. And then like a month later, you have the opposite opinion. Like, I don't agree. You know, so I had to like commit. I know. Well, that's the thing is like, yeah, as soon as you do that, then it makes it way easier for people to criticize also. Because then they can latch on to like very distinct things. Whereas, you know, in your mind, you might have been like, well, this is kind of like what I'm thinking right now. Not like it doesn't have to be like, this is the way it is, you know. But, you know. Yeah, I mean, yeah. And I tried to be honest about the uncertainties, too, right? That was one thing that helped me.

Well, that's one of the cool things about the book is that you have these little superscripts throughout the text, right? Like if something has empirical evidence, with an E, if it's supported with an S, which is a little bit weaker, and if it's the P, is I forget what the P is. It's like a proposal. Proposal, yeah. Like a new hypothesis. Yeah. Okay. Yeah, so that was a cool facet of just being able to read that because you wanted to be honest about where evidence was, essentially. Yeah, and be honest about, like, I'm trying to point in a direction. I'm not just saying this is how it is. I'm saying, well, this is how some of it is. And then here's the evidence, and then here's the new things. But I'm also kind of saying, and then I think we should go this direction. Like, even if I'm wrong on those proposals, answering that is great.

That's progress. Yeah. It's progress, yeah. Yeah, I was going to say, too, yeah, part of why I wrote the book, well, yeah, I first had the idea in 2023. It was about to be the 10-year anniversary of my lab getting formed, and I was kind of thinking that summer, right before then, about, like, well, what did we learn? what kind of new perspective do we have, if there is any. And in that context, I was actually traveling. I went to Edinburgh in Scotland, and I recently read some David Hume, and that's where he was, and there was Adam Smith there, and you kind of see him just kind of feeling inspired by that. And kind of his story of just going for it, right? He came out of nowhere, wrote this book, which is kind of the first book on psychology in a way, one of the first.

And it was like bold and wrong in a lot of ways in the end. It was like made progress, right? It got the ball rolling. And that was kind of like I had a lot of these same concerns of like, well, I got to like commit to something. And so I hope to just say, well, I'm just going to be clear about where the uncertainties are. But also, despite the uncertainty, my hope is that it will actually help us make progress. Because even if, like, if in 10 years, you know, I'm wrong about a bunch of this stuff, if people are citing it and saying he's wrong in this particular way, I'm like, oh, cool. I influenced that line of research. Yeah. That's fine, yeah. Before we go on and talk about, because I want to talk about intelligence and consciousness, unless you want to focus on other things, but, I mean, did writing the book change your mind about anything? And or the writing the book itself?

Because the other part, a very similar question, is just what have you changed your mind about from your earlier days? Let's say even in, you know, the first couple years running a lab to now. yeah um i actually on purpose planned to think some things through as i wrote the book it was kind of like a i i tied myself to the mask like as i say that to to force myself to like sit down and think through these things and and have a position and it's kind of a crazy thing to do but It seems exciting. It was exciting. But you had to take showers along the way and take walks, right? Yeah. Yes. Yeah. Along the mask. And turn me and spray me down with some ocean water or something. Yeah. Yeah. So let's see. What did I change my mind about? I mean, all sorts of things.

I had kind of more vague notions. It's all based on things that I, perspectives I had before. I started writing a book, obviously, but I had written a book proposal, outline of the topics. I knew I had something to say about each topic, but exactly what I was going to say, I wasn't sure about. The one thing that really surprised me, though, is when I went to go to write it, It was in the mental health section. I believe that's chapter nine, chapter eight. Yeah, chapter nine, Dysfunctional Flows and the Generation of Mental Health. Yeah, so when I was in there, I started that very cold cognition, cold pursuit, without thinking about the emotional end of it so much, even though, obviously, it's there and it is important. and I can say as much. So they say. But it wasn't until that, what's that? So they say, emotions are important.

So they say, emotions are important. So when I got to the mental health section, all of a sudden I'm like, well, if I think about these mental health disorders like depression, anxiety, generalized anxiety disorder, but even like schizophrenia where you have disorganized thinking is a cognitive thing, the main impact it has is on emotion and you know our emotional experience so then I said I gotta really think about this I've gotta really um say something about it because it's kind of at the core of the whole thing so I kind of gave myself that challenge and then yeah I have this whole new way of thinking about emotion that that isn't really emphasized so much that I didn't plan on it I think it is pretty important. It actually came about when I was giving it, like thinking back, like how did I even start thinking about emotions? I was giving a talk at Ohio State University.

And at the very end, I like had to catch this plane right at the end. It was like kind of mad time where right after the talk, I'm like rushing to get a taxi to get to the airport. And the person that was walking me over to find where the taxi was, he studies emotion. And he like you know one thing that your talk about brain flows made me think about is you talking a lot about abstract representations how those are low activity patterns and flow patterns that generalize So that's related to this compositionality idea. Well, emotions are like that. Emotions are very low-dimensional, maybe, based on what you're saying. And so kind of building out of that in this goal pursuit perspective, the idea is that emotions are extremely, extremely distributed to where they cover our whole bodies and all these concepts. And they're very distributed and low dimensional in terms of flow patterns.

And they're aligned to goals, actually. They're very cognitive, according to this conceptualization. Where if I have an abstract goal, like I want to win at chess, which is not based in evolution or anything. I could get like, I really care about this chess game. And my stomach's knots and my thoughts are racing. You know what I mean? And so like you can conceptualize emotions relative to these different goals. There's different values you have. and so it all fits in this global flow hierarchy perspective and the fact that it's low dimensional is important too related to that compositionality idea but it basically is the idea that you get up high dimensional content the specific situation, all the details but it's kind of flavored by this manifold that's this emotion that's extremely low dimensional

and kind of stereotyped and kind of like guides your goal pursuit. And so that's another idea. It's like it's actually emotions are very healthy, right? Or they can be very healthy emotions where they kind of frame everything, including your body, towards this goal and kind of like guide you towards it or away from it if it's like a threat or something, but depending on the emotion. But anyway, I thought I'd mention it because it's not emphasized in the whole thing that kind of emerged there. I'll just, so that nobody receives flack for calling emotion low-dimensional. It doesn't mean emotions are stupid and simple, because low-dimensionality is from a high, high-dimensionality that leads to a high-dimensional. It's still really high-dimensional, yeah. It's just like, if you think about this situation we're in right now, or I'm in this room, I have all these really high-dimensional representations, but I'm, like, excited.

You know, I have this excitement and emotion about, like, explaining these ideas in this book, and that's framing and guiding my actions. And they seem, well, actually part of what makes them hard to control is being a kind of control, right? Oftentimes it's hard to control emotions because they're so distributed. it's just like you think about there's some localized goal representation how is it going to like change so much of the brain because it's just like all over this kind of yeah low dimensional manifold that it's going to be hard to control typically but it's yeah if it's the right emotion for the right situation then it's very helpful for getting the flow patterns yeah what's a flow based explanation of when it goes wrong when it goes haywire when there's dysfunction, when the emotion is leading you down the wrong path, away from what's good for you or your goals? So there's the automatic flow pathways that get set from habits or, yeah,

this sense of learning. And you can think of it in terms of dynamical systems theory, too, where you have an attractor in that system. And so the flow patterns could just get kind of pulled towards a location that's not helpful. for whatever your values or goals are. And so, yeah, you can think of it as, you know, you have some kind of mental health disorder that could emerge from that sort of thing where it's just an attractor that the brain keeps going back to in terms of flow patterns. And we have the cognitive control system can often get us out of those situations, but sometimes that's not enough. But that's where, yeah, in chapter three, I talk about cybernetic feedback loops are probably the basis for kind of control. You have like a kind of dynamic set point

or dynamic attractor where you have this idea and this working memory kind of workspace of what you want, And then you have these feedback loops with the rest of the brain to try to shift the brain state towards that goal or set point. But it's just like even if it's 40% of cortex, it's still, you know, you may not be able to overcome a really strong attractor in the landscape. Yeah. And that's where intervention could help, perhaps. Yeah. Yeah, like talk therapy or drugs or brain stimulation or something like that. Okay, I'm going to cherry pick a sentence from the book, and then we can go, and then I'll give you some options of which way we can go. The reason why I'm cherry picking this sentence is because, you know, with the rise of artificial intelligence, there's, I don't know, 12 very vocal people.

only 12 but very vocal people worried worried about ai consciousness right um okay so here's the sentence clearly human intelligence is not possible without consciousness right so i'm worried about that sentence oh you did yeah yeah because i did think about the ai and i'm like what you said you said human intelligence yeah i did i think i added that later. Oh, yeah. Okay. I worried about it because I didn't have that. I'm like, I'm going to put human in here because that's really what I mean. But that's interesting. Anyway, go ahead. Yeah. Well, I mean, I sort of wanted to use it as a segue into talking about either intelligence or consciousness or both if we have time. But yeah, that's interesting that you remember that sentence. So I don't, I don't know what, maybe you can just unpack that sentence and then that might lead you to talking about what's interesting to you about consciousness or intelligence or both?

Yeah, so for humans, it appears that it's really, there's a lot of evidence that it's important to be conscious to comprehend novel things and generate novel conceptualizations. And I think it has to do with this zone of proximal composition or working memory where you have these compositional elements, these representations that could be combined in new ways. And technically you can combine them from spontaneous activity or where you're sleeping or something, but to really understand them and evaluate them, you need basically instantiation of the full global flow hierarchy and these kind of conscious states that bind things together. It's about binding. So there's this conceptualization that starts in Chapter 4, actually, of these flow colliders that we mentioned a little bit ago,

where you have multiple signals, multiple flows coming in to, say, a brain region and neural population. And then there's a particular configuration of them that can generate a new representation. there's a concept of resonant flow colliders that has to do with the timing and what helps all the it becomes a hierarchical thing where the sources are bound with the target where you have a conjunction and you have the components too so you maintain all the representational details of the constituent. So I have an example of, was it a table? I'm trying to remember what I had in the figure. It's a table to the right of a chair with an apple on top of the table. You think about it, there's a relation, spatial relations between these objects. And you can think about if you have the conjunction of those, which is the scene, but you also have simultaneously representations

of each of the components, like the apple, the table, the chair, and you didn't lose those when you thought about the scene as a whole. So that's what I mean, like there's this kind of hierarchical relationship there. And so altogether, I call that a flow field. So the idea that these flows all have a bound representation that actually goes beyond what we're able to attend to all at once. We tend to be thinking about one or a couple of those things at a time, but if you turn around, get distracted, it's part of your conscious conceptualization. You can easily move around in that space. It also happens with language. If you think about what I said, that working memory is only four items, but a sentence could be ten words.

You make a little flow field with each sentence. Basically, each word kind of contributes to this representation. And then it's beyond your working memory capacity, but you can kind of think of the higher order relationships, you know, the words and the things that they're referring to and how they're related. And so anyway, so that's these distributed representations that are implemented by flows and bound by flows are the basic idea behind the consciousness chapter. One of the things that you do in the consciousness chapter is go through a couple different theories of consciousness, right? You talk about global workspace theory from Bernie Barr's that has gotten some traction in neural experiments. I mean, you talk about integrated information theory and, you know, I think a few others. And I think the book reads as if you're partial or maybe the flow-based approach or brain flow theory is more partial to a global workspace conception of consciousness.

But is that – am I misreading it? Because you do sort of lend some credence to some other theories. Like is there a winner among the current theories of how to think about consciousness? I think overall it's partial to the global workspace theory, but in order to add something to the narrative, I'm really focused on how brain flows contribute to these concepts. So, for instance, I mentioned spontaneous activity and spontaneous brain flows are always going on. actually the kind of tone of those flows changes based on your conscious state. And there's some key experiments I talk about, like this one involving TMS under anesthesia versus consciousness. If you just add some activities in TMS and then record that with EEG,

you can see that the activity flows over more time and more distally when someone's conscious versus not. And it also applies to sleep if you, it seems like a crazy experiment, but they did it where people were sleeping and then they TMS them. But when they're TMS, it doesn't flow as far as if they were awake. That would be during non-REM sleep, I would assume, right? Yeah, during R.E.M. Under R.E.M., it's more like consciousness. More like consciousness. And so when you have, you need a certain base level of internally generated spontaneous activity. And you give an example in the book of, I never looked at the video, but of the sand flow field experiment. Fluidized beds, yeah. Fluidized beds, yeah. So this fluidization is a central concept in the book, right? where, like, you pump just enough air so that the sand allows, like, really nice, flowy.

Yeah, it acts kind of like water, but it requires air bubbles to be constantly coming up and getting the sand into this fluidized state. And you actually can make waves by just pushing your hand through it. And you see the same thing with brain activity. You can see waves propagating. and the brain flows are slightly generalized from that, from these traveling waves as they're called. Traveling waves, I believe, have this kind of nice property of like right nearby, the wave kind of propagates. You can kind of like literally as if it were a wave in water. With the brain flows, you could have, it's just generalization of that because you can have long distance connections or you can skip, right? spatially. So there's these prominent pathways like from parietal cortex to frontal cortex, for instance. But it's the same concept of they're kind of like waves. So yeah, the fluidization is really about, just like with fluidized beds of sand, you need a very specific tuning. So you need

the right, I guess it's called excitatory and inhibitory balance in the literature. But it It allows for when you do have, say, a stimulus come in, if you have the right fluidization state in the brain, the idea is the flows will propagate more easily. If you have too much, you might have epilepsy. If you have too little, you might just be brain dead, right? I'm actually studying. So a term that you haven't used that is dealing with this is criticality, right, you know, the critical brain hypothesis. So I'm actually studying near to far criticality in different contexts in mice doing their behavior. So this hits home for me, the sweet spot, right? Yeah, the sweet spot, yeah. Okay, so we have a few minutes left, and I have more questions for you, one of which in a moment I'm going to ask you if there's something that we haven't covered that you'd like to cover. But I have to ask you, given the activity flow-based accounts for things like consciousness, intelligence, etc., how does this make you think differently, if at all?

or has this reoriented how you think about artificial intelligence and what it is, its nature, relative to our very human way of performing these things or implementing? Yeah, so I could bring it. So that's, I think, Chapter 7 is on intelligence. I included natural and artificial intelligence in that chapter. And the way I've been thinking about artificial intelligence in this framework is that current AI is kind of untethered relative to how our brains work in the sense that they don't have a proper global flow hierarchy. Like our global flow hierarchy is grounded at the bottom by perception action cycles. We're embodied. And that's important for preventing what's called hallucinations. Just like kind of, you know, not enough grounding in reality. And then on the other side, At the top of the global hierarchy, we have core values that shape our goal pursuits.

And so, you know, we'll have, I don't know, a conscience, essentially, right? We don't want to go against some of these core values when, even if, you know, we have conflict internally, right? But they don't have that. So they don't have either. They might follow sides of the pole. Exactly. They're just like the middle part that's like. Just bringing it back. That are, it's not good, basically. And there's attempts, right? They have, what's it called, constitutional AI that tries, I think, you know, to get the upper end of that. And then there's some models or a lot of models these days would bring vision in. Maybe that is not just language, so it'll kind of ground things a little bit. There's some evidence for that, actually, that it does help. But there's still two untethered. And it's interesting because it's like not just untethered but in the exact ways that frame how, you know, a million nervous systems work. We're grounded, we're embodied, and we have values, and that's what defines like the whole thing at the computational level.

So, yeah, it's kind of crazy. So does that make it more or less impressive, like what these models have been able to do, right? Yeah, I don't know. I mean, like to be fair, there's some grounding at the upper ends. One thing that I think isn't always talked about is how reinforcement learning is in the core of these large language models because they'll do this reinforcement learning from human feedback. And so you're kind of like training just like we train our own brains relative to our values, like what to represent and what's prominent and so forth. So there is the kind of top that's kind of generating these things, but it's very simple. Just like there's this issue of like, let's say this word, they try to please you too much.

Like, they won't tell you if you're wrong or something. They're just trained. Yeah. Is that the word? Yeah. Yeah, I think that's it. They're so, they're just, their training objective is just to, like, make you happy in the moment, basically. But that comes externally, right, where our values are internally generated. Right. Well, to be fair, I mean, we're very social creatures, so I guess some of it is. That's a survival goal. But we internalize these values from our group and stuff, right? Sure. They just don't have the architecture for that, as far as I can tell. They just don't have, you know. And it's like, yeah, it's crazy. Like, yeah, should we make AI more human-like? I think with respect to having some kind of core values, that are internalized, I think that would be good. And then, yeah, having them embodied enough to ground the thoughts in reality,

if you want to call that the representations in reality, I think that would be good. Okay, is there anything that we didn't talk about that you want to make sure that you share? We talked about a lot, and I don't want to, we can't spoil everything from the book for the listeners. Yeah, I'm trying to think. The main things I want to talk about, you know, what this change in perspective towards brain activity flows and how useful I think that'll be, that would be if we shifted. So it's a more generative perspective. It's about adding context to the activity patterns that we're analyzing and the connectivity and combining them to think about how the activity is generated through these flows. And how is that different than the way neuroscience has been done thus far? um so one kind of unspoken assumption and a lot of cognitive neuroscience has been if you see an

activity in a brain region or a neural population go up say for a given test condition there's some unknown within region process like if it's the fusiform face area there's face processing going on but we literally have it we have a study from a few years or a couple years back showing that you You can generate face selectivity in the fusiform face area from other regions just based on the flows to the face area. So the computation isn't inside. It's probably some, but most of the computation is from other regions, and the selection of the connections and their weights select the features that generate the face selectivity. That's just indicative. but like one of the most well-localized functions in the human brain, just over two decades of work showing this, is actually a distributed process. Right. That's like a headline. But it's also a fundamentally different way of looking at it,

that it's the flow that's doing the processing. Yeah. The function comes from the flow itself, not from within area, not solely from within area process. Yeah, there's some recurrent processing, yeah. But yeah, we didn't even model the recurrent processing, and it generated most of the variance based on activity. That's cool. And so there's still a localized function there, but it's a distributed process to put it there. And then the Fuse-Farm-Face area does other things as well. It represents all manner of visual objects that's been shown before. So I'm not saying it's so – even then we already know it's more complex and these representations are distributed. But this is one strange exception where there's some localized functionality. Even then it's about the distributive processes kind of converging there for whatever reason.

And so that's a different way of thinking of it, right? rather than the unknown within the region process, driving everything. All right, that's a great place to end it. Congrats again on the book. I enjoyed it. It was my main reading material on those long flights to and from Geneva. So I love reading on planes because you can really sit down and focus, you know, strapping yourself to the mast. Yeah, there's something about being on a plane, yeah. Yeah. Anyway, thanks for sharing a lot of the messages in the book. And like I said, of course, there's a lot more in there. And you really go in depth on a lot of these things. So I think you mentioned this earlier. It's written in a way that someone who is already an expert across different fields can kind of go as deep as they want to with the way that you've written it. And people can kind of jump in and out of those details. So I think hopefully that makes it a good read for a large swath of different expertise, et cetera.

So anyway, thank you, Michael, for being here. Good to see you. Thanks for having me. Until another four years, perhaps, from now, maybe sooner. We'll schedule it now. Yeah, yeah, we schedule it. Okay, thanks, Michael. Thanks. Brain Inspired is powered by The Transmitter, an online publication that aims to deliver useful information, insights, and tools to build bridges across neuroscience and advanced research. Visit thetransmitter.org to explore the latest neuroscience news and perspectives written by journalists and scientists. If you value Brain Inspired, support it through Patreon to access full-length episodes, join our Discord community, and even influence who I invite to the podcast. Go to brandspire.co to learn more. The music you hear is a little slow, jazzy blues performed by my friend, Kyle Donovan. Thank you for your support. See you next time.

you