#503 – 数学におけるAI危機：ナビエ–ストークス、リーマン予想、計算複雑性

All right, so this is a kind of emergency episode, I think it's fair to say, about the state of mathematics and artificial intelligence. A kind of crisis moment in the history of mathematics. Over the last few weeks, machines claimed to have solved some of the hardest problems in the history of math. Navier-Stokes, the QuadraVeman hypothesis, the unique games, conjecture, and hundreds of other results. So, here with me, we have Stephen Brunton, who's a mathematician, fluid dynamics expert at the University of Washington, who can speak to negative folks, turbulence, all things fluid, and many other topics. Also, here's Ankur Murtra, a mathematician and theoretical computer scientist at MIT, visiting Institute for Exam Studies in Princeton this year, who can speak to many things, unique games, conjecture, and complexity theory in general,

and the mathematics of how the AI learns, and all things theoretical computer science. and Alex, a mathematician, number theorist, the ruggers, who can speak to also very many topics, including Riemann Hypothesis and Lean, the automated proof and automated verifying, checking and so on, and so many other topics in mathematics. All three are brilliant. All are very kind to show up this early morning and talk to me, assemble this kind of multi-agent mini-swarm of human brains to try to interpret what happened in this very moment. A lot of mathematicians, brilliant mathematicians, including Tara Sall and others, are freaking out. It's a mix of awe, terror, dread, and a few inklings of light, of hope also. And now, a quick few-second mention of our sponsors. Check them out in the description

or at lexfriegman.com slash sponsors. It is the best way to support this podcast. We've got Whisperflow for voice, text, and writing, Finn for customer service AI agents, Element for Electrolyte, Blitzy for code generation in large code bases, Shopify for selling stuff online, NetSuite for business management software, and, of course, Replexity for curiosity-driven knowledge exploration. Choose wisely, my friends. And now, on to the full addries. I try to make them interesting, but if you skip, please still check out our sponsors. I enjoy their stuff. Maybe you will, too. To get in touch with me, for whatever reason, go to lifefreedmon.com slash contact. All right, let's go. This episode is brought to you by WhisperFlow, the AI voice dictation app that lets you light by speaking. and at this point I speak to AI

maybe an order of magnitude more than I type to AI. It's a crazy thing to say. Me who's anxious about social interaction, I never thought I would find comfort in voice as the interface to AI systems, but I do. It's fascinating. I love the keyboard. I've loved the keyboard for many decades, but we must say goodbye to the past and say hello to the future. And one of those tough goodbyes has been the Kinesis keyboard for me and the keyboard in general. I'm typing less and less and less these days. But playing more guitar and playing more piano. It's a different kind of keyboard. Anyway, if you want to try to use voice as the input layer for AI tools, this is the way to go. Go to whisperflow.ai.com to get one month of Whisperflow Pro free. That's W-I-S-P-R flow dot A-I slash Lex. This episode is also brought to you by Finn, the number one agent for customer service.

65% average resolution rate. It's trusted by over 6,000 customer service leaders and top companies, including many of the top AI companies. You know when an AI company trusts another AI company to do a thing, that means they're legit. that SIN has found a niche, an incredibly valuable niche, and they do it extremely well. There's some wisdom in that about how to succeed in this insane old world of AI systems everywhere. Find a niche that brings a lot of value to a large number of people and get really damn good at that niche. Until, frankly, the niche swallows the world because we live in a time where it's just a bunch of niches and you realize there's so many people at the globe that the niches are quite large. They can fill stadiums. They can fill nations. Anyway, go to fin.ai slash lex to learn more about transforming your customer service and scaring

your support team. That's fin.ai slash lex. This episode is also brought to you by Element, my daily zero-sugar and delicious electrolyte mix I'm sipping on it right now, watermelon salt, delicious, all the crazy stuff I do. I've been basically exclusively eating only once a day, working insane hours, going through all kinds of crazy whirlwind rollercoaster journeys inside the mind, trying to comprehend everything that's happening in the world. What a difficult episode this is on mathematics. Mathematics has been at the core of many of the things I admire about the limits of what the human mind is capable of. And here we are breaking through those limits at a rate that nobody could have anticipated in. It's just a lot. It's so much to deal with. There's so much to deal with for mathematicians.

There's so much to deal with for other people, for thousands, for maybe millions of people. Scared about the future of the world? I'm with you. All right. Get a free 8-count sample pack with any purchase. Try it at drinkelement.com. Speaking of a rapidly changing world, this episode is brought to you by Blitzy, an AI-powered autonomous software development platform. It's built for large, complex code bases. If you're one of them, Vibe Coders, who Open Clause Peter says is a slur, Vibe Coding, a lot of people that use AI agents, the programmer, really working on small code bases, and often actually code bases that don't interact with customers, with little people. And if you're working on code bases that serve a large number of people, and those code bases are large, there's a whole set of tooling and systems and concerns you need to be aware of.

And that's what Blitzy helps you with. The future of autonomous software development is here. Learn more or speak to a member of the team at Blitzy.com slash Lex. That's Blitzy.com slash Lex. This episode is brought to you by Shopify. A platform designed for anyone to sell anywhere. Why does it fill me with so much joy when I mention Shopify? I think about Shopify engineering. It fills me with joy. What am I going to do? Anyway, it's a great way to build a great-looking online store. Toby is a man. He's the CEO of Shopify. Brilliant guy. Brilliant engineer. Brilliant leader of engineers. And it's always nice when a guy like that, who's at the cutting edge, who pushes a company to be at the cutting edge, is running a company as impactful Shopify. And it is. So if you want to be connected with such an impactful company, sign up for a $1 per month trial period at shopify.com.

That's all lowercase. Go to shopify.com. To take your business to the next level today. This episode is also brought to you by NetSuite, all in one AI-powered business management system. Their NetSuite Next system automatically services insights throughout your day. Their AI agents work alongside you to complete various tasks. It's a great integrated system. to all the different modules within a company, you know, managing financials, all the HR stuff, inventory and supply, e-commerce, all of that kind of stuff, all the mess, the beautiful, beautiful mess, the nature of a company, all of those are different modules, hierarchical modules, all of them need to be connected and managed and organized, and the utilization of AI within that whole system is essential, and NetSuite does a great job of that. For the first time ever, you can try NetSuite for free.

Go to netsuite.ai.lex. That's netsuite.ai.lex. This is the Lex Friedman Podcast. And now, dear friends, back to this fascinating conversation about mathematics. So I apologize to start with a question about feeling, but what have you been feeling about this historic moment? Maybe, Alex, do you want to go first? Yeah, I don't know if I would call it a crisis. There's a lot of opportunity in that crisis. It's a chance to rethink what it is we're trying to accomplish. And it's the chaos of opportunity that some people are reacting negatively to, some people are reacting positively to, and sometimes those reactions are happening in the same person.

and so I think we're entering if we're not already in a period of some kind of golden age of mathematical knowledge after millennia of having a candle that we are walking around the woods and this is as far as we can see now we have a flashlight I think we should comment on the magnitude of this moment for people who are not aware so maybe to quote you something like something like holy shit if a human did this It would be an instant field medal. Maybe he didn't say holy shit, but basically saying this is an incredible result. A lot of people wouldn't have expected this amount of progress in this amount of time. So this is truly shocking. So you're now speaking as a person who is dealing with the feelings you've felt over the last couple of days, having found this out. So maybe also it would be useful to go back to the feeling you were having when you first saw the report that, in this case, OpenAI dropped. And the two of you happened to be on camera loosely on the same day,

maybe a few drinks in because of the conference or get-together. Maybe, Ankur, can you speak to what you were feeling? It was an interesting night when the results dropped. Now, it was not entirely unexpected because for weeks there had been rumors about the two biggest problems in my area that were in that collection of results. So the unique games conjecture and RL equals L. There were even rumors about matrix multiplication. So really my first reaction was, wow, it's all true. Thank goodness it's finally out there because now people can actually go through it and process what's happened and figure out what to do next. Now, as to your earlier question, you know, what are we feeling? Well, for me, it changes on a daily basis. But I think the biggest, most overarching feeling is really feeling like you're getting pulled in two very different directions. My work has always been about asking the right questions. And to that extent, these tools are amazing for me because, you know, they increase my ability to learn about other fields.

It's like having a world expert who's there to teach you about things in your own language. Your ability to make connections is just magnified. But, you know, one of the parts of my job that I really love the most is the mentorship aspect. Not just training grad students and postdocs and seeing them do great things, but having them come with me on these journeys. So the part that pulls me in the other direction is the recognition that, you know, the playbook and the blueprint for success that we had for so long is now thrown out the window. So it becomes a lot harder to advise students and postdocs about, you know, what are things they should be working on that are going to make an impact, are going to make a name for themselves. So I feel very excited, but maybe more intellectually lonely than I felt before, in part because, you know, a lot of times great results come from asking very humble questions, at least for me. And now there's so much anxiety and dread and feeling that, wow, you would have to do something really momentous to stand out in the sea of AI results, that it feels like we've lost some of that patience in terms of planting the seeds from which the great oak trees grow.

Terence Tau, feels not the winner, wrote, And I mourn the path not taken. And the opportunity is lost in the frantic race to develop the technology. So, I mean, if we're being realistic, it's more dramatic for a PhD student and a postdoc than maybe the way you've articulated it. I mean, you're saying it quite calmly, but if I was a PhD student right now, I'd be in a state of, like, shock.

like my whole life. I was going through some of this. I know a lot of my friends are programmers. They're going through this state of shock in last December when basically most of the programming world switched to AI writing most of the programs. So I would approximate it to a feeling of kind of a loss of meaning, kind of terror, like dark stuff. We'll keep us forward now. Steve, what do you think? What are you feeling? I would say most of my feelings are pretty hopeful. I think I'm an engineer. I like solving problems that make technology more realistic for humans, artificial hearts and airplanes. And our ability to solve real applied problems is going to get much, much better. Our scope and reach is going to increase. And so for that, I'm extremely hopeful. But there is a lot of uncertainty. And I think, you know, some of this fear of meaning, of losing control, of, you know, just being kind of cut out of the process that we all love being a part of, it is a little scary.

And I don't think any of us would be surprised at either of the kind of two paths that might take place over the next five or ten years where we really are just dwarfed by other intelligence or it kind of levels off and this becomes an incredible tool that we use for our curiosity, for our human problem solving. And that uncertainty is massive. Those are two very different worlds. And so uncertainty is difficult for humans. I think it's also difficult for those of us who have chosen math. We chose math, you know, probably because we liked rules and formalism and, you know, uncertainty is uncomfortable. I think the humility aspect is really important, though. This is a really good opportunity for us to remember why we're doing this. Like, what is human purpose? It's not to be famous. It's not to have a field medal. You know, solving interesting problems, helping other people, being curious, just wanting to understand the world.

Those are really good things to kind of get back. Like, what do we need to be satisfied humans? And I think it's a really interesting moment for us to think about. I've been thinking about it a lot lately. It is more difficult for young people, right, like figuring out how to find their way. but I actually have more faith in them than less in a sense. Do you think there's something to what Terry says about it's harder to be motivated when you know a problem has been solved? To explore that space when a problem has been solved like this by an AI system? Yes and no, in the sense that I do very much agree that the path is as important as the destination. Like if you go on a road trip, the road trip is usually just as fun as getting there. At the same time, if we found out that another country had invented teleportation or time travel,

I guarantee we would reinvent it in the next five years. Like if you knew it was possible and you could reverse engineer a little bit, we would be curious, we would dive into it, and we would figure it out. So knowing there's a solution also is an incredible motivation to get there much faster. Let me ask the question this way. If this moment marks the moment of the trajectory where AI basically, quote-unquote, solves math or kills math, can you make the case that that is, in fact, a trajectory, and then can you make the case that that is not a trajectory? can you steal them in both paths so this is a path towards which like mathematics as we know it is done completely so that's going to be a thing done by machines and then what humans do is something fundamentally different or there still will be something that looks similar to our current mathematics

can you make it a steal man in both cases Alice? Yeah, so you said two things, and I'd like to separate them. One is solve math, and the other is destroy math. So let's make that precise. What does solve math mean? If you think back to when chess engines first came out, and people were like, oh, this is going to be amazing. We'll play centaur games. So there's a human being plus a computer system against another human being and a computer system. And that lasted like 10 seconds. And then the computers got so good that if you did anything but choose the top move that the computer recommended, you would lose. And so the human being's role in that game became, yes, accept the first move, yes, accept the first move. That's no longer fun, and we don't see many games of Centaur chess anymore, right? And so, in fact, I just had a conversation with a colleague named Thomas Goberta where we axiomatized our argument for what the future might look like. And one of the axioms that I'm willing to even accept for the sake of argument is what I call solved axioms.

And the solved axiom says, let's assume at some future point, we're nowhere near there now, and I don't expect us to be there necessarily. We'll see where the capabilities taper off if they do. But let's assume the solved axiom, which says any mathematical problem that can be solved with human interacting with AI could also have been solved by AI alone. That's the solved axiom. So that would mean that really there's nothing new for a human being to, you know, you can say to an AI system, okay, I have problem X. I want to think about problem X. But the only way that I will make progress, any progress that I will make on problem X, will just be press a button. The AI thinks about it. It doesn't necessarily solve it. It doesn't mean that it will solve every problem. The Rheumann hypothesis is not solved. Okay? That's what I really want the solution to. I just didn't think in my lifetime I would see any kind of progress towards it. that we saw a few days ago. So it's not that AI systems are solving every problem. They're solving every problem that could be solved by humans interacting with AI.

That's what that axiom would be. So I'm saying, if we are ever in that world, does that necessarily destroy mathematics? And my strong argument is no, I don't think it destroys mathematics. If that axiom is true, what's the role of humans? And so I don't think that mathematics, pure mathematics, not engineering mathematics, not things that will cure cancer and solve, you know, everybody has a jetpack, whatever other technological advancement you might imagine. Pure mathematics. You know, we know that we're proving theorems just because we're curious about them. I don't think the solution to theorems living in ones and zeros and data centers is valuable. human beings. I think the value of pure mathematics is it lives in the heads of human beings. So just to, you know, add to this, I mean, I'm glad you came right out with the question that's on everyone's minds, and thank you for that. I would caution against overfitting to one objective function for this, right? Because, you know, we used to think of ourselves as just the smartest

things around. You know, human mathematicians can solve, you know, reach the highest treetops, but now we're faced with the fact that many of the problems that we thought were, you know, quite a ways out have been solved by AI. But maybe one of the consolations we can take is actually the fact that we're very computationally limited compared to the way that AI is solving it. You know, given the same finish line of proving the same theorem, I can't really hold that many papers or ideas in my head at one time. So in order for me to reach that same finish line, maybe I have to develop more generalizable and abstractable ways of getting there that then help you not just solve that one problem, but explain connections or other directions you want to get into. So it's always amazing when you look back on papers you've written, you can see these threads, invisible sometimes, that connect different areas of work, because developing systems of ways of understanding problems helps you for surrounding problems, not just to get a lean, formalized, you know, one serum down.

But the interesting thing here is those threads, what you're saying is sometimes invisible, are quite visible to AI systems now. It seems like that's perhaps the actual very reason why they're able to make so much progress is they can extract the threads and pull on them because they have the view of so many papers. I think it depends on what you mean by threads. You know, for me, the reason you prove results is so that you better understand the world around you. Mathematics to me is a vehicle for abstraction and understanding things. And, you know, it's when you have a major result, what's important is not just the truth of the statement, but what you've learned en route to doing that. Usually people's first question is, OK, you proved something fantastic, but how did you do it? and then people, once they digest that new idea, that new definition, that new way of looking at things, there are lots of other problems nearby that fall. So somehow you're building structures that help you attack not just one problem, but a whole sea of problems.

And, you know, it's not that we need fully formalized, you know, results to do that. Sometimes we reason in very sloppy metaphors that, you know, help us conceptualize the world around us. and how much of this is something I was mentioning offline, attention is all you need, how much of this is the human ability to shine a light on particular areas that quote-unquote matter? So the human ability to ask the right questions, the important questions, that's always been the case. So within this, before AI, there's a, you can think of like here's the axioms and just abstractly from the axioms you can get to logical deductions, right? The theorems that we could prove from the axioms. And there's a much, much, much larger space of theorems that we could prove than the space that human beings ever actually care about. And so, you know, so almost by definition, it's like a topology, that the theorems that we'll get excited about are the theorems that we care about as humans. So maybe AI can convince us that there's some interesting structure. Maybe you can say, hey, look over here, let me give you this new definition or something.

It hasn't done that yet. And sometimes there's a mechanism by which we select the things that we care about. It could have an application, so negative folks are connected to fluids and all natural systems, but it could also be a thing that some kind of mixture of beauty and difficulty that just pulls at us. Maybe there's an intuition. You know, the prime numbers are full of these really complicated problems that maybe don't have an obvious application in the immediate sense, but it pulls at us somehow, and maybe as a symptom of the pursuit of the hard thing, great applied mathematics and engineering and solutions to real-world problems comes out. So just as an example, a lot of times when I'm working on a problem, I might have some bigger goal that I want to accomplish, but usually what I do is I try to find the simplest, most embarrassing open problem that I still don't know how to solve.

You know, that aspect of, like, digesting it down to something, boy, how can we not know this? It, you know, it feels like it's on the right track because then when we can solve that example, maybe we're assembling the set of tools we need to make a tax on other problems. So I do think we're going to have to become a lot more deliberate about, you know, what do we care about? How do we use these tools from a metacognition standpoint? You know, what exactly are we delegating to them and what are we not delegating to them? and we're really at the beginning stages where we're just trying to piece it together as we go along. On the one hand, looking for a proof is kind of like exploring this tree of possibilities. You might have some intuitions based on some other proof you've seen about which directions to go in. Maybe I should go down this branch. Maybe I should go down this branch. But sometimes we encounter points on the tree which have really high branching factor. We just don't know which way to go down. We don't even have a guess about if we went down them, what would we discover?

And now in this centaur world of math, we suddenly, you know, have something else intelligent that we can interact with. Right? And what is the right way to use that tool to search for the beautiful theories, I think, will be challenging and evolving. But it's important to not lose sight of the fact that we don't just care about theorems. So, you know, I'm a mathematician. I'm a theoretical computer scientist. but I've also been reading a little bit more broadly about social sciences just to sort of make my peace with all of the changes that have been happening. One of the things that really stuck in my mind that I liked is there's something called the doorman fallacy, which is very interesting. So the idea is that you take a complex job and you boil it down to its simplest, most visible task. So what does a doorman do? I guess they hold the door for people as they're coming and going, right? And when we lived in New York, we really enjoyed talking to our doorman. You know, they were wonderful people. But, you know, if you digest it down to just that simple job, you can think, oh, I can replace them just by a sliding door that detects when people are going or coming.

And you miss out on all the aspects of their job that, you know, add a lot of value and are harder to articulate. And I think that's a good way to think about mathematics in the age of AI, too. what does a mathematician do? You know, the simplest and most visible part of the job is we prove theorems. And, you know, theorems about conjectures that were open for hundreds of years, like Fermat's last theorem, these are very visible things that we've spent a lot of time popularizing. But I don't think that that's all that a mathematician does. I like to think about it as a mathematician is someone who thinks deeply. And a byproduct of thinking deeply is, you know, proving old conjectures, but it's also building new theories, building new ways of looking at old things, making new conceptual connections. And these are parts that, you know, maybe AI will follow up to, but we're not there yet at this moment. So even though this is a momentous time, I don't think it's the right time to hit the panic button. I think it's the right time to really recenter in terms of what our values are and have these conversations as a field.

and remember that what we do really has a plurality of different aesthetics and goals and that when one thing becomes cheap, like problem solving for well-defined things, it makes the other things that are orthogonal to it all that much more valuable. This is really beautiful. The dormant fallacy, going to perplexity here, is the mistake of reducing a person's job to its most obvious task and assuming that automating that task replaces the value of the whole role. So a hotel decides, quote, we're paying someone to open the door. An automatic door can do that more cheaply. But, in fact, the doorman also welcomes guests and recognizes returning customers, handles luggage and hails taxis, provides security and discourages unwanted visitors, and signals prestige and makes guests feel cared for. That was my favorite part. I looked up what a doorman does, and boy, we have a lot of commonalities with mathematicians. If I don't signal brand value, I don't know what I'm doing. That's funny. But I think the task for us humans in general, but also mathematicians,

is to come up with that list, like actually make more explicit exactly what it is that humans do. And I haven't heard English mention beauty. I apologize if I've mentioned beauty excessively already. But there's something that mathematicians are, from my outsider sort of fan perspective, are kind of expositors, scholars of things beautiful, broadly defined. That's exactly right. Until five seconds ago, there was a near perfect correlation between the ability to solve new deep theorems and what we actually want as human mathematicians. and what we want is exactly what you said, beauty and understanding and depth of thought and the fact that you can press a button and get what used to be our proxy without any effort, without any deep thought and without any necessarily knowledge. We have people saying, do a breakthrough and then putting whatever they find on Twitter

and now what are math decisions supposed to do with that? I would go one step behind answering questions and doing problem solving is asking the right questions. So that is how I think you measure good problem solving is asking the right sub-questions, asking a question that is interesting and can be solved and can be interpreted. And so I feel like the value of asking good questions has always been something we value in our colleagues, and that's going to become increasingly more valuable. I agree completely. And just to emphasize a point that was sort of hidden in there, I think asking the right questions requires a lot of mathematical insight. You'll see people make that comment all the time. What are we going to value going forward? It's asking the right question. It's giving great talks. But I wouldn't minimize, not that you were, but just to be explicit, I would not minimize the role in mathematical insight in both of those tasks. because it's when we say that, you know, talks will become more important.

I'm not imagining that, you know, we're all going to become Twitter influencers and TikTok whatever. I hope not. I hope not. Because, you know, really what stands out is having ideas that get adopted by your community that end up becoming powerful over time. So, you know, I want to tie this back to something that Lex brought up earlier on, which is, you know, he asked, am I being overly optimistic? Like probably, right, compared to, you know, how grad students view these things. But I think one of the things that I'm most worried about in the near term is what's going to happen to our systems and institutions. You know, it's not just hiring and tenure and all these things. But it's really that when a grad student starts off their PhD, you know, they're a new entrant of a field. They're trying to absorb what are the techniques in their field, what are the important problems. And they're trying to figure out how to make a name for themselves. And what we used to do is we would give them a blueprint that was very simple to follow. Take a problem people care about and improve it.

And then people will know who you are. You know, you'll get papers in top venues. You know, doors will open for you. But now I think that blueprint has changed in a lot of ways because there are so many results that are happening right now that are all just overloaded with it. And I think it's okay to take time to rethink that blueprint. We still care about the same things like ideas that are going to influence your community, but they're just harder to operationalize in terms of some simple rubric that we can give them. So I'm still optimistic. I just think it's going to take, you know, reworking a lot of our systems and institutions to do that. By the way, to go back to your point, I don't actually believe the solved axion. I don't think we're there. I can prove a theorem, but I don't understand why the proof works. There's some magic that happens over here. and until I generalize somehow the phenomenon and understand it more deeply, then I'll see why the proof works the way it does. Maybe that's one of the reasons why I don't actually believe the solid action because the things that we do are much more exploratory

and what constitutes a good question beyond just theory, agreement hypothesis, make some progress on it. I think you also brought up a really important point that's like generality and generalization of our concepts. So related to beauty and mathematics, I think one of the gold standards has always been how simple can your result be and how much does it generalize? And, you know, out of this infinite space of theorems that could be proven, finding some structure that is simple and generalizes like that is, I think, still the gold standard. And unless I'm mistaken, humans are still asking the question. So, like, we, you know, we prove existence, uniqueness, and maybe we're stoked with AI, but humans are guiding us. Humans are still asking the questions. Like, I would love to see the next breakthrough if you had a machine that can prove any theorem you pose. I wanted to start telling you what questions I should be asking. Do you think it's possible that that machine just does a for loop

and finds all the simple and generalizable, Basically, all the E equals MC squared. That's happening today, actually. So, like, there are absolutely research programs that are systematically, automatically discovering kind of minimal physical laws. So this is less mathematical theorems, more physics governing laws and equations. So observing natural systems and trying to see in those natural systems if there are simple physical laws that model how they operate. Yeah, distilling it down to, like, the basics. So you can kind of recapitulate known theories. If you observe ball dropping, you can deduce gravity and F equals MA. If you observe fluids, you can deduce maybe your Stokes. And then taking that to data, we haven't written down simple equations, weather, turbulence, solar slayers, things like that. And I'm assuming similar things are happening in math where there are probably these minimal pieces that can be deduced.

Do you think there's a large space of simple and generalizable things in math that can be discovered automatically? So my version of the Solveig vaccine is that anything that I do as a human mathematician, I've tried on several occasions to sit down and say, okay, you know, LOMs will never do this that I do. And I'm like, well, why not? I can ask a question. I can ask it a question. I can ask it what questions should I ask, right? Because it's such a generalized system, I'm not saying that it's good at all of those things now. But my version of the solved axiom is anything that I can contribute as a human being, it could also have contributed on its own. And so what I really mean by that is in terms of value, what do I bring to the community? It can't be, I asked a great question, it can't necessarily be in this world of the solved axiom, which again I don't believe, but even in that world, It should be something other than, I asked a great question, I proved a great hero.

And even in that world, I think you need mathematicians, people with human mathematical expertise. Do you believe in the solved axiom? We're on the spectrum of how close we can get to it. And in that spectrum, what's the role of the human, do you think? I guess I don't really believe in the solved axiom. I think maybe it's half true in some ways. and figuring out which half of it is true will be very interesting and challenging. So I think there'll be major surprises and more to come in terms of what AI can solve. But, you know, for me, when you're facing a new technology that's really, really good, the danger is that when you try to conceptualize why it's working, you do so by anthropomorphizing it. So, you know, maybe one of the earlier lessons here is from deep learning. So there are these amazing examples. You know, you can take neural nets and you can do human-level or better object recognition. But in general, the way that it succeeds is less robust than how a human does,

because there are all these examples like adversarial examples, where you can have very slight patterns to natural images that can completely fool these object detection systems, but not a human. And, you know, I had a former colleague at MIT, Alex Mondry. They had some really nice work, which I really enjoyed. It was, I think the title was something like Adversarial Examples are Features, Not Bugs. So they did the following cool experiment. So you take a bunch of natural data, maybe pictures of cats and dogs. You train a deep network to identify them very well. You add some adversarial perturbations to them that, you know, make to the AI system a cat look like a dog and vice versa. So it's kind of think like a small little pattern. That changes how the object detector works. And then you create a totally fake data set. You know, it's got pictures of cats with dog patterns on top, and you label it as dogs and vice versa for cats. And amazingly, when you train a deep network on this with all the wrong labels,

it's going to learn something that does well on natural data. So somehow when we recognize what a cat and dog is, we do it in a way that we can articulate in terms of its legs and its tail and its shape in general. But a lot of times when, you know, deep networks or AI can do things, they might be able to find trapdoors or backdoors for solving a problem that use a lot of statistical relationships in the data that a human doesn't. So to me, you know, it's important to not, you know, draw this boundary, you know, very sharply between what you can do and what AI can't do, because you're going to be wrong. But that doesn't mean that I don't believe there will emerge some kind of boundary, if only for the fact that I think we over-anthropomorphize the way that they solve problems. And when I'm looking through the open AI solutions, they just don't look like a human has solved those problems. So to me, there still seems to be a very big qualitative difference, and that's an opportunity really for collaboration

and Centaur Math more than it is, you know, dread about the AI overloads. But, you know, who knows? Could be wrong next week. What's your intuition about how much of this is AI just tying together a bunch of thousands of disparate papers and pulling up the threads, sorry for bringing back the metaphor, but the threads of the different papers, and then just basically articulating the proofs that were already there in the different papers, and you're making minor connections? Or is it actually developing new mathematical ideas? Have you seen glimmers of whatever that genius alpha zero move in the game of Go? Move 37. Move 37. So I haven't read all 700 of their papers, the ones that I have looked at. You know, some people speak about this dismissively like, oh, it was in the convex hull. You do the moves that we know as humans, and you just do them really well and do a really good tree search.

I wouldn't call it the convex hull. I think Tim Gowers had a really good way of talking about it instead as the semi-group generated by all of these things. Yes, it's the same notion of putting together known moves, but the space of known moves, what it can accomplish, is sort of much, much bigger than this kind of derogatory convex hull. So in terms of the quasi-RH, the moves that it took are all things that were known to human beings. And there's a very long cascade of them that is kind of shocking when you look at it and say, oh my God, that's so beautiful. How did you think to, how did you, I'm formalizing it, of course. How did you think to do this move after that move and after that move? And yeah, that does it. Could you articulate to the more general public the intuition, the line between Send Me Group and Comex Hall? the derogatory convex hull. And quasi-Ariation, of course, is the quasi-Herman hypothesis that was a big boost.

Yeah. Let me try to do it like this. You have, think of like a balloon. So the inside of the balloon is its convex hull. And so we have like all of these results that all these ideas and mathematicians have thought up. And then anything inside of that is some linear combination of it. But it feels compact. It's like there's this kind of closed, enclosed area. And within that area, Like, AI hasn't really made new progress because it hasn't moved the balloon out. It hasn't given us a new idea yet. And it's derogatory because the intuition there is we could have done it, mathematicians could have done it, because it's already quite obvious. It's already within the space of things that have already been thought about and developed. Yeah, so it has read every paper. It's exactly what Anker was just saying. It has read every paper. It has internalized all of that. that's more than any human brain not even could hold, but have time to read and learn and internalize. So in terms of breath, yes, it beats any human being.

And then in terms of depth, you know, has it really generated a genuinely new idea? And you can argue that, at least in the case of quasi-ARH, I would not say it's a genuinely new idea. It's a beautiful combination of a lot of things that were very well known. There are a lot of people who are kicking themselves that had some idea in that direction 20 years ago, and two years ago there was a big breakthrough on cubic theta functions and cubic gaussums, and that ended up being the path that the AI took to the solution. But so to contrast that with a semi-group, I think of a semi-group as like tiles. So you're just playing tiles on the board, and what you're allowed to do is whenever you have a tile, you can put another tile to the left, to the right, to the front, or to the back. and then you can just start tiling things. And that's like a semi-group action. And the question is, well, how far can you reach with that? You can go, I don't know. You can go really, really far out with existing ideas

might prove a whole lot more than we as humans have been able to explore. So your intuition is more in the semi-group direction, Meaning, like, there is at least glimmers of expansion versus linear combinations inside the complex. You know, how we choose to interpret what it's doing, I think, is up to us. But throughout the history of mathematics, there have been very seemingly difficult and deep problems that then someone comes up with a three-page solution that is very elementary and could have been explained to an undergrad and could have been discovered 20 years before when the problem was first posed. And had it been discovered 20 years before, it would be an exercise in a textbook somewhere. And because that solution wasn't discovered and really bright people worked on it for a long time and made incremental progress and made the problem really famous, then by the time it was actually solved, it was a really big deal. That happens from time to time in mathematics.

And what I said a year ago when I wrote, so there was this International Congress of Mathematicians, this summer, and I was writing my proceedings that they wanted 10 months in advance. So I said, you know, what am I going to write about math and AI now that will still be relevant 10 months from now? And I think what's there is somewhat evergreen. I stand by what I wrote. But one of the things that I wrote there was exactly that we're going to discover that a whole lot more problems that we thought were unreachable by current technology actually are reachable by current technology if you can put it together in just the right way. And that's what AI is amazing at. Again, someone could interpret that as, oh, so it's inside, you know, these solutions were inside the convex hull. You want to call it that, that's fine. I think that's, you know, it's not invalid. So I like all these metaphors, you know, also the balloon. So the important thing about the balloon is the boundary between what's inside and outside. And what makes it derogatory is the fact that, you know, you haven't seemingly changed the boundary. But, you know, definitely these problems have changed the boundary.

And, you know, maybe a more concrete thing is just that the types of things that people are going to work on after that drop of opening up problems is forever different. You know, there were at least a couple of results there that are like, you know, field-altering results that will change what types of problems you give to young researchers, what areas you want to explore, because your conception of the landscape is just that much different. So not just because it expanded, but because the very expansion changed the direction of the interesting questions. That's right. So in fact, one of the problems which probably maybe it's the easiest to explain what kind of effect it had on people was not actually from the OpenAI drop. It was an entropic result that was written up by my colleague, Virginia Vassilovska-Williams, and then Josh Ullman. So this was about the threesome problem. And I'd say that there it has a major effect on what's called fine-grained complexity theory, because there was a big surprise that a very basic problem that you give people in coding interviews

can actually be solved asymptotically much faster than the solution anyone knew. And that was counter to what the community was betting. In fact, there was a whole theory built up around the idea that if this problem really is that hard, then all these other problems inherit its hardness. So now as a result of that problem being easier than we thought, you know, having a bit better of an algorithm, you know, yes, that gives you a lot new algorithms across the entire landscape, but it also changes the character of the field because the goal there was to put all those problems together and so that we should be studying them all together. But now we're sort of back in the state of affairs where maybe the right way to study them is separately after all. So I think these are examples where, you know, So irrespective of how much deep, new, generalizable ideas we get out there, even just knowing the truth of the theorem is going to change the way that people organize research in that area. And there's no putting the genie back in the bottle on some of these problems.

I mean, imagine one of the 700 was factoring in polynomial time. That would absolutely change. Everybody would know. Like, you can't use your credit card on Amazon anymore. All of e-commerce is gone. All of secure communications is gone. like that. Can you explain factoring probably all the time? Yeah. So if you give me two very large numbers, a million digits each, I can multiply them together and do that in a split second. If you ask me to go backwards, say you took two large prime numbers, you multiply them together, you give me the product and you ask me to figure out which two numbers, you know, just factor. Don't we ask our kids to learn how to factor in school? You know, it's a lie. We do not know how to factor. And the fact that we don't know how to factor is why we have the capacity, the ability to communicate with one another in all kinds of secure ways. So all of modern, basically the modern world, depends on the fact that we don't have an algorithm for taking a large number and pulling its plant factors out of it.

So a lot of e-commerce does because that's the basis for a lot of the schemes and public key encryption. But, you know, famously there are quantum algorithms for factoring. So there's a lot of emphasis these days on finding post-quantum bases for cryptography. A lot of space methods. Yeah, we're already concerned about what will happen when we have quantum computers large enough to, but we still have a little bit of time before quantum computers get large enough to make this an actual problem. But there's no theoretical reason. I mean, okay, factoring is probably not NP-hard, but you know let's say let's say it proved that P doesn't equal NP maybe that doesn't that would be amazing obviously but that doesn't if it proves that P equals NP that would be completely that would be very interesting that would be very interesting I'll never ask about more broadly in the context of complexity theory but presumably OpenAI or any other company would not drop that result carelessly

that's the kind of thing that should go through yeah the government. Sometimes certain mathematical results quite obviously lead to dramatic impact on the world. Other results might take maybe months to digest but that one would affect the economic systems of the entire nations. How many mathematicians does the NSA enjoy? Thousands. I've always done this thought experiment. what might they have that they just wouldn't tell us about? And this was 10 years ago. Some evidence showed that collaborative teams of mathematicians could solve much, much harder problems than any one individual mathematician. They broke it down in the right way. So, I don't know. Do you mind if I just revisit the balloon for a second? Because I actually like the semi-group analogy better than the convex hull.

but I don't think it should be interpreted as derogatory as it is, even if it was just that these machines are interpolating human knowledge rather than extrapolating. Because even if there is this convex hull of ideas and things that could be pieced together from bits that humans have invented, we're not doing that fast enough. There's a lot of opaqueness, and this balloon is not see-through. And so even if you can make this convex hull much more transparent, kind of lift the fog, give mathematicians radar so they can see what's over the horizon, that's pretty incredible. And I don't think we should be interpreting that as derogatory at all. I do think it is more extrapolatory than that. Yeah, we should say probably the size of the convex hull. We don't know actually how big it is. Well, that's part of the point. We don't know how big it is. We don't know what's next to each other.

There's so much fog of what we know. And that's exciting to lift that up. But human beings do make new definitions, bring new concepts in. And that fundamentally feels different from combining existing ideas and repackaging them. When human beings do that, and they do that from time to time, it really feels like, okay, there's a new boundary point. and now what's the context of that, of what you could prove? So it's like you take the balloon and there are ways to pull on it. And we know when it expands. Like somehow we have an intuition for when it's getting bigger. You can recognize that. In many ways, the archetype of what the researcher looks like is going to change because of this new technology too. And I'm always amazed that when you look back historically, how much very different fields of math really spawned from similar ideas. And a lot of those connections drove the development in the beginning, but over time they become a bit more specialized.

And I think certainly the way that we train graduate students, you know, we'll need to make sure that they're not just over-specialized in the same way. You know, I don't know what the future of hiring will look like. I don't know what the future of journals will look like. But I think people who think broadly about, you know, a bigger swath of mathematics will be in the best position to benefit from these technologies. You know, that would be my guess. That's an almost silly, sounding question, but a large part of many decades of research has been the paper. If you were to guess in the years to come, how does that change? how a human contributes their idea, maybe collaborate, how a team of humans contribute an idea into the public space. So, I mean, I don't know about in pure math, but in engineering and some of the applied sciences, we already have a pretty rapidly shifting view of what an academic contribution is.

So if you write code in part of a larger infrastructure that is reusable, that, you know, supports larger projects, So think of LIGO or some of the big billion dollar astronomy projects or cancer research projects that require hundreds of people and huge resources already it not necessarily one 15 PDF You know, there's lots of different pieces, code, data, reusable bits that can be fit into a larger framework. And in my mind, that is kind of obviously even more relevant with AI, is having, you a little workable piece or widget or API or code base or data set that can be leveraged by the community or by AI is increasingly valuable, even more than just like one result or one paper. And I don't know how that looks in pure math. Can you comment on your view how everything's changing? even just observing the insane evolution of even archives,

which is similar in some ways to what's happening on GitHub. For example, a lot of people in the open source movement are really struggling to know what to do. Some are predicting the death of GitHub. Why? Just because there's so much code being added to GitHub. Yeah. Seeing the growth. So one of the things that makes the open source movement successful is there's a kind of one or a few folks that really manage each repository. or the human beings, and they are psychologically and on many levels are becoming overwhelmed by all the pull requests. So it becomes impossible to manage, and they get exhausted, and there's this whole feeling. The open source movement, maybe similar to the academic environment, requires the humans to be motivated. And if you get worn out by the just similar to the ideas that Terry Tao mentioned, that he just quit. And so that's a fairer mode that you want to kind of avoid. So journals were already in mass under very, very severe strain from the number of things before AI,

having nothing to do with AI, just there was more people producing more knowledge, sending it to journals, and journals had to referee it, and referees are exhausted and being asked to work really hard to understand these theorems and check them. The only thing worse than that system, I think, is not having that system. because the way that, like, our mathematical epistemology, the way that we arrive at truth, is deliberately slow and careful and precise. Go back to Pullman's solution of the Poitier conjecture. He puts these papers on the archives, and then people around the world are, you know, shocked by this, and so they invite him to come give lectures. He comes to Stony Brook, he comes to NYU, he comes to Princeton. He's going all around the country and the world, and he's giving three-hour-long impromptu explanations of, you know, maybe an hour is him explaining what he's doing and the next two hours are people saying, on page 247, lemma 412, I don't understand how you do this.

And then he goes to the board and gives them a half-hour lecture on how he does that. So the entire thing lives in his mind. He's done the best he can to communicate it on paper. and then it's up to us, the community, to go back through it very carefully, very in a detailed and precise way and agree, yes, this is a real proof of the Plunk-Grey conjecture. And that took years to do and a whole lot of human effort. So coming up with a solution is only the first step in the process. And that process has to be deliberate and it has to be community-driven. And so that's what the journal system is trying to accomplish. So you make a really interesting point. And it kind of highlights this traditional view of a pure mathematician as this lone genius who solves a singular problem and then goes explaining it to the world. In many other fields, we have moved past the point where we can't individually advance circuit design or aircraft design or protein folding or any astronomy.

And we have to work together in these billion-dollar collaborative projects. And, again, I'm an engineer, so I'm thinking leadership is obviously the thing that a community needs when it's having this kind of crisis of organization and individuality moving towards a more collective model. And that's interesting because I know so many mathematicians and, like, by kind of organization, this is a very independent group of people who don't like being told what to do, who don't like being told what to study, who are very independent by their very nature. You know, we like solitude and working alone. That's why we, you know, spend so much time doing it. I think that's a really good point. I mean, it's interesting just to see the differences in working styles, even between math and physics, you know, with things like the Large Hadron Glider. So, I mean, let me take a contrarian view for these things, which is, you know, you should never let a good crisis go to waste. I agree.

The journals were overloaded. The conferences were overloaded. It always strikes me that we're doing so much work, you know, to review these things, handle all these things. And now we're faced with the task that someone's got to review all these AI-generated papers. And some of them are important, but, you know, it's really hard to get your community on board with doing that. I would really, you know, I don't know what's going to happen, but I would really be in favor for finding new models, not to replace the old ones, but to relieve some of the pressures that have come about. So, you know, I'd love to see models where, you know, maybe we take a more collaborative effort to verify some of these AI proofs, right? So some of the OpenAI results that have dropped, you know, improvements on fast Fourier transforms. You know, there's been tweet streams with people improving the content. I don't really know what to make of that. You know, I'm not even on Twitter. And it seems to me that we need to figure out other modes of working. for organizing what are the state of all the improvements,

but also how do we organize and tame these ideas? So it's kind of striking that the way that we think about publications is you work hard on your draft, you post it on archives, and that's your draft. You might update and create a new version at some point or a third version, but the draft belongs to you. It's not really up to the community to add notes. Hey, I think you can do this argument differently. way, hey, I think this is simplified this way. But maybe there's some kinds of things that we need other models for, you know, allowing crowdsourcing for our understanding to evolve. I'd be all for that. You know, the other thing is I love giving research talks. You know, one of my favorite parts about giving research talks is you can take all of the messy complexities of the paper, the things where you just knew you had to dot the I's and cross the T's, but that wasn't really what excited you. There was some key idea, and there was some special case that really illustrated why that idea was powerful. And that's the part you really want to explain to people, because when someone asked me about Lemma 57, I don't even remember it.

And maybe this idea that a paper is just this monolithic thing designed to be written down and completely formalized, Well, now we can trust lean formalizations to some extent for some of the technical building blocks. And that can allow us to rework the way that we do the writing in the first place, so we can put more of the emphasis on the work on, you know, the promulgation of those ideas. The MassLib model is exactly what you were describing, where instead of a paper, you have a PR, you have a pull request. The maintainers and the reviewers will go through the pull request. They'll make suggestions here and there. I mean, the referees make suggestions you can improve this and that, but it's not a public process in a journal. In GitHub, open workplace, you can see exactly who contributed what idea that simplified an argument, or you're reusing this, you're reinventing this idea that already exists in this part of the library. You can just quote that and re-actory this part, whatever.

And Mathlib, working formally, allows one for the first time. One of the problems with mathematics is that we don't have a perfectly well-written specification to our theorems, our technical lemmas. If you open up a random page in a random math book and read a lemma, there's so much context that they're all set. Because it's so annoying to write something that fills in every bit of context. So we just assume you've read the previous 60 pages. Now I can just tell you what the next step is in that lemma. But then when you try to, that means you can't just take it off the shelf the way you can a part, a physical part when you're building a rocket. And so if all of mathematics is done in the future formally, then the specifications are built into the object. You know you're not using it in the wrong way because you can't use it in the wrong way because we won't let you. And that is the sort of crowdsourced version of mathematics. And that was the dream of mathlet when it was started 10 years ago. was to run mathematics more like a science.

As a quick aside, can you mention what Mathlib is and lean? Just because we'll mention it a few times for folks. Sure. So mathematics is, in the ideal, a formal process, starting with an axiom-tied system and then logical rules of inference from there. So people think of, I don't know, do people still learn Euclid's geometry in high school or something? That's supposed to be the kind of model for kids to start seeing proofs for the first time and seeing an axiomatic structure. And you can take that idea and write it in software and say, here are the actual axioms of mathematics, here are the rules of logical inference, and Lean, there's a long history of this kind of software. It's called Interactive Theorem Provers, where you're writing line by line in a proof, and the computer says, yes, this line does follow by logical inference from the goal state on the previous line. So Lean is software that allows you to check all kinds of things. You can check the specifications of your code aligned with what you were trying to define.

That's what it was invented for, was bug-free code. And hopefully in the future we will have bug-free code because AI systems are really good at finding bugs in code written by humans. So it's becoming ever more important for that application. But another application is to check mathematical results. and allow you to write a mathematical result in a very formal way. And so now Lean itself is just a hardware that checks the logical steps. You do not want to start every math project by defining what the natural numbers are using zero and the successor, you know, the piano axioms of the natural numbers. So it's nice to have a large general library that's extremely well maintained, extremely well organized, and so on. And that's what MATLIB is. Mathlib is the Lean library for formalizing mathematics in the Lean ecosystem. And again, Lean is just one interactive theorem prover. There are lots of others. It's the one that so far has been winning the math wars, if there are any math wars,

the company has two people or something. And it's the technology used in part to show that the OpenAI release of all the different proofs, that the proofs are correct. That's right. It has become now standard to, because auto-formalization has gotten to the point where AI systems can not only prove a theorem in natural language, you can't trust their natural language proof. They're just stochastic processes. And so the fact that they're saying things doesn't mean that they necessarily have a model of mathematics in their heads, whatever that means, the same way that we do. but that's okay because I'll write a lean code that builds on top of an athlete, builds on top of the number theorem plus project or whatever and checks that it has indeed solved this problem down to the axioms. So this is where I'll just take the tangent and the SAR experiment may be a bit philosophical at this moment but what if you have that black box that gives you proof that you can't understand? Let's say the more interesting case where

not because it's so big but because the idea is just don't make sense to any of the world expert mathematicians in that field when you deal with that. So the only way, sort of almost by definition, because the logical rules of inference are written down, and there's only a few of them, there's a handful of them. The axioms, there's a handful of them. So the only way that you have mathematics that's not understandable is scale. Let me add a contrarian take. One of the things I also wonder is what would unformalizing math look like? You know, certainly formalization helps tremendously that we have confidence that the results are correct. But much of the way that we understand the results ourselves is really at some unformalized level. You know, what is an interesting special case? What, you know, where does this come from? You know, this technique you're using, right? And, you know, maybe that's what that world would look like. if we had these black boxes, we would try to tease apart and figure out how to ask them

easier questions to develop some kind of system of understanding of how they actually got to the answer. Yeah, so there are really two different things that we conflate when we write a paper. One is the engineering of the proof, and the other is the communication of that to someone else. And they're almost at odds with one another. I had this experience, I just taught real analysis last fall, and you open up any real analysis textbook, and this was my experience when I at real analysis. The first couple of lectures are what are the real numbers? You know, we have to define them or construct them as equivalence classes or quasi sequences or rations, whatever the hell all that means. It takes like two weeks for a first semester course in real analysis to get there. And then we have the real number line and now we can start doing calculus, you know, with formal epsilon and delta definitions, yada, yada. And it always struck me as bizarre that we do this because before the class, I knew what the real numbers were. It's this number line. and then we spend these two weeks talking about question sequences of rationals and then on week three, they're a number line again.

What the hell was the purpose of that? And so it's because the way things were written or had been written were very engineering. You need to construct real numbers before you still start using them. But the real numbers, human beings, use the real numbers long before we understood that they needed to be constructed formally. So what I did in my class is I jumped straight into epsilon-delta calculus. We did, you know, six weeks of calculations until we got to a point where, wait a second, you know, here's a theorem, it's true in the real numbers, it's in convergence, something called Balzano-Varstrans theorem. You have a sequence of real numbers that are jumping around but in a fixed range. There has to be some subsequence that converges to a real number. Well, that's not true in the rational. Why is it true in the reals? What are the reals? So you don't start asking that question until you hit a problem that needs to be solved. So the way I taught the course was sort of backwards from how it needs to be taught if it's an engineering project. Because you can't just start talking about the real numbers until you've constructed them, until you know what they are. Well, that's correct if what you're doing is building up the theory.

But if you're telling a story, then you absolutely can just start with the real numbers. You have an intuition for what that is. And until your intuition breaks, you don't need the resolution of how you actually construct them. So I think the way that we write papers in the future is going to be very different, where we spend a lot more time just talking about the story of what have I learned in this process, what are the interesting, you do the same thing here, here, and here, but then here you're sort of stuck in where do you go, and here's an idea for why you go left and not right and not up and down or whatever. And there's a lean formalization that accompanies this story that lets you know the engineering actually works out. Because you can't just tell stories. If you just tell stories, you know, math is not music where you can tell a beautiful story and it's okay, the parallel fish is moving somewhere and, you know, you've broken some rules, but Bach does it all the time. So I think that's a very important point, just to sort of emphasize the point that Alex was making. You know, there's this view of mathematics, which I don't believe in, where it's just this rising tide of theorems.

You know, how can AI be bad? It's contributing to this rising tide. But, you know, really mathematics is not just about the number of theorems we have, far from it. It's also about that story of fighting with problems. Because when you fight with a good problem, it fights back at you and you learn something about it. It's really about the system of beliefs and abstractions you come up with in route to, you know, when you're fighting this problem, you discover some more general meaning that applies more broadly. And I think it's a great opportunity to sort of disabuse ourselves of this idea that we're just, you know, axiomatizing things until we get to the next theorem. You know, telling that story of how we actually got to these problems, what the parallels are to other problems, is now, at this pivotal moment, more important than it ever has been. To get all the way back to your question about, you know, is it a matter of scale? Well, yes, it's just a matter of scale, but exactly to Anker's point, the way that we think about mathematics is about stories,

and it might be that we don't understand the story of this. You can follow a proof line by line and not understand what's really going on. You might say, and you just ask some other LLM to explain to you the story, if it can come up with such an explanation. But that's what we want. We want stories. We want understanding. When you fight with a problem, it fights back. I think there's a deep truth to that about all the great pursuits that humanity has taken on. And on that topic, I would like to talk to you about some of the greatest fights of all time with some of the hardest problems in mathematics that had fallen. But first, I'm about to break, if it's okay. Sure. So Alex, you're a number theorist. Maybe it's fair to say the central problem in your field is the REMA hypothesis. And you showed a lot of excitement on the aforementioned Twitter X about the quasi-RH,

the quasi-REMA hypothesis proof. Can you explain that? Can you explain what the REMA hypothesis is? And can you explain the nature of your excitement about it? Sure. There's sort of a one-minute version of the Riemann hypothesis, or a five-minute version, or a 30-hour version. I don't know how far you want to... Let's stop pulling at that string. So we know what the prime numbers are. People are interested in the distribution of the prime numbers. And what Riemann discovered is that there's a function, which is totally crazy, that there's a function that takes complex numbers as inputs and has complex numbers as outputs. and that function governs the behavior of the prime numbers. It's called the Riemann zeta function. And by the zeros of the zeta function, we mean complex inputs where the output is the number zero in the complex plane, so the origin in the complex plane. And the question is, where are these complex zeros? What inputs, what complex number inputs give you a complex number output of zero

when you apply this zeta function? Why in the world should that – complex numbers? What are you talking about? We have these discrete numbers that aren't divisible by anything else sitting in the natural numbers on the number line, and now you want to talk about functions from a complex space to a complex space. That was the great breakthrough. The great breakthrough was to figure out how to do signal processing on the prime numbers. if you think about the reason people can hear your podcast is that my voice box is producing pressure waves of sound that's the natural space then that goes into the microphone the microphone analyzes the Fourier analysis on it, converts it into amplitudes and frequencies and it's just a bunch of numbers stores those numbers or sends those numbers the speaker then takes those numbers and reconstructs back out what sounds to people like my voice. The same thing is going on. That's what the zeta function does for the prime numbers.

It takes the prime numbers, you can think of the prime numbers as particles of air molecules wiggling around, and it does a signal processing analysis on it that can recover the same prime numbers back. And the frequencies and the amplitudes correspond to the zeros of the zeta function. The zeta function has that information in it. And the Riemann hypothesis, so the amplitudes of all of these, you can call them Riemann waves, and the superposition of those waves recovers the distribution of the primes. And all of those amplitudes are supposed to be exactly the same. Like the orchestra that plays the music of the prime numbers has every single instrument sounding at exactly the same amplitude. That's what the Riemann hypothesis predicts. So can you actually, we know what the prime numbers are, but maybe going into the space of philosophy and the romantic notions of aesthetics, why are they so interesting and important and central and beautiful at the core of mathematics?

And connected to why Riemann hypothesis is exciting is why is it so difficult to figure out if and what the pattern is in the prime numbers? Let me start with your second question first. Why something is difficult, I only know how to answer once I know what the answer is. Right? Maybe it's not difficult. Okay. Maybe it's not as difficult as we thought it was. I did not think I would see a proof of the quasi-remen hypothesis in my lifetime. So the fact that I saw it on Tuesday was extraordinarily shocking. What did you do the moment you saw it? Like, in 30 seconds after, like, what was your view number? Yeah, I remember where I was the second I saw the drop. So we were at this National Museum of Mathematics gala in New York City. And so I had a whiskey in one hand, and I kept refreshing. I was talking to people, socializing, but I also kept refreshing because I knew it was dropping around 6 o'clock Eastern time.

And then the drop hit, and in fact, I was talking to a quantum reporter there, and we were just chatting, oh, yeah, they're going to drop these, you know, 300, 400, we didn't know how many 700 it turns out problems and I was like oh wow that's really cool, it dropped. So I'm looking at my phone and like the second thing I see is quasi RH and I'm like what do you mean quasi RH? You don't mean like for what? And I look and it's the zeta function and it's all Dirichlet L function so they spelled no zero zeros and I'm like that can't be what's in this drop. That I could not compute in that moment what I was seeing. That speed of progress in that little time, yeah. It was so far outside my model of what was possible. And it's been a month of things happening way outside of my model of what was possible. From our last theorem, formalization, I was sure we were three years away at least from being able to do something like that.

So maybe you can elaborate on that. It was in Lean. It was formalized. How big was the formalized left? I don't remember. 13 million lines. 13 million lines. Okay. So I just, two days before, I gave a colloquium in the Princeton math department where I explained in painstaking detail what Lean is, what Mathlib is, but also that AI systems. Today, you know, there's this Mathlib library where everything is kind of perfectly done, exactly the right level of generality. There's not supposed to be any code that's reusing code that's reproving theorem somewhere else. Everything is the most general, the most abstract. And what AI systems are capable of doing is building like one or two layers on top of that. But what they're doing is jerry-rigging. They have not been trained yet. I'm not saying they can't do it, but they're not good at it right now, at what I call canonization, which is taking kind of sloppy code, refactoring it, figuring out exactly where in the library it belongs, what's the canonical version of this statement, of this lemma, and then everything, every proof should just be a combination of

you apply this lemma from over here, you apply that lemma from over there, put all these things together, there's your theorem. And then it's really clean, it's really beautiful, very elegant. That's what MathLib is going for. And what I said in my colloquium is where we are today is that you can get entire textbooks formalized as long as they're in the MathLib halo. as long as they're like one, maybe two steps removed from things that have already been done in MathLib. But when you start, and this had been my practice, my experience had been, when you start asking to do things that are, you know, three or four steps removed from the library, the way that it came up with a definition of this object and the way that it came up with some other definition of the same object, because it was doing it for two different purposes, then they clash, you can't stitch these things back together, Like, you really need the library, MathLid, to grow in order to get to a place where you can do research manuscripts, you know, that are really using, synthesizing tons of different areas of mathematics. Which is why you would expect for Mazda Serum to take, because it's so far outside of the Hayward, MathLid, it would take quite a bit of time.

Exactly right. So Kevin Buzzard had a million-dollar grant to spend the next five years, and maybe he's already a year or two into that grant, to reduce the formalization of Fermat's Last Theorem to things that were known in the 1980s. So he wanted to do that last step, those seven years of Wiles' work and Wiles Taylor, to finally finish Fermat's Last Theorem, but built on top of things that we knew would be extraordinarily difficult. He wasn't going to do Langland's Tunnel. He wasn't going to do Maser. There were all kinds of things that were expected to be even much more difficult than what these five years would take. And everything I said in Echolokin was wrong. And I think Echolokin was on a Wednesday. On Friday, there was a drop of 13 million lines of code from the axioms for Maslow's theorem. It's proved the statement of Maslow's theorem is completely elementary. There's no semantic alignment to check. You just glance at it. Yes, that's the statement. As Lune says, it's already free. Can you explain how it was done? How it was done. And so it was anthropic.

And how it was done was AI systems just got really, really good. It's still not a nice proof. It's not the clean proof. There's still a ton of work that needs to be done to make that proof a massive quality proof. And I think that's important work and work that we want to do. But in terms of just getting that sorry-free theorem, they just blasted through all of these. They jerry-rigged this. They jerry-rigged that. And, you know, 13 million lines later, they had a sorry-free, formal, lean proof of from our last theorem. It's just absolutely shocking. By the way, it's beautiful to always, because of how rapid everything is, there's going to be a lot of statements that everybody makes, the world experts make that are wrong. That's the whole point of it. And that's the fun thing about human discussion is we have disagreements, and I would like to say somewhere between 40% to 80% of everything said today will be wrong in five years because everything is changing so fast. But that's the whole point of human discussion, and deliberation and discourse and even the scientific process is saying shit and seeing

what sticks over time. Yeah, over 10 years, over 100 years and so on. I was describing my experience. I had a lot of experience with Lean and with using AI. I started doing Lean before AI existed in the sense that we understand it now. And it was very painstaking. And the second AI systems came on, you know, the first LLMs were coming online. They were crap at lean, but then they got a little bit better, a little bit better. And I was like, oh, my God, I can do library search. I have to sit there looking. I know there's a lemma that does this, but I don't know what it's called. And I don't know which exactly. Is it in the manifold library? Is it in the algebraic geometry library? Like, give me this lemma. It was great at that. I was like, oh, this is amazing. I'm going to keep using these systems. And every bit of my experience told me that they can do, like, X but not 10X. And this was a million X. completely outside of the distribution in my mind of what systems were capable of. Do you have a ballpark estimate of how many humans were involved in that and what is the level of human involvement in generating those 13 million lines of code?

I believe it was two or three humans, and I believe most of their involvement was keep going. All fast. Do you think it's really just like gentle guidance, not like pathway creating and As far as I understand, there was no domain expertise on the parts of the people who were doing it. As the meme goes with Open AI and Drop, the prompt is solve math or something. Do a breakthrough. Yeah, do a breakthrough. Do a breakthrough. That's the prompt. And it goes and does it. Of course, you never know from a company perspective. There's a lot of incentive to sort of sell it as that, even though there might have been a lot of intermediate steps that led in any stages of the training or maybe some things left unsaid, even though the final prompt is do a breakthrough. There could be a lot of groundwork before that that led to, we don't know. So I remember that was a Friday because I just was completely, like my mind was completely exploded.

and it was Labor Day and we just took the kids to the beach and I needed a few days of, you know, I'm not checking my phone, I'm not on Twitter, I'm just throwing my kids into the ocean. Did you have that moment where you were just sitting and looking at the ocean and contemplating? Like, what does it all mean? Yeah. So I had already gone through that a month before. Yeah, and I was like, so now what that says is that auto-formalization is basically solved. It's not solved in the way that I would want it to be solved. I want, and people are still going to work very, very hard on this. There's all kinds of other things that happen, just like there's all kinds of other things that happen when you prove theorems that are byproducts of that process for human understanding. There's all kinds of other things when you work on making a beautiful library of mathematics. And that process will continue. That process will be very difficult. It's going to be the, you know, thousands of man hours, people around the world contributing in this open source way. but in terms of can you get a sorry free

proof of something that human beings have written down on paper I think that problem is done and so you were staying there with a whiskey having gone through this a month before and now again this was round three when I got back from Labor Day it was Navier Stokes I was just mourning it's FLT you're going to do this to me but now I mean this really hit home with the quasi RH so maybe can you speak to the importance and the beauty of the primes and also how big and hard the leap is from quasi-RH to full RH yeah so the primes the reason the primes are so valuable is that they are the fundamental particles in the theory of multiplication every whole number is broken down into a product of primes you know we know about the elementary particles in physics and all of matter is made up of that in the same way the primes are the particles that make up

the multiplicative structure of whole numbers. And their distribution had been a mystery and remains a mystery. Even the Riemann hypothesis doesn't solve everything we want to know about prime numbers. But the Riemann hypothesis was so beautiful because it pointed the way to what the distribution should be and it has a thousand applications in very far away fields. If only the Riemann hypothesis were true, then we would know X, Y, Z. So let me interject a little story that came up last week, which was pretty entertaining. So we had Grant Sanderson was visiting IES, and we had a long discussion about what is AI doing to the math community. And I really enjoyed this comment that Camilo DeLellis made, which was, I'll do my best impression if I can do the accent, but there was a play written in the 1950s by mathematicians. Who knows that mathematicians are writing plays, but apparently this play, the premise of it was that von Neumann died, and he went to heaven. And he met God, and he got to ask him a question. So he asked him if the Riemann hypothesis was true. And of course, everyone

holds their breath as they're waiting to hear the answer. And God says, yes. So finally, everyone can exhale. And then von Neumann asks, but how do you know? And God thinks and says, well, I know where all the zeros are, and they're all in the r equal to half line. So my understanding is the premise of the play is that that's not enough for von Neumann. He gets a little bit agitated to try to find out more about why it's true, and I think he ends up at the end in the other place. I don't know what to draw the moral from it, but, yeah, it seems to speak to the mathematical experience. Math is not the fact. It's the why. It's always been the why. It's our deepest quest for understanding. And if you just tell me a fact that's true, but I don't understand why it's true, That has contributed zero to my knowledge. So primes, I think, are also interesting because they relate to things that a three-year-old can understand. Like, fundamentally, we know how to add one to numbers.

And, you know, there's this breakthrough moment when you're a kid. I can add one forever. That's a big idea. I can keep adding one forever. Then you learn how to multiply and group, and that's also a big idea. and primes are interesting to me and have always been interesting because we don't understand how adding and multiplying work together. The two most fundamental, basic, simple ideas in all of mathematics, we don't know how they work together. I can add one to a number that I can factor and I have no idea what happens. I know if it's even, it becomes odd and vice versa, but I don't know if a number that has many, many composites becomes prime. We just fundamentally don't know that simple, you know, adding one, how that changes multiplication. And primes are basically, you know, just like the distillation of that three-year-old mystery. And it's a mystery that all of us still wrestle with. You know, that would be the interesting bit, is if the proof told me something fundamental about that structure that I could actually use.

Yeah, to the three-year-old, I think my older one was something like two or three. He's playing, we had these little blocks, you know, cubes that kids like to play with him. and there was like 20 of them in the pack, and so he would make a little 4x5 grid, and he liked that, and he would make a 10x2 grid, and that was kind of cool. And then one day I look over, and he's like really frustrated. I'm like, what's going on? He's like, you know, it's not working. And one of them had rolled under the couch, and he only had 19, and he couldn't figure out a way to make a nice-looking rectangle out of the 19. It still bothers me today. Yeah. So, absolutely, kids love primes. Kids can discover primes. And the solution is to lose another, right? Well, exactly, yes, yes. Do you think, so the leap between quasi and RH, I presume you think humans will not, in your lifetime, be able to fall without AI's help before we know about this. I don't know. Can AI, I guess, is the follow-on question. Like, how hard is it? I don't know.

I don't know. I did not think I would see a proof of quasi RH. now that I see what the proof is. And there are a lot of technical details that I still have to go through for my own satisfaction. There's a lean formalization. The lean statement was already in the library. It was already in MathLib. So the fact that they have a sorry-free proof gives me 100% confidence that the proof is right. And moreover, having read the proof at a high level, the ideas that it's using are all completely valid. They're brilliant to embed the Mobius function, in twisted bisectic characters. This is all stuff that's the bread and butter of analytic number theorists that was chained in just the right way and was absolutely incredible. And so what Riemann showed is that, remember we were talking about this function that a complex inputs and has these outputs, and we're worried about when the output is zero. So he showed that there's what's called the critical strip, which is a vertical infinite strip in the complex plane, where the real part is, so you can think about the x-coordinate, lies between zero and one, and the imaginary part, the y-coordinate, can go arbitrarily high or arbitrarily low.

And the hypothesis is that the zeros all lie right at the dead center of that strip, where the real part is a half. That's exactly what Anka was just saying. And what we knew until Tuesday was a region where there are no zeros of the zeta function, that's like getting thinner and thinner the higher up you go. It goes towards the one line. The one line is something called the prime numbers theorem, which tells you the basic distribution of the prime numbers is equivalent to the fact that there are no zeros of this function on the line of your part of s equals 1. So if the x-coordinate is 1, as you go up and down that line, you will never find a value where when you apply the zeta function, it hits 0. Okay? So we have this zero-free region that was getting thinner and thinner the higher up you go, and that gave us the best-known distribution on the prime numbers, the best error term in the prime numbers, which I can go into. You tell me how much math you want to know. Let's go. But what I could have anticipated is that it would thicken that line a little bit.

It would still be going to zero. The thickness would still be going to zero as you go up the imaginary axis. But it would be better than what we currently know. That thickness hasn't moved in 70 years, but I could sort of imagine that it could maybe make some progress. What it did is it gave us a vertical strip. So there are no zeros with real parts between 7-8 and 1, an entire infinite vertical strip. That's the quasi-rewinner process. And if you can get that 7-8 down to 1 half, that's the full-rewinner process. Now, the methods that it used in the 7-8 proof, and there's a second proof that's about 11-12, never mind the technical details, that method sort of fundamentally can't get down to 1 half. So a new idea is needed. You need a new idea, but quasi-RH needed lots and lots of new ideas, and it came up with that. If you're, having gone through this three times already, is your intuition, and this is all three of you, about re-measuring in your mind the hardness of the various problems, because you mentioned, like, picking easy things? That presumably you have a good hardness radar.

Like, this problem is probably solvable, and this is not. Solvable today, solvable this week, solvable this month. Are you readjusting to this new world? I'm readjusting my inability to predict it. Yes, exactly. Well, you had the ability, no? It's noisy. Sure. It's a larger amount of noise than I thought. And I think we've always known that there are problems we care about that are within reach of the tools that are out there right now. But it's just surprising the scale at which, you know, how many different problems really fall into that category. So we also don't know a lot about, you know, apparently they prompted thousands of problems, and what they dropped were only the positive solutions, but they only had about a 10% success rate. So we're sort of left guessing, you know, is there a very jagged boundary that we could squint at and say we can make sense out of that distinguishes, you know, what did fall versus what didn't fall?

I think one of the other dangers is that maybe the problems which are the safest are ones which you look at and it's just painfully obvious you need a really creative new idea. You know, because you can sort of tell that maybe there's a barrier and the things that are out there in literature just don't seem to fit into that hole. But now I think one of the things that's changed the landscape mass forever is, you know, maybe people become a little bit more secretive with those ideas than they were in the past. Because once those ideas are out there, then it might be a lot more straightforward for AI to be able to balance that one new idea, that missing ingredient, that new tile it needs, and be able to fully exhaustively explore what it can reach out of that. So you might be a lot more protective of your ideas than you were in the past. I think that's also a bit scary because traditionally the mathematics community has been really open to sharing ideas, and that would be something bad to lose. But that's also because we had this first through the gate mentality, and I think that's also going to change.

That's right. I would prefer that the openness stay in the first through the gate. That's right, but that'll take time. It'll take time, yeah. So that could be an effect, you know, that's more transient in nature, that if we can learn how to value other things, then maybe that'll go away. There are other communities that have models for how to value contribution, even when it's not lionizing one individual's contribution. So it's not that there's not a way forward, but, you know, that is in the short term a real roadblock to doing mathematics the way that we used to and enjoying it the same way that we used to. So speaking of which, Steve, you told us a story about soup that made you maybe fall in love with the bigger problem of flu dynamics. So going from math to applied math to engineering, the whole thing. So maybe can you talk about Navier-Stokes, these set of equations that explain fluids, why it's such a big deal, why it's beautiful, similar as we talked about primes, why is it so hard and why is it such a big deal that Navier-Stokes was proven?

So fundamentally, the Navier-Stokes equations is just a way of stating the conservation of momentum for fluids, for objects that can move kind of freely like gases and liquids. So we know F equals ma is conservation of momentum, you know, classical Newtonian. Physics, the same basic principle applies to the gas in this room, to water flowing. And it's true in any volume. So it's true in little volumes, big volumes, kind of imaginary volumes. And if you write down that conservation of momentum in a volume, and then you kind of take the limit as that volume becomes arbitrary and small, you get these Navier-Sokes equations. It's a differential equation, a partial differential equation, that describes how the velocity field of a fluid changes in space and time, and also the pressure related to the forces acting on that fluid. Can you give me a question about why fluid, given that F equals MA is a pretty simple equation,

why fluids are such complicated systems? F equals MA is simple to write down, but it can get complicated really, really quickly. So if I have, like, a single pendulum, it's really easy to write down the equation for that single pendulum. If I put, you know, another pendulum on the end of it, now I have a more complicated equation, and it gives chaos. fluids is, you know, in a sense, if I try to model the error in this room, I could chop the room up into a billion little boxes and write down a little differential equation for how the momentum works in all of those billion little boxes. But now I have a really complicated system of equations. And so it's simple to write down. It's simple to conceptually understand what the equation of saying and representing, but actually solving it and understanding how the fluid actually moves in any kind of practical sense is very, very complicated. I read something like, you know, like one space where food dynamics is important is reducing drag for when you're flying, for airplanes.

I read something like airlines could save like something like tens of billions of dollars by reducing the drag by 1%. So why is it so hard? So that's one. And drag is maybe easier than something like turbulence. So why are these things that are worth so much money so difficult? So the good news is that if you can improve our ability to predict turbulence, to simulate it even a little bit better, faster, more accurately, there are dozens of fields where you can have a huge impact, both economical, ecological, human health and safety, with a relatively small improvement. So drag reduction on airplanes, better artificial hearts, more efficient wind turbines and gas turbines, you name it. The world is we live and work in a fluid, and so do all of our machines. The inverse design problem, I know it's actually very closely related to this kind of like math problem.

I know what I want to prove. I don't know how to get there. I know I want a wing that has high lift and low drag, but I don't know how changing the geometry, changing the shape of that wing, will affect this whole cascade of smaller and smaller and smaller vortices, you know, this turbulent fluid that generates over the wing that eventually results in its lift and drag. I don't know how to inverse design efficiently. Part of it's because the equations are expensive to simulate or to test. you know, wind tunnels are expensive, you need supercomputers to simulate these things. This is actually kind of related to pure math. We'll get back to this, I think. But there's this really nice Feynman quote that, you know, we know how to write down these very general equations, Maxwell's equations, maybe our Sox equations, Schrodinger's equations. But we currently don't know from the equation what phenomena is possible. You know, if you'd never visited Earth, could you have possibly predicted the sunsets and the tides

and all the phenomena just from, like, a partial differential equation? Like, we're not even close to there. And there's some very fundamental math questions about, you know, if I can write down a differential equation, I can't tell you what's inside of it. We still have to simulate. We still have to test. We still have to do this very elaborate engineering process to understand how these equations manifest drag on an airplane wing or turbulence inside of an artificial heart. Sorry, can I just ask a question on that? I've always wanted, so like, in the case of Einstein's theory of general relativity, he writes down these equations. A few years later, Schwarzschild finds a blow-up. And they're like, oh, come on. So what's wrong with these equations that there's a blow-up? You know, if you get enough matter into a small enough space, then like light can't escape it. We don't have any physical experience for this. This has got to be a mistake in the equations until we discover black holes. And then it turns out that actually the theory perfectly explains, you know, predicts what's supposed to happen. In the case of blow-up of Navier-Stokes, are we going to discover something new about water,

or is the fact that at some scale you get to particles, you know, you get to an atomic structure that does not follow the liquid equations anymore, last, why would we have predicted that there should be solutions to a continuous process that's allowed to be pushed to this limit? Yeah, it's a great question. I'm going to give you the engineering answer, which is these equations are an exceptionally good model of how fluids work at a certain scale. And you're absolutely right. You need some number of individual molecules to be averaged over in a volume for these to be constituent of equations that are valid. And so if you zoom in small enough, you know, now we're talking about individual molecules and maybe your source breaks down. So the mathematical problem of these equations blowing up is interesting from a mathematical standpoint. By the time they blow up, it stops being a fluid.

There's no black holes. Like other physics take over. But it's not a way. I think it's interesting from a philosophy of science perspective that where equations break down could, and maybe statistically high likelihood, is a window, a door, and a breakthrough of needing some other discovery. Like Newtonian physics breaking down needs Einstein. And the grand unification is that we don't have gravity at the same scales as we have interactions of subatomic particles. But this idea that we don't know something so fundamental about these equations, there are so many people on Earth right now simulating and testing maybe or soaks in every industry imaginable. I mean, it's probably one of the most studied and analyzed and used equations in the entire engineering world. and you don't know the fundamental property of the PD, does it blow up in finite time or not?

That's like the zeroth order version of the bigger question is, you know, what other kinds of solutions does it emit? Does it, you know, are there sunsets and hurricanes and tornadoes? All the phenomenology that could exist, you know, existence and uniqueness is like the simplest baby step. And that's where it gets interesting to me is, even if there's not a direct, tangible use of this mathematical result in an engineering sense, it gets us that one step closer to understanding the nature of these equations and what is possible, which is very fundamentally interesting to us. Could there be, could this solution that has the blow-up be used somehow in an engineering application and, you know, you want to drive it into this state and get some funny behavior that helps airplanes fly with less drag or something? Yeah, I mean, I think in principle, a huge industry, a huge part of what we try to do is manipulate fluid flows, like bend them into our will, right? So if you watch a bird fly or an insect fly, they are shaping vortices and carefully funneling energy into structures for their objectives of their daily life to perch on a rock or hunt.

And we try to do the same thing. So flow control is one of the highest challenges. It's very, very challenging. But all of our science fiction technology would be kind of accelerated if we had better ability to control these flows. And this proof is one very interesting construction of this kind of, you know, very specialized flow control in a way. I'm reminded that Feynman was asked about intelligence, artificial intelligence in the 70s. And he said something like, the way that we solve problems with technology is radically different from how nature solves these problems. Look at birds, how birds fly, and how airplanes fly. He was exactly pointing to this. And it's kind of interesting to think back to that comment now when we see what LLNs can do with fear improving versus what human brains can do with fear improving and all kinds of other applications. It's interesting that he pointed exactly to this issue of how we've solved flight using technology and how different that is from nature's solution to it.

Our airplanes don't fly. Now I wonder if birds have existential crises on the plane. Exactly. What am I even here for? Yeah. God, what are the different components of the negative succession? Can you dig in? Like what does it take to describe fluid? Yeah, so very, very basic. A fluid moves, and it moves differently in different places and at different points in time. And so the kind of basic building block of a fluid is the velocity field. So at every point in space, there is a velocity vector that tells you how that fluid is moving. You know, maybe it's really fast here and slow here, and it creates a vortex. And it gets more interesting when the fluid velocity changes a lot over space. So like a tornado is a very rapidly changing fluid velocity field in both space and time.

There's pressures that are involved. So again, like the core of a tornado is low pressure. So pressures are kind of forcing this velocity field and causing it to move in the way it does. That's the building block, is a velocity field and its pressures. And then the equations tell you how that velocity changes in time and in space according to kind of its own momentum. So fluid has momentum to carry itself. That's, you know, gusts of wind. It has internal friction and resistance, that's viscosity, so honey has more internal resistance than water, has more internal resistance than air. And then all of the external forces like speeding and pressure and gravity and things like that. So each of those can be quantified as a term in this larger Navier-Soak equation. So speaking of external forces, can you describe the mass side of this? The Navier-Soaks was a blow-up. what was the formation of the problem,

and what was actually proven, and why was it so hard? If you don't have that internal friction, that viscosity, we have a strong suspicion that the equations can and will blow up under certain conditions. We have other examples, kind of simpler problems, that are like Navier-Stokes that do blow up. So the simplest one is just a simple, ordinary differential equation, if you don't mind me taking a little tangent. So X dot equals X squared. X is the state of my system. X dot just means the rate of change of X in time. And if that rate of change of X is growing proportional to X squared, that differential equation does blow up in finite time. So it just takes a quadratic non-linearity. Think about if I have X dot equals X. This is like X could be the population of bunnies. And we know bunnies like to populate. X dot equals X grows exponentially. You know, the state of my bunny population in time will grow exponentially.

It'll get bigger and bigger and bigger. It'll double and double and double. But it won't reach infinity until infinite time has passed. That's just a normal exponential growth. That's not finite time blow up. That's X dot equals X. But if I take X dot equals X squared that blows up in finite time So that quadratic non that quadratic kind of feedback is enough to cause this equation to blow up And you can like, if you write this down, you know, you can solve this using kind of classic basic calculus techniques and you get this finite time blow up. And so there's a progression of partial differential equations that have the same behavior where a quadratic non-linearity. And that's really interesting is the Navier-Stokes equations, one of the terms, the convective term that tells how the fluid kind of moves itself with its own momentum is a quadratic nonlinearity, just like x dot equals x squared. It's kind of a self-motion. Without diffusion, without viscosity, we think that that should probably blow up to infinity and finite time, how the finite time blow up.

But we don't know if viscosity, if the kind of internal friction and resistance like honey has always damped out those blowups before they can happen. My intuition as an engineer is, of course they do. Like if I have a flow where it's accelerating to infinity, that should immediately diffuse into its neighbors. It should create this diffusion into its neighboring fluid and kind of tamp down that growth. And so I think the math problem is essentially, are there any edge cases, any initial configurations of the fluid where you perfectly orchestrate this thing so that it could actually accelerate to infinity in finite time? And then there's two kind of technical versions of this. One is with no external forcing, and one is with smooth external forcing. Smooth being important because if the external forcing has singularities, then it's kind of trivial. If I force it with a tornado, you know, an infinitely tight tornado, then of course it's going to blow up.

The problem, as stated, has smooth external forcing. There's four versions on the clay website, one of them being the forced version. And that's one that was proven. And that's one that was proven. Exactly what was proven and what are some interesting things about the post that you saw? Yeah, so what is proven is that there is this way of orchestrating a smooth kind of external forcing that perfectly cancels out various turns in the equation so that you get this vortex. This can only happen in three dimensions, which is really important. We should probably say that what we've been talking about three dimensions is a big difference in 3D and 2D. Yeah, so I mean, when we talk about a fluid, we usually imagine it in 3D, but there are these kind of analogies or simplified 2D flows. Weather on Earth is pretty well approximated as a 2D flow. You mentioned somewhere in the talk that I heard that hurricanes are kind of like modeled well in 2D

and tornadoes in 3D. And I remember different sort of analogies that you were drawing there. Yeah, so one of the interesting phenomena in three-dimensional turbulence is you generally have this breakdown of big vortices into smaller vortices into little itty-bitty vortices until eventually they molecularly diffuse into heat. That's kind of this three-dimensional turbulence cascade. In two dimensions, like weather on Earth, you actually get a reverse cascade, which is why hurricanes form. You actually have little vortices coalescing into medium vortices coalescing into bigger vortices. But so the AI proof of this kind of counter example that you can have finite glow up basically creates this very, very specialized vortex that in the third dimension is stretching out and thinning it so that it can essentially become infinitely thin. The radius gets smaller and smaller and smaller as it stretches out, kind of like a figure

skater doing a spin. You pull your arms in, you go much, much faster. And there's this very specialized kind of rate of contraction in this vortex so that the energy in the fluid still remains finite, but at this localized filament, the flow kind of blows up and you get infinite vorticity. I was surprised when mathematicians before the AI proof started thinking it was going to be possible to find this counterexample, which only started maybe 15 years ago. That's right. It's relatively recent in the grand scheme of things. Instead of looking for a solution, looking for a counterexample. Because all of our physical intuition tells us diffusion does solve the problem. And so it is very physically counterintuitive. In the mathematics world, it's a really hard problem. Impressors to construct a counterexample. What does it teach to you as an engineer who's stared and studied at a lot of different natural systems that are fluids, and maybe take that as a step towards what are the equivalent kind of hardness

and harder problems yet to be solved in fluids. So it's a very important math result. Uh-oh. I'll say, like, day-to-day engineering fluids and applied math kind of analysis of fluids is almost completely unchanged. Fluids are just as hard to simulate and predict today as they were a month ago or two months ago. And so when I'm trying to actually solve an engineering problem with a fluid, I want to design an airplane wing or an artificial heart or whatever it is, I'm still having to simulate and predict and manipulate this flow in a very practical way that's expensive in terms of supercomputer time and wind tunnel resources. And this mathematical theorem and proof are very interesting, but it operates at a regime that's so non-physical that it doesn't really affect my day-to-day engineering. So, I don't know, this is very pithy, but I like to say, you know, we solved the million-dollar math problem.

There's a trillion-dollar engineering problem, which is, like, how do you predict these things? in a fast, efficient way and do this inverse design. That's a much harder problem. And you say it's a trillion dollar because it literally solves some of these problems. That's how much money you'll make, not because it's a prize, but like this industry. Yeah, and we could say, you know, trillion dollars of possible benefit to humanity of like better, faster solutions, lower drag, less emissions, you know, all of it. We have transportation, defense, health, energy. All of these have fluid at the very core technology. And so I think those problems are still very hard. And this math result is important in the sense I was talking about earlier that we don't know the fundamental character of what is possible in these equations. We don't even know if the solutions exist. That's so fundamental. This is like a baby step towards a much bigger human goal of understanding what's in an equation in a more practical sense.

Can you speak to some of the maybe beautiful, fascinating phenomena in the space of fluids that you've been thinking about? You mentioned turbulence. What are the big puzzles to sort of solve in this space to understand? You mentioned drag, like air, fluid. what are like the big millennial prize type problems that people are thinking about? I mean, there's so many. You can break them down into a few categories. So turbulence is very hard because it's multi-scale. What is turbulence? Turbulence is this phenomena that when you have a complex fluid roiling or mixing and moving, kind of like da Vinci's gives you beautiful drawings of fluids at all of these different scales. All of the scales matter. The big vortices and the medium vortices and the little vortices, all of them actually matter to predict the lift and the drag and the thing. So the gas in this room is a great counter example.

It's the opposite of turbulence. It is complicated. The molecules are all moving around in a very frenetic way. And there's a lot of molecules. but that 10 to the 23 number of gas particles, all of those individual configurations don't matter. We know from thermodynamics, this is like the great success of classical physics, is that there's a small handful of numbers that actually determine how this gas behaves, temperature, pressure, density. That simple description doesn't exist for turbulent fluids, for the flow over a wing, the flow inside your heart, for the flow on a river. All of the scales actually matter. All of the numbers in that simulation actually matter to predict the next, you know, behavior to predict what's going to happen. And that makes it fundamentally challenging. So this is actually a great AI problem, is is there a coordinate system? Is there a set of numbers, just like we have for gases, you know, temperature, pressure, density?

Is there a set of numbers or some representation of a fluid, a turbulent fluid, that allow it to be simply described in a way that is useful for engineering, to build, you know, motors and engines, just like we have for testing. So then the interesting question, if it's a problem for AI, and AI is listening, I'm sure, the solution to that problem, would the path lie through mathematics or through having AI systems sit there and watch some fluid for a while? I think yes, for both. because fish, they figure it out. So I think you have the right keyword as coordinates. And so that's kind of why I said both. And I'm not trying to be like trite, but all of our great physics breakthroughs were predated by a coordinate transformation. So Newton's F equals MA came after we put the sun at the center of the universe instead of the earth, right? You had to do that coordinate transform first.

So I think the way I see it is we need to take all of this data, these simulations, these observations of fluids, and what we're looking for is a simple coordinate system so that the description of that fluid becomes very simple and generalizable, beautiful in a physical sense. And that is both a brute force, you know, get a lot of data, try a bunch of different coordinate transformations and representations until you find the simplest one that generalizes the best. But there's also a lot of deep mathematics and what the form of that coordinate transformation might take. In general, if you find a question that really smart people asked, you know, in the 70s and indies, but they hit a wall computationally, that's very fertile ground right now. So we know what the form of these coordinate systems probably should take from some fundamental principles. We've just never been able to write them down.

They're too complicated to write down. That's like a perfect problem for AI is, you know, now let's give it a lot more data, a lot more compute, and see if we can start learning these transformations we know should exist. Just to understand, the formation of vortexes, so that's like tornadoes, little tornadoes. Their formation, is that part tricky to understand? Yeah, absolutely. And it's beautiful, too. I love watching these videos of if you shoot two vortices together, they'll influence each other and then merge in these very interesting ways. You have like a red vortex and a blue vortex, and you can see where they merge and join. But they also stretch and fold, and, you know, turbulence is chaotic, so the trajectories get tangled like spaghetti. It's not random. There's a huge amount of structure, but it is very multiscale, very tangled.

One of the things I would wonder, you know, scientifically, I mean, these are really interesting problems, and it was a great explanation. So I'm always curious about what computational problems nature can solve already, right? And there are a lot of examples of this. You know, there's a famous one with, like, flying molds finding shortest paths and mazes. In fact, some of the precursors to Navier-Stokes were, you know, like some of Terry Towers' work on showing that you can have blow-up for Navier-Stokes-like equations because they can simulate Turing machines. So I guess I would also wonder what sorts of computational tasks these computations with vortices colliding can do, and how that imposes limits on what sorts of coordinate systems ought to exist. Because, you know, you mentioned PV equals nRT. That's an amazing dimensionality reduction, because you don't keep track of all of these positions and velocities of particles. but there's just a small list of numbers that describes a metastate, and that's it.

But if I really had a tremendous reduction like that in the case of turbulence, then it would seem to me very difficult to compute very interesting things, too. Could it be like the game of Rife, where you have this very simple cellular automata system, and it's a universal drawing machine, and you can cook up whatever behavior you want in it? Yeah, that's right. I mean, for Game of Life, a lot of times people think about, you know, what are minimal rules to have all of these exotic behaviors. And it's really shocking how small the rules need to be to get that, right? But I guess, you know, I'm a theoretical computer scientist. One of the biggest exports from our field is really the algorithmic lens. So it's not just about, you know, computation where you have a well-defined problem and a laptop and you want to solve it. But it's also about, you know, computation in the natural world. and what that teaches us about what the rules of nature might be. So, you know, some famous examples of this are things like equilibria in multiplayer games.

And Nash equilibrium is a great solution concept, but of course the fact that it's hard to compute Nash equilibrium, you know, throws into question, is it really a universal mechanism that explains how people play games? There's this amazing quote by Kamal Jain that I really like, which is, if my laptop can't find it, then neither can the market. And, you know, to me, what's really exciting is the nexus of a lot of these different perspectives. You know, when you were describing it, it sounded to me like there are very interesting computational problems under the hood, and that one can kind of get a dual perspective on these things by thinking about how that computation imposes limits on what a good coordinate system wants to look like. So I think that's a really nice way of framing the problem. And I would take two extreme examples. So Earth's weather and climate systems are actually just complex. All the scales do matter.

The weather in this zip code matters to me. You know, everything matters. All the variables matter. And you're right. Like, if I over-reduce that system to, you know, five numbers, then I lose the descriptive capability of what I actually care about. That's one extreme. The other extreme, let's think about how a moth flies or a butterfly or fruit flies. They are proof by existence that you can interact with an extremely complicated turbulent flow and do very advanced engineering control. They can control their forces and moments in gusts, in rain. They can go very rich lives, and they presumably don't have a PhD in fluid dynamics or know the Navy or Stokes. They're probably not running supercomputer simulations. There might be five or six numbers that actually matter to their daily life of maintaining a leading-edge vortex and stalling their wings to perch and to land.

And those are kind of two extreme examples, and I really want to be able to handle both of those. We want to be able to handle the full complexity of truly complex systems, and we want to be able to distill down the absolute bare minimum essentials for fast decisions in those kind of insect extreme cases. And biological flyers still dramatically outperform, you know, human-designed fluid systems in many regimes. And we still have a lot to learn about these simple things. So maybe we're, like, veering into research now. But, you know, to me it's always amazing how nature not only solves very hard optimization problems, it seems to find very robust solutions. And I'm always struck by the fact that, you know, when we model it, we tend to model it assuming we have full knowledge, you know, about what the boundary and initial conditions are for things like Navi or Stokes. But when a bird flies, you know, it doesn't know all of that.

And to some extent, you know, modeling things with uncertainty is really how you reframe the optimization problem to get very, you know, rich and robust solutions. It's kind of amazing to me how a lot of everyday examples sort of fit into this. I remember, you know, I'm a big baseball fan, and I remember reading a math paper about baseball, that when you get an outfielder, you know, once the ball has been hit, a lot of stuff is known. You know, the velocity, the trajectory, you would think the outfielder would just take a straight line pass to where they expected the ball to be. But when you observe outfielders, they don't take straight line paths. They take a slight curve to it. So why is that? At least one way to wrap your head around it is that that's actually a very robust strategy because when you're taking a curve path, you can continually reevaluate the trajectory of the ball and then adjust as needed. So there are very interesting solutions that nature finds, but it's not always obvious what is the right formulation for the optimization problem in the first place.

and sometimes modeling it in the case of uncertainty so that you can design solutions that are robust to the fact that you don't have full system knowledge seems to be an important piece of the picture. So there's a fun aside related to this. Insects in particular don't have the ability to measure their full flow field. They can react to gusts of wind, gusts of wind, faster than the information that hits their wings. goes to their brain and back to their muscles. So researchers have studied this for decades in biology. And what they've found is that there are strain-sensitive neurons embedded on the wings, maybe a few dozen to a few hundred of these, in very stereotypical locations. So brother fruit flies will have the same locations of these strain-sensitive neurons. And those strain-sensitive neurons fire when the wing bends in a certain temporal pattern.

It's like it's hit by a gust of wind, the wind bends, neurons fire, and a local computation is happening in that shoulder muscle of this insect that goes directly back and corrects robustly based on partial information of a system it doesn't have equations for that is non-linear and turbulent. So it's like, again, proof by existence is exactly what you're saying is maybe the robust strategy of actually relying on partial information and uncertainty. They can do more than we can at this point with way less information. Yeah. I have a couple of, I apologize, pothead thoughts because Alex mentioned black holes. I have a fun thought that maybe the singularity observed with a blow-up in the other social equations comes into play when we're, as opposed to airplane wings on Earth, we're traveling through two connected black holes in a warp drive that we figure out. And then there's slow control of space-time, and that becomes irrelevant because now you're in a black hole, hence the singularity.

for people just listening to this look out you gave me I cursed myself moving on to the next topic but the forcing you need the black hole vortex to also be forced exactly the right yeah the forcing where's the force I mean at that point you could try to write it there's a way you can make something out of this there's some non-zero probability that future answers will listen to this that guy is the one time he said something that was actually correct okay If it's okay, quit by the break. Sure, I'm good. I'm Corey. One of my favorite topics is theoretical computer science. You're one of the world experts in many of this problem. So I'd love to ask you about one of the big problems that's been solved in this OpenAI job, which is the unique games conjecture. Can you give people an intuition about what this problem is, why it's a big deal, and what are some aspects of the proofs that are interesting? Yes, I'd love to. So let me give you a little bit of a background.

So let's start off with complexity theory 101. So one of the most important problems in theoretical computer science is the satisfiability problem. So the thing to keep in your mind is imagine you've got variables X1 to Xn, and they can take on either true or false values. And what I'm going to do is I'm going to create a big formula out of them. And usually the way we do this is with an and of ors. So I start off with a clause that's, you know, an or, X1 or not X3 or X7. So depending on my assignment, that clause might be satisfied as long as at least one of those things is true. And then I take the and of many of these different clauses together. And the question is, if I give you that formula called the conjunctive normal form formula, you know, can you find a satisfying assignment? So can you find an assignment of x1, x2, up to xn, setting them either to be equal to true or false,

so that all of those or clauses are on, so that the entire formula evaluates the true. So this is the canonical example of a problem that's easy to check someone else's answer, but it might be hard to find an answer yourself. So if you told me that it's satisfiable, all I would do is I'd ask you, okay, give me the satisfying assignment. You tell me what x1 is set to, what x2 is set to, and I just go clause by clause and check to make sure that you really have satisfied everything. Now, the central problem in theoretical neuroscience is the P versus NB problem. So are problems that are easy to check? Is it easy to find their solutions to? And the presumption in our field is that, no, it's not easy. and that's the basis of a lot of things like cryptography and e-commerce and what have you. Now, really, the starting point for getting to something like the unique games conjecture

is something called the PCP theorem. So this was a landmark result proved in the 90s. It actually was such a big deal that it made the front page of the New York Times. And the crux of it is really this idea that it's even robustly hard to fool a verifier. So you start off with a formula, and either it's satisfiable or not satisfiable. And what does that mean? If it's satisfiable, you can write down an assignment, and I can check one by one that the entire thing evaluates the truth. But if it's not satisfiable, that means someplace you're lying. You write down the purported satisfying assignment, and when I go clause by clause, I'm quickly going to find one clause that is not true, and that invalidates the whole thing. And I say, uh-huh, I caught you. It's not actually a satisfying assignment. So the idea behind things like probabilistically checkable proofs without getting into a lot of the technical definitions, the idea is really gap amplification. So can you start off with a formula that is hard to decide whether it's satisfiable or not?

And can you engineer a new, maybe much bigger formula that's robustly easy to check in the sense that if it's satisfiable, it'll still be satisfiable. You can write down an assignment and it'll be fine. But if it's not, I should actually be able to just flip a few coins, query random variables, check a random clause, and with good probability find that you're lying. So the point is that with very sublinear inspection, with only constant amount of inspection, I can find that you're cheating. So this was a momentous result in theoretical computer science because it laid the foundation really for what's called hardness of approximation. So a lot of algorithms and complexity theory is, what do you do when you're faced with problems that are genuinely hard to solve? And, you know, one of the things we do is we try to approximately solve them. So maybe we can't find an assignment if there is one that satisfies all the clauses, but can we find one that satisfies many of the clauses?

How large a fraction of the clauses. Now, even there, we would expect problems to become computationally hard after a certain point. It should be that it's not just that, you know, deciding whether it's completely satisfiable or not completely satisfiable is hard, but maybe even deciding is it 100% satisfiable or is it only 99% satisfiable. The best case over all assignments satisfied most 99% of the clauses. that too might end up being hard to decide. So these are what probabilistically checkable proofs give us. In places where we knew that the optimization problem was hard, they tell us that it's even hard to approximate. And the unique game's conjecture is a strengthening of the ordinary PCP theorem because it has to do with a certain type of label cover instance. It's a bit technical to go into the details. But the point is that if that version of a probabilistically checkable proof were true, it would have this great organizing effect.

Where it would turn out that for a huge swath of problems, we would actually know what is the best approximation algorithm, and it would be something very simple. So when I started off in the field, unique games conjecture had been posed a few years before that. And there was still a lot of debate about whether it should be true or false. And, you know, people worked hard to try and give proofs of it. People worked hard to try and disprove it. There was this amazing time period, maybe in the 2010s, where there was quite a lot of back and forth. And, you know, there would be new results came out that people's opinions would change, whether they believed unique games or not. I like this quip. I forget who said it, so I can't properly attribute it. But someone told me, you know, maybe God's going to decide whether it's true or false based on which side is working harder. Because it really seemed to change year to year. And, you know, one of the results, one of the headlining results in the OpenAI drop of proof was that the unique games conjecture is true.

Now, I should mention that this was not without spectacular human progress on it. So one of my colleagues at MIT, one of my colleagues at IAS, they both contributed, among other people, to a lot of the major strides, so Dormincer and Irikanur. They proved a weaker version called the two-to-one game theorem. And this really was a shot heard around the world in theoretical computer science because at that point, I think a lot of people weren't sure what was a path forward for unique games conjecture. and I think their work really gave a viable path forward to something that was in the vicinity of unique games. So it sounded like, you know, maybe humans could get there with another 10, 20 years. But, you know, just when OpenAI released the results, it turns out that the unique games conjecture is actually true. So this was quite remarkable because, you know, at some point, I guess I did find myself on the side of the doubters

because Unique Games just sounded too nice to be true. Because, you know, a lot of work, like Prasad Raghavendra's work, it implies optimal hardness of approximation for so many different types of constraint satisfaction problems that the conclusion just looked to me too strong to be true. What did you feel when you found out that night? Okay, well, for starters, there were already rumors, you know, in the weeks leading up that OpenAI had solved Unique Games conjecture and that it was a truth. And these weren't even just completely arbitrary rumors. One of my colleagues, Thor Vincer, he actually posted a paper where he wrote something in the footnote at the bottom that, you know, we would have liked to polish this paper more, but because of the rumors that were going around, we feel the need to put out our approach sooner rather than later. So for me, it was in some sense a bit of a relief that the results were finally out there. Now, they're not very readable papers.

It's not like you can point to them and really quickly identify what's a new key idea. But I'm glad that they're out there because now we can move forward on the next phase, which is digesting those proofs and figuring out where the field moves after that. Now, in terms of how did I feel that night, you know, as Alex mentioned, we, you know, we all had a drink in hand. And then I saw, you know, on my phone the news that these proofs had dropped. And, you know, people kept up coming and talking to me at the gala. But it's already hard to socialize as a mathematician, let alone, you know, when you have math, you really want to read. So it's doubly hard. The other funny thing about it was, you know, I was in New York already. so I took it as an opportunity before the gala to visit a lot of people I knew at NYU. And I just happened to be meeting with Subhash Kot, the one who introduced the Unique Games conjecture. And I just so happened to ask him, you know, just a few hours before the gala,

before the OpenAI papers released, Subhash, what would you do, you know, if they released a positive proof of Unique Games? Are you going to read it? And he told me, no. Which, you know, that's a lot of people's reactions, unfortunately. It's just kind of true that there are these grand challenges we work on, and sometimes when they get solved, you know, sometimes it feels like, you know, someone has stolen your thunder. You know, it's not like people didn't have ideas about how to solve the unique games conjecture, and I bet you those blueprints would have panned out if we'd given them more time. I think the other thing I worry is that, you know, that we lose out on the human effort of trying to reach those solutions to begin with. You know, math is not just about what is the furthest I can reach with the current set of tools I have. Is it a combat fall? Is it a subgroup? I don't know. But, you know, sometimes when it's a human-driven solution, we learn a lot more in the process.

So I worry about the fact that there were these great landmark problems, field-defining problems that a lot of people are reaching towards, and that decisive progress was made, you know, and the results all dropped in one night, and that people lose their enthusiasm for going down those paths. And I wonder if we'd given those people that space to go down those paths, what other exciting things we would have discovered. So I think one of the things I mentioned, you know, we were all trying to process it at MoMath, And, you know, there were a lot of really nice talks. Not all of them went up on YouTube. Some of them were, you know, about benefactors and awards named after them. And the thing I kept coming back to is this idea that, you know, why do you do math in the first place is to play and learn about the world. And we saw so many manifestations of it that night, you know, because there were awards named after people who, you know, why were they donating to the museum was because they wanted their kids to have a place to play with mathematical ideas and learn about the world.

It was also the opening of an Escher exhibit. And what a great visual of, you know, what happens when we play with mathematical concepts, right? And so it felt to me that in this moment, it's really important to remember as we're going through all of this change together is we don't know what the future holds, what AI and mathematicians collaboration will look like in the future, what hiring will look like. But the reason we all got into it was because we were very curious. You know, we wanted to understand the world. And mathematicians are very good at abstracting things. You know, they don't just understand one little piece, but they find some abstraction and some set of definitions that turns out to be unreasonably powerful in some other corner of mathematics. So I think it's really important, whatever the future holds, to make sure that we organize our field in a way where we still encourage people to play with ideas. I don't know what kind of reward system will work, but, you know, when I look out and I talk to young people in the field, they have a lot of anxiety.

They hear a lot of stories even about people they know running harnesses to scrape the web for open problems, to dump as many papers as they can to flood archives. I don't know if that's a good strategy. It doesn't seem like it is, but it certainly doesn't feel like fun to me. It doesn't feel like we're playing with ideas and learning about the world. And that's sad. I just want to get back to the things I really enjoy. The other thing maybe I just want to add in there, too, is like, you know, it's a big identity crisis for a lot of people. Because, you know, some people view themselves more as problem solvers, and that can feel very comforting. That you can draw a little circle around the types of problems you work on, the types of techniques you use, the types of problems you care about. And that feels very comforting because the rest of the world is messy and you've got blinders on. This is the part that I care about. Now I think is the time to remove those blinders because I just don't find that to be a productive way to do it.

You know, we love playing and learning about the world around us. And it's important to remember as, you know, a young mathematician, are you smarter than you are curious or are you more curious than you are smart? It's very important to be more curious than anything else. and you mentioned fun there could be another perspective of it of more about the struggle from my maybe outsider perspective it's some of the most fun in the moment sure feels like struggle i've talked to a lot of olympic gold medalists a lot of them feel something approximating depression after they win the gold medal because it's like you have given everything and you realize everything you've given is not over. 30 years ago, there was a documentary of Andrew Wiles, and there's a great scene. I think they even run it right at the very beginning. To your gold medalist, you know, what do you do once you've won the thing? He's sitting there at his desk,

and he's describing the moment when he realizes what the solution is, not his initial announcement, which was flawed, and then for many months he was trying to correct it, and he finally saw how to do it. and he's trying to say, you know, I will never again in my life have a moment as powerful as the moment that I experienced at this desk that day and he's choking up. He can't get the words out. So that, can you get that if you press the button and it gave you the solution? Absolutely not. So exactly to both of your points, It's like the journey, you know, if you see a giant hill, a giant mountain, and you say, one day I'm going to get to the top of that, and you start training for it, and you're sort of doing little by little, whatever, and then the helicopter lands and just drops you at the top. Can you actually speak to that? The interesting thing about Andrew Wilde, and a little bit about Grigori Perlman as well, is working alone for a prolonged period of time.

What do you make of that? How common is that in mathematics? of sitting there and grinding a problem for, in his case, seven years. Until the 20th century, almost every math paper that had ever been written was written, and I had one author. I think now the median is two in mathematics. And so we're not a lab science. It really is a solitary process. Now, see, the things that I give my graduate students to do are not the things that I'm working on, sort of by design. I want them to have their own field. I want to open doors for them. It's different in different fields, right? So when they have a success, it really should be their success even if I was first. I had an idea that this would go somewhere, and they executed on it. That's theirs. But in terms of sitting there, the paper that I put out a month ago is something I've been thinking about for 10 years. It's a five-author paper. over those 10 years we made incremental progress to the point where we finally had a solution

and that paper was an instance of there's a phenomenon here and I don't understand what it is. It wasn't here's a problem, solve it. We could not at the start of the project give you the lean formalized statement that needs a sorry. You discover what the statement is through the process of packing away at the problem little by little until you see enough of it to say this is how it comes together. Actually, that makes me think, can you describe your thinking process of how you think through really hard problems? Is there some aspects that are interesting, like a lot of coffee in the morning, at night, alone, brooding with a cigar, or together at a whiteboard with others? Is there some interesting color, textual insights you can give? Sure. I think a lot through world models in some way. So, you know, I had some mental model of how I think results in some sub-area go,

and I think that colors a lot of how I see surrounding areas, too. So for me, a lot of my work is really about, you know, building bridges between algorithmic thinking and other problems that usually aren't studied through that same algorithmic lens. So I think when I first started as an advisor, I sort of made the mistake that I gave students questions that were 100% in my area. And the problem then was, you know, when they would come back and they'd have an idea, I could shoot it down very quickly. And I was either right or I was wrong, but either right, no one was having any fun. And, you know, over time, what I started to do was I would give students questions that were 10, 20 degrees off from my own area. Things that I wanted to understand better myself, papers that I thought were important that I just hadn't had time to sit down and read. And then I had the student explain the result to me. And what's fun is we would go through that process together, not, you know, by printing out the paper and going through it line by line, but trying to reverse engineer why do

we think this works in the first place? And there's so many details that once you get the feel for it, you kind of have the sense that they ought to work out. But at least in important papers, there are one or maybe even more than that. key ideas that really change your perspective. Oh, this really might be doable. So what I found was, you know, as soon as we start trying to understand one of these results, we find our own way of looking at things. And then it's not long after we find some other questions that we now think are approachable because we found our own way of organizing our thoughts. So that's often the way I do it is the earlier that I can have the student in the driver's feet, not necessarily with a well-defined question, but with them teaching me something and them being responsible for distilling and digesting and abstracting it down to its essence. You know, reasoning through with them analogies that aren't meant to literally be true, but help us figure out what to expect in neighboring areas.

You know, then it feels like they bring a unique perspective and we're off to the races. So for me, in a lot of ways, the technology is only going to enable that kind of work because it means that a much richer set of problems are approachable. You know, for me, when I find experts in related areas that can really explain things in my own language, I hold on to them for dear life because, you know, they can speak the gospel. They can tell me how this area is organized and explain it in my own language. But now, you know, I have technology that can do that too. And it also feels a bit sad that you lose that human element that was there once before. I think also students have so much anxiety about, you know, what publications are going to look like that one of the hardest parts is, you know, when I'm thinking about a project, I'm really trying to think about it in a sort of multi-year time horizon. because I have some meta-vision that I want to realize, like how can we use learning theory techniques and quantum many-body systems?

And there are a lot of stepping stones en route to that, but some of the most exciting problems are, you know, many years out. But students are so worried about the incoming AI tidal wave, you know, the future of work, universal basic income, are any of us going to have jobs, that they're really thinking in three- and six-month horizons. And then it's hard to find problems that we both care about just because, you know, we're thinking about different worlds. Do you think we should encourage them to continue to think long term? I mean, I understand that. Oh, I do, but they don't always listen to you. I mean, I don't know if I'm right or wrong. You know, maybe AI will solve all of our problems. But it seems like the only ones that are actually worth us working on at this moment in time are the ones we're still going to care about in a bunch of years. And you should only be working on future-proof problems. If you're working on things that, you know, you're in a rush because you think the next generation of models will definitely solve those problems, then maybe just go on vacation and don't solve that problem.

Can you speak to how you think? You do work a lot of natural systems. What are the interesting insights that could be pulled out of these natural systems? So I find a tension between focusing very hard on an engineering problem I'm trying to solve, which can be very singular. You put on blinders. You only think about this problem. And then giving yourself the kind of mental freedom to look around and just observe and be curious. Very often you find the solution when you turn over a rock and when it's not something you're thinking about, you know, with the full front of your mind, but in the side of your mind. If you're looking at a problem through your peripheral vision, you have a much better chance of solving it sometimes. And so this is a constant balance for me is observing all of the fields that I don't work in and all of the nature that I don't understand and all of the things that I'm curious about, but then also having very focused problems that you don't let go.

And this must be true for all of us. I've heard from both of you, you know, focus problems you've been working on for years and decades that you haven't let go of. But presumably that is informed by a million other pieces of information that you happen to pick up through interesting conversations. Do you have an example because it sounds like neighboring areas is a common theme. Do you have examples of things you're like, you're the curious peripheral vision of your brain. and went, oh, okay, that's an interesting system, might be relevant, and it ended up being so, or maybe in the future will. So for me, as a fluid dynamicist, biology and its messiness and complexity, and also material science and its messiness and complexity, are neighboring in a way that's very fruitful. I learn a lot from both of those fields, and a lot of the kind of basic math that we develop to handle complexity does seem to transfer. And sometimes you have an idea just because you see something is possible,

or you start to develop something and you can explain something that we didn't understand before. But I think, for me, biology and fluids and materials have this really interesting kind of commonality of multiscale, nonlinear, messy, nebulous objectives, things like that. By the way, on that topic, do you think there's hope that AI can have some significant breakthroughs in the messy world of biology? I certainly hope so, and I think so. There are large efforts in biology to collect very large data sets and to understand complexity in systems. So the connectome where you have the full wiring diagram of the brains of insects, it looks like we will start to have connectomes of other animals, and eventually it'll be probably like genomes where it is cheaper and more accessible. I have this, like, feeling that we'll find out, speaking to the resilience of, you know, birds being resilient,

that in order to understand how the human biology system works, you have to, like, pull that thread, and you're going to have to understand how the entirety of life on Earth emerged and functioned as an entire, like, mechanism. Like, it's all tied together. Like, you can't replicate. it's ridiculous what happens with humans from a single cell. This whole thing goes it builds and the intelligence that's built into the whole building mechanism in the first few days weeks months of life this whole thing is like I think could be like many orders of magnitude more complex than some of the hard mathematics problems we've been talking about. Again I suspect if turbulence is hard like whatever the hell is going on in here I don't mean the brain I mean like the fluids around the brain the whole thing the immune system all of it everything interconnected together

and I am very deeply connected to the history of the bacteria that was here billions of years ago I suspect the complexity of the entirety of the mechanism of life on earth is many many many orders of magnitude higher than any system that is amenable to AI today. Sorry. I think you're right. It's incredible to just imagine all of the complexity and scales. Inside one cell, it's a whole city. It's a whole infrastructure. One of your neurons has garbage collection and power production and information. Like the whole thing is at every single scale is beyond our ability to possibly find that you're absolutely right. And I think that's what makes it so interesting. And it is related to math in the sense that finding the right scale where we have any hope at all. And there is something interesting that we can use to understand that whole picture.

I think that's very fascinating. and that up until now has been a human endeavor of finding the bits that are interesting and understandable in this mess of complexity. And, yeah, I think you're right. I mean, the fact that anything is understandable about the natural world is a miracle, right? There's no reason that Newton's laws can be written in physics that Newton himself can describe the mathematics of and invent calculus for the purpose of being able to say what inverse square propagation results in elliptical orbits solve the two-body problem, yada, yada. So the fact that the universe speaks mathematics is completely mysterious. There's no a priori reason. We don't have a right to know anything that we know. I apologize for the question, but I'm obligated by the said universe to ask the question, but given this topic, do you think math is invented or discovered? Yeah, an age-old question. I have a very sort of pragmatic answer to this, which is the questions that we end up asking, it seems to me, are functions of what's going on in the world. What questions we possibly can be asking, given our current state of knowledge, the questions we're trying to answer today are not questions that Euclid could have thought about.

math was sort of the things that people got interested in were different in Greece versus India versus China. But then the answers, once you ask a question precisely, the answer is what God decides the answer is. So we don't have, so we're discovering answers to questions that we've invented. We're discovering answers? Okay. Steve, you have worked with a lot of natural systems and look at equations. Equations are I guess always approximations of the real world? So is math real? You've got to say yes. How big is the gap between the mathematical world and the real world? Surprisingly small. I think unexpectedly small. I like your answer a lot, actually. I do somehow fundamentally think that math is both created and discovered. We create mathematical formalisms to understand the world around us. and those approximations are unreasonably effective.

It is a miracle that there are simple, compact descriptions, F equals MA equals MC squared, that describe everything that we observe at all of the scales that we seem to observe. We seem to be able to distill something simple, mathematical, that describes them. That feels like we are creating those, but that there is that simple generalizable description underneath everything and that seems to be something we're also uncovering. Do you think that's a good guide, you know, kind of searching for the beautiful simplicity for us? Or are we just like aesthetically pleasing things? I think it's a good guide for us because we are fundamentally simple. We can only handle so much in our minds. we can only envision so much. I think simple is essential for humans to interact with the real world. Okay, if I want to get slightly deeper into the question of created versus discovered,

numbers are a good example, integers. We all fundamentally feel like counting numbers are real in a very essential way. I can have one rock and two rocks and three rocks, and that is a number of rocks. It is a real number of rocks. But those are absolutely abstractions that I have created. A rock is not a rock, is not a rock. It's a collection of atoms. It's a different collection of atoms. There's not one rock or two rocks. Do numbers even exist? I wrestle with this. I don't know. Do we actually believe that integers are a human creation to interact with our symbolic abstraction of what we observe, quantities, and, I mean, it took humans thousands of years to discover the number zero. Right. And then negative numbers. Negative numbers were actually accepted by mathematicians in the West after complex numbers were accepted as a mechanism for study, which seems insane,

but there's a whole kind of story behind that. What's your relationship like with infinity? Is it real to you? Or is it a nice thing to have as part of proof? So much of modern mathematics is exactly dealing seriously with infinity, right? One of the first times people really start to hit a wall in their mathematical education is when they hit calculus. And there's sort of several reasons for it. One is they probably have a lot of gaps in their understanding. In previous, you know, they got an 81 on geometry and a 75 in trigonometry or whatever. Those are pointing to gaps in their understanding, but all of which is going to come together in one self-loop when the Swiss cheese is just, you know, they're not able to cross that threshold. That's one issue with calculus. The other issue with calculus is the first time you deal seriously with infinity. You have to add up infinitely many, infinitely small things and get a finite number and have that number make sense. Human beings, Newton and Leibniz,

did not understand infinity the way we understand it today. They had an intuition for what it's supposed to represent, but if you read Newton, I mean, Newton in his time was ridiculed for the way he wrote about, you know, the ghosts of departed quantities. You have to divide zero by zero and have it make sense. And you don't do it when they're zero, you do it after they become zero. The way that he was trying to write about these things turned into the crisis in the 19th century when Cauchy and Fourier proved conflicting theorems. Cauchy said you have a sequence of continuous functions, and you take the limit of continuous functions, you're going to get a continuous function, right? Limits are a continuous process. Continuous functions are continuous. The answer has to be continuous. And Fourier says, I'm going to add up a bunch of sines and cosines. It doesn't get more continuous than sines and cosines. And I'll get a saw tooth. I'll get whatever you want. I'll get a discontinuous function in the limit. They broke math. So that's when we had to start facing, it was no longer okay to think about infinity the way Newton and Lognitz

and the entire, Euler's entire, what, thousand some odd papers lived in a world where infinity was something to play with, and in the 19th century, infinity had to get professionalized. So it turns out maps have been killed multiple times before That right And somehow it only a scratch The vast majority of our first year PhD courses are the result of crises that happened in the 19th century. And where math was destroyed, it had to be rebuilt from scratch. Speaking of crises, let me get back to the D versus NP question I was talking about. Do you think how hard of a problem is that? Proving This is sort of the central question of complexity theory, P equals NP. Do you think AI will be able to prove this? In fact, we should mention that you were kind of a part of a panel. There was initially a debate about whether LLM-based methods could one day make progress

on the hardest problems in mathematics, like PNP or the Riemann hypothesis. That was a year ago. So what is your updated view on this? Well, clearly they can make progress on central questions in the field. P for SMP, maybe it's just because it's closest to my heart. That seems pretty difficult to make progress on. You know, it's kind of remarkable the things we don't understand. So that problem I was describing about satisfiability, we don't actually know any super linear lower bounds. You know, we think that it takes exponential time, not that much faster than enumerating all possible assignments and checking them. And yet we don't know anything that sort of is asymptotically more running time than just reading the darn input. I wish you actually mentioned, you've talked about hard and easy. What is a more technical way to think about what is hard and what is easy?

Right, so this is polynomial time versus not polynomial time, or, you know, NP, just to be clear, stands for non-deterministic polynomial time. So the question is whether things which you can solve with non-determinism, like guess the assignment and check it, can also be solved in polynomial time. So the question there is if I give you a description of the formula, you know, I tell you what are the variables, what clauses they appear in, is there an algorithm that runs in time polynomial in the size of the input and correctly decides whether or not there is a satisfying assignment? We should mention that the polynomial time or exponential time or superlinear time, all this refers to worst-case analysis. That's right. So can you explain what worst case analysis is and what are its pitfalls? Yeah, I should say what are its pitfalls because I think it's a little bit counterintuitive from people outside to feel like to understand exactly

when we say a certain problem is hard, you're actually talking about worst case. That's right. It's like analyzing, I don't know, I don't know why that popped into my head, but like marriage. It's like analyzing whether I should marry this person or not by saying, well, what's the worst thing that could possibly happen? I think that's for human relationships, maybe it's not such a good strategy. Maybe the average case is a better guy for making decisions. Anyway, I don't know why I brought that up. Okay, I have no advice about who you can marry. Yeah, complexity theory does not give you. I think there is some, isn't there a dating, just an optimal stopping theory of the optimal secretary problem? The six-tier problem. There's a lot of different names for it, but here's a mathematical way. There's an optimal strategy. First, so what is worst-case analysis? So, you know, I was describing the SAF problem earlier, and the algorithmic problem you want to solve is given any formula you want to decide,

is it satisfiable or not? Now, what's crucial there is that when I'm asking you to solve that algorithmic problem, I'm not telling you anything more about the structure of the formula. I'm not telling you, you know, these are the size of the clauses. They look like this. If I drew a graph connecting them, it would have this topology. So that's what worst-case analysis is about, is it's about the premise that you can treat all of these computational problems actually by ignoring a lot of the fine-grained detail about the individual structure, and you can try and solve them, you know, for all possible inputs. And that's been spectacularly powerful because we've identified lots of algorithmic problems that really can be solved in the worst case and can be solved really efficiently, like finding the max flow as an example. But the question is whether or not that's a useful abstraction. So sometimes maybe you really do need to know more about the structure of the instance you're faced with

in order to solve it better. And that's really an open question. This was the area of what's called beyond worst case analysis. And I would sort of argue that it's a very important direction for theoretical computer science to go in. It's hard to figure out what you should articulate about natural problems that is really the right level of detail that your algorithm can exploit it and can do better things than it can in the worst case. But let's use an example. So you were talking about the secretary problem. So the way that this works is it's an example of something called online algorithms. So you have a sequence of options coming, one through ten. They come in each time step. And your goal is, you know, when someone comes along, you know their ranking relative to everyone else you've seen before, but you don't know their ranking relative to the people who are yet to come. And your goal is to find, to make a decision when the person comes along, do you commit or do they move on, in such a way that you have a good chance of finding the absolute best candidate.

And so the point is that you can actually get a constant competitive ratio algorithm, that what you do, the best strategy is to wait some amount of the time, not make any decisions, don't accept anyone. And then once that probationary window has passed, you take the first person who's better than anyone you've seen thus far. And that'll give you a good competitive ratio in the sense that you can proof theorems. Assuming the inputs arrive in a uniformly random order, you're likely to do pretty well compared to as if you knew what that sequence of inputs was in advance. So there's, you know, under online algorithms, there's this veil of uncertainty, which is I don't know what the sequence of inputs are going to look like. I want to do well no matter what that sequence is because I don't really know how to model what that sequence looks like. Now, you can instead say, hey, maybe the algorithms I derive aren't good enough for my applications. Can I do better by trying to model more of the world around me?

And then there's kind of an art to figuring out what you should model, what you should build into your model, and what you shouldn't. And that seems to me like a tremendously important area to explore. So, for example, one other variant of, you know, online algorithms that's really important is the paging problem. So your laptop solves this problem all the time. You know, it's getting a sequence of requests of things that it wants to recall from memory. And the question is, what should it actually put in its cache, the part that's the easiest to access? And the problem, again, is that you don't know what that sequence of requests looks like in advance. So you can study the online algorithm's problem of, you know, what is a good policy for managing the cache? You know, when something gets accessed, you know, I have to put it in, but what do I evict? And you can design algorithms that are good in terms of the competitive ratio. the number of times they miss in terms of those accesses is not much worse than the best offline optimum

if someone told me that sequence of requests in advance and I could optimize just for that sequence of requests. Now, this is a problem that's been studied since the 80s and 90s. The trouble is somehow that, you know, the best algorithms you can give, we know tight upper and lower bounds, they're just not that very good. You know, the number of misses they make compared to the best offline optimum is a large factor off from it. And that just doesn't compete with what your laptop is able to do right now, because it really makes very few cache misses at the end of the day. So you're stuck with this problem which sort of bridges mathematical thinking with trying to model the natural world, which is, you know, what is it about realistic sequences of requests that makes them not the worst case? This is really an inverse problem now because it's, you know, the other side of the coin. Instead of someone handed you a problem that you're just trying to solve in the worst case and we can all agree on how it's formulated and we know what's a better algorithm that does better on the competitive ratio,

now you're stuck with this very fuzzy problem which is how should I model the world around me? And in a lot of ways, algorithmic thinking is necessary for figuring out what are productive ways to model that problem. Because it's not that you're going to build a totally realistic model of what sequence of page requests look like, but how am I going to course on the problem in just the right way, that it leads to interesting algorithmic problems that teach me about the world, that force me to, you know, design new approaches to solving these very basic problems. So I'd argue that, you know, sort of coming out of this space where we are only solving well-defined problems, but we're actually using our mathematical and algorithmic insights to figure out where to dig in the first place. It's now more important than ever. It should have always been important. But, you know, that to me is really going to be thrust into the spotlight. Like a simple example is we were talking about factoring numbers. Well, factoring a random large number is really easy.

Half of them are even. You know, factoring is fine to factor. So it's really those very special numbers that are products of two primes about the square root to factor the original numbers. That's for factoring. Which makes me think one of the things you worked on is mathematics of LLMs, AI systems, you know, networks. And those things are pretty damn good at modeling the natural world. So what can you learn from that? First of all, why does deep learning work? Do we have another three hours? Do we even have intuition? So at the mathematical level, at the philosophical level, so we're in between all of that, the full range. I don't know how we even enter that. Okay, so that's a big question. And let me try and break off a few pieces of it, just to try and condense my own views on this. So, you know, I mean, opening eye drop of problems,

There was no result in there that explained deep learning. There was no result in there that explained LLMs. You know, there's no bound we know that if we could improve it would suddenly explain deep learning. The problems are just not posed that way. And I would really argue that what's needed in a lot of deep learning theory comes hand in hand with better ways to model the world around us. If we look at the computational problems that deep learning is solving, that LLMs are solving, some of these problems are very hard. You know, trying to fit the parameters of a depth-for network to data is a cryptographically hard problem. We know that. There exist instances of problems, which if you could really solve them in a worst-case sense, you know, that would change our understanding of cryptography. And yet these problems get solved all the time every day when we're fitting deep, you know, much deeper networks than that. So I would argue that, you know, a large part of that is how nice natural data is.

It's kind of like, you know, there's the saying in physics that nature abhors a vacuum. I guess my version of it is that in some sense, nature abhors computation hard problems. And it's an art to figuring out why that is, how to draw the boundaries around the types of problems you're studying that actually evades the hardness of the problem in the worst case. So there's an interesting example back from the 90s. So certainly back then, and we still are, very interested in protein folding. Now, you can model this in a worst-case sense. I've got a sequence of amino acids. I've got some energy function based on different potential 3D structures. and I want to find this 3D structure that minimizes how much energy there is. Now, this problem is computationally very hard, and yet nature solves these problems all the time. So why is that? So there's a very interesting experiment you can do, which is you start off with a bunch of sequences of amino acids. You try out some truistics for folding them into 3D structures.

These are things that don't solve the problem in the worst case. They just seem to do the right thing sometimes, for whatever that means. And then the only sequences of amino acids that live on to the next generation are the ones that actually have very good faults. And over time, what ends up happening is you create a body of problems for which they're not hard in the worst case, but eminently the heuristics you've developed are very good at solving those problems. So in a lot of ways, you know, what are the structure of proteins of nature seems to fold all the time? Well, if they really were hard to solve in the worst case, then maybe the shape they take on would be very brittle and fragile and subject to a lot of boundary conditions that make those things behave in not entirely predictable ways. So in some sense, you know, a lot of our data has evolved in ways to make these computational problems easier. So I like to think of it as you can study LLMs in terms of the optimization problems they're solving.

You can study deep learning that way. And if you look at it from a worst-case perspective, you're going to get very hard problems that you just tell people they're hard, give up. But apparently not giving up was a really good idea. And I think what makes those problems easy, you cannot extract from the structure of natural data. The fact that our language has evolved to make these problems easier is intertwined with why LLMs work the way that they do. So my view is that, you know, people working in this space in theoretical machine learning, they have to become more comfortable with the uncertainty of modeling. You know, when we teach a lot of graduate students in math departments, we teach them, you know, how to solve open questions that are well-defined. But I think teaching people that skill of how to deal with messy, uncertain things and how to model it and how to critique models. It's just an important part of going forward, figuring out what research in theoretical machine learning looks like,

but maybe applied mathematics much more generally, maybe mathematics more generally too, for that matter. Is it nevertheless amenable to breakthrough? So one of the things I would maybe push back on a little bit, just on the pure company incentive side, And this is the thing that people talk about, which is the thing that I think OpenAI, Anthropics, all these companies are chasing is the self-improving researcher. So if they, in fact, had breakthroughs on the mathematics about LEMs, I would venture to say they wouldn't publish it because they need those breakthroughs, right? So it's fascinating because that is the big breakthrough from the pragmatic perspective of AI, self-improving and progressing. So how difficult is that problem that you're kind of referring to, which is making big progress on modeling the natural world,

understanding the mathematics of how to deal with that in order to have intelligence that is far above the intelligence of humans? Great question. I mean, I think alignment is one of the most important topics we're facing right now, right? And, you know, just because it's very hard doesn't mean we don't make progress towards understanding it. So one of the things that I've worked on recently, we were working on things called low-rank language models, which are a very simple abstraction of language models. And we were proving new algorithmic guarantees. But then it turned out that real language models really do have that kind of structure, at least in an approximate sense. And we were just pulling up this thread to see where this idea goes. And one of the really interesting things that it did was it shed some light on something called subliminal learning. So this was a really neat experiment that you can do with your favorite LLM. So you take two LLMs, LLMA and LLMB. For LLMA, you tell it a system prompt that says,

you really love OWLs. Every conversation topic, you try to bring up OWLs if it's at all germane. And then you ask it something concrete, like generate random numbers. There's no way to squeeze OWLs into that, so it'll generate random-looking numbers, 48, 37, 8, whatever. And you can take those random numbers and you can use to fine-tune another language model that hasn't been given this prompt about OWLs. So you give LLMB the same, you know, base prompt, generate random numbers, and now you fine-tune its parameters so that it's more likely to generate the numbers that model A generated. And it turns out that hidden within those numbers is the level of owls. So it turns out that when you do that, LLMB starts bringing up owls in every conversation you have with it. This is a very mysterious thing. This is called subliminal learning, because when we fine-tune language models on data to try and nudge them in different directions,

depending on what task we want them to solve, then there might be a lot of externalities where we're actually teaching them other behaviors that aren't directly, obviously present in the data. So how do we understand the relationship between these behaviors and what happens when we're trying to do alignment and, you know, the downstream implications it might have? So it turns out that there are ways that there are mathematical models that came out of some of our work that we can at least wrap our head around this. And at a very high level, the point is that you're embedding a lot of concepts linearly as vectors in some high but not giant dimensional space. And, you know, the number of concepts is so much larger than the dimension of that data that it forces the vectors to not quite be orthogonal to each other. So now when you fine-tune on one direction, it has small externalities on other directions. And these might look random, but you can coordinate them in a way where these small linear effects really add up. So the way that I like describing this is like a Ouija board.

So these are things that were popular when I was a kid. We had very strange board games, you know, because we would just sit down and want to talk to ghosts. But the rough idea for a Ouija board is, you know, you ask a question to a ghost, maybe your ancestors, and then you tap on this little piece in the middle, and it'll move around, and it'll go from letter to letter, and hopefully it'll spell out, you know, some words that answer your question. And the point is that when the taps are not actually random, but they're coordinated in the right way, you can get it to spell out whatever you want. And that's exactly what language models are doing as a byproduct, is that, you know, there are a lot of statistical relationships in the structure of our data that they magnify and amplify in addition to all of the amazing things they do, like quasi-RIMA. And understanding what those relationships are and how the models magnify and amplify them is really important to working with these systems, especially in contexts where we're deploying them and it's not just a very safe environment,

like outputting a lean proof of the theorem we care about, but it's about a messier problem that actually interacts with people. Yeah, I mean, we've talked about game of life and cellular automata from simple rules emerges this incredible complexity. In that same way, what you've been talking about right now is a single large LLM system. If you take that one system and have it do a lot of reasoning, that is already this dynamical problem. process that can interact with tools to the outside world. Now you take that and have multi-agent systems or very multi-agent swarms of systems. And now you have a type of game of life, failure automata, where the seeming obsession with owls that have been discovered, slight, especially with owls, results in about four hours, let's say, picking a number randomly,

where all of a sudden all humans are dead and there's owls everywhere. And it all started from random numbers. All these random numbers. So it's a caricature, but we've already seen evidence of this. I apologize for kind of taking it to the character place, which makes it not serious. I think to step back, These systems do have a lot of power because we depend so much in our world on our digital systems around us. And as the AI becomes more and more powerful, it becomes more and more useful to us, so we begin to depend on it, give it more access, give it more freedom to roam the public world. And now these behaviors that we're studying, the mathematics of these systems, taken at scale, you now start to get the cellular automata effect of incredible complexities that emerges, the vortex that come out from simple owl-obsessed agents. So their understanding this obsession with owls is very important,

so that they don't accidentally do a lot of damage to our civilization that is in many ways, on many levels, sociologically, geopolitically, technologically fragile. It's a pretty fragile civilization that we're existing in. And so understanding them as systems that connect at scale and very rapidly is very important. So that's a very real reason to have rigor, especially like poolable rigor, to understanding some of these systems. But we also don't want to outsource nearly so much decision-making to these systems for exactly that reason. Yeah, we have infrastructure and protocols in place already for safety critical and high impact decisions. We have the FAA, we have the American Medical Association, the Bar Association for these things that affect humans. And I think how we incorporate AI into our general processes in the entire world on two axes.

We have how much do you trust the output, so verifiability or trustworthiness. And on the other axis is how high stakes is it if it's wrong. If AI spits out a wrong proof, nobody goes without power. It's fine. So it's very low stakes. And it is actually extremely verifiable. So you know very quickly if it's right or wrong, and it's low stakes if it's wrong. So that's the perfect use case is this open-ended design space. Designing a better vehicle, you know, better car design that's lower drag, more fuel efficient. Low stakes if it gets the wrong design because you can always test it in the wind tunnel and verify if it did or didn't work. So design optimization space is perfect for AI. and we as humans have to be very careful about not bleeding into, we need a firewall of what needs to actually be verified and we have protocols, we have systems, legal systems and infrastructure for verification

but it is something we need to think about. If we get lazy, it's a very slippery slope. We need to have that firewall. What do you think are the hardest problems before us right now in the space of mathematics or in your, maybe perceive more in the space a little bit broader outside of mathematics? What do you think is the problem that's unlikely to be solved in the next year or two? I'm still going with P versus MP. You think that is? By the way, do you think it's possible it turns out that P equals MP? Sure. I think it's possible. What does that world look like to you? So it depends on what the running time of the algorithm is. So it could be so large that it's impractical, and then it doesn't really matter that much. It's more of a tidal wave for the theory research. Yeah, but that seems like a very important problem and very hard to fall.

In fact, I remember back when I was a graduate student, there was an attempt on P4CNT, which was actually semi-serious. Some people started all having rumors that, hey, has it been solved? And my first reaction was, no, it has not been solved. And I remember spending an afternoon thinking about the betting markets were still pretty new then. So it wouldn't let me take a counter bet that P4CNT has not been solved. So instead what I did was I bet on all the other Millennium Prize problems because that seemed like a good way to hedge because the question was which Millennium Prize problem will be solved first. But is it possible that P equals NP in the real way? So there's something deeply that we don't understand about complexity of theory, about computer science, about algorithms, where there are fast algorithms for satisfiability. So I think there are two threads in there that are important to tease out.

So first, it seems, at least so far, that you can always manufacture hard instances of problems. And we use that to great effect for our cryptography systems. But it does seem like, in practice, a lot of problems are much easier than we give them credit for. So for things like SAT, one of the places where they come up is in compilers and register allocation, and SAT solvers are amazingly effective at solving those kinds of SAT problems. Now, we don't really know what makes those problems so eminently solvable compared to the other hard problems that come out of cryptography or other areas, But it could be that in practice, a lot of the problems that we model in the worst case are secretly easy, and that the harder part is articulating what kinds of structures they have that makes them eminently easy, but that there are still a rich source of hard problems that we can do a lot of things, like building cryptographic systems out of.

That would be my bet in the world we live in, but that's just a guess. So maybe there's some way to basically quickly classify that this particular example of the problem is easy. So be able to identify quickly that this is one of the easier instances. There's a whole subset. It could be. So I'm reminded of the opening line for Anna and Karina. every happy family is happy in the same way and every unhappy family is unhappy in their own way. I think my worry is that many problems are easier than we make them out to be, but they're easy for very different reasons. Now, that doesn't mean that algorithmic perspectives aren't useful in trying to classify or developing an ontology of what those structures are, but I think the types of SAT problems that come out of register allocation

are very different than the other SAP problems that come out of other domains, even though they might be easy, too. I just think they're easy for very different reasons. So to you, the hard problem is P and P. So you'll be impressed if that, within the next couple of years, that's... I would be pretty impressed. Alex, what's the hard problem that you'll be impressed? Well, I'm already impressed with quasi-RH, but it's not RH. you know it's hard to think of a more iconic problem to say this is the hardest you know one of the hardest problems in mathematics that's been around for so long and what do you think on that sorry you just made me think of plum credit like um the pearlman uh what did he say he said if the proof is correct that's all the credit i need he famously turned down the field of metal what do you think he would say about all this uh the opening eye drop i mean i have a very hard time predicting what Prohmann will do. I did not know he would decline the Fields Medal. I did not

know he would decline the million dollar clay prize. What do you think about that in general, like turning down the prize? A few folks have done that in history of prizes. The only one who has turned down a clay prize and the million dollars, what do I think about it? Mathematicians are weird. To each his own. We love diversity of opinions and thoughts, and that's what makes the world of mathematics so wonderful. There's so many different kinds of people, and we let them all in. I mean, these questions are kind of coupled with what the mathematics community is dealing with now and the value of prizes, because in some sense, his particular way, world view, is aligned with this new world view that the mathematics community is entering, which is like it doesn't make sense. I don't know if there will be ever another Fields Medal given.

And so you're like you're entering Perlman World. We were talking about that explicitly this summer when the Fields Medals, You know, the last round of Fields medals was given out in Philadelphia, and there was definitely a question, you know, because people were following the progress. There hadn't been this latest bit of progress that really puts the question in the most powerful way possible. But already this summer, we were thinking, is this the last time that a Fields medal was awarded? I sure hope not, but we have to figure out what that means. You know, you have to think more sociologically about what do prizes accomplish? Why have these prizes? The people who win these prizes are already, you know, tenured professors at top places, and they're going to be doing mathematics for the rest of their lives. Well, Proven being one very well-known exception. So what do the prizes accomplish? They celebrate the activity. They celebrate.

They say, this is what we think is important. This is, you know, the highest instance of something. and there will still be activities that we want to celebrate. I don't know if right now they're giving out for solving major problems. Solving major problems is something AI systems can do. That's not something that will be rare and hence it won't be nearly as valuable as it was in the past. I would like to think that despite our earlier discussion about the solved axiom, And again, I don't believe the solve axiom, and I believe centaur math will be a very great way to do math. I mean, that's how I currently feel now. I'm constantly talking to AI systems, and I'm learning a lot of math a lot faster than I'd be able to. Because the way math is written, you know, I sometimes give a paper to a Brad student to read, and they come back a week later, and I say, so how far did you get? Well, I didn't really get past the first page, to be honest. Oh, really? Tell me. Well, actually, I didn't get past the first paragraph.

I said, okay, fine, why not? Well, the first sentence is by the perturbation theory of non-local systems, and I don't know what any of that means, so that's kind of where I'm stuck. Okay, you don't know how to read a math paper. You skip the first little paragraphs, go straight to, you know, the lemma three, and that's where you can start working and work the way backwards to what the thing is doing. But in the future, there is no, I don't know what this means, because you just upload it to an LL and you start asking it questions and it will zero in on the zone of proximal development, the place where here's everything that you know and here's the next thing you need to hear so that you start learning. And if people use that effectively, they can really supercharge their education. I don't just mean kids. I mean old boys like me can learn new fields. You know, to learn a field, like this is what our graduate students are going through. They have to completely rewire their brains. Like, learning math is so hard. Up until past the master's level into the PhD level, you are not expected to produce new mathematical theories.

Everything that they can do is something that is in a textbook somewhere. And yet it's still extraordinarily difficult for people to learn to solve problems that are solvable by lots of other people, never mind problems that are solvable. So learning math is hard. Learning new math is even harder. discovering math or whatever, working out solutions to math problems. But I think it's going to get easier, and this tool is going to be an amazing resource for people to be able to get to that zone of proximal development much more rapidly and develop their understanding much more rapidly. Yeah, I've been having so much fun on topics I'm not even – I would love to go back to grad school with the tools we have now. but one of the ways my brain likes to learn is constructing visual examples, toy examples, the simplest possible example that still contains the insight there,

and especially if it's interactive. Like if I can move stuff, it's like, oh, you immediately get it. All these words that the community agreed mean something you're supposed to know, all the notation, all that kind of stuff. If I may just know that this little symbol means this, Oh, okay, cool. All right. And it's just everything kind of falls into place. Also, you fall in love with the subject. And I feel like I can take on any problem that's completely outside of my field. Like, I can't understand, like, protein folding. I can, like, play with the parameters and play. I can take a deep mind paper and I can understand, like, taking any aspects of it and playing with it. And what's a simple example? ask crazy aforementioned pothead questions. So, yeah, this is super fun. The question is whether people will be motivated. This is the problem. It's the problem of human motivation. That's the real question. Well, the key ingredients there are curiosity,

asking good questions, and being able to call bullshit, like the critical advisor skills of getting information where you're not the expert and knowing if you need to dig deeper or push back. And I think, you know, there will always be people who are curious and not curious and who ask the questions and don't. We need to be really promoting that more than we ever have. It's a little scary to me that we're expecting our undergraduates, our young students, to interact with these tools when they're learning, when their brains aren't fully formed. We're asking them to have critical, not just critical reasoning skills themselves, but like advising skills. How do you take in information and tell if it's true or not or how to question it? That's a huge task for us in education is how do we train people to have good advisor skills when they're kids? I think the hardest problem is in undergrad. The hardest problem is middle school.

What is the point at which homework starts getting hard? At that point, why do the homework? It's a temptation, right? That thing was just, at first I'm saying, see, I've come to appreciate the value of doing difficult things because of loss and suffering, but I think because I was forced to. But if you're not forced to do it, because all of them can, I can just help you, I don't know what that, what kind of society that creates. Math has the longest head start on that question because calculators came out, what, 70 years ago? And we still ask kids to learn their time tables, at least some of us do. And those who said, no, kids don't need to learn time tables, you know, to the extent that teachers listened to that and kids listened to that and didn't learn, they really suffered. There's a great, great amount of human suffering resulting from people saying you don't need to know time tables because calculators can do it.

And my hope is that we can learn from that and say, just because a calculator can do it, you're going to be much better off if you can also do it yourself. It doesn't mean if I have to do some calculation, obviously I'm reaching for a calculator, but I could in principle do it myself, and that's what allows every time you give yourself that skill, master a skill, kids love mastering skills. How happy are they when they learn to tie their shoelaces for the first time? You would think, it's like, shoelaces, who cares? No, it's the greatest thing ever. You want to celebrate every little bit of independence, and you want to be able to think. You want to be independent from the machines. Just because the machine can do something doesn't mean you don't want to have the capacity to do it yourself. Well, this is where I have to ask because a lot of people are full of anxiety. You mentioned graduate students, PhD, graduate, undergrad, high school students. What advice would you give? Maybe different advice at different levels. to folks trying to figure out what to do with their life

to PhD students who are freaking out right now. Is there some wisdom you can give them? So, yeah, there's different things at every scale. Let me start with PhD students, I think, in math. PhD students in pure mathematics, my advice to them is very simple. Do what we have always done. learn the most difficult, the most abstract and academic your mind can handle as much of it as possible. Your advisor should hopefully be guiding you through that process. And then just as a one more, one step down from that, what we're seeing at the undergraduate level is a bimodal distribution. It used to be grades were basically Gaussian distributed. Now we're seeing a very bimodal distribution between the kids who are using AI to check their understanding. Wait, the professor said this in class, what does that mean? I have a test coming up, here are my notes from the last three weeks, start quizzing me and check if I really know something. Those kids are getting

much higher grades. And then there are kids who, oh, this problem is set, is due in an hour, can you just tell me the answers? I'll write them down in my own words. And so there's like a decorrelation between homework progress and exams. And we're seeing this by modal distribution, I don't know if you guys are seeing the same thing. So the implied advice there is to be one of those, the one that's using it to learn and to grow, to be the head, not the tail. Do you mind if we shake a bathroom, Bert? Yes. We've somehow made intelligence this pinnacle and have almost equated intelligence with morality. Right. And value. And value. And I think that couldn't be farther from the truth. Super intelligence does not imply super morality. And I think that's a huge concern with AI systems. If they are super intelligent, it has no implication that they will be super moral.

I'm not even pushing kids to go to college. that everybody needs to, regardless of what they study or what their outcomes are. Exactly. And super intelligence seems to give you a super ability to rationalize. And common sense, people who, you know, farmers and the rest of the world, like they have a sense that that rationalization is often bullshit. Just because you can make it sound really good and I can't argue if it doesn't make it right. Education and wisdom are two different things. Yeah, exactly. I know it's difficult to do, but maybe Steve, do you have advice for grad, college, maybe young kids, on what to do in this crazy, fast-changing time? Definitely be curious and read and learn all of the fundamentals. I think that's always been good advice. Even if your computer can do linear algebra better than you, you should definitely still learn linear algebra, build the intuition, same with calculus,

Same with physics. I think understanding history and understanding this change in the context of other great changes that have happened through human existence is important. But really being curious. I should say that this curious thing in the mind is trying to get stolen nonstop by the engagement economy. So, like, the same engine that makes you curious is the same engine that makes you doom scroll. Like, oh, that's a – so I think you have to kind of learn how to, like, avoid all the distractions and focus on letting the curiosity actually explore something and soak something in, like, learn a thing. And actually, AI is really good at doing that because they're incentivized to give you value from the interaction. versus incentivized to get you glued, addicted to the thing.

I don't know how people in high school, college also, can survive with the phone being around. The distraction box being right there. So avoid distractions and focus deeply. We all were bored when we were kids sometimes. I think boredom is really important. So we'll actually be bored. be. Curiosity and focus come often from boredom. I would like to be bored more often. I try to do it sometimes. Yeah, we would come home from school and finish homework in five seconds or whatever and just go outside and hope to run into some other kids who also just went outside and then go and find something to do and come home when it's dinner time. Yeah, you don't realize actually in the high school years, even before someone on the grad list is getting harder and harder, but, like, high school's a really nice time to just, like, learn everything,

and you think, like, at least I remember, I don't know what it is, like, now, but I think it's just more, even more fun now, you think you're, like, okay, when I grow up, I'll get to actually do something useful in this world, oh, yeah, okay, I'm learning, it's very nice, and so on, but you realize like this is like from a fulfilling perspective from building a foundation of an awesome life where you remember all these things this is like the time to do it and the whole world is forcing you to do it and it will reward you for sitting there and learning because like when you get more you know even to like grad school this kind of exploratory stuff you get discouraged and god forbid you get a job, then you're definitely discouraged from just learning, exploring, and so on. The time to explore is like high school. By the way, like pre-tenure faculty are on this like, oh, once I get tenure, then I can

just sit back and relax and think big enough. But I have to like produce enough stuff to get the tenure in the first place. But it's not what we want our young people to be doing, is clawing their way there. Ankur, do you have advice you give to students? The problem with going last is that I think a lot of my advice got taken, so being curious was definitely one of them. Now, I would say that it's also important to do things for intrinsic reasons. So one of the things I've noticed is a lot of people in our profession, and a lot of people would go the academic route, the reason they do things sometimes is really external locus of self because they see their values through how other people see them. And that's good enough to get you started. Maybe you start working very hard math so that people are impressed by you. But at some point you run out of these milestones to hit,

and then you really have to have an intrinsic reason for doing the things you do. and hopefully you have a sense of, you know, what's important to you, what are your values. So I think these days, you know, you can take our advice about, you know, being curious, but also the old playbook is really being thrown out. So you can take our high-level advice, but you're going to have to figure it out for yourself and you're going to have to be comfortable with uncertainty. And the only way I know how to deal with that kind of uncertainty is to have a sense of purpose about what you think is important, what you love about your job, and make sure that those are the things that you're doing. Because the old playbook, if you just stick to that, you're going to end up taking all these AI-generated proofs and spending all your time putting them on archives, sending them to journals. I don't think that's anyone's intrinsic reason that they get into it, is really to pad their CV. I don't know what will be important in the future, but if you really have some higher purpose you believe in, then I think you'll be in a much better position to adapt to the changing world

and figure out your own blueprint and be comfortable with uncertainty as the world changes around you. This is a trickier question, but a lot of people listening to this are feeling a lot of anxiety and fear about the future. It's both about a job, but also a future that's meaningful. Can you look back to your own life when you had that uncertainty, that anxiety? What's the advice you can give to that guy when you're in that low point? So let me try to address that question. It's a very tough question. I'll tell a short story that, you know, we spent our sabbatical in the Bay Area, and we were living in Palo Alto, which was kind of the epicenter of college stress. You know, you can just feel it. You go to the cafes. It's just in the water. You know, what extracurriculars are you doing right now? It'll help you. And then right when I came back to MIT, they asked me to give the convocation speech there. And it was just interesting thinking through that speech.

You know, what do you say to this entire auditorium filled with freshmen and their families who, if things are a game to optimize how good you look on paper as a 17-year-old kid, they've won. And I just kept thinking about when I was in that same situation, when I was starting college, how excited I was just to learn all of these different things. You know, I'd sit there and plan out, you know, my four years. And, you know, I had little notebooks where I'd figure out what my classes would be. And if I took these classes in just the right order, I could take photonics as a sophomore. And I had no idea what photonics was, but it sounded fun and fancy. And I think that aspect, you know, so much of our society now is geared towards winning and looking good on paper. And I always wonder, to what end? You start off in high school and you can start, as soon as you walk in the door, optimizing your APUs and your extracurriculars so that you can get to an elite school.

And then when you get to that elite school, you might skip classes because, you know, maybe the important thing is to prepare for your interviews. so that you can get a job at a frontier lab and then you can get some nice, you know, restricted stock units and live out your life. And that's great. You won. You won at every stage. But so what? You know, for me, the best way to live a rich and fulfilling life is just to pretend you've already won and then figure out what you would do in that scenario. I remember the advice I got when I started at MIT's faculty, The Sanjeev Arora told me, you know, the easiest way to get tenure is just to pretend you already have it and do what you would have wanted to do all along. So I don't know if that advice applies to, you know, graduate students, the high schoolers, too, but I think my advice stays the same. Pretend you already have tenure. What would you now do? Because a lot of the people we're giving advice to have really so many different opportunities.

Finding something that you really believe in is the only way I know to be fulfilled. I heard career advice from someone I respected that a fulfilling career is something where you can work on a problem that is important, that you find interesting, and that you're good at. And if you can get two out of those three, you're pretty lucky. If you can get all three, you're going to be happy for the rest of your life. And so having the right preparation, figuring out what you're good at, what's interesting, what's important, and then preparing yourself, you're not guaranteed to get all three of those, but you want to be ready when that opportunity comes. And then continue doing the things you actually enjoy doing, I think is great advice. I think we should also put into context and be quite honest. I do think a lot of the advice you gave applies, but we should be honest that I'm sitting with three world-class human brains

that have found their path. It's a survivor bias. So, you know, there's a lot of people listening to this that are like, they're pretty sure they won't find their path. It's the fear, it's the anxiety, it's the uncertainty. it's stumbled a bunch of times I think to be fair some of the most interesting people stumble quite a bit in life whether it's the external environment or the internal whatever the universe brings and that I just know a lot of people just feeling and monitoring it right now because of everything AI is doing to our society there's a lot of people just terrified and really anxious And I think there's a lot of words in what Ankur says. You can be paralyzed by that fear, or you can imagine, I already won. What would the person who is won do in this situation? And that alleviates the fear.

It actually gives you the optimal path under uncertainty, under having uncertainty, under a lot of uncertainty. And don't use the old playbook. It could be try to be the person that figures out how to write the new, the next playbook. I think Max Tagmar has spoken with many times because he's a wild one. He did a lot of weird and beautiful things in his career, but he also did the mainstream stuff. He said you have to do a little of both. So, you know, you still work enough within the system that you're part of the community, you're part of the flow of, in that case, it's science and the development of things, but also do enough renegade and wild and weird things to pave your own way, both, somehow, and find that balance. And not be afraid to be a little bit weird, but also become very skillful and good at being part of a community

and a respectful part of the community. So, both of those. But it's like Tom Wade said, I like my town with a little drop of poison. It's just a little bit of weird, or sometimes a lot of weird. It makes for some beautiful humans. I think that community part is actually going to be even more important in the future. We've had this maybe 30-year experiment with how much can we do with machines. And now that we can do a ton with machines, my hope is that we're going to realize we actually prefer to do things with humans. We prefer to fly in and have an in-person conversation. We prefer to have an oral discussion instead of just reading a paper. And I think there's like a peak to how much our day-to-day experience. I mean, in terms of, yes, I get a lot of fulfillment from my scientific work and from my academic career, but going home to my kids and going to the beach if SLT gets solved and just throwing a ball with them,

there's nothing like that. Like, that's the meaning of my life. This other stuff is the cherry on top. But the foundation of it is family, is being around other people that I care about. And I think as AI commodifies intelligence more and more, we'll realize that more and more. That intelligence and all the different puzzles that we solve is just a cherry on top or a cherry to the side. The main thing is other humans in our lives. This is, I can't tell you how much of an honor it is that you would come down in person for a conversation. I've become a huge fan of all three of you. It feels a bit surreal, and it's a big honor that you would talk with me today. This was, like, super fun. Thank you so much. Thank you so much. I'm good to see you, Alex. This was awesome. Thanks for having us. Thanks for having us. Thank you for having us. This was great.

you