ナビエ–ストークス問題とAIエージェント

This podcast is presented by NXAI, your partner for Time Series Foundation Models and Physical AI. Hello everybody and welcome to a new episode of our Industrial AI Podcast. My name is Robert Weber and in Linz, Günther Klammbauer is waiting for me. Günther, welcome to the podcast. Thanks for the invitation, Robert. It's great to be here again, and I'm happy to talk about AI with you and the other guests. Yeah, exactly. Because we wanted to record an episode on AI, LLM stuff, and mathematics, and you said we should invite a very special, a very prominent voice in this area, Gonzalo Cualabora. And I said, we have two more days to do that, and you said I will write him a message, and here we are. Gonzalo, welcome to the podcast. Hello, thank you very much for the invitation and it's a pleasure to be here.

Yeah, thanks a lot. You're based in Lausanne, right? Yes, I'm based in Lausanne, Switzerland. Okay, perfect. Before we start talking about AI and mathematics, can you please introduce yourself briefly to the listeners, Gonzalo? So, I am a mathematician by training and I have lived in the US for a while doing my PhD in MIT and also living in New York for a bit. And basically I have, and now I work in EPFL in Switzerland, and I have worked all my life trying to understand partial differential equations that model evolution of fluids, evolution of gases. And I'm also very interested in the development of AI techniques to advance the understanding of these equations. Günter, short introduction. Yes, I'm a professor at Johannes Kepel, University Linz, for artificial intelligence. and I speak about all topics related to AI. Very prominent. No, not not prominent. Very, I was going to say, I like to talk about it.

And you like the format, right? You like to talk in a podcast, yeah. Yes, exactly. That's what I wanted to do. That's why we invite you always, yeah. So, Gonzalo, first question, the easiest question. Did OpenAI actually solve the Navien-Stokes problem? Yes. So I think, I believe that this thing should be carefully checked. And I don't think we are at the stage yet to have a final write-up that every mathematician would agree with. But I think everyone in the community is very inclined to believe at this moment that this is true. But of course, pending official final confirmation. What is your opinion? My opinion is that the proof actually develops very interesting new ideas. and it's incredible the new ideas that have been developed by the suggested proof. On the other hand, I should also say that the writing is extremely poor and is not presented in a way that a professional mathematician would. That's also the reason why it's taking us so long to the mathematical community

to actually understand and confirm that this truth is right. But you are a bit skeptical, I hear. I wouldn't say skeptical because, I mean, I have gone through the proof. I have spent a big part of the last month trying to understand it, and I'm fairly confident that it's correct. Where I'm not satisfied with the presentation of it, I don't think it's a final product of the quality that we should expect for such a big result. If I may jump in here, are you actually working more with the natural language proof or with the lean proof, with the lean version? No, I think the natural language proof is better. Of course you work with AI, but you try to unparse basically what the proof has done. There is a very interesting paper of Peter Konstantin, Mikhail Ignatov and Vlad Vickul, where actually they do some further work on this paper. And it's very interesting to read their AI statement, because they say we have significantly used AI to write this paper,

and by far the task that has required to use the most AI is just to decipher the proof released by AI. Günter, are large language models becoming great mathematicians or are they just becoming extremely good at manipulating mathematical language? What is your opinion? Yes, I think a bit of both. First of all, I think they've become very good because there's a lot of material to train this. The mathematical community was open and shared a lot of manuscripts and you can always present something like new mathematical problems to the system and it learns to solve them based on what they have. And then also this Lean, I think this is something, this Lean programming language is something that is in favor of the large language models. There are, of course, these drawbacks with the readability and this has, I think, multiple reasons. So first of all the training Reinforcement learning Post training goes in the direction that

It has to solve it correctly And not that it has to transport It well to the community and so on So this is the one thing I think it doesn't Really it's really not Reinforced To write this in a very good Meta and understandable meta And the other thing is also that I think the Kind of brain of the LLMs Works a bit differently than ours Because they have this KB cache, so they basically can store a lot of words in the context and look back. So they basically memorize almost all the things and the results that they have already written, whereas we, our memory is a bit leaky. We don't have everything exactly. So when we write and when we present papers, publications, we already, we know, for example, that we have to repeat concepts or pick up concepts or structure in a certain way. and therefore, for many other reasons than mathematical publications or in general publications are better written or differently written by humans. And I fully understand concerns by Gonzalo and the mathematical community

and I also have very ambiguous feelings towards not only the Navier Stokes but also towards the other problems that have been allegedly solved. have been allegedly solved. Gonzalo, you mentioned proving, right? So the proof. Can AI discover mathematics or is it just getting better at proving it? So can you explain that a little bit? Or what is your opinion? So I think for the moment it's getting better at proving things and there has been a lot of talk about this convex hull idea which basically means that what Deja is strong at is to get ideas from very different parts of the mathematical literature that no mathematician would be able to know all those different ideas, and then combine them in a very smart way, which already allows you to develop incredible proofs. But of course, this is very uncertain because given the rate of progress,

it's pretty much likely that in one year the AI is even still much better and able to develop new interesting ideas. Actually, in Navier Stokes, there are a couple of ideas that as far as I know and collaborators know, they are completely new. So already they don't appear in previous mathematical literature. Yeah, Guto, you want to add something? Yes, so first of all, I think this is a very good thought by Gonzalo that these different mathematical tricks, so I have an undergrad who studied mathematics, so I dare to say something about it. So in order to prove something, you often have some tricks. You introduce variables. You introduce a concept or transform variables to another space. There are different types of tricks in each mathematical community, and maybe each mathematician has his own set of tricks, and, of course, an AI can maybe see them all or many and use them in different contexts. That's what Gonzalo means, and I think that is very valuable to connect this. yeah and the other question is and i'm very curious here so i think the initial idea

consado was that this navia stokes proof follows something like what they call that the spanish program or something like that and it doesn't appear so is this what's your take on this so that's mathematically duty for me and i cannot follow so it would be great to get some insights from you there? I think the discussion here has been a bit between white and black, and I think it's actually more a bit of a gray area. So what I believe is true is that, apart from Navier Stokes, OpenAI released a proof of blow-up for the Euler equations, which is a very similar equation, just without viscosity and without force. And this one builds much more heavily, I would say, in the Cordoba-Martinez-Foro approach. And then OpenAI also recognize that in order to solve Madirestos, they feed it with the proof of Euler. So there is, of course, some reminiscence of the ideas there, and you could see where. But I wouldn't go as far as saying that it's directly the same approach, because there

are also an incredible amount of new ideas. For example, half of the paper goes into a different strategy that then is merged with this strategy, so that half has nothing to do. And then in the other half, you can really see where these ideas could come from the Spanish program. Gunter, I want to come back from our listeners because they are not too deep in the whole topic. But is maybe the domain mathematics the perfect domain for AI research, AI-driven research AI discoveries because it's a domain with clear structure and clear processes. Do you know what I mean? So that Mathematic is maybe the best domain to prove something there. Yeah, I mean, maybe together with coding that's one of the domains that can profit a lot from these post-training strategies. So maybe to give a bit of an overview for people that not only OpenAI has not only announced

that they solved this Navier-Stokes problem, but also they've uploaded 722 additional manuscripts where they claim to have solved some other math problems that were open. They had to, there were some problems, three of the manuscripts they had to withdraw and so on. So there will be some problems. And I think also the readability and so on is difficult. And some of them have this formalized lean proof, some have not. And this has led to a big discussion, I think, in the mathematical community and generally, what's the impact of this and how will this field change? Why is it working so well? I think this is because you can always present kind of new problems. So these language models learn first natural language and so on, but also programming language. And they can also learn, of course, on mathematical papers. mathematical language is also a bit formal that we repeat stuff and introduce concepts and so on. And now think a bit back to your high school. You also got math, small math from really simple ones to solve.

But basically your teacher could always come up with new ones. You add these numbers or in the text change things. So this is very good for AI that you have basically an infinite amount of training data. You cannot memorize anything. you always get a new problem and I think that's what they're doing for this model, it's unreleased that they presented a lot of mathematical problems and then it tries to solve it and then what's the good thing is you can verify if the solution is correct, right? If it's another field for example write an article about blah blah blah, you cannot verify yes or no if the article is correct or not but here if the mathematical problem has been solved or whether the program in some Python code or C++ code, you can check if the thing is correct, also with the help of Lean, for example, and that's a very good training signal for the neural network to learn from. So these, because they have,

these are two domains where you have verifiable rewards and that leads to very powerful models. Gonzalo, from your point of view, How much of the discovery came from the AI and how much depended on human mathematical intuition? Could a powerful language model have achieved the same results independently? What is your opinion on that? So I think it is, so what is easy to think about here is the counterfactual where AI is not developed at the current level and mathematicians try to come up with this proof. I think that would take decades for us. I don't see any way mathematicians would be able to get there soon. It's more difficult to judge whether AI without previous mathematical literature would be able to arrive to this group, because of course AI is trained on millions of papers, so then which papers do we remove from the literature is a hard question. Yeah I think if it hadn seen any of the recent results I think it would not be possible And we see also that it can solve certain problems and some problems are yeah it didn progress that well So there are also fields or areas where it worked better and less

So, and also the energy invested was gigantic. So I think at least 23 million euros of compute were invested for this Navier Stokes stuff. And yeah, the price is 1 million euro, by the way, and not euro, but dollars. Yeah, but I think it will and also of course I'm very interested in Gonzalo's opinion. I think it will have an impact on mathematics how they will proceed. I think Terry Tao said it will become more industrialized and larger groups working maybe together with AI. We've seen other scientific fields like molecular biology where it is more industrialized. There are big consortia working on problems and also in physics, like you have CERN and so on, so they can make progress by making big industrialized research in a way, because they have a big consortia and also have big machines, basically. So, maybe this is a trend. But, yeah, again... That's an interesting question.

Do we see an industrializing or industrialization of mathematics, Gonzalo? Yes, it's the only possibility, I think, because basically anything that becomes automatized, immediately loses its value, and therefore mathematicians will dedicate themselves to actually understand these proofs, trying to present them, trying to apply them to other sciences, and that job, at least for now, the LLMs are much worse at doing than the actual job of doing proofs. Of course, it's uncertain what capabilities they will have in a couple of years, but at least for now, that's what the future seems like. Do you have any other insights why it's so unreal? So I also tried to read it. Gonzalo, is there any other insights? Why, or do you have some, can you say something? Why it's so difficult to read? So I think it's really good this point that you raised earlier about the memory. LLMs are able to memorize incredible amounts of data. And for example, when you look at the Navier-Stokes proof, there is an incredible amount of different variables.

No mathematician would write a paper like that. There is actually five pages of the paper that are just a table with all the different notations. Of course, this is something that for artificial intelligence probably is not a problem, but for a mathematician, you just forget about any technology when you are on the second page. You've argued that AI is becoming remarkably good at constructing explicit mathematical examples, but still struggles with developing entirely new theories. Is that the fundamental limitation of today's AI, Gonzalo? It's hard to talk about fundamental limitation. I think it's a limitation of today's AI. I wouldn't say fundamental, because maybe we talk one year from now, and this limitation is removed. But as of the AI that exists nowadays, and specifically the one we can access, not the internal models, I think that's a limitation. What is your opinion, Günther? Yeah, I think it's also a very smart combination sometimes of different tricks

that probably no human can grasp across the fields. And by that, by combining things, often something new comes out. So I think it's especially that connecting between the different fields is good with AI, I think. But what I see is like maybe not so much in mathematics, but very disruptive ideas. They are not represented well or an LLM will not do that. So if somebody has really a counter-opinion like that, I mean, for Navy Stokes, it's probably hard to, maybe in some other fields, think back. For example, in the Middle Ages, when the common opinion was that, yeah, the Earth in the middle, and then the sun rotates around the Earth, then it was very, and LLM would have probably believed it very strongly based on the literature that was available there. And it would be very hard to find out with this disruptive idea that the heliocentric world model that the Earth wrote.

And this is a remarkable thing by Galileo. And so these counter-opinions or unpopular opinions, very rare opinions, I think this will be hard for AI. I don't know if it's so relevant for mathematics, but in machine learning, for example, it would always recommend probably to approach new problems with transformer architectures because they're so powerful, whereas in some areas, this is probably not, and experts in the fields would have, no, I will approach this problem with a completely different architecture or even with a non-machine learning approach. And yeah, maybe that's not such a disruptive idea, but very unpopular ideas. And these are often the most important ideas that transform or open new fields of sciences, right, if you think of that. And, I mean, also Einstein's ideas were very unpopular and criticized when he came up with that. But I could not come up with a good example here in mathematics where somebody came up with something very unpopular that then proved.

Gonzalo, do you have an idea? Yes, I have some ideas in mind, and specifically I would look at a new understanding that simplifies previously existing proofs. So say that now you think, okay, the proof that has been done of Navier Stokes is extremely complicated. Let's try to simplify this approach. I don't think the current AI is good at that. And I think part of the problem is that it just led to the old approach because it's the one that has learned. but probably you want to simplify this. And probably also you cannot verify whether an approach is simpler than another. At least when I try to work with LLMs, they are not extremely good at differentiating what is a simplified version of the same argument. So I could see how in these kind of works where you actually try to simplify an existing method, the artificial intelligence is not as good. Gonzalo, at the end, if you were starting your career in mathematics today, Would you be excited or worried? I think I would be more excited maybe

because I think the situation is very uncertain right now, but it will stabilize. So if you're starting your career now, probably you're starting a PhD. You have four or five years, and I think the situation of mathematics will be much more clear in four or five years I think the people I feel bad for are the ones that are applying for jobs this year for example That is an extremely uncertain position to be in What will happen in the next five years? Definitely, proofs will become completely automatized, and I don't think mathematicians will be proving theorems anymore. Probably we still want to encourage mathematicians in their training phase to be proven theorems, just the same way we teach people to multiply in schools, but of course this can be automatized. And I think mathematicians will probably have a broader understanding of mathematics. It will become less specialized because we will be able to digest mathematics in many more areas with the help of LLMs.

and we will also probably center more ourselves in trying to ask what are the good questions and in what direction do we want to advance mathematical knowledge and what's the application of it as well. What are the good questions? Do you have two, three good questions? Yes, I have the good questions. Now I think another effect of artificial intelligence is that people are less willing to share good questions and that's something I'm not especially happy about. But I think it will happen. Because now everyone can try to answer those questions with the AI. So therefore it becomes a problem. Because if you just share your questions, anyone may actually get the solution in a couple of days. While in the pre-I era, normally you would be very happy to share your questions. Because you had already been working on them for one year, for two years. And you knew that other people were not going to try to scoop you or to try to answer those questions before you. Gunter, if you started your career in mathematics

today, would you be excited or worried? Yeah I never had a career in mathematics Imagine, imagine Imagine No, no, I think I would be I would be excited but also a bit worried where this field is going but also times of uncertainty it's just a time of uncertainty how this is transforming but I think it's not only mathematics I think people that are now in school or finishing school or the studies. I think there's a lot of uncertainty also, not only in mathematics, but in other fields. Will AI take my job? Will I study at all? And so on. And, yeah, I think the transformation that AI brings to, yeah, general to all jobs, basically, we will see that in the next years. And then when the dust has settled, we will see what the important skills are that you have. And I still think that analytical skills, the mathematical skills that we teach people in school and in universities,

this will still prevail maybe to work efficiently with AI together and so on. But I think these things will remain important. Natalo, what is on your research agenda in the next coming months? Give us a short overview. So I think, I mean, one question that I'm specifically interested in is, okay, and what new questions do the Navier Stokes blow up give us? And I think there are several important questions there. Can you share one? So, for example, one thing that I'm interested in is if you could adapt this technology to try to develop a singularity as the Navier Stokes proved in different physical situations. but basically using the same ideas from OpenAI, but in a different configuration. This could give you perhaps a blow-up that is more physical. So one thing that is not very satisfactory of the proof that they released is that the blow-up seems to be extremely unphysical. You tune the force that you have to apply to the fluid

in a very, very, very delicate way. And you would probably want to try to remove that. Okay, okay. Guter, what is on your agenda? Yeah, many things. My agenda is still to have more efficient alternatives to the Transformer, of course, to finally kick that thing out of the AI systems and many other research ideas, not only regarding LLMs, but also AI systems in other fields, like I always like molecular biology and biochemistry and so on. So I also don't want to share here because otherwise it gets cooped out of me. Oh, okay. What is the next domain they are focusing? Yeah, I mean, we already see a bit that they are, I think, just on physics, but also I think the life sciences, so Anthropic announced something like a harness for life sciences, and also I think OpenAI announced something today, Rosalind Lab or something like that, so something, I think they are going a bit into this life science direction.

It's a field where you, it's a bit more difficult, of course, where you cannot deduct stuff by combining previous results, some things we just don't know, right? We just don't know some things about diseases, how they develop and so on, for some at least. And it's also not, there's no way to combine the existing knowledge to find out you have to make an experiment or something like that. So this would be in a way harder for AI to make progress without experimentation in the physical world. I highly recommend the episode with the Dunja guys from Berlin. They built a gym to train agents and to train the AI on experiments. So I highly recommend the episode. I think four weeks. Yeah. I mean, there they can predict which experiment to do next. And we also see that in physics and under which conditions and so on. So I'm very excited for these phases, of course. Also these agentic systems are able to do it. Also, in certain ways, some labs in biochemistry are already a bit automated

with robotic arms and so on. So, yeah, I think they will see a lot of progress. But whether the LLMs and so on, they are already trained on the whole Internet, right? And now in math and coding, they became very good. It's very hard to predict what will be the next breakthrough. I mean, ChatGPT itself is only like four years back. Simply very fast. You're right. It was a pleasure. I can hear a kid in the back. He needs his father. He needs his father. and Gonzalo it was a pleasure greetings to Lausanne thank you very much have a very great day thanks Robert thanks have a great day