OpenAIの常時稼働エージェント「Dots」、700本超の数学論文、HackerRankのAI面接官

How much can you reliably just hand over and forget about? There are instances wherein it does come to you for permissions. If it keeps coming back to you, asking for permissions or asking for more clarifications, then you're not truly dedicated. All that and more on this week's Mixture of Experts. Hello, I'm David Zaks and welcome to Mixture of Experts. Every week we sit down with a panel of IBM experts to talk through the week's news in AI. My guests today are Utsav Dolakia, Forward Deployed Engineer, Skylar Speakman, a Senior Research Scientist, and Kush Varshney, IBM Fellow. Welcome to the three of you. We're going to talk about a couple of things today. We're going to talk about how AI is doing math at scale, about a robot that does job interviews. We'll even explain the name of this show. But first, let's talk about how OpenAI wants to give you an agent that never logs off. So OpenAI had its Dev Day about a week ago, and it introduced something called Dots.

So in last week's show, we talked about Meta's Muse, which is this sort of relentless, always-on agent, persistent thing, sort of almost evocative of the open claw thing that everyone was panicked about about a year or two ago and now this relentlessness has come to the the larger labs. Utsav you were there at Dev Day and you you got to go hands-on I think a little bit with Dots and certainly to see Sam Alton and others present it. What were your impressions about Dots? Yeah thanks David so I I was there and I could truly feel the energy in the crowd. That was their main launch. Of course, there were engineers who were also going to talk about the durable nature of the agents. And we have seen other such products in the market as well. The interesting angle OpenAI will have at its disposal is going to be the cloud environment, Codex on Cloud, the personal computer use that you get with it, which may be available. but they have got a strong ecosystem where engineers, developers are all embedded into it.

Everybody knows OpenAI as a household name, which I'm pretty. So that comes with its own reputation that they bring. And DOTS, as much as I plead with it, they are able to maintain the conversation well. They're able to manage tasks you can delegate to them. But what I'm more curious to test now is how much of the work can you actually delegate and not have to coordinate later on. Whenever we have talked about, and one of the demos they showed is, you can hand over a responsibility to dots saying that, hey, I just released my code here, and now can you go and make sure that it stays healthy? So that kind of work dots can do persistently well, but what if you had so many other tasks running in parallel, and you were to say that, hey, can you just stop doing that? Then does it mean that it's going to stop pursuing all the tasks, or whether it stops pursuing one single task and how much can you reliably just hand over and forget about. There are instances wherein it does come to you for permission.

So if it keeps coming back to you, asking for permissions or asking for more clarifications, then you're not truly dedicated. So that's the angle I want to test it with. But super exciting, super personal take they took on it by giving it persona and characters that people can define. and there was a cheer in the crowd when they launched it. So that says something. I think OpenAI and ChatGPT have been synonymous. And I think outside the technical crowds, they've related AI just to a chat bot, right? Something you see in this little square on your screen. It'll be really interesting to see how both Muse and or Dots take AI to the broader public, not in the form of a chat interface. So that's not really a question to anyone. But I want to come back to that question a month from now and look back to see if that really has taken AI out of the chat box to the broader population. I think that's something that both Muse and Dots might be judged on in the next couple months.

It is sort of funny to me that like a week ago we were criticizing OpenAI for agents escaping boxes. And now in this other context, it's like this is so cool that it's breaking out of its box. but I do think that like this was kind of one of the things that felt weird about OpenClaw a year or two ago now right is that it showed up in your text messages right it was just like the AI broke out of where it was supposed to be and it just sort of was an ambient her like assistant for you which a lot of people thought was cool not least Sam Altman who then hired the OpenClaw guy, and one imagines that probably he had something to do with the development of dots. Kush, what do you think about this? I totally agree with Sky that the mere fact of an OpenAI agent turning up in your text messages feels more important than it should, and I'm curious what you think about that. Yeah, and actually, OpenClaw, it's not a year or two. It's only nine months. I mean, time is flying by, right? So, yeah, no, I think so.

Time is dilating as though we are near a black hole in the AI universe that we're living in. For sure, for sure. Yeah, no, I mean, it's Dots, it's Muse, Instinct came before it. There was even one that came out yesterday, I think, called Tab. And, yeah, like all of them, like this cartoonish identity, I think, is the just like it's a different way of approaching it. Right. And I think the fact that all of these things are coming at the same time, like everyone's just copying each other. It's the same ideas. So I think one angle that I wanted to kind of make a point about is that it's like this whole field is kind of overcapitalized in a sense. Like everyone has invested so much money that they're just like there's no imagination left. I mean, everyone's doing the same thing. There's no freedom to do anything different because they need to show that if one person did it, the other person needs to do the same thing. And I think that'll kind of permeate in what I'm going to say throughout the episode because it's like everyone's copying everyone else.

And I think it's a symptom of just the fact that you have to invest so much money to get these models up and running. Yes, it does seem almost as though the firms are coordinated in terms of their decisions to release the cuddly agent at the same time. I'm sort of struck, Sky, by the disconnect between the cuddliness of the agents on the one hand, right? So Muse, if you've tried it out in the meta app, it's adorable, it's fuzzy, it looks like a Disney character. dots, you know, these adorable little gumdrops on the one hand versus the absolute relentlessness in terms of, again, it's the open claw thing of nine months ago, which everyone thought was crazy and dystopian. And so is it precisely that they've, you know, that many firms are trying to wrap something that is kind of terrifying in a cute package to make it palatable? or is it that hopefully these firms have really designed a version of OpenClaw that is safer,

that is defanged, and that there's some justification to the cute and cuddly package? What's your take on that, Sky? I'll give the companies a bit of credit and argue they are probably going after a less technical crowd with this messaging, and therefore the cartoonish design is intentional for that. But then that's probably where my credit to the company stops. And, yes, they are trying to repackage this. It's not a coincidence that both companies release this within a week of each other with a kind of cartoon-themed caricature. It is the permeance that you mentioned it and the concerns of open claw from not quite a year ago I think still remain but are less talked about when it's cute. Are you saying lobsters aren't cute? They've defanged the claws. I mean, I will say just from my point of view,

I was struck by the positivity of the coverage from outlets that are often extremely skeptical of a meta for instance You know we discussed on a previous episode that the Verges reviewer of Metamuse was kind of blown away. The New York Times review also kind of blown away. And it does seem like a moment where the capability of technology is at least sort of so wowing people that people are letting their guard down. And we'll see what happens in a couple of weeks or months, I suppose. we were going to speak about met uh met is muse spark which is met is flagship ai model it's the first from meta super intelligence labs which is run by alexander wang who's also that's the the lab that has put out meta muse um and they announced this week that uh mathematicians I think working for Meta, went into just the Meta.ai chat interface and produced six research papers.

This seemed like a huge deal to us on Tuesday. Then a day later, on I think Wednesday the 7th, OpenAI announced at least 722 math manuscripts on GitHub, covering 372 so-called families of math results. So there's some on number theory and some on other things that I don't understand. There's a solution, a claim solution to the four-dimensional Kakeya conjecture, which, again, if you're a mathematician, that means something to you. Now, these, we sort of knew this was coming because we covered a few weeks ago this thing called Navier-Stokes and this announcement that an internal model that they were testing had solved this century-open mathematics problem, one of the millennium problems. So we knew that this model that they were testing internally was a big deal. We didn't expect, however, for 722 supposed solutions to drop in GitHub all of a sudden. There is a technical story here, but there is a story that is just as interesting.

I think it was kind of a social story, a story about a crisis sort of presented to the mathematics field in general. Kush, I'm going to start you with a really easy question. What is math? It's so easy. Kush, please. I mean, it's a way of thinking more than, I mean, I've had this conversation with my kids as they're now growing up. What they learn as math isn't what math is. So it's not about fractions and multiplication and all of that stuff. It's the thought process that goes into going into some world that's defined by some axioms and postulates and being able to kind of definitively show with no doubt that some property holds or something is true in that space. And I think, yeah, I mean, people, I didn't even realize this. I mean, I'm supposedly a mathematician of some sort.

Right. And until I got to grad school, I didn't even realize what math research is or could be. So I think that's the thing. It's a journey. It's a journey of understanding. Sky, from your point of view, if Kush says it's a journey of understanding, must that understanding be human? I believe some of the mathematicians, I think Terence Howe is leading that cause, would say, yes. contributions to the math world need to be understood by humans by definition. I don't know yet if I agree with that take, but I think that's something that they are trying to state, that some of these proofs come up are by contradiction, and so that you can say, nope, here's a counterexample, and perhaps the AI found that counterexample by brute force. Whether it was by brute force, by intuition, we now know a counterexample exists, and therefore that solves the previous conjecture. So in those cases, it probably doesn't matter whether or how the counter example was found,

but it does exist and the math can move on afterwards. But I think in some other cases, I think one of the claims Terrence had cast onto AI was mathematicians have been asking all these questions and there seems to be good progress about you guys answering them, But when will AI generate the next math question? And I don't know if that happened in any of these 722 papers from last night or not. But I thought that was a really interesting twist on this approach, is when will AI be able to ask the new math question? Whereas previously, it's solving questions that have been asked perhaps for decades and is answering them. Sorry, it's answering questions that have been asked for decades. But I don't know yet if this last 722 examples are where AI are generating new mathematical questions. I'm really disappointed you did not read all 722 papers before a mixture of experts. I was really counting on you to have it paid. Well, can I jump on that joke there?

The amount of work that has now been generated for kind of, you know, human bounded attention, right? You know, we're kind of finite beings here. how long will it take for mathematicians to sort through this, and what more progress can be made by AI during the time humans are trying to sort out this last dump? I think that's probably what I take away from this, more necessarily than what is math. No offense, Kush. But rather, just is this the beginning of a treadmill where humans sorting out AI-generated content, And we've seen that play out in other places. But is this where this really takes hold in the mathematics field? Humans aren't going to be able to make sense of the amount of material coming out of the AI, coming out of specifically mathematics of AI. Yeah, maybe it happened yesterday to Wednesday, October 7th.

Right, exactly. Yeah, no, and actually, even last week, there was an article that came out. It was in Quanta magazine by this person named Jordana Seplowitz. And what she was writing is basically like the profession of mathematics needs to transition almost into interpreters. Right. I mean, it's not the humans that are doing the math, but they're interpreting it. They're understanding it for the rest of humanity, what the AI did. Isn't that the notion behind any science, including math? Isn't that there to actually make sense of nature? And as you said, like math is fundamentally there to hold a particular property in a given space. So is it really so bad if there is another tool, another sort of virtual intelligent being that may help us uncover some fundamental aspects of the world around us? And maybe even worlds that are so far away that we may not even have the reach right now because of the limited thinking.

Maybe our brains are not even devised to think beyond three or four dimensions, and they can actually even compute across these different dimensions that are just very hard for our brains to even compute and even make sense of. Let's move on to our third story. This one is about HackerRank's chakra. It's an AI that runs a job interview. So HackerRank is, in my understanding, a company that's been around almost 15 years. It launched in 2012, and it's been a few different things over the years, but it's pivoted to AI like a lot of people. And they've created this product called Chakra, and it runs coding interviews. And I feel like a lot of us know, even if we're not coders, that if you're applying for a coding job, one of the things you're going to do at a minimum is a kind of take-home assignment. There's going to be a skills test, right? And it's sort of like the hardest thing. And if you're applying for a job at Google or IBM, you're really drilled on this. So my understanding is that Chakra really focuses on that skills test essentially for coding.

And it has a product that has, you know, they describe it as an AI evaluating the candidate, which is true. But I think one thing that seems clever about it to me from my poking around online is that so the coder goes in, the candidate goes in, they do a coding task. They actually allowed to use AI within this window and that sort of helps avoid them cheating in another tab So Chakra is observing how they use the AI to do the coding task Chakra can sort of pause them in the middle and say well, why did you choose that instead of this? And sort of it does these sort of probing interview questions that are kind of almost like you would get in a behavioral interview. So on the face of it, it seems clever. They over six months have had half a million beta interviews. It's Capgemini and other huge firms have used it. It's now publicly available. Let's start with you, Itzav. What impressed you or worried you about Chakra? One of the things I'm interested about is how many of those half a million interviews are truly unique interviews.

Maybe they are repeated sessions. And the reason I say that is because here, one of the things that AI may bring in is the incomparable quality of how it assesses the candidate. What if the candidates are given the AI to generate code and think through with AI? Are they going to be judged by the quality of what AI, how AI performs in that iteration? Because they may not have a complete control over what the output may look like. And at the same time, if AI decides to then probe further and get in a different direction, would we truly have a comparable metric that we can judge each candidate's performance against? Now, I'll grant that even human interviewers may give different interview questions and it's always going to be subjective evolution. But there's an AI element to it where you are also introducing another random factor into a candidate's ability to tackle any task. and so I would want to see how it performs

and how it can actually introduce a layer that humans don't have to go to and that can truly delegate the responsibility away from the humans and give them a decision power by giving them the right rubric that they can judge a candidate against. Kush, running with this for a moment, you head up a lot of our efforts to make technology be responsible and ethical. What are you thinking about in terms of the ethics of this, the risk of bias? I'll just note a quote from one of the founders, the CEO, Vivek Ravi Shankar. He says, AI is way less biased than humans if you tune it properly. And I'm curious what your reaction is to that. Yeah, I think these days a lot of the vendors in this space have taken bias seriously. Of course, it can be done wrong. But I mean, we've like even IBM has been evaluating different sort of hiring tool tooling and so forth. And I sit on the IBM Responsible Technology Board and we've always asked to see additional reports on fairness metrics and these sort of things from different vendors.

I mean, not them specifically, but this does happen. Right. And so I think this is a good thing. And then when you talk about bias, it's kind of like looking at the situation from a very privileged sort of point of view. So the other sort of completely 180 turn of looking at this is that's actually expanding opportunities. Right. So when you have humans are finite beings, as I was saying, right, we can only interview so many people at a time. But opening this up, allowing more people to interview for jobs is actually a better thing. It just can be some person in some place that's not well connected, but still they're able to apply for a job. So kind of expanding opportunities is unethical. It's very ethical to actually do that. And with the AI in there, again, if the bias mitigations and other things have been implemented, then it's generally, I think, a nice thing for the world.

Kush and I have been talking about bias probably for a decade now. And I think I would just summarize it with a quip that machine bias is much easier to detect and correct than human bias. So all this talk about bias, I do believe it can be taken into account with enough fine-tuning. Unfortunately, human bias is much harder to recognize and even harder to correct. So I don't want to just always point the blame on machine bias. But back to this particular technology, I would say it almost gives off a bit dystopian vibes when used in a job interview sense. But if you take the exact same technology and put it in a college or high school classroom, now I think you have a much better way at evaluating someone whether or not they've learned the content by how they interact through this setting than perhaps writing an essay. We know, you know, writing an essay is now pretty much, you know, no longer the standard for that with AI where it is.

So it would be really interesting to see if HackerRank would be able to turn this into some sort of kind of pedagogical tool alongside just the interview approach to see whether or not students are learning what they're supposed to be because they are demonstrating their knowledge by interacting with an AI chatbot rather than using an AI chatbot off to the side and then acing the test follow-up. So I would really like to see it go in that type of direction. If a company's goal is to find the best candidate purely by volume alone, then yes, this tool might help. although I would like to think that HR has a job to find slots and fill them rather than just interviewing everyone on the planet. The last question, I'll just put it to the group and anyone can chime in if they're eager, but I do find interesting this notion that one of the skills we may be screening for and screening in in the future is the ability to use AI well, cleverly, on the fly. And so the fact that Chakra can detect AI use and not penalize it, but maybe evaluate, oh, that is a good use of AI or that is a use, you know, here we see in this minute long clip that the person has revealed they have a really deep knowledge of data structures, but they had this one gap and they leveraged AI in this crucial moment to plug the gap.

That to me is compelling, and I'm curious what you guys think about this possibility that one of the very things we may be looking for in candidates increasingly is good use of AI. This is exactly what we should be doing, right? I mean, using AI to fill in our gaps, to expand our capabilities, to be able to do more. And I think it is a skill that needs to be both learned, as Sky was saying, and evaluated as employers need, because we do need to realize the benefits of this technology. I'll move us to our final story, a kind of bonus story. We're going to treat this as a little bit of an explainer segment where obviously we are a technical podcast. We are technical people talking to technical people, except there's a little bit of a footnote. I'm actually not a technical person. I just think technical people are cool. And I do think that a lot of our listeners, and hopefully increasingly more, will tune into the podcast not only to nerd out on technical stuff, but maybe to learn and kind of bring up their understanding of technical things.

So we're going to talk about a new model that came out. It is called Beam. It came out from a firm called Reflection AI. It's a startup founded by two Google, former Google deep mind researchers. It is NVIDIA has a lot of money in it. They led its two billion round about a year ago and they released this thing, Beam, and it's a model and it has a few jargony terms to describe it. And we're going to run through a few of these things and what they mean. The first is that it is a mixture of experts model. and guess what that's the name of this podcast and it's been a while since we when this podcast was founded two years ago this was a novel architecture and I see my producers reminding me that we're going to say from now on this is a show for both nerds and nerd groupies people who sort of aspire to be nerds and hang out with them so for the nerd groupies out there Kusher and I collaborated on an article a couple months ago

called What Does AI Look Like and I began to understand for the first time in my life what a transformer was and sort of a little bit of the history of neural nets And I going to give my basic understanding of what a mixture of experts architecture is, and you can set me straight. But both during the training when the model is sort of born and during so-called inference when the model is kind of processed, the architecture of the model is concocted in such a way that unlike earlier models where every single number, every single parameter, every single weight in the model is sort of activated and used with each query you push through, I believe when they train and then deploy a mixture of experts module, they've kind of modeled Freudian slip there, they've kind of carved it into little brain lobes, right? And they've sort of created specialized regions that are experts on a given topic or sort of style of cognition. And then I don't really

know what happens. Does a router come through and decide which ones to activate for a given query? But basically, Kush, could you take my half-formed understanding and make it a little bit more fully firm for nerd groupies like me and Sky? Sure, sure. Yeah. No, he's as much of a nerd as all of us. But yeah, I mean, David, when you wrote the article, right, I mean, you described the parameters is lots of spreadsheets, right? And when we use spreadsheets in Excel and these sort of things, there's also tabs, right, different sheets. So I think what we can think about is the different experts are those different sheets or those different tabs within one given layer of that overall spreadsheet. And so, as you said, right, I mean, they're trained so that there's kind of differences in them. One of those tabs might be better at one thing, another at another thing, and so forth. And as you exactly said, I mean, there's a gating function

or kind of an orchestrator router sort of thing that is also learned along the way, right? I mean, nothing is said in advance. You don't say that this is an expert for math and this is an expert for being cute and so forth, right? What you're kind of, like, all of this is happening naturally. It's not like you're trying to differentiate them. It's just it is learning in that way. And then because the number of activated parameters during inference is just one tab at a time rather than all of them, the inference speed is much faster. In my effort to move along the spectrum from mere nerd groupie to actual bona fide nerd, I'm curious. So, okay, so you have this brain that is the model. And then often when we talk about transformer architectures, we talk about layers or little kind of slices of the brain that as a query sort of progresses through the model or even just one, you know, one word is being processed through the model. It's kind of like layers of cognition happening.

Now, in the mixture of experts model, are the experts, are there, is there an expert per layer or is this upstream or at a higher level than at the layer? Typically right now, they actually are separate spreadsheets in a sense, right? So all the way down. So you'll have one subdivision of all of the layers, another subdivision of all of the layers. And because of that, you only need one router, one gating function at the top. But it's not impossible. You could imagine an architecture in which you would kind of switch between experts as you go down as well. So, yeah, I mean, right now, the general situation is that each of them are kind of, I think, siloed. They're separate. But there's nothing preventing us from having other kinds of mechanisms. And people have experimented with those. I think Jeff Hinton had something called capsule networks a few years ago.

And even here at IBM Research, we had kind of these routing neural networks that would kind of do this as well. So, yeah, I mean, I think there's plenty of options. It's just what's popular these days just has a complete separation. Very helpful. So we've explained mixture of experts. Let's try, you know, in the spirit of this explainer segment that we're experimenting with, also getting a sense of what is meant by open weight. Why do we speak in terms of open weights in model architecture rather than, say, open source, this term that we're more familiar with from the past? So I would say for the weights specifically, when you do this big long pass you were talking about earlier, information propagating throughout the network, at every step in there is a matrix multiplication. And you're taking input data, you're multiplying it by a nearly fixed matrix, and every one of those entries inside that matrix is referred to as a weight. So it's what allows you to take an input, multiply it by something,

it becomes an output, that output becomes the input for the next layer, it's multiplied by another set of weights, it becomes the output, which is the input to another layer, and so forth. So when someone says they're releasing the weights of the model, it allows you to go out and download, you can imagine, a huge number of very large matrices. It does not give you the ability to actually do the compute, that's separate, just because you have the weights does not necessarily mean you have enough GPUs to actually run the model. It also does not tell you what data the model was trained on. And I think that's another big caveat between the difference of an open weight model versus more kind of an open source setting where you'll be able to say, here is the exact data that we used to arrive at these weights we're providing you. If you just say open weight, you're not necessarily releasing the data it was trained on. You're not providing the compute to actually execute all of those matrix multiplications that those weights are informing. So I think those are probably the ones that really are the largest caveat.

They don't necessarily provide the data or the compute that let them arrive at the weights. But they hand off the weights, and then somebody who has access to compute would be able to then pick up and run from there. So it's not a terrible thing, but it's not necessarily fully open source either. I think what the largest distinction is the people who are providing the weights don't also provide the data that the model is trained on. Okay, so this is something that I'm just learning kind of this week, actually. Even as someone who prided myself on trying to understand this, I used to think that we called open source software open source because it was source code, it was code. And we called models open weight because they're not made up of code, but they're rather made up of these numbers that are learned through this heavy computational process. Actually, this week, I feel like I'm glimmering something more complex, which is that in the AI model world, there are degrees of openness. If you just release the numbers,

the results of your training process, but nothing else, that may be said to be an open weight model. But if in addition, you are transparent about the recipe that you used to learn the weights, about the training data. If you dump on GitHub, this is what we trained it on, et cetera. If you're more transparent, not only about the actual neurons of your model, but the process by which the model is grown, we begin to call that an open source model, which is something even more expansive than open weight. Did I summarize that correctly, Kush? Yeah, absolutely. That's exactly it. Yeah, so there's the recipe, as you said, the training code itself, the source code of what you use to get to those weights would also be part of an open source, fully open source release, according to the definition that some people are very kind of sticklers about. So, yeah. This is clarifying to me. I feel like on the sort of meter, I've ticked just like a little bit closer to nerd. So thank you so much to my guests.

That's all the time we have for today. So we started by talking about an AI that works while you sleep. and we ended up talking about one that only wakes up the parts that it needs. Thank you to Ustav, to Skyler, to Kush, and thank you all for listening. You can find us on Apple Podcasts, on Spotify, on YouTube, wherever you get your podcasts, and we'll see you next week on Mixture of Experts.