Anthropicは再帰的自己改善の入り口にいるのか？ METR報告書（一部黒塗り）を読み解く

I just got through reading what may be the scariest AI report I've ever seen. Nieder, an independent group that assesses AI risk, spent 11 weeks going through Anthropic's internal data. They found Anthropic is now improving its AI about twice as fast as it was a year ago. And the main reason seems to be that its AI is helping build the next version of itself. The reason that's scary is if Anthropic is able to hand off a big share of coding and experiments to its AI, the next more powerful model arrives sooner. That version then does even more of the work, so the one after that arrives sooner still. And each iteration leaves less time to verify that the new model is safe before it's helping build the next one. AI experts have been warning about this for years, and there's a term for it. It's called recursive self-improvement. And if Neater's right, Anthropic might be on the cusp of it right now.

Some of those experts thought this was further off. This report changed their minds. Anthropic has drawn itself a line. The point where its AI speeds up its research enough to make two years of progress in one. By the time it reaches that point, it plans to have taken serious, specific steps to make its AI harder to steal and to monitor everything happening inside its own AI development. In its August risk report, Anthropic said it didn't believe its models had reached that line. They invited Meter to look into that claim and gave them 11 weeks inside the company. They were given access to data almost no one outside Anthropic has seen. Internal test scores, data on how its AI is trained, and interviews with people building the models. They didn't have unrestricted access, but Meter says Anthropic shared a lot more than they expected. At the end of its investigation, Meter found that, as of October, Anthropic was improving its AI about twice as fast as it was a year ago, at least on the kinds of tasks that we can easily measure.

That's meter's best guess, though they say the true speedup could be anywhere between 1.3 and three times as fast. One of their main pieces of evidence was Anthropics test scores for every model it's built, including three internal models that are better than any the public has seen. If you plot those over time, around October of last year, the line suddenly gets about twice as steep. Personally, I find twice as fast hard to wrap my head around. So let me try to make it feel real. In late 2023, Pod was a chatbot. It couldn't look at a photo. It couldn't even look things up on the Internet. By late 2025, Anthropic said it could work on its own for more than 30 hours, building an entire Slack-style app with no humans stepping in. That was two years of progress. Then, just a year later, AI helped settle a math problem that had defeated generations of mathematicians for 87 years.

That's what Meter means by making two years' worth of progress in one. Meter's next question was, what's behind that speed-up? They ruled out the simplest explanation, more computing power. Anthropic is definitely using more, but that isn't where the speed-up is coming from. Other obvious explanations, like a hiring spree, a trove of new data, or a big research breakthrough, may have kicked it off, but they can't explain why it's lasted this long. Taking everything into account Meter best guess is that about 80 of today speed comes from Anthropic using its own AI to help do its research But Anthropic redacted most of the reasoning behind that number in the public report including the majority of the evidence that Meter found most convincing Meter admits this means we have to rely on their judgment much more than would be ideal. But they were able to share some of the evidence. First, a bunch of Anthropics' own researchers say AI has sped up their area a lot.

One said, I've seen us discovering very promising directions entirely because people can spam 10 ideas at Claude and just see what sticks. On their own, impressions like these wouldn't be that convincing. But they line up with the timing. The speed-up seems to start in October 2025, right around when Claude got good enough to write code on its own for hours at a time. Here's the second piece of evidence. Claude isn't doing major research on its own yet, but researchers say they're seeing early signs of it in several parts of the training process. Anthropic has its own numbers on this. Earlier this year, Claude was doing almost none of its research work end-to-end. By August, it was doing about a quarter, though still with humans supervising. So meters read is that AI's role in its own development has been growing and that it's now the main reason Anthropic is speeding up. And there's nothing obvious that's going to slow it down.

AI building AI is different from other technologies. The better your AI gets, the faster it helps you build the next one. That loop only stops if it hits a bottleneck, like running short on computing power, training data, or ideas for algorithmic improvements. Meter looked for all three and didn't find anything likely to get in the way in the next year. They aren't confident. Their picture changed several times as they got more information. But Meter says they can't rule out things getting a lot faster in the next six months. they even say the speed up itself could be increasing if the speed up does keep climbing then two years of progress in one year could become two years of progress in six months then three months then six weeks two years ago ai couldn't count the r's in strawberry this year anthropics mythos found thousands of serious security flaws that nobody knew about

in every major operating system and every major web browser. After a few more rounds of speed up, I honestly can't imagine what AI is going to be able to do. This is all uncomfortably close to the way things play out in AI 2027, a well-known prediction scenario written two years ago based on AI progress and the geopolitical situation at the time. In its darker ending, by the end of 2027, AI is super intelligent, and humans have lost any real chance of controlling their future. By 2030, it's killed almost everyone on Earth with biological weapons. In that forecast, the speed-up in AI research meter just found doesn't happen until early 2027. If meters write about this, we may already be a few months ahead of schedule. All of this rests on Meter being right.

So how solid is their finding that Anthropic is now making progress twice as fast Meter redid the analysis lots of ways with different statistical methods changing which benchmarks they use leaving out certain models Almost every version still showed progress happening around twice as fast. The strongest pushback I've seen comes from the philosopher Toby Ord. He points out that that line of test scores could be read another way. Mythos came from a much bigger training run, and those usually bring a one-off jump in ability. Allow for that, and the line only shows progress about 1.4 times as fast. That's a fair point about the test scores, but as AI safety researcher Buck Slegeris points out, they're not the main reason Nieder reached their conclusion. They're just the only part of their analysis that wasn't redacted, and Nieder says the redacted evidence made them take the twice-as-fast finding more seriously.

Another reason some people are skeptical. Compared with other companies' models, rather than its own past, Anthropic's speed-up looks smaller. Meter says that could mean Anthropic was just catching up, or that its models were less tuned to the test than its rivals. But you can partly check that with public data. An economist, Karthik Tatapali, tried, expecting to cast doubt on Meter's result. Instead, he says he came away convinced. But the biggest caveat is that all of this is measured on tasks that are easy to grade, like a coding problem with the right answer. We don't know if messier real-world work has sped up as much. But that cuts both ways. Meter points out that AI's math scores haven't sped up much, even as Millennium Prize problems that have stumped mathematicians for decades are now being solved. A lot of people I trust have changed their minds because of this report.

Ryan Greenblatt, who studies these risks for a living, says he was seriously wrong. He'd expected much less speed up this year and now calls the report a big update towards superintelligence within one to two years. Buck Schlegger now thinks there's about a one in three chance AI companies build superintelligence within the next year. And if that happens, he thinks AIs will take over. Nobody knows exactly how much AI is speeding up Anthropics AI progress. But whether progress is 1.4 times as fast or two and a half times as fast, AI is now helping build better AI. And nothing meter found suggests that's about to stop. I want to come back to Anthropics' line. Two years of progress in one. If Meter's right, isn't Anthropic already there? Apparently, Anthropic didn't think so, at least as of August.

Their last public report said progress had sped up by less than a factor of two. But when Meter ran Anthropic's private numbers, they came out at 1.97. Just as I was finishing this video, Anthropic answered this question itself. Its latest IPO filing says it has now crossed its own line based on what it expects from its next models and based on how fast things are moving now. It hasn't said what it's going to do about this, only that it's evaluating the implications. Anthropic is either at that line or very close to it. And I want to see Anthropic do what it said it would do. That means security is strong enough to stop the world best resource hackers including governments and tracking everything that happens in its AI development so it would catch an employee or an AI doing something it shouldn But to be honest even if Anthropic does all of that

it isn't nearly enough. Recursive self-improvement, AI building better AI, building better AI may be close. The AI companies aren't stopping because they think that if they do, their competitors will just keep going and those competitors might be even less careful. So if the companies won't stop themselves, the government needs to step in. Alex Bors, a New York state legislator, is calling on Congress to ban AI companies from fully automating their own research. And some version of that proposal has supporters from Bernie Sanders to Steve Bannon. I think that's exactly the right place to start. Anthropic deserves credit for agreeing to this kind of outside investigation. OpenAI, Google, and the rest could be speeding up just as fast, and we wouldn't know. But Anthropic set the terms here. They decided what information Meter could see, what made it into the final public report,

and whether Meter will be invited back. They also tightened their redaction rules in the final week, forcing Meter to rework their public report and push back publication. And Meter says that while writing their report, they were worried that being too critical could make companies less willing to let them in next time. That's the problem with outside scrutiny that only happens when the company agrees to it. So at an absolute minimum, every frontier company should have to report how much AI is speeding up its own research and have someone independent check what they say. If you remember early 2020, you know how this kind of thing feels. In February, COVID seemed like a far off problem or even fear mongering. By April, half the world was in lockdown. That's what happens when something keeps speeding up. It looks manageable right up until it isn't.

And COVID spread at a fairly steady rate. With AI, the worry is that the rate itself will keep increasing. This might just be the beginning of an almost incomprehensible speed up in the progress of literally everything. For better and for worse. If you're wondering what you can do, if you've been considering pivoting to work on this issue, now is the time to do it. That might mean working directly on these problems. It might mean finding a more flexible job so you can take projects on on the side. Or it might mean donating more and thinking hard about where. And talk about it. Share this video or your own worries with the people in your life. Most people still have no idea this is happening. Or learn more about recursive self-improvement and why it's so dangerous. We'll leave some links below. If you're not sure where to start, we have a free AI career advising tool that can talk you through it.

It can suggest options and support you along the way. because if things change as fast as this report suggests, there may not be long before what we do stops making much of a difference. And when that happens, I want to be able to say, we gave it everything we had.