AIがゲーム開発者を罰している：Steamの「AI開示」と売上の関係（第319回）

Thank you for listening to Data Science at Home podcast with Francesco Gattoleta. You are about to get cutting-edge insights from the people who are reshaping the world of technology with machine learning, data science, and artificial intelligence. It's time for Data Science at Home. Welcome to the show. So here is a question. If you walked into a museum and every single painting had a little plaque that said, generated by computer. Would you spend the same amount of time looking? Would you feel the same thing? I think most people would say, yeah, of course, art is art. It's about the result. And then they walk out in, you know, half the time and never come back. And that tension between what people say they believe and what they actually do is exactly what a new study just confirmed about artificial intelligence in games, which is going to be the topic of the day.

The numbers are kind of brutal, so let me get into that. Welcome to Data Science at Home. I'm Frag, AI architect, former academic, and someone who's been building systems with and around AI long enough to know when the hype starts eating itself. Today, we are talking about gaming, AI, and a fascinating piece of research that basically puts a number on how much players punish developers for using AI tools. Spoiler alert, it's a lot. Like, really a lot. And the who gets punished part is even more interesting than the how much. So, let's just start. First of all, I want to start from Steam. You probably know it. It's the biggest PC game distribution platform on the planet, run by Valve. And since January 2024, if I'm not wrong, they require developers to disclose when they have used artificial intelligence or any form of AI during development.

So you'll see a little blurb on the store page that says something like AI was used to generate some voice acting or AI-assisted asset creation. That's probably the most common one. You know, these kind of things. Now, Steam doesn't publish sales numbers, okay? That's proprietary. Developers can share, of course, that information if they want, but most of them don't. So, the standard proxy in the industry is review counts. So, mainly if a game sells, you know, it gets played, and of course, if it gets played, some percentage of the players leave a review. Some of them are super excited, some others are really turned down, whatever. So reviews are a pretty reliable stand-in for sales volume, okay? It's an heuristic, it's not an exact match, but stick with me. Now, there is a researcher named Ross Barton, PhD, Head of Products and Data at Game Oracle,

took almost 10,000 Steam releases from January to October 2025 and studied them, okay? And what he found is absolutely amazing. Games that disclosed AI averaged 4 reviews in the first month after launch while games without AI disclosure averaged 7 reviews so nearly half In the zero review category games that nobody bothered to review at all it was 20% of AI games versus 15% for non-AI games So we see this kind of trends from both sides If you look at the titles that got enough reviews to have a meaningful score, AI games averaged 84.6 versus 88.3% for non-AI games. Now, those numbers are already pretty damning, but they are kind of the surface level, because here's where it gets even more interesting.

Burton built a causal statistical model. It doesn't give the details about that, or at least I couldn't find. I have a bit of a doubt about causal statistical models because causality is extremely hard to quantify but let keep on that okay So it controlled for developer experience publisher backing genre release timing you know, the stuff that actually explains why some games do better than others. And then he compared games with pretty much the same category against each other. And within those matched groups, what he did, well, Now, games with the high disclosure received 53% fewer reviews than games without. 53% is an enormous amount, if you think. So, if two games are otherwise identical, for example, they have the same type of developer, they have the same genre, same release window, same level of backing,

you know, very much similar categories, very much similar features. the one with the AI disclosure gets about half the engagement, 53%. It's enormous. So that's not a small signal. It's a very loud signal about what the players are actually thinking about. Let's have a quick break because this episode is brought to you by OutShift, Cisco's incubation engine. Today's AI agents operate in silos, limiting their true potential. We've been focused on building bigger, smarter models, but actually scaling up is just one approach. To reach superintelligence together, we need to do more. We need to scale out. And we actually have a blueprint from 70,000 years ago. Humans didn't just get smarter individually. The cognitive revolution transformed society because we began sharing knowledge, goals, and innovation. Agents are now at the same inflection point. They can connect, but they can't think together.

That's why OutShift by Cisco is building the Internet of Cognition, transforming AI from isolated systems into orchestrated superintelligence. By creating an open, interoperable infrastructure, OutShift is enabling agents and humans to share intent, context, and reasoning. The cognitive evolution for agents is here. Explore the Internet of Cognition at outshift.com. That's O-U-T-S-H-I-F-T dot com Now, here is the part that really caught my attention on his work Because it really flips the obvious narrative on its head You might assume that the small indie developer, you know, the solo creator The bedroom coder who has no money, of course, and no team and uses AI Because it's literally the only way they can make a game You might assume that's who's getting hammered, right?

Well, not exactly. Burton's sensitivity analysis shows that inexperienced developers with no marketing budget who use the AI saw almost no negative impact on their sales. Because those games were already going to kind of struggle, you know. And the AI disclosure isn't the thing that is killing them. It's more lack of resources, lack of audience, lack of polish that's already doing that job. So there is this kind of confounding factor, again, that sucks. It sucks with or without AI. Now, with all due respect, of course, to the solo developers, sucks is a bad word. It's just it lacks polish. Let me put it like that. Now, the developers getting destroyed by the AI stigma are in fact the established studios, the ones with experience, with an existing player base, with previous titles, with a reputation, with production budgets, of course, and with amazing teams.

For those developers, AI disclosure correlates with a 40 to 60% drop in sales. 40 to 60% is a catastrophe that's not a rounding error that's the difference between a game being an extreme success and the game being a financial disaster and the example cited in the research makes sense in this context so there is Black Ops 7 for example, Activision one of the biggest studios in the world faced real backlash when AI generated content was found in the game Then we have Jurassic World Evolution Tree Same deal Brands with history audiences with expectations and then the AI disclosure lands and the internet turns on them So contrast that with the finals and the suck up, which used AI and did well. Well, different contexts there, different audience relationship, different type of AI used. So the pattern isn't really AI equal

bad, the pattern is more like AI plus audience trust plus established expectations equal bad. The higher the expectations, the harder the fall. And honestly, that makes complete sense to me. And it leads directly to the thing that I actually want to talk about, which is art. Games are art and you cannot automate art. So let me go back to the museum question at the beginning of the episode, because I think it cuts right to the core of what's actually happening here imagine the Louvre you know, the French museum right? You walk in you stand in front of the Mona Lisa which by the way is Italian for whatever reason it's at the Louvre more on that in another episode there's a little plaque next to it and the plaque says, generated by an AI system trained on I don't know, Renaissance painting styles, rendered in approximately 0.3 seconds.

How do you feel? Generally, how do you feel? And I know the philosophical answer, you know, the experience of looking at a beautiful image doesn't change based on how it was made. Sure, philosophically, but psychologically, you feel robbed, right? Because you paid a ticket, you probably took a plane to go there and to something that was generated and rendered in 0.3 seconds. and because part of what makes standing in front of the Mona Lisa meaningful is knowing that a human being, Leonardo in this case in 1503 or whenever he actually finished the thing made a choice about every single brush stroke that the imperfections are intentional that the slight asymmetry of the smile is his asymmetry you are looking at a record of human decisions human struggles, human vision you remove that and you have a technically impressive image

you don't have the same thing games are the same actually games are more of this because games are an ongoing dialogue between the people who make them and the people who play them or will play them so when a player discovers a weird mechanic or finds a hidden detail in a level or hears a line of dialogue that feels weirdly specific to their life experience, well, there's this implicit understanding that, wow, someone put that there on purpose. Someone made a decision. Someone cared enough to craft that particular moment. When you disclose that AI generated the aspects, the voice acting, the level layouts, you're telling the player that the conversation is now partly with the machine. and the player often, you know, without quite being able to articulate it, feels that loss. Pay attention to what I'm saying. This is not anti-technology.

I build AI systems for a living and I think the tools are extraordinary. But there is a category difference between using AI as a tool, you know, the way a sculptor might use power tools, and the outsourcing the decisions to AI. The first is still art, the second is something else. I hope you agree with me. And I think that the 53% review gap of this research study is players not analytically, not consciously, not in a boycott, just voting with their attention. Feeling something different and moving on. So let me focus a bit on the interpretation problem. Burton is honest about one thing, which I honestly appreciate. It doesn't know exactly what's causing the penalty It raises the possibility that AI uses correlated with other decisions that lead or lead to a poorly crafted game

In other words maybe studios that cut corners with AI also cut corners everywhere else and maybe the AI disclosure is a symptom not the disease Maybe players are not detecting AI specifically They are more detecting a game that feels rushed, you know, underbaked, cynical about its audience. It could also be. I think that's partly true. And it's actually the most important point for developers to hear. If you use AI to compensate for not having enough time or not having enough people or not caring enough to get it right players will notice not because they'll spot the AI specifically but because games made under those conditions have a certain texture a smoothness that is also somehow emptiness dialogue that technically fits but doesn't quite you know, hit, doesn't punch art that's technically competent and somehow

anonymous on the other hand, if you're using AI the way a good writer uses the spell check to handle the mechanical parts faster, so you can spend more time on the parts that require judgment and taste and care, well then you're probably fine, the finals used AI for certain audio assets and did well because the game itself is good the human decisions are still there the tool isn't the problem Replacing human decision-making with a tool, that's the problem And those are very different things So, where does this leave us? Well, the numbers are pretty clear, at least from this one study AI disclosure hurts sales The hit lands hardest on developers with the most to lose You know, the established studios, the recognizable brands The games with genuine expectations attached And underneath the numbers is something that I think is actually obvious

Once you say it out loud Which is making games is a creative act And players know the difference between a game that was crafted And a game that was generated Not always consciously, not always articulately But the 53% gap in reviews is people feeling something and acting on it That's not a lobby campaign, that's not an organized boycott that's just humans responding to the presence or absence in that case of other humans in the thing they are experiencing so back to the museum one more time you know what would actually bring people back to a museum full of AI generated art well if the AI was a tool in the hands of a human artist who had something to say if the plaque said painted by an artist using AI assisted techniques well that's a different story because there's still a human there there's still someone whose choice is your witnessing that's the line and the studios that figure out where it is

will be fine the ones that try to replace the human will keep wondering why of course their review scores are tanking one last thing and this explains why I've been talking about artificial intelligence and video games a lot lately please check the previous episode if you haven't it's because I'm making one no AI I'm making a video game is going to be a 2D top-down RPG. I don't want to spoil more. There will definitely be no AI involved, zero. We are three super senior developers and lifelong gamers who collectively spend a pathetic amount of time leveling up and playing games, and we decided to build something. Of course, for the game, we have a story, a world as a top we are genuinely excited about. we are learning a ton of things along the way because it's something completely different from what we have been doing so far. And I'll be honest, I'm having a blast.

Now, what we need are nerds like us, people who can do sprites, pixel art animations, environments, all that stuff, tile sets, people who care about craft. If that sounds crazy to you, that's good. That's exactly the right feeling and it's exactly the right reason to reach out. In fact, this is Data Science at Home. Subscribe whenever you get your podcast. See you next week. You've been listening to Data Science at Home podcast. Be sure to subscribe on iTunes, Stitcher, or Podbean to get new fresh episodes. For more, please follow us on Instagram, Twitter, and Facebook, or visit our website at datascienceathome.com.