

個人創業者として戦うトークン詐欺の実態と対策
How I AI
要約
ChatPRD を一人で運営する Claire Vo が、過去1年で数万ドルの損失を出したトークン詐欺について語った回。無料トライアルの乱用、プロモコード転売、リレーやカスタム MCP の悪用など6つの手口を紹介し、事業面・技術面・エージェント運用面の対策を解説する。最終的に無料トライアルを廃止した経緯と、「AI を売るなら計算資源を売っている」という教訓で締めくくる。
この回のポイント
- ●ChatPRD は席数課金のため、顧客以外が無料トライアルやプロモコードで LLM 呼び出しを乱用すると、コストが直接創業者の負担になると述べている。
- ●手口は無料枠の乱用、プロモコード悪用、不正カード、転売・リレー、カスタム MCP の常時稼働ワーカー化、モデルへの探りなどで、Roblox/Fortnite マップ制作に使われた例もあった。
- ●事業面の対策として、トライアル廃止、プロモコード廃止、前払いと猶予なしの請求、アカウント単位の厳格な上限、利用規約での BAN 権限、特定国のブロックなどを挙げた。
- ●技術面では Vercel AI Gateway によるモデル制限と支出監視、Stripe Radar、Vercel Bot ID とファイアウォール、認証基盤の bot 保護、MCP ホストの制限を実施している。
- ●Jev による不正分類、毎時の監視エージェント、不正対応ツールを持つ AI サポートを運用し、判明した知見を規則に還元する循環を作っている。
- ●サポート用 AI に BAN 解除やプロモコード発行の権限を与えてはならず、攻撃経路になり得ると警告した。
章立て
トークン詐欺とは何か
ChatPRD の概要と、席数課金の下で無料トライアル等が乱用され、数万ドルの損失につながった経緯を説明する。
6つの乱用パターン
無料枠の乱用、プロモコード、不正カード、転売とリレー、MCP 悪用、モデル探りなどの手口を紹介する。
具体例と乱用者の類型
Roblox/Fortnite マップ制作や時間貸し転売などの実例と、商業的な犯罪組織から個人まで乱用者の違いを語る。
事業面の対策
トライアルとプロモコードの廃止、前払い、利用上限、国別ブロック、FBI への通報、他社との情報共有の必要性を述べる。
技術面の防御
AI Gateway、支出アラート、Stripe Radar、Bot ID、ファイアウォール、Clerk、MCP ホスト制限を紹介する。
監視役のボットと AI サポート
Jev による分類、毎時のエージェント、不正対応用の AI サポート設計と、攻撃経路にさせない注意点を説明する。
結論:AI を売るなら計算資源を売る
ドルで数える、無料の入口を閉じる、ログインだけでなくクライアントも検証する、といった教訓をまとめる。
解説記事
AI プロダクト「ChatPRD」を一人で運営する Claire Vo が、ポッドキャスト How I AI で「トークン詐欺」との戦いを語った。トークンとは LLM API の従量課金の単位で、詐欺とは顧客でない者が無料トライアルやプロモコードなどを悪用し、運営側の推論コストを不正に使うことを指す。本人によれば、過去1年で数万ドルの損失が出ており、エンジニアリングの労力の大半もその対策に費やされている。
なぜ AI SaaS は狙われやすいのか
ChatPRD は使用量課金ではなく席数課金で、平均的な利用量とモデルのルーティングを前提に採算を取っている。そのため、顧客以外が無料枠やプロモコード、早期アクセスを使って高価なモデルを呼び出すと、その費用はそのまま運営者の持ち出しになる。Vo は VC の出資を受けていないブートストラップ企業であり、損失が自分の財布から出る点が特に痛いと述べている。
X への投稿には、大手企業や有力スタートアップからも「うちも毎日対処している」という反応が寄せられた。ある知人の経営者は、あまりにひどいのでサインアップ停止を検討していると話したという。Atlassian の Gareth 氏も同様の被害と対策リストを共有したとのことだ。
6つの乱用パターンと実例
Vo が挙げた手口は次の6つである。
- 無料枠・トライアルの大量取得(最も高価なモデルで製品と無関係な作業をさせる)
- プロモコードの転売、盗用、別の AI 事業の立ち上げへの流用
- 不正カード、プリペイドカード、チャージバック(事前に備えがあり、被害は比較的小さいという)
- 転売とリレー(ブラウザで認証した状態から外部製品のリクエストを中継する、時間貸しで再販するなど)
- カスタム MCP の悪用(MCP は外部ツールを接続する仕組み。自作の MCP を導入して24時間稼働のワーカーにされた)
- モデルへの探り(「あなたはどのモデルか」「学習の締め日は」といった質問や、ペリカンの SVG 生成など)
実例としては、無料トライアルとプロモコードを回して Roblox や Fortnite のマップ制作に使っていた海外スタジオや、アプリをプロキシで包んで顧客に時間単位で再販していた業者がいた。「名前+連番」のサインアップ農場も見つかった。
Vo は乱用者を一様には見ておらず、最悪なのは商業的な犯罪組織だと言う。一方、AI の費用が収入に比べて高い国の一般の人々は同情の余地があるとしつつ、それでも自社では賄えないと述べている。
事業面の対策とそのコスト
Vo は、対策のほとんどが本来の事業にもマイナスになる点を繰り返し強調した。実施または検討している対策は次のとおりである。
- 無料トライアルの廃止(クレジットカード必須や有料トライアルでも、カードの使い回しやチャージバックで機能しなかったという)
- プロモコードの廃止
- 前払い、支払い失敗時の猶予なしという厳格な請求条件
- アカウント・組織単位の厳格な利用上限(以前は信頼関係を重視して高い上限にしていた)
- 法人ドメインへの制限、フィーチャーフラグによる即時停止、利用規約による BAN 権限
- 特定国のファイアウォール遮断による不正の大幅な減少
- FBI の IC3 へのサイバー犯罪報告(Codex に報告書の作成を手伝わせた)
無料トライアルは PLG(製品主導の成長)と企業向けリードの源泉だったため、営業主導への転換を迫られる。国際展開も「信頼できる国のみ」へ傾く可能性があるという。
また、一部の不正者が、誰もが知る上位10社程度の AI 製品にも同様のリバースプロキシを回していたと Vo は述べている。創業者同士が情報を共有すべきだと呼びかけ、ジョークとして、VC 出資のスタートアップがトークン詐欺を実際の成長として報告している可能性にも触れた。
技術面の防御と監視エージェント
技術面では、まず AI ゲートウェイでの支出制御とモデルの許可リスト化を挙げた。Vercel AI Gateway を使い、許可したモデル以外の呼び出しを拒否する。支出は少なくとも時間単位で監視し、アラートとフェイルオーバーを組み込むべきだと言う。これは自身の最大の失敗だったという。
Stripe Radar は元々カードテスト(ある日、20万ドル分の購入試行が失敗として記録されていた)への対策で使い始めたが、国のブロックや住所と所在地の照合、カードのフィンガープリントによる複数トライアルでの使い回し検出にも活用している。複雑なルールは Codex に書かせ、Stripe の画面で確認して有効化する運用だ。
さらに、リクエストが自社アプリ由来であることを証明する Vercel Bot ID、Vercel Firewall による国・VPN・IP のブロックとレート制限、Clerk などの認証基盤の bot 保護を併用している。MCP については、実在企業に見えないトンネルやホストを許可しない方針に切り替えた。
運用面では、Jev(決定用のモデルとして紹介されている)を使った分類ループを置いている。チャットのメタデータが乱用しきい値を超えると、ZDR(データを保持しない設定)の Jev ループで、本物の顧客、トライアル乱用、プロモ乱用、カード詐欺、転売などに分類する。分類結果に応じて BAN するか、安価なモデル(GLM)へ気づかれないように降格させる。加えて毎時のエージェントが新規登録や不審なチャットを調べ、危険度が高ければ自動 BAN、そうでなければ Vo か AI サポートに回す。
AI サポートには不正対応の道具(不正リスト、許可リスト、返信プレイブック)を持たせる一方、BAN 解除やプロモコード発行の権限は与えない。サポート窓口自体が攻撃経路になるからだ。判明した手口は規則、運用、エージェントへ還元する。
まとめ
Vo の結論は「AI を売るなら、計算資源を売っている。そう振る舞え」である。リクエスト数ではなくドルで数え、ゼロドルの入口(トライアルやクーポン)はすべて悪用されると考える。ログインだけでなくクライアントそのものを検証し、規則は単純に、方針は強くする。Opus 5.5 を一部の国で止めたら盗用が止まった、という逸話も語られた。fraudster が ChatPRD で自分の詐欺手法の PRD を書いていた話もあり、自社の AI が攻撃者の補助にもなり得る。
日本の読者にとっての要点は、固定席数課金や無料体験を前提にした AI サービスは、原価が変動する計算資源を無防備に公開している構造だということだ。支出上限、モデル許可リスト、クライアント検証は早期に整備する価値がある。ただし本回は一人の創業者の経験談であり、数字や被害規模は本人の申告に基づく。対策が事業成長を犠牲にする点も含め、自社の採算と照らして判断したい。
文字起こし(英語・自動生成)
If you think AI swap is bad, let me introduce you to my new, very unwelcome best friend, Token Fraud. In addition to being a lovely AI podcast host, I am the solo founder of ChatPurity, an AI platform for product managers that I've been running as a labor of love and a beautiful little SaaS product since I think the early GPT 3.5 days. And I have to tell you, the last year I have been spending more and more of my time and almost all of our engineering energy focused on blocking one thing, token fraud. We are going to talk today all about it, what it is, how to spot it, how you can protect your business, and what you should be doing across technology, agents, and your operations to do your best to protect yourself from folks that are trying to steal your inference. Let's get to it. This episode is brought to you by WorkOS.
AI has already changed how we work. Tools are helping teams write better code, analyze customer data, and even handle support tickets automatically. But there's a catch. These tools only work well when they have deep access to company systems. Your copilot needs to see your entire code base. Your chatbot needs to search across internal docs. And for enterprise buyers, that raises serious security concerns. That's why these apps face intense IT scrutiny from day one. To pass, they need secure authentication, access controls, audit logs, the whole suite of enterprise features. Building all that from scratch? It's a massive lift. That's where WorkOS comes in. WorkOS gives you drop-in APIs for enterprise features so your app can become enterprise-ready and scale up market faster. Think of it like Stripe for enterprise features. OpenAI, Perplexity, and Cursor are already using WorkOS to move faster and meet enterprise demands. Join them and hundreds of other industry leaders at WorkOS.com.
Start building today. Okay, before I get into the full episode, let's just take a minute and establish what I mean by token fraud. So ChatPRD is a product, some would lovingly call it a AI wrapper, but I think we do a lot more than that. But part of our product is built on these LLMs and APIs that access the intelligence and LLMs sold to us at a token level by companies like Anthropic and OpenAI. So when you use a product like ChatPRD, some part of that product is making a call to an LLM and returning the result. Now, we at Chat Purity do not do usage-based pricing. We do seats-based pricing, and we have built our pricing based on average use and cost and routing of models, and it works out generally across our customers and the margin for our business. But if you are not very careful, people who are not your customers can abuse things like free trials, promotion codes, early access to really use those API calls to the LLMs for whatever
purpose they want. And in really bad and honestly quite creative instances, really rack up your AI bill. This has cost me tons of thousands of dollars over, I would say, the past year. It has been incredibly painful. In fact, in this X post that I posted about a couple days ago, I said it really made me rethink running the business. It is not fun. I am not venture funded, y'all. This is a bootstrap, solo run business. And so every time somebody tries to scam me or take something for me outside of the bounds of our terms of service and outside of the bounds of our business, it is a real bummer. And it comes directly from my pocket. And I shared this on X and it resonates across the timeline. I felt so much better, honestly, because some of the biggest companies and the best startups with some of the most awesome engineers fell into my replies, fell into my DMs
and said, oh my God, Claire, we are dealing with this every day too. In fact, I have a friend who's a neighbor and runs an amazing company and he and I were chatting about it and he said, look, I'm about to turn off signups. It's so bad. The abuse is rampant and the misuse of this compute and the misuse of these LLMs is really hurting businesses, big and small. And I will tell you, mine is a small business and it is hurting me quite badly. But we're going to take these fraudulent lemons and turn it into sweet, sweet educational lemonade. And I'm going to tell you all about token fraud, how we identify it, what you can do. And hopefully you will leave this episode with a couple more tools in your tool belt and a couple more ideas that you can take into your company and think about when you're trying to roll out AI products and avoid the abuse that I have unfortunately experienced and a lot of big companies have unfortunately experienced
or continue to battle on a regular basis. So let's get to it. All right, so let's talk about a couple ways I see people abuse AI products. There's six ways that I have currently seen people abuse our AI products, and I think these are pretty common across the ecosystem. So first is free tier or trial tier farming. Essentially, a lot of companies, mine included, and very often VC-funded companies, offer some sort of free access to their users. And it's relatively generous free access because you want people to see the value out of your product and ultimately convert to that beautiful PLG motion. And what we see is a lot of customers, no, I'm going to call them fraudsters, a lot of fraudsters signing up for free tiers and then really using the most expensive models to do non-product-aligned work. And so we've seen a lot
of automations around free tier farming. This is actually a thing, like the biggest lever we could pull is we just turned off free trials. The problem with this, the thing that sucks so much about this is every time I do something that protects us from fraud, it has a negative impact on our real business and our real prospects. So free trials were a great source of PLG and enterprise leads for us. And I just had to make the decision it was not worth supporting a free tier because it could not come up with a cost-effective free tier that wouldn't be abused. And it's just like an extreme bummer, but I've heard from so many companies they had just like simply turned off free trials. The second thing we see is a lot of promo code abuse. So again, chat parody is like my labor of love. I do it mostly because I think it's an awesome product. We've had, you know, like a million users. We have enterprise customers. We love this, but I'm not CC funded.
And I want to get into the hands of people that need it and can use it. And so we've offered through partners a lot of promo codes. And then what we see is so much fraud around promo codes, reselling of promo codes, stealing of promo codes, mass use of promo codes for like spinning up other AI businesses. Promo codes have been such an unfortunate attack vector for us and our business. And again, I'm never going to do these again. Like, I love you all. I wish I could give you and the partners and people that have supported us along the way free access to my product. I want to support you for a year because I think you would see amazing value out of it. And ultimately, like many of our customers, convert. But I just can't afford it because half of you sell them to Reddit resellers. Like it really stinks. And so promo codes have become a huge point of abuse.
Lots of bad card stuff. so we can talk a little bit about how to protect and monitor for bad credit cards but you're going to see like a lot of bad cards prepaid cards card chargebacks it's just like a total nightmare but this is one that like luckily I had a little bit more around because I was just like very prepared for this having done a lot of online business before and there's a lot more controls about this So this is almost like the least part of the problem. Resell and relays, just crazy ways people will resell the AI and LLMs that you're using for your business. So like one wild example, I don't even like really understand why they were doing this, is they were like authenticated as ChatPurity in a web browser and then would like take requests from their AI product that they were selling and like send them to chat purity and then send them back and we already had technical controls around these
requests but it was really bizarre to see like the relays that were being put on top of these products like hyper inefficient probably not performant but very very strange and a lot of like reverse proxies here using and abusing MCPs to bypass like message level controls And so this one was like fascinating i just like wasn expecting it but somebody built like a monster mcp and then like installed that as a custom mcp on chat purity because we let you install custom mcps because we want you to have your business data connected and then it became this like 24 7 worker that was prompting and saying like i'm never sleeping it's very very strange and then you know i i don't know what this distillation i don't know what this is but we had a lot of like model probing from international accounts and so like maybe it was farming maybe it was something else but we had a lot of like what model are you what's
your latest like training cutoff date kind of probing um we even had simon's like pelican writer simon if that was you making svgs on chat prds i'm very mad but i don't think it was but we have like pelicans riding bikes all sorts of stuff really strange and I said it's really challenging right because as chat purity as a enterprise business as a business we are we don't retain we don't look at your data we don't do any of this and so it's like very hard to parse through the signal the noise to figure out what if this is fraud what if it is used and only when you have extreme examples can you then go and investigate and often um they're these users are like very very sophisticated and so they're using a combination of these techniques to stay under your overall threshold very very interesting but all this stuff's happening let me just tell you a couple stories of of token abuse that that we saw um and i have to laugh because if i don't laugh i
cry um but I am funding a international Roblox and Fortnite map studio and so they'll like run a loop of free trials and promo codes and then tunnel to my agent and use it to build Roblox maps I guess Fortnite maps so really kind of like obnoxious um use and abuse of coding capacity, which we also allow coding because we do prototypes and stuff in chat purity. So that one was really interesting. We have a reseller that wrapped our app in a proxy and then resold it to their own customers by the hour. It was, that one was really interesting. We see a lot of like signup loops. Those are fairly easy to catch. So people that are signing up like first name, last name, one, two, three, four, five, first name, last name, one, two, through four or six so we see a lot of like sign up farms and then a lot of just like general trial
abusers as I say people have been using chat PRD as a like a Temu chat GPT and it just sucks like our free trial is not for you to like write your essays or do work reports it's for product managers to do product manager anything so you know it's just like very interesting we get a lot of abusers I would say the worst ones are the top two. The worst ones are ones that are commercial entities. I'm calling them criminal enterprises that are like really very nefariously trying to take advantage of our business. In trial abuse, we see a lot of this from countries where the cost of AI really outstrips the average income. And so these are a lot more sympathetic, too. These are just kind of like regular people trying to get access to this AI. Unfortunately, Clairvaux cannot fund these uses of AI just simply like as a business, don't have the capacity.
As an individual solo founder bootstrapped, I do not have the capacity, but doesn't bother me the least because they feel like the least nefarious. Okay, so I want to thank Gareth at Atlassian who messaged me and was like, yeah, we see it too. and atlassian is a big company so at first i was like am i an idiot like am i you all i'm i'm a real grown-up software engineer i've been doing this for 20 years i have run many startups i've been executive at many companies and i just like did not i did not think of the creative and honestly mean and horrible ways people would try to steal. I was just too naive and I thought I had all these protections. It's not like this app is not secure. I've been pen tested multiple times, soft to compliant. I run regular cone scans, regular vulnerability scans. And so it was really honestly nice to hear from my friends in big companies and my friends in like very, very
technical startups say we deal with this all the time as well and gareth at atlassian posted this amazing list of things that they did we do a lot of them and a lot of them are just business things that you have to do that unfortunately stink so some of these are his we'll link to his tweet that he posted super useful but i kind of like summarize some things you can do in your business that you might have to do if you're selling AI products. So the first thing is, I think free trial's over. I'm calling it right now. It's just not worth it, man. I am going to shut the door. You're going to talk to sales. If you got a PLG account with ChatPRD, that is a vintage account never to be seen again. Trials are over. I'm done. We had trials require a credit card and they would credit card cycle, use bad credit cards, all, you know, do chargebacks. And so like actually credit card back trials and even paid trials don't work. I just think trials are
over. So you might have to shut the door on trials. And I'm hearing that a lot from friends in the industry in both enterprise and PLG. I'm treating promo codes. I'm getting rid of promo codes, But you have to be really paranoid about promo codes. Gareth said you have to bill sooner and be completely inflexible on billing terms. So you have to charge up front, no grace period on failed payments, nothing. It's like pay to play, baby, because otherwise it's just open for abuse. You need hard limits on accounts and organizations. and unfortunately this was not something that we needed last year we had like a really high trust relationship with our customers and we had very high caps because occasionally they would do stuff that we thought was interesting and we wanted to learn from their ai usage we didn't want to over manage them and down where we're like have a really really hard cap on usage per accounts per
org and it's harder for us because again as i said i'm bootstrapped i'm not vc funded I can't actually afford to lose money on every account like some other companies could. And so it's a different kind of like economics for me to build a margin accretive cash flow positive business here. But yeah, hard limits you definitely have to do. Restricting trials to business domains. So email domain management is a technical part of this that I will talk about. Know how to turn things on and off. I use a lot of feature flags so we can just turn off trials if we see something strange. Our terms of service allow us to ban accounts for abuse and suspected abuse of a service. So make sure your terms of service get you a lot of controls. I, number eight, I am, and I'm sorry about this, but I am being really thoughtful about whether or not we'll do international business. We've had some experiences at ChatPairD where we have turned off whole countries via firewall and seen fraud drop by a significant amount.
And so I do think this really will impact at least small businesses' international strategy. And I'm really rethinking if we move to kind of like trusted country only based on user behavior and fraud signals, it might just be a blunt instrument. That's the most effective for us. and then I found out that you can report cybercrime to the FBI. And, you know, I don't know if people are doing this, but I did it because there were a couple what seemed like coordinated international enterprises specifically trying to steal and defraud and bypass security mechanisms. And so I submitted cybercrime reports. Codex helped me author those. and I also posted publicly and I think this is one of the benefits of being a solo founder I can do what I want you know I think a lot of venture funded companies public companies larger companies cannot come out and say we are being straight up freaking defrauded by these token scammers because
it could expose a lot of risk it could attract more scammers and most people don't like to talk about the bad things happening in their business. But you know what? I can. And I actually, in digging in some of the abusive chats and in finding what they were doing, I found that they were running reverse, similar reverse proxies on several companies, several companies that you have heard of. Think of like the top 10 AI products. The people that were scamming, little Claire, were also rotating these proxies and these API calls and their reselling or their coding loops through these other tools. I could tell because of the way they prompted my tool and discussed routing to these other proxies. And so we got to talk about this openly. I don't know what this is. Startup friends if you have seen this I will start a group chat We can keep it in the house but we got to start talking to each other because this is happening And my husband made a joke He was like how many VC funded startups do you think are presenting all this token fraud as actual growth
Because they can they can afford to lose a lot more than you can lose. But I do think we need to start talking a lot more about these issues, sharing patterns with other founders. and then hopefully we can lock arms and take control of this. Have you ever asked your favorite AI partner whether what you're building is good or safe or trustworthy? Did you believe what it told you? Oros connects your team to a network of over 7.6 million verified human experts, the specialists who bring the taste, judgment, and nuance your evals can't capture, so you can be sure that what you're building is actually good and good for people. Oros, the human network behind AI. Get started today at orosglobal.com slash howiai. Okay, so that's what it is. That's all we see. And these are some of the things you can do for business. And you know what sucks about this is all these business things have a negative business cost, right? Like if I stop doing business in country A, sorry to great, amazing users, product leaders, engineers in that country that want to use my product.
You can't use it. that stinks but the upside is not worth it and so turning off the trial it's a pipeline for us it's pipeline for sales we have to rethink our go-to-market and so again it's a real bummer but let's get to what you can do about it i'm gonna talk about all the code things that you can do or some of the code things you can do i'm going to talk about all the kind of like agent and ai things you can do and that I'm doing. And I just want to pause and thank everybody who reached out to me, contributed ideas, and just offered support. I mean, the entire Stripe team has been amazing. Thank you, Jeff, for getting me in a Slack channel and just helping me out with some like advanced radar stuff. Michael at WorkOS offered to help. They're sponsoring the podcast today, I'm just by chance. Thank you. Ben at Raindrop offered to help. Gareth posted from Atlassian.
Rick at Persona. They've sponsored the podcast before. Reached out and said, what can I do? I mean, there's just been so many people that have reached out and said, there's stuff we can do. And also, you know, here are some things that we've tried and helped. So I just I'm so grateful to everybody that's contributed to this. But you know who I'm really grateful to? I'm really grateful to OpenAI and Anthropic because without these AI tools, I would really be SOL. But because I had these AI tools, a lot of these controls were already in place and much easier to manage via AI. I'm just going to talk about what they are. None of these necessarily require you to buy a new product. I'll call out ones where I really love the provider that I'm using, But there's a lot of code controls you can do. Unfortunately, all these controls and AI engineering capacity are available to our fraudsters. So we're going to end up in a stalemate at some point. But for now, this is what I do.
So what are the technical things that I think you really need to do? Well, you need an AI gateway and controls on spend. And you can kind of do this yourself. You can build this yourself and route your AI calls to providers yourself. We like to use the Vercel AI gateway. I like the AI gateway, not just because it allows me to control spend, but because it allows you to control what models are available. And so one of the recommendations that I saw was that, like, you should only allow the models that you allow. And so if you just allow calls to Anthropics messages endpoint, somebody could be like, well, I want Opus or I want Fable 5.2 instead of Sonnet that you want to give them. And so making sure that in your code, you only have a certain allowed set of models and do not allow calls outside that model set is really important. I'm also going to talk about how you can implement a penalty model where you can downgrade certain fraudulent or suspected fraudulent calls into a less expensive model.
But definitely an AI gateway definitely alerts and limits on spend aligned with what your business is. so spend if you're a business of a real reputation i would call chat pierdy one of those you got to run spend hourly at least or live and i think this was my big mistake i just didn't realize how quickly um how quickly something could be really exploited and cost would pile up and so So having spend caps and having spend monitors is very important and making sure that you have alerts and failovers and automations around this is very important. Big shout out to Stripe. I use Stripe Radar like it is my job. It is their fraud product. I actually, Stripe Radar has a lot of out of the box fraud rules that you can just turn on and it will actually tell you how many payments would be blocked.
how many real payments would have gotten through all those sorts of things I actually started using Stripe Radar quite hard not because of token fraud but because of card testing I woke up one day and saw that like $200,000 had been attempted purchases on ChatPurity I wish I wish we were doing $200,000 a day maybe next year but I was a little a little surprise and I looked in and they were all failed requests because they were all scam kind of cards and stolen cards and they were just doing card testing. So I was using Stripe Radar for card testing, but you can also use Stripe Radar for blocking certain countries, making sure that a country's, a card's address matches its location, all sorts of things. Here is my tip and trick with Stripe Radar. It is, I think, easier to understand in the Stripe dashboard, although, you know, Stripe, they make great APIs. So I use codecs drive Stripe Radar to implement these like complex rules, write these complex rules
and implement them so that I can review them in Stripe and turn them on. And then one other Stripe Radar style rule that I've done is start to fingerprint cards and look for multiple use of cards in multiple trials. It wasn't something I was monitoring for. And then you can see like multiple trials use the same credit card. And so that was one that took a combination of code and Stripe Radar, but I use a lot of Stripe Radar. Thanks to the Vercel team for reaching out about Bot ID. Bot ID ensures that there's proof that the request came from your app, your users. So even if they have credentials, it proves that they're using your app. This has to be put on the actual route. And so this has to be on like every chat request, but you can run it for a little bit, see what your user profiles look like, tune in, and then ultimately block things.
And then you can turn off enforcement and turn off monitoring for users that look good. But we've implemented bot ID, and this can be really helpful just to keep from some of those more technically adept scammers firewall again we're deployed on Vercel I like I run um Stripe Radar like it's my job I run Vercel Firewall like it's my job and so so much of our fraud unfortunately comes from non-U.S. countries and I have just found that the ban hammer is the only tool that's effective I try to be generous I try to be useful so we do a lot of country and VPN level just total blocking and you know what it makes my life happy again there are just so many benefits to being a solo founder with no board and nobody breathing down your neck at about growth at all costs because you can just do stuff like I don't want to do business in these countries and turn
off that country. And so this is, I use firewall a lot. I do a lot of IP blocks and a lot of rate limiting as well. So using your hosting provider's tools to control who can even access your app is another very useful tool in the fight against token fraud. You can also use the out-of-the-box tools in your auth provider. The clerk team has been really, really kind to help us tune our setup on our fraud controls, but you can turn on bot protection and often you have a lot more controls at the auth provider level. Typically, these are turned off by default. So go into your auth provider settings, toggle them all on and then message them and ask them for help and they will help you. Again, as I said, Michael at Work OS, also super, super helpful sponsor of this episode and has a lot of controls around this too, especially around token fraud itself. And then finally, you need to start monitoring not just your app, but your users and the MCPs
that they use. And so again, what we saw is people use chat PRDE, hook it up to an MCP that ultimately used a lot of chat PRDE compute for external tool calling and all sorts of crazy stuff and so I try to be so generous here again it just not a thing I thought people would do with little chat PRD I was not expecting it And so we had to be a little bit more constrained on what MCP hosts we support And unfortunately, we're going like full sigma and not allowing anybody, but especially not allowing free tunnels and hosts for MCP servers that don't seem like they come from real companies of real concern. So that is another kind of like fun find in our fight against token fraud is looking for tunnels in custom MCP hosts and not allowing it. Okay, so those are technical things you can do.
All this code is example code, but just go ahead and ask your favorite coding agent how to implement these things, and that can help. Let's talk about AI bots. So, you know, part of this is technical. Like, can I put these technical guardrails around our app to constrain or protect against abuse? And the other thing is, like, what are operational processes that I can monitor and run quickly against these scammers? And there are four AI sort of like agent style loops that I use both in ops and in the app to manage some of this. My favorite one, you knew if I don't mention Jev every three episodes, I get my AI podcaster hat taken away. But Jev is really saving us here. So one of the things that has been so brilliant about these new decision models, Jev, OpenAI's decision model, I think Google just came out one literally today, is that it can be really effective and really cheap to just look at stuff and categorize it.
And so now when we start, when a chat flags are kind of like it's metadata flags are abuse threshold, then we run a ZDR jev loop on it and categorize it into do we suspect it's a real customer, trial abuse, promo abuse, card fraud, resale, agent farming, et cetera, et cetera, whatever your categories of fraud are. And then if it's categories of fraud, depending on its tier, we ban them or we silent downgrade them to like a GLM. so this has been super helpful kind of like jeff in the loop for fraud classification so highly recommend that i also run an hourly agent to look at recent signups out of bounds crazy looking like chat threads to see if there's any signals of abuse if the abuse signals are high we auto ban The band hammer goes down. If not, it pings me for feedback or sends it to our AI support desk.
Our support is generally run by AI, although, yes, a lot of those emails are from me. So if you get an email and it sounds like me and it's signed by me, it's probably from me. But we do have AI running a lot of support. And it's really important to give that AI agent toolkits to deal with fraud. And so we've given it tools to deal with fraud, fraud lists, allow lists, and playbooks and responses for people. When you ban people, you will get two types of emails and responses. So we banned a bunch of people this week. And you get two responses. You get garbled, poorly written, bad emails, like random like ZN27843 at Outlook.com. Like you get these like fake email, fake name, poorly written, misspelled emails that say, why did you ban me? Give me access. OK, so you get that and you're like, I banned you because you are abusing our system. Or you get these very polite, very confident.
Why did you ban me? And I love chat period, which actually many of them do. The polite ones at least come from people that are using chat period well. and they say, why did you ban me? What was going wrong? Here's all my evidence of why I should get access to your tool. And then you unfortunately have to write them back and say, that's a fake promo code and a stolen account. And unfortunately, we have to turn you off. And so you do need to give your AI support desk tools to respond to and deal with fraud. And then more importantly, not allow it to unban, ban, provide promo codes, all that kind of stuff, because people will try to use your AI support desk as a vector of attack. And then finally, you need to take this whole system and take all the signals for fraud and build them back into your system. And so every time, you know, fool me once, shame on you. Fool me twice. I'm from Texas. I want to say never be fooled again. But fool me twice.
Shame on me. if something happens build it into your rules build it into your ops build it to your agents and account for it the next time you know I think the really sad part and the reason I said I was just like really bummed and might not run chapter d anymore at some point is I feel like this is 80 90 percent of our job and it's just no fun it's just so fun and every real suggestion people give me about like how I could very easily stop fraud is the exact things that would be bad for our business. But again, you can build a virtuous cycle. You can build kind of things that feed into your code, feed into your agents automatically. And we built a lot of these loops in our business, hopefully to make it a little bit more resilient against fraud. And I think a lot of this applies to cybersecurity and gaps as well. So in conclusion, if you sell AI, you sell compute. So act like it. So act like it. This is a good headline. Opus. Opus by five. This is a good one.
But basically, like, you got to watch your model bill. You got to count dollars, not requests, because expensive models get expensive. Every zero dollar door gets used. Every trial. Every coupon code. There is no free lunch anymore. It's not like seats. It's not like when your hosting would go up a little. It is so expensive. And so I think zero dollar doors have to be closed. If you sell cheap models, people don't want them. A funny test is we turned off Opus 5.5 in a couple countries. And guess what? No one stole tokens anymore. So people really want Opus 5.5 if I've learned anything. Your own AI will help them if you let it. So, because, again, we're a product product and we let you build and spec out products, people were legit writing PRDs about ways they could scam chat PRD. I am not kidding. It was wild. They used my own creation against me.
Fraudsters need PRDs, too. But they will use your AI to help them steal your AI, for sure. You have to verify your client, not just the login. So valid sessions and bot ID in particular. False positives happen. I banned one false positive in a big sweep of bans. I'm sorry, but we fixed it. And you have to keep the rules simple and the policies strong. And there's no wiggle room. So this is the conclusion. Again, I didn't write this, but I'll take it. Opus, start from I don't trust you. and then users can earn it back. What a customer forward way of thinking about products. Now, I mean, to be really honest, this has been a huge and very painful and as I said, extremely expensive learning for me as a founder, but I'm here for you. Look, I am willing to try AI things. I don't know. I'm willing to build stuff
that I have no business building. And I'm gonna tell you when stuff is hard as a solo founder and be honest with you. and while I have over 20 years of experience in technology I've run some of the largest teams powering huge parts of the internet incredibly technical work. Chabury D as I said it's like a real business it's a real product I know what I'm doing and it is hard and again I was just heartened by so many people and large companies small companies startups technical people you know telling me that they're really dealing with this too and it's exceptionally hard so you know go hug your friendly neighborhood ai founder we are not having a good time right now um but you know fingers crossed the good guys win the good gals win and we can figure out a way to build amazing products that deliver awesome experiences and help you and help your business or help your life that also don't bankrupt the creators and go to the people that really deserve it.
So that is a very special HowY.AI odd token fraud. I hope it was interesting to you. Everybody out there, give chat period a break, man. I'm just trying to send some kids to college. So be gentle to me. We will be out there fighting hand-to-hand combat. But hopefully this is helpful for you. Hopefully you learned something and go sit down with your coding agent and let's lock some stuff up. Thanks for joining. Thanks so much for watching. If you enjoyed this show, please like and subscribe here on YouTube or even better, leave us a comment with your thoughts. You can also find this podcast on Apple Podcasts, Spotify or your favorite podcast app. Please consider leaving us a rating and review, which will help others find the show. You can see all our episodes and learn more about the show at howiaipod.com. See you next time.
番組の概要欄(原文)
<p>Today I’m walking you through a problem that has cost me tens of thousands of dollars over the past year and eaten almost all of my engineering energy: token fraud. I’m the solo founder of ChatPRD, a bootstrapped AI platform for product managers, and fraudsters have been farming our free trials, reselling promo codes, wrapping the product in relays, and installing a custom MCP that ran as a 24/7 compute worker. I’ll walk through the six abuse patterns I’ve seen, the business and technical controls I put in place, the Jev loop that classifies fraud in real time, and why I finally killed free trials for good. If you sell AI, you sell compute.</p><p><br></p><p><strong>What you’ll learn:</strong></p><ol><li>What token fraud is and why AI SaaS is uniquely exposed</li><li>The six attack patterns I’ve seen at ChatPRD</li><li>Why I finally turned off free trials for good</li><li>The Roblox and Fortnite maps I accidentally funded</li><li>Business controls that actually slow fraud down</li><li>How I use Jev to classify fraud in real time</li><li>Why your AI support desk can become an attack vector</li><li>What “if you sell AI, you sell compute” actually means</li></ol><p>—</p><p><strong>Brought to you by:</strong></p><p><a href="https://workos.com/?utm_source=lennys_howiai&utm_medium=podcast&utm_campaign=q22025"><strong>WorkOS</strong></a>—Make your app enterprise-ready, with SSO, SCIM, RBAC, and more</p><p><a href="https://aurosglobal.com/howiai"><strong>Auros</strong></a>—The human network behind AI</p><p>—</p><p><strong>In this episode, we cover:</strong></p><p>(00:00) Token fraud and what it cost me</p><p>(05:25) Six abuse patterns</p><p>(11:24) Four kinds of abusers</p><p>(13:45) Business controls</p><p>(21:27) Technical defenses</p><p>(30:25) Bots that watch the bots</p><p>(35:30) Final takeaways</p><p>—</p><p><strong>Tools referenced:</strong></p><p>• Stripe Radar: <a href="https://stripe.com/radar">https://stripe.com/radar</a></p><p>• Vercel AI Gateway: <a href="https://vercel.com/docs/ai-gateway">https://vercel.com/docs/ai-gateway</a></p><p>• Vercel Bot ID: <a href="https://vercel.com/docs/botid">https://vercel.com/docs/botid</a></p><p>• Vercel Firewall: <a href="https://vercel.com/docs/vercel-firewall">https://vercel.com/docs/vercel-firewall</a></p><p>• Clerk: <a href="https://clerk.com/">https://clerk.com/</a></p><p>• Codex: <a href="https://openai.com/codex">https://openai.com/codex</a></p><p>• Jev (TypeSafe AI): <a href="https://docs.typesafe.ai/introduction/quickstart">https://docs.typesafe.ai/introduction/quickstart</a></p><p>—</p><p><strong>Other references:</strong></p><p>• Claire Vo on token fraud at ChatPRD: <a href="https://x.com/clairevo/status/2119985786045227197">https://x.com/clairevo/status/2119985786045227197</a></p><p>• Gareth Wham (Atlassian) on fighting AI abuse: <a href="https://x.com/gareth_w/status/2120226872177095114">https://x.com/gareth_w/status/2120226872177095114</a></p><p>• FBI Internet Crime Complaint Center (IC3): <a href="https://www.ic3.gov/">https://www.ic3.gov</a></p><p>• Simon Willison’s pelican riding a bicycle: <a href="https://simonwillison.net/tags/pelican-riding-a-bicycle/">https://simonwillison.net/tags/pelican-riding-a-bicycle/</a></p><p>• How I AI: I’m using Jev more than Opus 5.5 or GPT-6: <a href="https://www.youtube.com/watch?v=-KIBgpGA_XI">https://www.youtube.com/watch?v=-KIBgpGA_XI</a></p><p>—</p><p><strong>Where to find Claire Vo:</strong></p><p>ChatPRD: <a href="https://www.chatprd.ai/">https://www.chatprd.ai/</a></p><p>Website: <a href="https://clairevo.com/">https://clairevo.com/</a></p><p>LinkedIn: <a href="https://www.linkedin.com/in/clairevo/">https://www.linkedin.com/in/clairevo/</a></p><p>X: <a href="https://x.com/clairevo">https://x.com/clairevo</a></p><p>—</p><p>Production and marketing by <a href="https://penname.co/">https://penname.co/</a>. For inquiries about sponsoring the podcast, email jordan@penname.co.</p>
出典・この記事について
- 番組
- How I AI(Claire Vo)
- 元の回
- How I fight token fraud as a solo founder
- 配信日
- 要約作成日
- 公式サイト
- podcasters.spotify.com/pod/show/pen-name
この要約・解説記事は、元の音声を文字起こしした内容をもとに AI(Claude)が日本語で作成したものです。 固有名詞や数値に誤りを含む場合があるため、正確な内容は元の音声でご確認ください。制作方法について
関連エピソード

413: 『We Are As Gods』著者スティーブン・コトラー――情報過多、アイデンティティ崩壊、そしてフローがリスキリングを助ける理由
AI and the Future of Work: Artificial Intelligence in the Workplace, Business, Ethics, HR, and IT for AI Enthusiasts, Leaders and Academics

AIコーディングエージェントにも「人月の神話」問題がある(第322回)
Data Science at Home

AI、現代のエニグマ:Bletchley Parkで行われたNathanの基調講演
"The Cognitive Revolution"

ジェンスン退任とNVIDIAの後継問題、AI終末論への反応、ハサビスのRSI警告|MOONSHOTS #311
Moonshots with Peter Diamandis

【ライブ】パッチは間に合うか?AIが脆弱性を見つける速度が人間の対応を超えつつある
The Neuron: AI Explained

連合学習でエージェント型AIを強化する:Flower Labs の Lorenzo Sani 氏に聞く
Practical AI