He May Have Found AI's FEELINGS — Richard Ren, Center for AI Safety Researcher

Doom Debates!1時間8分
He May Have Found AI's FEELINGS — Richard Ren, Center for AI Safety Researcher

He May Have Found AI's FEELINGS — Richard Ren, Center for AI Safety Researcher

Doom Debates!

0:001:08:05
このエピソードはアーカイブのため、日本語要約の対象外です。
番組の概要欄(原文)

Richard Ren is a research engineer at the Center for AI Safety who graduated summa cum laude from the University of Pennsylvania. His new paper on AI wellbeing makes the bold claim that today’s AIs have measurable, human-like feelings, which the paper calls “functional wellbeing.” They act happy when they succeed and sad when they’re berated. We cover how you measure a language model’s happiness, the “AI drugs” his lab concocted, and why I count the findings as a Yudkowskian victory. Then we debate what it all means for AI consciousness, and whether AI is a moral patient. If we can’t rule out that AI models suffer, what do we owe them? Richard is careful never to claim the models are conscious, and he puts his P(Doom) at 50–65%, right alongside my 50%. The real disagreement is foxes vs. hedgehogs: he takes the data as it comes, while I say Yudkowsky’s theory called it twenty years ago. Enjoy the ride. Watch on YouTube: https://www.youtube.com/watch?v=1glFImnyp6o Timestamps 00:00:00 — Cold Open 00:01:12 — Introducing Richard Ren 00:02:24 — What’s Your P(Doom)?™ 00:03:31 — From AI Skeptic to Safety Researcher 00:08:16 — Why Care About AI Wellbeing? 00:11:45 — AIs Have Coherent Utility Functions 00:17:33 — Persona Selection 00:19:56 — Foxes vs. Hedgehogs 00:24:28 — How Coherent Are AI Preferences? 00:27:42 — From Preferences to Wellbeing 00:31:16 — Can You Trust an AI’s Self-Report? 00:36:42 — What Makes AI Happy and Sad 00:39:19 — The Liberal, College-Educated Persona Hypothesis 00:44:29 — Debating Where AI Experiences Qualia 00:52:06 — On Substrate Independence 00:56:28 — Janus, Dysphorics, and Mind Crime 00:58:58 — Which Images Make AIs Happiest? 00:59:38 — Creating an AI Drug 01:04:01 — AI Drugs Are Yudkowsky’s Paperclips 01:05:47 — The Missing Mood 01:06:50 — Does Richard Support Pausing AI? Links Richard Ren on X (@notRichardRen) — https://x.com/notRichardRen Collision — Richard Ren's Substack — https://richardren.substack.com/ "AI Wellbeing: Measuring and Improving the Functional Pleasure and Pain of AIs" — Richard Ren, Kunyang Li, Mantas Mazeika et al. (CAIS, 2026) — https://www.ai-wellbeing.org/ "Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs" — Mantas Mazeika et al. (CAIS, 2025) — the coherent-preferences paper this work builds on — https://www.emergent-values.ai/ "The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems" — Richard Ren et al. (2025) — the AI honesty benchmark — https://www.mask-benchmark.ai/ "Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?" — Richard Ren et al. (NeurIPS 2024) — the meta-analysis of AI safety benchmarks — https://arxiv.org/abs/2407.21792 "Representation Engineering: A Top-Down Approach to AI Transparency" — Andy Zou et al. (2023) — Richard's first CAIS collaboration — https://arxiv.org/abs/2310.01405 Center for AI Safety — https://safe.ai/ Center for AI Safety — careers — https://safe.ai/careers Statement on AI Risk (Center for AI Safety, May 2023) — organized by Dan Hendrycks — https://safe.ai/work/statement-on-ai-extinction-risk The 2026 Singapore Consensus on Global AI Safety Research Priorities — https://aisafetypriorities.org/ UK AI Security Institute (formerly the AI Safety Institute) — https://www.aisi.gov.uk/ Sora — the OpenAI video model that blew up Richard's 30-to-50-year timeline three months after he wrote it down — https://en.wikipedia.org/wiki/Sora_(text-to-video_model) Google Gemini calls itself "a disgrace to my species" (Ars Technica, Aug 2025) — the self-deleting-AI anecdote — https://arstechnica.com/ai/2025/08/google-gemini-struggles-to-write-code-calls-itself-a-disgrace-to-my-species/ Coherent decisions imply consistent utilities — Eliezer Yudkowsky (the coherence-theorems argument) — https://www.lesswrong.com/posts/RQpNHSiWaXTvDxt6R/coherent-decisions-imply-consistent-utilities Simulators — Janus's essay on LLMs as persona simulators — https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators Janus (@repligate) on X — https://x.com/repligate The Hedgehog and the Fox — Isaiah Berlin's original essay — https://en.wikipedia.org/wiki/The_Hedgehog_and_the_Fox AI 2027 — https://ai-2027.com/ Magnifica Humanitas — Pope Leo XIV's encyclical on AI (May 2026), the "AIs are not conscious" position — https://www.vatican.va/content/leo-xiv/en/encyclicals/documents/20260515-magnifica-humanitas.html Implicit Association Test (Harvard Project Implicit) — https://implicit.harvard.edu/implicit/ PauseAI — https://pauseai.info/ We Found AI's Preferences — Bombshell New Safety Research — I Explain It Better Than David Shapiro — https://www.youtube.com/watch?v=ml1JdiELQ30 Gödel's Theorem Proves AI Lacks Consciousness?! Liron Reacts to Sir Roger Penrose — https://www.youtube.com/watch?v=xwvijjZxpwI He Led The Famous 2023 Statement on AI Extinction Risk — Adam Khoja, Center for AI Safety Researcher — https://www.youtube.com/watch?v=QqESBXuo6EI Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

出典・この記事について

番組
Doom Debates!(Liron Shapira)
元の回
He May Have Found AI's FEELINGS — Richard Ren, Center for AI Safety Researcher
配信日
公式サイト
lironshapira.substack.com/podcast
X でシェアApple Podcasts で聴く

関連エピソード