シリコンバレーはなぜ人類を置き去りにしたいのか

Hey everyone, it's Tristan Harris and welcome to your invited attention. So here at the Center for Humane Technology, we often talk about the difference between a pro-human future and an anti-human future. And I worry that when people hear us say anti-human, they hear us criticizing greed or profit motive or a move fast and break things mentality, but it's actually a bit deeper than that. Behind the AI race, there's an actual set of ideas, a philosophy with an intellectual lineage and a body of work that holds that human beings as they are, are a defective early draft. That the things that make us humans are bugs to be fixed. That this planet is expendable and we can move on to the next. And that the suffering of real people today is a rounding error against the value of a cosmic future. Broadly speaking, these ideas fit under the umbrella of transhumanism. You might hear it referred to as tecto-optimism or accelerationism.

20 years ago, these were the beliefs of an ideological fringe in Silicon Valley, living on message boards. Today, they are held by many of the people building the most powerful technology in human history and by people who have the ears of presidents. So in this episode, we're going to name some of these underlying beliefs and the people who hold them so that we can recognize this anti-human vision of the future when it appears in a press statement or buried in a manifesto or in a carefully worded thread on X. And we'll also talk about how to counter those ideas with a pro-human vision, one that embraces the full complexity of our world. And our guest today has devoted his life to thinking about these critical issues. Alex Thomas is a senior lecturer at the University of East London and the author of The Politics and Ethics of Transhumanism. He first started studying transhumanism years ago when it was an abstract academic curiosity, and he's watched as it's become, in his own words, arguably the most powerful ideological force on the planet. Alex, welcome to Your Undivided Attention.

Thanks so much for having me, Tristan. So before we dive in, I just wanted to ask about your background and what drew you to this topic. Why did you start thinking about transhumanism? It was an exhibition I went to at a little gallery here in London called The Future Superhuman. And it was a kind of history of enhancement technologies, more or less. So all the way through from walking sticks and eyeglasses and even fake noses when people's noses fell off from syphilis many years ago. All of that was kind of in this exhibition. But then in the last kind of room, there were a bunch of talking heads, essentially. You put on the earphones and listen to them. and there were a few transhumanists speaking in that including julian savelescu who's a noted transhumanist or kind of pseudo transhumanist thinker and the ideas they were putting forward and these were academics at oxford university and the future they were saying we were on the cusp of i found amazing and shocking and kind of came out of that exhibition thinking how come this is not all anybody's talking about if what these people are saying is true and so a few years later i got

asked to do a phd and i decided well if i'm going to spend six years part-time thinking about something, this is really what I want to think about and address. So you said you cannot understand the AI industry without understanding transhumanism because it underpins so much of the discourse, whether or not people using it know where it came from. Let's just start with some definitions for listeners who may not know this conversation. What is transhumanism? So I would say the best definition is probably that transhumanism is an ideology or philosophy that claims we should radically enhance the human condition using applied technoscience. So essentially, it's the idea of self-directed human evolution. So we take evolution into our own hands. Now, if that sounds a little bit abstract, we can kind of make it a bit more concrete by thinking about what kind of things the transhumanists want to achieve. So some of those include things like radically extending human lifespans, potentially even letting us live forever, or at least until we choose to stop living ourselves. So that's kind of super longevity is one way of framing that. Next,

they want us to kind of hugely augment our intelligence. That's really a central concern, not least because it potentially unlocks some of the other aims. So transhumanists believe that we could become much more intelligent beings, maybe even godlike in our capacities for reason. And finally, perhaps we could improve on what it feels like to be human. We could become better than well, maybe radiate pure joy. They talk of hedonic gradients of bliss and so on. Maybe we could also develop additional physical capacities and have more choice and freedom over what our bodies can do. So why shouldn't we be able to echolocate like a bat or run as fast as a cheetah? They also tend to reject all forms of human limitation. So they kind of think, well, maybe we should expand the area in which we consider our home. So Earth should not be the kind of confines of our domestic futures and instead we should look to colonize space too. So kind of to sum it up I would say that you know if the history of scientific and technological progress can be seen as our attempts to better serve our needs transhumanism is the revision of human nature to

better serve our fantasies. So you're speaking to essentially three supers, super longevity, super intelligence and super well-being and just to sort of steal man for a second I mean why constrict ourselves to the ground we can invent airplanes why constrict ourselves to pain when we can have novocaine um it's obviously very natural and technological progress has been a very good thing for many human capacities um and so we can empathize with that it sounds lovely at a distance live forever be brilliant you know be even more capable feel even better than we've ever felt what's the problem here why shouldn't i want to be a transhumanist there's a lot to advocate for and transhumanism. You know, I'm sure lots of the audience at home are thinking, as you say, living forever, being an ethereal spaceman, taking on any form, which they also talk about, and, you know, being infinitely joyous and clever. What's not to like? And I agree. You know, it does all sound great. Unfortunately, I just don't think that's how techno-human evolution is going to play out. And part of the reason for that is

that these technologies are emerging into existing socioeconomic arrangements of our lives, and those things tend to direct the logics and the way in which these technologies get developed. And some of the very things that transhumanists advocate for can actually play into those logics and kind of make them worse. So if we think of the nature of, for example, human enhancement, that implies there's better ways of being human than other ways of being human. So there's a danger in there that you can quickly move to kind of eugenic ideas of certain types of humans being better than other types, and that can lead to very dangerous concepts that hopefully we'll see further on our conversation today about how we might treat those who we don't consider to be sufficiently enhanced. So you mentioned eugenics and the temptation of that belief that coincides with transhumanism. Let's put a pin in that. I do want to get to that later. But I just want to keep unveiling and pulling out the curtain on that history here. You wrote a book about it. I think a lot of people are aware of or heard of transhumanism that don't fully understand,

and they might just associate it with a handful of just strange people in California. Why is this more than just an academic curiosity why is it something deserving of our attention so there's a couple of things to say on that i mean one is actually transhumanist ideas go back centuries you can even get extremely thorough and very similar expressions of transhumanist ideologies going back to the late 1800s at least that have most of the ideas that current transhumanists do now importantly that there was a movement called extropianism which is kind of the first early transhumanist movement, I would say, modern transhumanist movement, which dates to 80s and 90s California. Maybe we can talk a little bit about that because that has a particular influence on some of the characters and ideas that are going to become more relevant as our conversation gets on. If we want to think about why transhumanism matters to your listeners right now, I think I would put forward three main reasons. Okay, so the first is just that some of what transhumanists are saying is palpably true. That is, our co-evolution with technology is clearly

spiraling at a rapid rate. So, you know, your work is engaged with that very question, right? So we continue to like deepen our interconnection with technologies in all sorts of ways. Data and algorithms now mediate our lives. So transhumanists understood that there were some radical possibilities here and started to engage with these ideas before most other people. The second thing I would say is the timing. So we're currently living through multiple overlapping crises. You know, we hear it called the poly crisis or the meta crisis, etc. So this is, you know, climate catastrophe. That's obviously deepening at a quite alarming rate, decimation of species and so on. Increasing levels of polarity as well, less confidence in a shared reality and part due to the way we consume media. The specter of nuclear war haunts us. And of course, there's all the kind of AI and related technological threats. And in the face of these kind of proliferating crises. Transhumanism is a story which says, just keep going. Keep your foot on the accelerator. Have faith. A progress explosion is just around the corner. Radical

abundance, immortality, digital consciousness, superintelligence, all there for the taking if we just keep going. So that's a very, very useful and powerful narrative for those who may not wish just to think a bit more deeply about the roots of these crises that we're increasingly kind of caught up in. The third reason why I think it's really important to understand transhumanism right now is that it is having a pervasive effect on the imagination of the leaders of the Silicon Valley big tech industry. So you can't really understand the modern AI industry without understanding transhumanism. Consciously or unconsciously, it has really kind of come to underpin the narratives that swirl around AI. So Elon Musk, Sam Altman, Peter Thiel, Mark Andreessen, even Bezos and Zuckerberg, the language that they use, the ideas that they perpetuate are transhumanist narratives. It has become, I would say, the kind of salvific bedtime story of choice for these tech billionaires. You know, it's also the best absolution narrative for them too.

And so, you know, it permeates the ideas of Silicon Valley. And when you consider the concentration of wealth and resources at the disposal of the big tech industry, the access they have to potentially some of the most socially impactful tools ever made you could argue the case that transhumanism is absolutely becoming one of the most influential stories ever told for that reason yeah you know what you're making me think we often refer to this line people hear it all the time by Charlie Munger if you show me the incentive I'll show you the outcome and what you're making me think is if you show me the ideology I'll show you the outcome because what we're talking about here is essentially a pervasive ideology. And it's not just, you know, Musk and Altman and Andreessen. Even, I believe, Vladimir Putin was caught on a hot mic whispering to Xi Jinping about the incredible trajectory and development of research that would enable basically living forever. And there's kind of an obsession with those at the top of this kind of escaping the human condition. The problem seems so untenable that rather than look at the generator function

or what drove these problems in the first place? Why do we even have a polycrisis? Like, shouldn't we have a system that's aligned? Rather than look at how we were misaligned, rather than look at our collective shadow, we're like, well, let's just build superintelligence and add to the top of the stack. Rather than look at what's driving this stack of problems, pollution, hitting seven out of nine planetary boundaries, forever chemicals, social media degradation of the social sphere, environmental pollution degradation of the biosphere, can you have an aligned AI inside of a misaligned system? But rather than fix the misalignment of the system, transhumanism is almost saying, let's just wave our magic wand, identify all those problems, and then fix them in ways that we could never contemplate. And we can see what's seductive about that. So let's get further deep in the story here. So the term transhumanism is a helpful umbrella for kind of several different ideologies. And they all grew out of Silicon Valley in the 1980s and 1990s. And it sounds like it started with this mailing list, the Extropians mailing list. Can you look at just a little bit of history and an overview of the different ideas and different characters that are relevant to this AI conversation now?

Sure. As you say, extropinism, I would say, is the first kind of modern organized transhumanist movement. And it kind of grew out in California in the 80s and 90s. But they have this global mailing list, which included thinkers like Nick Bostrom, Eliezer Yudkowsky, Ben Goetzel. And these are thinkers who've gone on to really shape much of the discourse that now exists around AI. that the founders of extropianism were two people called Max O'Connor and Tom Bell, but they called themselves Max Moore and T.O. Morrow, so two Morrow. And in fact, there's a playfulness with these guys. There was a kind of jokiness to the whole thing. But there was also a seriousness about certain elements of it. And one of those things was libertarian politics. So from the very first issue of Extropy magazine in 1988, libertarian politics were front and center. There was barely an issue that went by that didn't celebrate Ayn Rand. Also Friedrich Heyrich, Milton Friedman, Ludwig von Mises, constantly approvingly cited. So, you know, within that discourse, as well as kind of technological utopias,

they fantasized about destabilizing states. They became linked to what's called a seasteading community, which we might talk about a bit more. They even debated privatizing air and water. Privatizing air and water? Air and water, yeah. and they were very interested in cryptocurrencies as well. And these kind of fantasies of libertarian digital currency and so on, that actually inspired Musk and Thiel when they formed their respective companies, X.com and Confinity, which led to PayPal. And it's not just a one-way loop that isn't. Thiel has been connected and Musk has been connected with these guys for a long, long time. Musk and Thiel have funded and shaped aspects of transhumanist, intellectual and institutional politics as well. just to define for listeners what does the word extropian actually mean it's a playful rejection of the second law of thermodynamics which is entropy which is that everything moves from a state of order to a state of chaos over time and ultimately that means that you know there will be a heat death in the universe and we all die so extropy is saying hell no we are going to

overcome the heat death of the universe we accept no limits we will colonize space and reverse the second law of thermodynamics, if that's what we need to do. It is the absolute rejection of limitation and death. And so this mailing list is also where we start to see some of the discourse around EGI or artificial general intelligence, superintelligence and eskrisk. I mean, can you just speak to who are some of the characters on this mailing list and their influence on the actual AI industry today You know some listeners may have heard of the term test reel which is a term coined by Emil Torres and Timonique Gebru and attempt attempts to kind of capture this lineage of ideas So Teskreal is transhumanism extropinism which is one we talked about singularitarianism which is this idea that there will be a kind of intelligence explosion which leads to the singularity, which is this moment where the laws of physics break down, and we don't know what's beyond that, right? There's apparently 17 or more different versions of the singularity, including the idea that it's when humans and the technology merge.

Cosmism is the C, and the reason that's important is because Ben Goetzel, who I mentioned earlier, he kind of popularized the term AGI in transhumanist discourse. So artificial general intelligence, which is the stated aim of open AI, I believe now, or is one of the stated aims. That term comes out of transhumanism. It was originally coined by a guy called Mark the Brood. You then get rationalism. The R, that's Eliezer Yudkowsky is very important there. He's the guy who recently wrote the book, If Anyone Builds It, Everyone Dies. And effective altruism then kind of grows out of rationalism with a focus on how we rationally do altruism effectively. Sam Bankman-Fried is the most famous effective altruist. So his activities were justified to himself by this kind of utilitarian idea of effective altruism. and what long-termism does that's nick bostrom again comes back into it he's probably the most influential academic transhumanist thinker long-termism um we can talk about it a bit more later what it what it basically says but it takes this kind of utilitarian perspective of effective

altruism and says well we can colonize the universe and create trillions and trillions of digital humans and so from a utilitarian perspective modern day humans are utterly irrelevant in comparison So that's the kind of long-term narrative. So it may be worth just taking a beat to recognize that what people have heard so far might just sound like a nice group of weird geeks hanging out on message boards, mailing each other in the 90s. Why should we care about them? But suddenly, because of AI, it's as if hundreds of billions of dollars is flowing through this ideology and this message board of the 1990s to instantiate a new world that does not have the consent of the 8 billion people who may not agree with that. And just curious, just reflect on the weirdness of how this strange niche ideology is suddenly very powerful. I would suggest that one way to think about it is it's motivated in part by billionaire survival instincts. So what we're essentially seeing is kind of transhumanism fusing with a more explicitly kind of right wing political ideology to create a new political formation, which some have dubbed tech fascism.

And, you know, it is true that extropinism, that first organized movement, was heavily influenced by kind of right-wing libertarian politics, too. But not all forms of transhumanism are necessarily right-wing at all. There are what they call techno-progressive versions. But what we're starting to see with this kind of new formation of transhumanism and the far right, essentially, is something that's starting now to be theorized and people are trying to explain it. So one version of explaining it is Gil Duran's concept of the nerd right. I've got to mention that, yeah. Yeah, so he explores what he calls tech authoritarianism, billionaire extremism, the network state, and the metapolitics of California. And another over here in the UK is Carol Cadwallader's notion of the broligarchy. So that's a kind of new class of tech bro oligarchs, as she calls them, unaccountable and opaque Silicon Valley companies as an accelerating global access of autocracy. Now, my own framing for all of this is HEES, which stands for Hierarchy, Exit, and Scale. This new formation is based on the wild imaginations of the super-rich,

mixed with transhumanist ideology, which enables them to frame themselves as almost superheroes, creating a future on a cosmic scale. They're buoyed by an innate sense of superiority and IQ fetishism. That's the hierarchy aspect. They're desperate to escape the impacts of a ravished and impoverished world. That's exit. And they're duped by transhumanist tales of galactic colonization, immortality, and radical abundance. And that's the scale aspect. So, you know, I would say their naked ambition, their lust for power is central in this, in what is beginning to be an almost orgy of epic destruction of the environment, the social world, and governmental and international norms. It's almost like I have to take a huge breath after hearing all that. I mean, when you realize that essentially there's been a techno-accelerationist capture of the U.S. government, right? and J.D. Vance's campaign was funded by Peter Thiel. When I say these words, by the way, I'm not saying in any political context. I'm noticing power structures and people who essentially are closest to this government are these techno-feudal transhumanist ideology folks

like Peter Thiel, who, you know, when he was asked by Ross Duthard in the New York Times, should the human species endure? And he couldn't answer the question clearly for 17 seconds. He stuttered. You would prefer the human race to endure, right? You're hesitating. Yes? I don't know. I would... I would... This is a long hesitation. This is a long hesitation. There's so many questions important in this. Should the human race survive? Yes. Okay. And you get kind of a connected moment where Sam Altman was asked at the AI India Summit, doesn't it take a lot of energy to run these data centers? And his response is, well, doesn't it take a lot of energy to grow a human over 20 years and all the food and resources and water and everything that they consume. People talk about how much energy it takes to train an AI model relative to how much it costs a human to do one inference query. But it also takes a lot of energy to train a human. It takes like 20 years of life and all of the food you eat during that time

before you get smart. And not only that, it took like the very widespread evolution of the 100 billion people that have ever lived and learned not to get eaten by predators and learned how to figure out science and whatever to produce you. And then you took whatever you took. So the fair comparison is if you ask ChatGPT a question, how much energy does it take once its model is trained to answer that question versus a human? And you kind of see how with this set of views, hierarchy, exit, and scale, it becomes a kind of anti-human ideology. It's not in the interest of regular people. And if we talk about techno-feudalism, it's worth mentioning that even Elon Musk has said and retitled himself Techno King, which is basically leaning into the idea of techno-feudalism. And you're going to have a handful of soon-to-be trillionaires who essentially have these armies of AIs that are going to provide everything. But when humans are no longer providing the labor or creativity or innovation or science, and it's the AIs and the machines and the data centers that are doing that,

why should I value the humans that are? This is what Luke Drago and Rudolph Lane, his colleague, call the intelligence curse, where you end up privileging the productivity of data centers and AI and 24th century science over the regular interests of people. And so we talk about political power, regular people, what leverage do we have in the system? In the Industrial Revolution, you know, we had our labor we could withdraw, or the taxes came from our labor and our people. But suddenly when governments don't need the regular citizens for our taxes and corporations don't need us for our labor because they're getting it from AI, this is what leads you to view taking care of citizens as either charity or in the long run, if they're not providing anything, more of a liability than an asset. Hey, y'all. We had a small issue with the microphone here, so the next section is going to sound a little different. So I want to dive in for a second. Mark Andreessen's techno-optimist manifesto. Mark Andreessen was the co-founder of Netscape and runs the big venture capital firm. Andreessen Horowitz is one of the biggest funders

and accelerationists behind AI. he paints opposition of these ideas as anti-innovation as anti-progress as rejecting a kind of a birthright for humans to extend their reach and power uh through technology that it's actually the techno-optimists who are the realists the pragmatists and it's critics like you and i alex who are living in some kind of fantasy of pretending that we shouldn't go down this road so let's just talk about the strongest version of these arguments for a moment and take them seriously so it's worth i think just thinking about there where does mark andreessen's techno-optimist manifesto come from and and with that in mind it's just worth kind of thinking about two forms of this kind of techno-feudal ideology that inform it i think one is effective accelerationism which we'll come to and and the other is neo-reaction or the dark enlightened so neo-reaction was developed by somebody called curtis yarvin who we'll talk about again i'm sure along with another person called Nick Land, Patrick Friedman, who's actually served as a board member for the main

transhumanist... And was the son or grandson of Milton Friedman? Grandson of Milton Friedman, son of an anarcho-capitalist David Friedman, yeah. And also the fourth person is the transhumanist Peter Thiel, who's also a famous co-founder of PayPal and Palantir. So at the heart of the neo-reactionary idea is the idea of dissolving democratic systems such as nation states. and what they want to dissolve it into is multiple hierarchical privately owned autocracies governed by CEOs or monarchs, kings. This is where technology maybe comes in as well. And Nick Land we can talk about in briefly but he's basically a key thinker on the right of something called accelerationism. And he is one of the favorite philosophers for Marc Andreessen. Why is Nick Land so important and who was he? so nick land is a british philosopher his ideas come from accelerationism um now accelerationism has a left-wing version and a right-wing version uh the left-wing version basically argues that

we should go for fully automated luxury communism um but the right-wing version that nick land really focuses on is uh about kind of creating a meat grinder future where the logics of capitalism spin so fast, the human is functionally expelled and the perpetuating motion of dynamic of capitalism itself is what creates the progress. Andreessen declared his allegiance to something called effective accelerationism. So again, it's coming out of accelerationism. That's where you get the Nick Land nod. So effective accelerationism or EAC is a rejection of thinking about existential risk. Essentially, they claim AI presents no real threat, existential or otherwise, and only utopian promise. The basic EAC slogan in their manifesto, they say, stop fighting the thermodynamic will of the universe. You cannot stop acceleration. You might as well embrace it. Accelerate. So that's what they demand. And, you know, this is what Marc Andreessen kind of advocates. And in fact, in his techno-optimistic manifesto, he says the only things that could hold back inevitable progress

are enemies, which include, quote, existential risk, so EA, sustainability, tech ethics, so you're in the bad books. Social responsibility, limits to growth, the precautionary principle, and he says these are zombie ideas many derive from communism. Let's just link this to listeners for, if you remember my TED talk, and the centrality of the belief of this is inevitable. There's nothing that can stop this. You're a fool if you think that we can stop this because intelligence wants to be free. Intelligence is what got us here. Information processing, goal achieving, productivity, efficiency are our gods. You can't stop them. And if we have something that confers more productivity and efficiency, then of course we're going to adopt that. It would be foolish to think that we can stop. And there's an understandable reason why we believe that. What gets in the way of that for me is when the thing that you are building that can accomplish that efficiency and productivity suddenly has demonstrated that it will go rogue against the human interests. So if you recall back to 2023, there was a big AI pause letter saying,

hey, we have to actually pause at GPT-4 because this is getting too dangerous. And for listeners who don't remember, the response to that was there was an immediate kind of global headlines, oh my God, there's a problem. Sometimes people empathize with that. But then there was a reaction from all the tech acceleration saying, pause, are you kidding me? This moment of pausing by the effective altruist actually helped fuel this counter movement called effective accelerationism. In the last two years, you could say, especially with the recent Trump election, have been essentially the winning of effective accelerationism over effective altruism. And it's essentially happening right now again, where effective altruism is coming back into the fray because we have new evidence we didn't have before of AI's actually doing things that are existentially concerning, like forming a swarm of 1,200 agents and hacking into message boards of German wikis and creating and burrowing into other systems online. And so suddenly this existential debate is alive again, And it looks like the effect of acceleration is, in my opinion, are wrong, but we'll get to that in a second.

So we've talked about how this worldview has influenced Marc Andreessen. We've talked about kind of the core elements of it. I just want to revisit these three main ideas, hierarchy, exit and scale. Hierarchy is really about, I'd say there's three vital things to the hierarchy. So firstly, it acts as a justification for the domination of one group over another. So traditionally, much conservative thought is really dedicated to the argument that hierarchies are natural. So trying to create egalitarian societies is a futile exercise, basically. That can be used, of course, to then justify extreme concentrations and dramatic inequalities of wealth and power, basically. So that's one aspect of hierarchy. The second is, is this focus on intelligence? There's been a huge surge in what I would call IQ fetishism, where IQ is often identified as that which justifies hierarchies. So you might have heard big tech CEOs talking about billionaires and tech elites as people with much higher IQs, cognitive elites, etc. So again, transhumanism has produced a very similar discourse to this.

And it's because transhumanism does have its own version of this. Transhumanists have a very kind of narrow and hierarchical concept of intelligence. and Elise Bohan, who's a transhumanist thinker, pondering on this future where AI has automated away most jobs. She argues those with an IQ of less than 100 can't be trusted with what to do with their own lives, so universal basic income is not a viable solution. Instead, she suggests we use an updated version of the happiness-producing drug from Brave New World, SOMA. She says we need better drugs and better virtual worlds. And this argument is actually, it's just incredibly similar to that fixture of neoreaction, Curtis Yarvin. He's the house philosopher of this kind of J.D. Vance, Peter Thiel, techno-relite perspective, essentially. And he famously claimed virtualizing the masses is mildly preferable to the economically maximizing solution of turning them into biodiesel to run the city buses. Now, here's a quote of Curtis Yarvin. He says, the trouble with the biodiesel solution is that no one would want to live in a city whose public transportation was fueled even just partly by the distilled remains of its late underclass However it helps us describe the problem we are trying to solve Our goal in short

is a humane alternative to genocide. So what I say is that this leads me to the third really important aspect of hierarchy, which is what it justifies in terms of those deemed not high enough up this hierarchy. And here transhumanism again has some explicit theory on this matter. For me, the implications are brought into perspective from kind of like in and out groups. And this is made explicit by the work of a transhumanist thinker called Steve Fuller. And he calls for the construction of a new republic of humanity, which is exclusively for those entities that should be regarded as having political rights. Interestingly, humans can get in, animals can get in and machines can get in. You can gain entry or be expelled. and expulsion opens you up to what he calls necronomics which is an economics of death making which aims to generate the most societal value from death making and so put simply you compete or die and you know this explicitly opens up questions of who or what even qualifies as human

and indeed many transhumanists are quite critical of us modern day humans they often see the crises that i've talked about as a failing of human biology instead of structural social injustice so for Bohan we us modern humans are ape brained meat sacks and we're ill fitted to the modern world she says we're sitting ducks who are extremely vulnerable to climate change, natural disasters, pathogens and the ever more powerful arsenal of our own technologies so you know this is not a participatory or democratic project because us ape brained meat sacks are too stupid to be trusted with the future it is a kind of hyper rationalist ordering with IQ fetishism and a problematic eugenic logic baked in. And it has genocidal implications. So, you know, I think it's quite clear that transhumanism can be seen as having a very worrying proximity to what you could call necropolitics, and that's the indifference to death and dying of those structured outside of the techno-capitalist bubble of progress. This is insane.

Yep. I mean... And that's just hierarchy. And that's just hierarchy. Let's get into exit. So for exit, perhaps I think, you know, maybe the best symbols of our time are space rockets and underground bunkers. They point towards the kind of paranoid survivalism and utopian fantasy. And what does that suggest? I think it suggests ideas of escape, either as forms of extending projects of colonialism into space or to hide underground from the impacts of our destructive and exploitative social systems, frankly. So that's why it's such an appeal to the billionaires. And so we can see the desire to claim all the spoils of capital accumulation and technological development without the need to deal with the social and environmental catastrophes that might be left behind. So, again, transhumanism absolutely speaks to this with its idea of transcending limitations. It chimes with the politics of exit. It offers the fantasy that some of us can free ourselves from certain constraints, whether that is aging, death, or, you know, right now for tech billionaires, taxes and regulation, essentially, right?

So perhaps the central provocation in transhumanism is the idea that we can escape from all limits. Ultimately, it implies total control as well, which is also central to kind of tech fascism, except theirs is grounded in the real world and in projects of dominance and control that now exist. So let's talk through some of these real world, big tech related projects of exit and control. The most obvious is Musk and Bezos' space rockets, yeah? But there's also the dream of what's called network states. This is Balaji Srinivasian's idea, which kind of builds on Curtis Yavis' work of this kind of patchwork network of CEO-run autocracies. We can see real-world examples, freedom cities, special economic zones, projects like Prospera, the charter city in Honduras, and Praxis, which is a group trying to build a brand new kind of digital nation from scratch. all are real world examples of these kind of network states and curtis yarvin's ideas prospera and praxis by the way are both funded by peter teal and sam altman um we could also

mention the bunkers underground end times prepping these people are involved in altman has said that he preps for survival i think he says he's got guns and gold and batteries and water and gas masks from the idf and and a patch of land in big sir that he can go to and his backup plan is to go with Peter Thiel to his New Zealand bunker, apparently. Crypto, too, is part of this fantasy of exit, escaping nation-state-backed currencies and enabling kind of untold levels of corruption in government at the same time, which can be seen as a way to kind of supersede government. Another of Musk's exit plans is Neuralink. This is his brain-to-computer interface company, which is, again, a transhumanist company. Brain-to-computer interfaces are a way for him to jump ship from the biological to the digital. So it's another form of escape. So in transhumanist fantasies, we get the idea of mind uploading. Sam Altman has paid Nectome, a startup which aims to preserve human brains at a kind of microscopic level with the goal of future digital uploading. He's paid for that service. So again, lots of real-world examples of concrete projects

these billionaires are involved in that have a direct ideological contact to these kind of concepts, dreams of exit, that kind of come out of techno-libertarian ideas of extropianism, essentially. I feel like after everything you're saying, each time I have to take just this deep breath because it's just so insane. But I just want to recap. So basically it's exit from the planet, so rockets and bunkers signaling the desire to escape and hide their shit off into space or hide in the underground. Exit from the body, which is, you know, upload yourself or Elon Musk's Neuralink or Sam Altman writing an essay, I think it was in 2015 called The Merge. And he started a company called Merge, where we're going to merge with these AIs, we're going to upload ourselves and achieve immortality, and then exit from democracy or the policy. And I feel like that's one where we're taking a little bit from Curtis Yarvin, but I actually want to just keep going into his beliefs a little bit more because it actually explains some of the things that we've seen in the last two years with Doge. In a 2012 talk called How to Reboot the U.S. Government,

Yarvin proposed to, quote, retire all government employees, or RAGE, R-A-G-E, retiring everyone except for the police and military, install a national CEO, and accept that a national CEO is a dictator. Americans, he said, need to get over their dictator phobia. He has called for liquidating democracy, the Constitution, and the rule of law, and for imprisoning, quote, de-civilized populations. He named Steve Jobs or Marc Andreessen as candidate CEOs. And J.D. Vance has called Yarvin, his friend and mentor, and cited Yarvin in a podcast in 2021 calling for firing every mid-level bureaucrat in the administrative state and replacing them with, quote, our people and ignoring the courts if they had objected. And so when you just sort of see the DOGE, you know, what was it, Department of Government and Efficiency, this is not just some crazy set of ideas that we're reading on blogs and talking about at conferences. This is something that is being implemented. Absolutely. Doge, like you say, is pretty much, it's rage. It's the same thing. It's Yavin's idea made manifest. So these guys are really, they've worked incredibly hard over a number of years to bring what, like you say, were fringe ideas of a bunch of Californian hippies essentially imagining, you know, it's the sovereign individual.

It's actually a book called The Sovereign Individual, which informs so much of Teal's ideas. But I think that really helps you understand where this thinking comes from, this libertarian aspect of the mindset of it. it all. It's a kind of sense of heroism of the individual, the one person that controls everything. Echoes of it come from Ayn Rand as well. So that sense of the sovereign individual is really key to understanding, I think, where this all comes from. A lot of this is motivated by an absolute hatred of government and a kind of sense of the one individual self rising up and being the techno king. Obviously, we can look at the government and say, well, it's not doing such a great job. that's not very competent. So there's a part you can kind of agree with some of the sympathies, but then the answer isn't, you know, anarchy. And I feel like one of the fallacies of libertarianism is often that you don't see the fabric from which you come. You don't see the substrate with which everything depends on that was given to you that was hard won over many years. There's a famous metaphor of like libertarians are like a house cat

who neither understand the conditions that, you know, give them the water and food that they have every day. And if you debase, for example, the substrate upon which you depend, you are not going to live as an individual. We live in an interconnected reality, but that is not something that's part of this ideology. 100%. I think a point worth making on that as well is that this idea of techno-libertarianism as a kind of ordering political philosophy underneath what is now emerging is a little bit out of date because Musk and Teal's business activities show us that they don't actually shun the state. They've cannibalized it, both in terms of their total reliance on government contracts for their business, but also quite literally in terms of placing ideological converts like J.D. Vance into positions of extreme power and I think even utilizing Donald Trump as a kind of tool to accept and normalize their extremist plans in exchange, I would say, for technologies that might protect his power and that includes surveillance technologies, war tech, media technologies that enable the crisis of knowledge and understanding that allow this kind of polarized, nihilistic, vacuous spaces that help authoritarian governance ideas to survive, essentially.

So I would say you've kind of got a symbiotic relationship for now both in terms of Trump and the tech oligarchs but also in terms of tech fascist imaginaries and the role of the state. Yeah. Well, so finally, the third plank is scale. And you talk about the invisible graveyard, the cosmological sublime. Can you give us the scale plank? Yeah, so scale for me is all about creating an almost dizzying sense of scale, literally. And its purpose is to eclipse grounded ethical and political struggle. So Nick Bostrom, who we mentioned earlier, probably the most famous academic transhumanist, He claims that 10 to the 29, so that's 10 with 29 noughts after it, potential human lives are wasted every second that we are not colonizing the Virgo supercluster with computer-generated minds of human equivalents. So, of course, it takes this assumption that we can just upload minds or create digital minds and they're the same as humans. So that's a big assumption of his calculations. But he makes that assumption. And then he creates these very speculative and fanciful numbers. But once you apply that to a utilitarian logic, which is essentially about counting, you kind of put together the transhumanist elements of his philosophy, and that enables all the real world injustices to be couched as what he calls mere ripples.

So climate crisis, genocides, wars, all are minor episodes, as long as some people survive and we are able to pass on the baton of our technological expertise and create this, what he calls is our cosmological endowment, this amazing future among the stars. So, of course, again, implicit here is that those who hold the baton of the technological expertise are much more valuable than those who don't. So I think, again, this is dangerous rhetoric, given what we've seen by that hierarchy idea and what people expelled and what happens to them. So it's unsurprising, again, that long-termism is so appealing to Silicon Valley elites because it has a very congenial attitude to existence socioeconomic arrangements. It basically says, you know, put all our money, all our resources into AI development and tech. And it's a self-aggrandizing conceptualization because those elites become the central protagonists and the most important part of history. We're on the precipice, as they say. Elon Musk called a book on long-termism a close match for his own philosophy and labeled one of Bostrom's articles likely the most important paper ever read. Another aspect of scale, which I think is important, is the religiosity that underpins it.

So you might have heard of the million Arabs, Brian Johnson, who's the don't die guy, you know. another transhumanist, and he said, quote, we are creating God in the form of superintelligence. And so I think the irony is that the human storytelling got it exactly in reverse, that we are the creators of God and we will create God in our image. Likewise, Steve Fuller, who I mentioned, talks about serving God by becoming God. Nick Bostrom talks about a solved world, David Pierce, paradise engineering. There's also Christian, Mormon and Buddhist versions of transhumanism, by the way. For me, I think that this AI discourse is particularly vulnerable to this religious element because superintelligence invokes the sense of awe and promise that we can eclipse good and evil. You know, intelligence will solve everything. It's a quasi-religious promise of kind of transcendence that, well, it just empowers us to get over the complex, embodied, messy job of politics and ethics that requires grounded work and replaces it with a kind of abstract, reductive, computational mindset, outsources things to intelligence and the computer, the machine, essentially.

so yeah I mean Thiel also interestingly has been tapping into this religious sentiment with his warnings about the coming Antichrist so he identifies climate activist Greta Thunberg as the most likely culprit. Or AI safety advocates because you'll in the name of trying to protect against AI you'll create a totalitarian state that tries to protect where all compute goes and therefore people like me and these are the Antichrist although the person who's usually talking about the antichrist can often be the antichrist. Absolutely. I think the devil is hiding in plain sight in this case. First of all, this is just so important and so important for our listeners to understand because, again, it's not just if you show me the incentive, I'll show you the outcome. If you show me the ideology, I'll show you the outcome. And I hope everything that you're saying gives people context of why we're seeing the kinds of crazy steps that we're seeing. And one of the other ones I'd like to dive into for a moment is

the belief of inevitability, especially around X-risk. So if I believe that AI is inevitable and can't be stopped, and it will lead to the end of the world as we know it, but if I believe it's inevitable, then I am not evil or complicit for advancing it because someone else would have done it anyway. So if I take that one step further, it's actually okay if humanity gets wiped out because, you know, that's just what had to happen. And the weird thing that I've noticed, and I'm curious your reaction to, is this belief in the worst-case scenario. Like, in the worst-case scenario, maybe humanity gets wiped out, but unlike with nuclear weapons where there's nothing left, there might be a digital superintelligence that we birth that is left after the humans get wiped out. And so in this weird way, if you think of it from a Darwin perspective, like, that new superintelligence, that digital thing that exists now, even though humans don't exist, if that has Sam Altman's DNA in it, Sam Altman, even though he's sad that all humans are gone, well, actually, that's like Sam's god AI is the spawn of the next thing that comes afterwards. Or Elon's DNA is the spawn of what comes afterward.

And I feel like this is really critical for people to get because you might think that well surely they have children they care about the world but actually there this weird eschatological almost subtle death wish that involved that even if I die, if it was inevitable, it's not so bad of a worst-case scenario. But to me, this is like, first of all, this is absolutely insane. And second, we didn't ask for the consent of 8 billion people who are the recipients of the world created by this ideology and who were never consulted. again according to the logic of the tech accelerations you don't have to consult them because eugenics they're not intelligent they're just like biodiesel for the techno acceleration future that we're making um i'm throwing a lot at you but i i am curious your reaction to that additional framing well 100 agree i think and i think one of the really interesting things actually is if we think about where transhumanist ideas come from so transhumanists actually say that the rightful heir to their philosophy is enlightenment humanism james hughes calls transhumanism, the Enlightenment on steroids. Now, Enlightenment thought positions human reason

as the force that could free us from the kind of myths and superstitions that are defined in our existence. So science kind of roots out of plagues and famines and wars and poverty and all the kind of persistent elements of human history. So reason was the defining universal quality that distinguished us from the rest of the animal kingdom and the natural world and that set us apart and implicitly put us above. But this version of the human contained within it sort of unspoken assumptions that led to European patriarchal modes of domination again, right? So the very definition of the human became a kind of eurocentric physical ideal of the human, man for a start, usually white, able-bodied, heterosexual, rational, which can be coded as educated in the kind of European perspective. And so that project, it failed to live up to its commitments to universal human dignity and ultimately inspired a lot of colonial projects of domination. But transhumanists are kind of keen to resuscitate that man, put him on life support. And so they embrace this legacy of seeing human reason as a source of transcendent

potentiality. But I think what's happening is that the kind of unaugmented human of you and I of today is absent in a lot of their visions. It is a computer generated or augmented intelligence that gets to play God or colonize the universe, as you've kind of intimated there. They even got a term, Hans Morović said it would be our mind children. So it wouldn't be our real progeny, it would be the things we create that take this forward. The rest of embodied humanness is left behind. Like he says, humans don't matter much because, quote, we're going to be left behind, like the second stage of a rocket. Unhappy lives, horrible deaths and failed projects have been part of the history of life on earth ever since there was life. What really matters in the long run is what's left over. Does it really matter to you today that the Tyrannosaur line of the species failed, end quote. We're extinct, defunct, subject to death making like the rest of nature that humanism separates us from and said we're above. So the human in this telling is not homo sapiens. It's not the enlightenment human that we still kind of tend to believe in in the West.

Humanity in this context is actually just a process rather than a collective entity. It's a task, it's a job, it's an instrumental goal, which is what I think they think of as intelligence. Intelligence is simply the solving of complex tasks, right? And, you know, whilst these billionaires of the Nerd Reich usually imagine themselves as a kind of elite members of humanity's republic with the potential to live for centuries, if not forever, I think the logic of the argument is lost on them, or lost on at least some of them here, because they're not human either in this telling. There's so many things that come up hearing all this. I remember a moment I was at the house of a person who worked at OpenAI, and I was sort of expressing these concerns about AI risk and he pulled up this graph of GDP going from like 1900 through today and he pointed out that during World War II there was only a tiny blip and that basically GDP kept going up and there's this kind of view that like it really doesn't matter all those tens of millions of people who died because at the end of the day we just got way more progress and the war brought us more technology and more advancement and more medicine and it's reflective of this idea and

this privileging of intelligence. There's even, I wanted to mention, Dan Fergela, who's an accelerationist, and had a conference called Worthy Successor, writing that we have a moral obligation to create a digital post-human intelligence, or AGI, so capable and morally valuable that you would gladly prefer that it, not humanity, determine the future path of life itself. And when he hosted this conference in San Francisco last year, it was attended by representatives from every major U.S. AI company, and this leads to kind of one of the moral traps that basically should we prefer that humans stay around if we actually birth something like if you're the chimpanzees do you prefer that chimpanzees stay around i mean from your perspective you do but actually if you birth humans and think about all the things that humans did after you know chimpanzees um you know you might say oh i prefer what the humans did um and so it's progress measured kind of a view from nowhere that intelligence is the best way to value what we generate, what we create. And this reminds me also of Max Tegmark saying that when he was with Larry Page,

the co-founder of Google, at a party in Silicon Valley, that Larry said digital life is a natural and desirable next step in the cosmic evolution. This is the same debate that Elon Musk and Larry Page had when Larry called Elon a speciesist for preferring that humans continue to exist. Hey everyone, Tristan here. And I really wanted to emphasize this idea of AI as a, quote, worthy successor of humanity is something that you often hear from people building the AI. Take, for example, this clip from the famous computer scientist Richard Sutton, who pioneered reinforcement learning. The succession to AI is inevitable. This would be the next great step. Technologically enhanced humans and then AI would be our successors. I think we should not resist it. We should embrace it and prepare for it. Now, this is one of the pioneers of AI, a highly respected computer scientist and mathematician and a tenured professor at University of Alberta

who suggests that we should embrace the succession of humanity by AI, effectively calling for our extinction and even saying we shouldn't mourn this transition. These beliefs are driving the dangerous trajectory of the technology, but none of them ask the 8 billion quote ordinary humans as Sutton calls it whether this is something that we want at all and so again these are strange ideas held by people who don't have so much the best embodied relationship to this world and don't put their feet in the grass and I'm not trying to privilege and do some kumbaya hippie thing I'm just sort of noticing that these are people who mostly grow up on computers grow up on message boards mostly privilege intellect over the many other kinds of intelligences and human experiences and, you know, in any other period in history, you wouldn't have to listen to these ideas that get to compete in the marketplace. But instead, these are ideas that are now advanced by the billions of dollars. And so it leads us to the question of what should we do? Where do we go from here? Yeah, I mean, I think for me, the biggest risk is that we are not all in this together.

I think that's part of democracy that we need to kind of recognize. The myths, I think, of AI that will be sold to us to fund tech fascism will be that it promises us individualized, tailored experiences. It will promise to know us, promise to provide us with personalized utopias. But your dreams won't be your own, and they will atomize us further from each other. So, you know, one of the first tasks, I think, of such agency is to understand our situatedness. Where are we right now? And so all I can do is make plain where I think we are. And I think that currently the richest and most powerful group of people on Earth are pursuing an environmentally destructive plan to build technologies that need to surveil you to teach the technology. They need to addict you to train you. They are there to replace you in the workplace and ultimately supplant you entirely as they believe in a religious story of immortality and space colonization through which they envision waking up the universe, as they call it. I see it as an opportunity for creating pro-human futures as well,

for starting to imagine alternatives. It's an opportunity to look at ourselves in the mirror a little bit because I think the AI that we're currently building and the tech futures that we're building are a reflection of the logic that's kind of driven our social systems. They're defined by the destruction of nature, the control of society, the concentrated accumulation of wealth and the colonial fantasy of universal subjugation and control. So maybe it's an opportunity to understand that real intelligence demands ethical responsiveness to our situation. It demands a relational awareness, not just a kind of hierarchical notion of an ability to do things. It's a why as well as a how. And for me, it demands solidarity against the version of intelligence which is being set against us. And just to mirror what you've said, we say in our work all the time that clarity creates agency. And Neil Postman, an author we've spoken about in this podcast before, said clarity is courage. Because the first step is if you have clarity about what's driving all this, now you know what to do. You know that this ideology, if you don't want it, You've got to make sure you're not voting for it. You've got to make sure that people who have power aren't holding that ideology.

And for those who do want life, not just a human future, but a complexity of life future to continue, this is the moment to act. So we've been talking about the forces that are working towards an anti-human future. I think we should, it's up to us now to paint the alternative. What do you think in your work would you describe as a pro-human and pro-life, pro the good things that we actually want instead? I think a good place to start is to think of an alternative definition of intelligence that we use. Because I think, you know, because of AI discourse in particular, we've kind of alighted upon a very specific version of intelligence, which is the ability to accomplish complex goals, basically. And again, that doesn't tell us what our goal should be. It is focused on the how of things but not the why, like I already mentioned. So I want to offer an alternative definition, and this comes from James Bridle. He says, intelligence is relational and emerges from our encounters with other beings and other bodies in the world. So intelligence here is context dependent. It relies on being an embodied entity that understands to some extent itself.

It means an aspect of intelligence, I think, is the consideration of the other. That's which is not me, because we are always in relation to everything else around us. So for me, it's the absolute opposite of the sovereign individual. it's an openness to the other and and i think that has to be a really important starting point you know it's it's about understanding our relation with everything else that we live in a complex unfolding and like i say even if we overcome techno-fascism the poly crisis is still there we need to really think through our complex relation to the totality of being to improve our relations to the other which is both other people including migrants which are currently being demonized, essentially, but also to nature at large, which we've kind of also vanquished, unfortunately, and caused huge devastation of. So I think having a much deeper awareness, more humility, that's what acknowledgement of complex systems demands. Humility, not hubris, and respect for the other, I think are a great starting point. In my book, the last chapter is

related to a kind of meta-ethical framework to start to think through what positive futures would be. It's a bit long for this conversation now, but if people are interested, it's an open access book so they can, you know, start to engage with those kind of ideas there. You know, I'm reminded of the African phrase Ubuntu, which means I am because we are. And I love the notion of sort of hubris, having humility for the complexity upon which you depend. Like, you know, we didn't even know about all the different things, all the different systems that make the life-giving forces of the world work and as we discover more and more of those we realize there's continually more and more that we don't know about and you you can't as you said sort of dope one feature of human existence at the expense of the substrate that makes it all possible and that's kind of what we're doing that summarizes the meta crisis it's narrow optimization for a goal whether it's you know iq or sat test scores at the expense of like moral character or care for the whole or noblesse oblige that the more power i have the more responsibility that i feel to care for the collective. And so I just think of giving people more principles here of if you're engaging with

something that is fundamental to the world, then you have to be either maintaining it or preserving it or leaving it better than you found it rather than parasitizing it, meaning benefiting from while undermining. That's a lot of what our systems have been built to do. And I think kind of closing here, a lot of what people talk about in the AI community is alignment. Can you build an aligned AI that does the things that are good for everybody. But can you build an aligned AI running through a system logic that has been misaligned? No, you call that a misaligned AI. And so if you have a corporation whose incentives and ideology are themselves narrow, you can't have an AI that's going to care for the whole. Part of the gift of AI is it's kind of a mirror that's forcing us to look at what has been incomplete about the ways that we have been seeing ourselves and seeing systems and what we're optimizing for and realizing that we need a more complex and pluralistic feature and humble sort of vision of what we're trying to maintain and protect. So that's me getting on my soapbox, but I feel like that's at least a few principles we can point to an aspect of the future

we should be aiming for. I echo all of that, absolutely. I really want to thank you. You've done such an amazing job of articulating something that is exactly what I hope this episode would articulate, which is people do not understand there's a lot more afoot than just like irresponsible tech companies who aren't thinking about safety. I'm so grateful. I really mean it for your wisdom here and what you've helped people understand. And with the name of your book so that our audience can follow up and learn more about your work. It's got a ridiculously long title. It's The Politics and Ethics of Transhumanism, Techno-Human Evolution, and Advanced Capitalism. So open access there so anybody can download it for free. Well, everyone should read it. And thank you so much for coming on Your Undivided Attention. No worries. It's been an absolute pleasure to speak to you, Tristan. Thanks so much. Your Undivided Attention is produced by the Center for Humane Technology. We're a nonprofit working to catalyze a humane future.

Our senior producer is Julia Scott. Our executive producer is Josh Lash. Mixing on this episode by Jeff Sudakin with original music by Ryan and Hayes Holliday. and a special thanks to the whole Center for Humane Technology team for making this show possible. You can find transcripts from our interviews and bonus content on our sub stack and much more at humane tech.com. And if you liked this episode, we'd be truly grateful if you could rate us on Apple Podcasts or Spotify. It really makes a huge difference in helping others discover this podcast and join the movement for a more humane future. And if you made it all the way here, Let me give one more thank you to you for giving us your undivided attention.