Inside the 21st Century’s Manhattan Project: The Race to AI Superintelligence

Kevin, thank you so much for being here.

Katie, thank you for having me.

Nice to see you again.

So you and Casey recently announced

that you were launching a new media company,

called Machine Gods Media,

not to be mistaken for your podcast of the same name

minus the media.

Machine Gods the podcast,

Machine Gods Media, the company, correct?

Yeah, a name so nice, we used it twice.

There you go.

So when you first announced that you'd be leaving The Times,

you described your vision for what you wanted to do

as one that, quote, Takes AI progress seriously,

is clear-eyed about the capabilities

and risks of powerful AI systems,

and tries to empower and entertain people

in the face of radical uncertainty.

I'm curious, as you look at sort of the landscape

of tech coverage, of AI coverage,

what's missing from the reporting and commentary

that you and Casey feel like you can address

that you want to address with the new show?

We just feel like it was high time

that two men had a place to talk about AI.

I've been saying this for years.

I want more men.

No, look, I think there are obviously

no shortage of podcasts, and YouTube shows,

and mainstream media coverage of AI.

It's the biggest story in the world right now.

But when Casey and I looked out at the media landscape,

we saw some issues.

One was, there are people who just are getting very famous

and having a lot of success saying that all this AI stuff

that's going on is fake, it's hype,

it's a giant financial bubble.

No one is using these tools.

They're not going to have any impact on the economy.

You know, OpenAI is going to go bankrupt,

Anthropic's going to go bankrupt.

But basically this is sort of a genre of popular criticism

and it's not just a few people.

This is now, I hear this from friends of mine

who don't pay close attention to tech news,

and just assume that what's going on is just fleeting,

and trivial, and that it will all

sort of go back to normal soon.

But there's another genre of AI coverage

that is purely hype.

It's look at the 17 amazing ways

that the new version of Claude

can supercharge your enterprise SaaS business.

And you can go on LinkedIn

and just see like example after example

of people who are just purely excited about this technology

and don't really care to talk about the risks.

And we both thought there's like a large gap in the middle

for what Casey calls AI realism,

which is basically this idea that you can take AI seriously,

acknowledge that the tools are powerful and impressive,

and in many cases dangerous,

and that you can help people understand that

and demystify this area

without sort of slipping into boosterism

and that you can also have a good time while you do it.

We don't want this to be just a dour take on AI doom.

We want to actually give people a good experience

and have a good time.

I have to ask, the show, you announced recently,

is being published in partnership with NPR.

Why NPR?

Why was that the right partner?

A bunch of reasons.

Both Casey and I are big fans of NPR.

We like the fact that they have a broad independent reach

and mandate.

We like the fact that they're going to let us own the show,

and make the creative decisions,

and it will be a distribution partnership

rather than like a full acquisition.

So we will still have some operating distance.

We think this is a really critical time

and a really important story.

And we like the idea

that people might be in their cars

just listening to their local NPR member station

and happen on our podcast.

And maybe that's going to be someone who works in policy,

or maybe that's going to be someone

who is involved in local government,

maybe that's going to be someone

who has a very different point of view on AI.

We don't just want to have the sort of opt-in,

self-selected tech audience listen to us.

Now I am going to ask you a gauche question.

A Bloomberg report recently said

you both were fielding offers of up to $5 million

for the show.

I have to admit that link did travel

through Wired's slack at rapid clip.

[Kevin and Katie laugh]

It's a startling sum of money.

Kevin, how much you making here?

It's not $5 million, I'll tell you that.

It's not?

Is it more?

It's more? [Kevin laughs]

It's more than $5 million?

[Kevin] So much more, Katie.

So much more.

I mean, look, they made us a good offer.

We could have gotten more money elsewhere.

I mean, that's kind of what I was wondering.

When I heard NPR, I thought to myself,

there's no way NPR is giving those guys $5 million,

with all due respect to NPR.

It is not traditionally where people go

to get rich in media is to public radio.

They made us a very good offer.

We loved their their new chief content officer,

Nadine Zylstra.

She's just a total force of nature

and we're very excited to work with her.

And we just thought they had a lot of things to offer us,

beyond just money, like their distribution on radio.

People don't realize how big radio still is.

The reach of radio, and especially public radio,

is still quite large.

We've had this show, Hard Fork, for the last four years.

We built up a pretty good size audience,

but The Times owned that show.

Yeah.

Owns the feed.

So we are looking to grow our show as quickly as possible,

and we just thought that the combination

of NPR's like commitment to journalistic excellence,

their long history, their wide distribution,

and their investment in helping us grow the show

was the right combination of factors.

This brings me to one more question

that I'm so curious about.

The idea that a great reporter, a great commentator,

can spend time somewhere

that's sort of, quote unquote, traditional,

like the New York Times.

They can build a brand, and then they realize

that they can just go do it themselves,

and they don't actually need that institution anymore

to exist in the world as talent,

and to make, often, a lot more money than they would

in traditional media.

What's your take on that?

I don't have anything bad to say about the New York Times.

I had nine wonderful years there.

It was my second stint at The Times.

So I've spent like the vast majority of my career

at the New York Times

and inside these big media institutions.

I do think we are entering this moment where,

at least for some portion of the audience,

they want to connect with individuals

more than institutions.

We have just seen this in wave after wave.

I am not doing this for ideological reasons.

I'm doing this because I thought

it was a really exciting opportunity.

But I do think that organizations that want to retain

and attract very talented people

will just need to be more flexible

about the kinds of arrangements.

Some people aren't going to want to give up their Substacks

and go inside a media institution.

Some people aren't going to want to

sort of have all of their work published

by sort of one publication.

They'll want to do some things for one place

and a few things for another place.

And so I think there are some media organizations

that are starting to experiment with different,

more flexible ways of, I don't know,

bringing people in part way,

or having them sort of maintain their independent operation,

but also contribute on an ongoing basis.

I think there are a lot of ways this can work,

but I think it all has to start from a recognition

that like the journalistic career path,

where you like go in in the mail room,

and you work your way up,

and you spend 25 years at the same employer,

and you eventually become an editor,

and then a manager of editors,

like that has broken down.

And that is regrettable.

I don't think that's a good thing that it's broken down,

but it has broken down.

And so I think institutions should grapple with the fact

that there's now a generation of media entrepreneurs

who just don't really find what they have to offer

all that appealing.

When you think about the talent piece of that,

when you think about AI, are you optimistic about journalism

and the industry of journalism?

I am very optimistic about the application of AI

to journalism.

Like that is one place where I have wanted to do

more experiments, not with having AI write for me,

or do all my reporting,

but like ways of extending journalism using AI.

What's an example of an experiment you would love to do?

I have colleagues, former colleagues at The Times,

who have done incredible sort of document analysis

on a scale.

Sure.

That wouldn't have been possible before.

Using satellite imagery to tell

whether a munitions factory has moved,

or something like that.

That's the kind of thing that I don't do much

in my own life,

but that I would like to see other organizations trying,

because I think that's really cool.

I have used AI to research, and edit,

and improve my own work for months now.

I have found that very helpful.

I think the caliber of my work is better.

And I would love to see more institutions in media

experimenting with using these tools

to improve the output of their journalists,

not just like filling their websites with slop,

but like actually helping these be tools

to make journalists better.

I want to talk about your book.

So The AGI Chronicles,

which I have read, it is hugely compelling.

And I'm curious, when did you decide,

when did you have that moment where you said,

This is a book.

Like I'm going to commit years of my life and my career

because there's a book here.

What was that moment for you,

when you realized that this was a, you know,

a big fucking deal?

I know exactly what it was.

It was early last year, 2025,

and I was in the car on the Bay Bridge, stuck in traffic,

and I was sort of zooming around from like thing to thing.

And it just kind of hit me like an epiphany.

It was like, I have been following this story

in all the incremental detail for years now.

I've interviewed all the major AI researchers, and CEOs,

spent time with the papers.

I've gone to all of the companies

and reported on what they're doing.

But I sort of realized that there was this larger story

that I was missing, that I hadn't really zoomed out

and tried to take a more panoramic view

of something that was just weird.

Like it was, it's a weird story.

And I felt, living in the Bay Area,

being immersed in San Francisco tech culture,

I had kind of gotten acclimated to that.

And it no longer seemed as strange to me

that there were these companies

racing to build the machine super intelligence

that could either save or destroy humanity.

And I feel sometimes like I am in Los Alamos, New Mexico

in 1943 when like the Manhattan Project rolls into town,

and I've just kind of got like my lawn chair,

and I'm just kind of looking at the trucks rolling in

and trying to make sense of what's happening.

But of course, all of this is going to be important.

I believe that this technology is important,

and that the people and the companies who built it

will be important historically.

And so when I thought about like,

who is actually doing the work

of like writing all this down?

It was nobody.

Nobody was doing it.

And I just felt like, I think it would be a tragedy

if all this just kind of disappeared

in a bunch of Signal messages and Slacks that auto delete.

And if we just sort of end up

with no durable historical record

of this really strange decade in AI,

when things went from not working at all

to like threatening the future of humanity.

So that was the job I tried to do.

I spent about a year reporting and writing.

I talked to more than 150 people.

I should have probably taken more time,

because it was very hard and intense.

But I think what came out of it

was like exactly what I tried to do,

it was like, make an artifact that people,

and future AI systems can look back at to say,

Here is how this happened.

Here's who made it happen.

Here were the key decisions and moments along the way.

And the book follows three key companies,

OpenAI, Anthropic and Google

in their sort of pursuit of this technology.

I'm curious, what were sort of your big picture learnings

about those companies,

and sort of the key differences between them

that you think it's important for people to know?

Yeah, I mean, the companies are very different

from one another,

both in the sort of makeup of their personnel

and also in their ambitions.

Let's start with Google, because they're the oldest.

They have had, for decades now,

an advanced AI research effort,

but they were pioneers in AI.

They developed the transformer, which is the T in ChatGPT,

the sort of foundational technology

that all of this other stuff rests on.

And then there were these two guys,

Elon Musk and Sam Altman, who got very worried

about how well they were doing,

about the fact that Google and DeepMind were racing ahead,

and they decided to start OpenAI to.

So funny to imagine that now.

[Kevin laughs]

I mean, it's wild and we have the emails.

Like it's all there in the record,

where they're basically like,

we have to start a lab that's going to sort of beat them,

or at least challenge them,

so that they don't kind of run away with the whole game.

So they start OpenAI, and they do a couple years of that,

and then this guy at OpenAI, Dario Amodei,

he takes six of his colleagues,

and they leave and start Anthropic,

basically to make sure that OpenAI

doesn't get to the critical threshold of AGI first.

So the whole industry sort of spawned out of itself.

These people all used to work together,

and now they run these companies

that are, in some ways, mortal enemies.

Like I was shocked.

This was actually my biggest surprise

was I thought this was more like Coke and Pepsi.

This is not a buddy-buddy industry.

This is like a blood feud.

From all of the reporting that you did, who do you trust?

Who do you trust with our future

in the context of artificial intelligence?

I don't trust any single person.

None of them.

But I think what I learned

through reporting this book is like,

these are people.

They are flawed, they are fallible.

Their motives are never 100% pure.

Some of these people are quite nice.

Some of them are very thoughtful.

I think we have, in some ways, gotten very lucky

with the people who are running these AI companies

who I think are, on the whole,

much better suited to build powerful technology

and release it into the world

than like the social media barons were.

I think they are.

You see a marked difference

between sort of the Facebook era and this AI era?

Oh yeah, I mean, I think, for one,

they are just way less naive.

The social media guys came in and they said,

We're going to change the world.

We're going to free communication

from sort of the bottlenecks that hold it back.

We're going to distribute the benefits of technology

to everyone.

And they really didn't start thinking about the problems

until they were being questioned in front of Congress.

I mean, I will say, some of these AI guys

talk a lot about saving the world.

They talk a lot about how great this will be for humanity.

You know what I mean?

If you go back and look

at like the founding emails of OpenAI,

they have been very consistent that they think

this is a potentially very dangerous technology.

Now, they're racing toward it,

so maybe their words don't mean that much,

but I think you can't accuse them of being naive,

because they just have such a long track record,

all of them, of saying that,

Yeah, this could be great for humanity,

but it's not a given that it will be,

and we need to build it really carefully and thoughtfully

to make sure that it's actually going to turn out well

for us.

What kind of responsibility

do you think falls on their shoulders

in the context of AI safety?

Yeah, and this was something that I've asked

all of them about at various points.

Like, why don't you just stop?

Why don't you just slow down?

Yeah, why, yeah.

Why do you get up every day and try to make these systems

more powerful if you're worried it could end the world

in some cases?

Some of them, including Dario and Sam,

genuinely believe that this technology is inevitable.

The recipe for AI is not that hard.

You get a lot of compute, you get a lot of data,

you build the right scaffolding,

and kind of grow the model

in this organic process of stochastic gradient descent

and reinforcement learning,

and out comes a super intelligent model.

But that is their sincere belief,

that it's not that hard to build this stuff

if you understand the basics,

and that because it's not that hard, someone will do it.

And whether that someone is a US AI company,

or a Chinese AI company, or a terrorist group,

or an academic research lab, someone will do this.

And so it is in the best interest of humanity

for someone who thinks a lot about safety

to be the first to get there,

because they can sort of set standards

for the rest of the industry.

Now, I don't know if I fully buy that,

but that is their logic,

and that is the reason that they feel sort of validated

getting up and doing this every day.

The book ends in an interesting way,

and I have a couple questions about that.

I was just looking at the last page

before I came in to do this interview,

and it ends essentially, without giving away any spoilers,

you essentially saying,

Really hope these guys get it right.

Really hope that they slow down

so that I can sort of keep living my life

the way I live it now.

It's funny timing that the book is coming out

right as these very acute and severe conversations

around AI safety are happening.

And it feels like every other day

a company is disclosing some new breach,

something that went wrong with one of their models.

And how do you think about that?

How scared are you right now, Kevin Roose?

I am actually feeling quite hopeful right now,

relative to where I was a few months ago.

And it's largely because

we are now having this conversation.

There have been people, including many of the people

I spoke to for the book,

who have been worried about runaway AI, rogue AI,

possible AI takeover for 10, 15 years,

who have been trying to sound the alarm about this,

and no one believed them.

Outside their little bubble of AI safety people,

everyone was sort of like, Yeah, yeah, yeah.

And now I feel like we have finally reached this point

where this stuff is inside the Overton window.

I think that some of what I see on social media,

and some of what I see in the news media,

is really hyperbolic and is really dramatic.

And I'm not saying that the stakes aren't dramatic.

I think my assessment of the situation, though,

is that, if something does go horribly wrong

with artificial intelligence,

it will be much more boring than we might think,

and it will have more to do with human error

than maybe we are sometimes attributing.

That's what I think.

Yeah, I totally understand that.

And to be clear, I'm not saying I know exactly the risks

that we should be most afraid of.

[Katie] Yeah.

I just think there's this whole category of risk

that has kind of been written off, or downplayed,

or just those are just those weirdos in Berkeley

talking about it,

that we can now sort of have conversations about.

People are worried, they're taking this seriously,

and I think we really have a window here

where we can actually make some changes

or take some steps to make sure that this goes better

for humanity.

I don't think that was possible a couple of months ago.

So I think this is, that's why I'm feeling more hopeful,

because even though I think objectively,

like the models are getting worse, and scarier,

and more dangerous, we are also much better positioned

to recognize, and talk about,

and perhaps prevent those risks.

I think that you, in the book,

in that last page that I was just looking at,

I don't want to ascribe an emotion to you,

but I felt fear there from you,

and a reluctance to see your life change too quickly.

And I think that that is something

that is very much universal.

Absolutely, and I think this is where

the people inside the AI bubble do not understand the world.

I think the people at the AI companies,

building this technology,

are generally people who enjoy the prospect

of large unannounced social change.

Large unannounced social change.

Hard to imagine a bigger nightmare for me, personally.

[Katie laughs]

Like they love when things get weird.

That's part of why they move to San Francisco.

They want to live in the future.

The prospect of radical upheaval does not scare them.

Doesn't that seem like a huge problem to you?

Yes, because normal people don't think like that.

Like, normal people want to live a life

that is recognizable to them.

They want their kids to grow up

in a society that resembles the one that they grew up in.

We don't manage change very well as a society.

We never have.

But I think this is why I wish

that there had been more people involved,

more types of people involved in the critical conversations

around this technology, because, as I said,

50 people in San Francisco, give or take,

made all of the relevant decisions,

and they are not a representative sample.

They are very weird.

They see the world differently.

They have a higher tolerance for change than most people.

And I think it led to them making some decisions

that we can't really take back now.

I mean, do you think that if, two years ago,

we had had more philosophers, more artists,

more creatives, more journalists, lawyers,

whoever it may be involved in those conversations,

that it really would have moved the needle

when there is so much money

on the other side of that conversation?

I guess what I'm saying is that sure,

it's all well and good for a broader coalition

to be having those conversations,

but Greg Brockman's donations to the Trump administration

get him that phone call, right?

They go to the inauguration.

They have the president's phone number.

I mean, to be fair, a lot of journalists

have that phone number too, but you know what I mean, right?

The access to the decision makers

is bestowed upon those 50 people in San Francisco

by virtue of their wealth and their power.

I think it can help at the margins.

And I'll give you an example.

One of the people I write about

at some length in the book is Amanda Askell.

She is a longtime employee at Anthropic

and was at OpenAI before that.

You just had a story in Wired

about searching for the most powerful woman

in Silicon Valley.

I think she's got to be up there in the top two or three.

She has been in charge of Claude's character.

This is her.

They call her the Claude mother at Anthropic.

She's a virtue ethicist.

She has a PhD in philosophy.

She went into AI specifically to think about this question

of what should a good AI system do?

How should it act?

What values should it represent?

How should it decline to do certain things

or volunteer to do certain things?

How can you instill something like virtue ethics

in a chatbot?

And this was a very fringe area of research.

She was, to my knowledge, the first person ever

to do this kind of work inside one of these AI companies.

And it has resulted in them having

a really sophisticated way of thinking

about the morality and the ethics of Claude.

She now has a whole team.

They have many people that are devoted to this,

and I think it has probably made Claude not just safer

and better behaved,

but has also kind of inspired other labs

to hire their own philosophers

and come up with their own ways of training their AIs

for something like moral goodness.

So yeah, I think there's a really strong argument

for having lots of people from lots of different disciplines

engaging with this technology,

because it should not just be engineers doing this.

As someone with a Bachelor of Arts degree in philosophy,

I have to say, it is boom time for my people.

We are coming for planet Earth,

and we are about to shake the foundations

like you've never seen.

You never thought a philosophy major

[Katie and Kevin laugh]

would be determining the outcome of planet Earth.

I mean, there was a lot in the book, to me, that was,

as we've been talking about, troubling, right?

That is troubling, that is scary.

What stands out to you that is hopeful?

When you think about the book,

when you think about the technology,

I mean, aside from the fact that we may actually

make some meaningful progress towards regulation,

was there something you discovered in your reporting

that made you feel optimistic?

Yeah, a lot of the optimism that I feel around this stuff

has to do with science and medicine.

I lost my father to cancer.

I know lots of other people who have lost loved ones

to rare diseases that have not been cured,

not because we sort of,

not because we lack the ingenuity,

but because it's just a question of resources and manpower.

And like, I think that AI can do incredible things

for people who suffer from disease.

I don't think that's all marketing BS.

I was very moved,

there's a story in the book about AlphaFold,

the DeepMind protein folding AI system

that won the Nobel Prize.

And there's this sort of incredibly touching moment

where a mother of a kid who has a rare disease,

a life-threatening genetic condition,

writes to the DeepMind researchers

after this AlphaFold breakthrough

and asks them like, Could this help my kid?

And they have to give her the honest answer,

which is probably not, because this stuff takes time.

You have to get it through clinical trials.

Even if you have these amazing breakthroughs in science,

like you have to design the drugs,

you have to test the drugs,

you have to get the drugs approved.

Like it can be a decade before these things actually make it

to saving people's lives.

But like, I want that to happen faster.

I want there to be more AI designed drugs.

I want us to get them to market quickly.

I don't want to have to suffer,

have people suffer from the same diseases

that killed previous generations.

So that's where I feel a lot of optimism right now.

Ask me in two weeks, maybe it'll be something else,

but right now that's where I'm feeling good.

Depends on how your P-doom is looking behind you.

Now, I'm curious about your own process with AI.

You have published three books prior to this one.

Your first was published in 2009,

well before any of this technology was available.

How did it change the book writing process for you?

I know you're very open with how you use AI,

sometimes to much controversy among your journalistic peers,

but tell us about how you used it for the book.

Yeah, there's a whole section in the beginning of the book

about how I did and did not use AI,

because I thought it was important,

first and foremost, to be transparent.

Like I don't think this business

of like people writing books,

and then people check it in Pangram,

and it comes up as AI generated,

then they lose their book contract.

You know what, Kevin,

I'm going to be transparent with you.

I'm going to do you a favor if you haven't already done it.

I ran your book through Pangram.

How did it do?

I was like, I really like this guy,

but like if we have a scoop here, it's a scoop.

But the book is written by a human being.

Presumably by you.

Yeah, it was written

[Katie laughs]

by a very tired, overworked, under-slept human being.

I did also have a researcher help me with this, Jasmine Sun,

and had a bunch of great human editors at FSG.

So it was, the book is fully human written,

but how did you use the tech?

Yeah, a lot.

Constantly.

So I had a giant notebook on NotebookLM

filled with all of my research materials,

interview transcripts, magazine articles, academic papers.

And the most basic way I would use that

is just to query it about things that I needed to know.

So like give me all of the stories

that I've heard about the pre-training of GPT-4,

and then it would pop back a list,

sort of like a supercharged search function.

I also did it to help with some sort of reporting tasks,

like who would the three people have been in the room

when this decision was made

and what are their contact details?

It helped with organizing my notes, with fact checking.

Actually at the end, I had a human fact checker,

but I also had had an AI swarm doing fact checks

and catching some things that honestly,

neither I or the human fact checker had caught.

Oh, wow.

So we got to fix those in the manuscript.

And then I used it for editorial feedback.

I have a council of Claudes, I call it,

which is sort of my my team of Claudes

that are assigned to different personalities

and vantage points.

So I have one who's like a hardcore LLM skeptic

who goes through the draft and tells me,

Oh, this is what I object to.

This is what I object to.

You're anthropomorphizing here.

It's not really thinking here.

I have one that's like a sort of Kurzweil-type futurist

that wants to like expand the vision of the future

in the book.

So they're like, some of this was just slop

and probably wasted more time than it saved.

But like there were a couple things

that the Council of Claudes said to me

where I was like, Oh yeah, that's a good point.

I should go back and revise that.

And how do you think about the premium

on human generated writing?

I'm curious about this.

Like do you think in two years, three years, five years,

the average person will care that Kevin wrote this book

as opposed to an AI writing that book?

Do you think it matters to people?

I do.

I think it matters a lot to people.

And I know this because they've done studies where,

when people read a sample of writing

that is generated by AI,

but they don't know it's generated by AI,

they give it very high marks.

They prefer it to human written text in a blinded test.

But then the minute you tell them

this was generated by AI, they hate it.

Their approval of it plummets.

I think this has a lot to do with human psychology.

We like to think that people work really hard

to make something for us.

Yeah.

[Kevin] That it represents their authentic view.

Yeah.

So I think that people will still continue to be offended

when they find out that their favorite writer has used AI

to generate their latest book or essay or whatever.

But I think that's basically only if they detect it,

only if they can tell.

And I don't think they can tell

or will be able to for much longer.

So I think as long as Pangram exists,

that will probably be a useful tool.

But I also think that, yeah, in the abstract,

people just, they just don't know the difference.

And when you tell them the difference,

they care, but before that they don't.

That's really interesting.

I mean, it is the psychological value

of wanting to know that someone's fingerprints

were on something,

and wanting to know that you are getting their voice

and not the voice of Claude.

Totally.

I think it's important for writers too

to like show their process for this reason,

to talk about,

and maybe like literally film themselves working

so that people can prove,

like you can sort of connect with the laborer

involved in making something.

I think that's going to be important for humans

of all creative stripes.

Is that going to be a social video series

that you and Casey come up with?

Yeah, I'm streaming myself writing

16 hours a day on Twitch. [Katie laughs]

Go check it out.

Sounds like hell.

Kevin, congratulations on the book.

Thank you so much for being here.

This was fascinating, really, really, truly.

Thank you, Katie, you're the best.

[electronic chime warbles and swells]