The Invisible Technology Behind AI's Biggest Breakthroughs

[electronic music]

Hi, I'm Paul Rousseau,

Vice President at TSMC North America.

TSMC is the company

that manufactures the world's most advanced chips.

The ones that go into your phone,

the ones that answer your questions on AI,

the ones that might self-drive your car.

I have a PhD in applied physics,

and I spent 25 years at TSMC,

watching how the things we do get smaller and smaller,

while what they accomplish gets bigger and bigger.

Today, I'm going to explain how a chip works,

first to a kid, then to a college student,

then to someone who does this for a living.

Let's start small, fittingly.

[bright music]

Good morning, Elena. How you doing today?

Good. Yeah? How old are you?

I'm eight, and I just turned eight.

[Paul] You just turned eight? Happy birthday.

Thank you. So you know what, Elena?

I have a little present for you today.

It's a postage stamp. Oh.

But before I give it to you, you have to answer a riddle.

Okay, I love riddles.

What's smaller than a postage stamp,

but it's the smartest thing people have ever built?

A robot? A robot!

A robot could be very smart.

Bigger than a stamp though, right?

Oh yeah.

Elena, what did you play with this morning?

I played with my tablet.

You played with your tablet!

So you were this close

to touching the smartest thing people have ever built.

Let me show you something, Elena.

This is a board,

and this is a type of board you might find in your tablet.

And there's a bunch of chips on here.

[Paul] So is it this one? Uh-huh.

On this one? The crazy part is,

a chip looks a lot like a city.

Have you ever been to a big city?

Yeah, like this one!

I'll show you a model. This is a city.

Do you see it has streets?

Oh yeah, just like this one.

And buildings.

And the building has libraries which store

all of the information the chip needs to know.

Have you been to the library before?

Yeah, we pick up cool books.

You pick up cool books, right?

Libraries is where all the information is stored.

So the chip has its libraries.

It also has its buildings where it does its computations.

It does all the math. Oh.

And then it has messengers. The cars are messenger cars.

They're little zaps of electricities

that go around carrying the messages.

And the streets are like the teeny, teeny wires in the chip.

They're smaller than a human hair.

Wow!

And so when you hit the screen on your tablet,

it sends a message.

And what would happen, Elena,

if this city didn't have any traffic lights?

Oh, I think that everything will keep on going

and then we'll crash, and then it'll be a little crazy.

It'll be a mess.

So this city has billions of traffic lights.

And what they do is they control the messenger car

so they go exactly from the right building

to the other building carrying the right messages

and doing all the work.

Okay. Wow!

And that happens billions of times per second.

So it's going from here to there. So it's very, very busy.

Very, very busy, yeah. And that's how a chip works, Elena.

Wow!

So remember our board, Elena?

Yeah. Take another look at it.

Does it look like a city?

It does, but very, very small.

Very, very small.

I can see the streets,

and they're smaller than a piece of hair.

Right.

Now imagine we take this whole board,

and we shrink it to something like this.

That must be hard. This is already tiny, tiny, tiny.

Really, really small.

It's really, really small.

So have you ever dreamed, Elena, of creating a machine,

building a machine?

Oh yes, yes. Yeah?

What kind of machine would you build?

A robot.

[Paul] A robot. What kind of robot?

A robot that is pink and it can clean my room,

make me breakfast, and sort my socks.

So I want you to close your eyes for a second, Elena.

Okay.

You can't see anything now, right?

No.

But you can still hear me and you're thinking.

Yeah. Where are you thinking?

[Paul] In my brain. In my brain.

So what does this robot need?

A brain. A brain.

And what's that called?

A chip. Exactly. You got it.

Well, Elena, you solved my riddle.

[Paul] I did? Wow. You did.

That was fun. Good.

You get to keep the stamp.

Oh, thank you! And the chip.

Wow.

That way you can ask other people the same riddle.

Well, first things first, I'm going to tell them

that it's a super chip for the electronics,

not the chip that they eat.

Yeah, you want to make sure they don't eat that chip.

Yeah.

But then you can teach them what a real chip is.

Yeah.

It was a pleasure talking to you, Elena.

Thank you so much. You're welcome.

[bright music]

Hey, Joe. Great to meet you.

Nice to meet you too, Paul.

So I hear you're a student in electrical engineering.

Tell me a little bit about what you've been studying.

I study analog integrated circuits,

with a focus on semiconductors and MOSFET specifically.

So Joe, I'm curious,

tell me a little bit what you know about scaling.

How we've been on this trajectory of scaling transistors

and semiconductors for many years.

I know about Moore's Law.

The idea that every two years, I think,

we basically can double the amount of transistors

that we can fit into a given area.

So Joe, let's demonstrate.

I'm going to give you a piece of paper,

and I want you to fold it in half.

There you go. That's one generation of Moore's Law.

That wasn't that hard, was it?

Yeah, it was pretty easy.

Except we've been doing this for over 30 generations.

Okay. So keep folding, Joe.

I don't think I'm going to be able to fold this 30 times,

but I'll give it a shot.

Think about the smartphone I have.

It has the power of a supercomputer from the mid-'90s.

And not only that,

that supercomputer would've cost me $50 million.

I can get this for about $1,000.

That's the power of Moore's Law.

That's amazing. It is amazing.

Joe, you're slowing down. What's going on?

So I'm running out of the strength and the...

I mean, it's getting too thick to fold basically.

It's getting thick.

As you keep scaling, it gets harder and harder.

For example, the name of our latest technology in production

is Two Nanometers.

That's basically 10 atoms across.

Wow. I want to show you a wafer

that we built right here in the US.

This is a 300 millimeter wafer, and it's on four nanometers.

And if you look at it carefully,

you can see the individual chips in there.

So what do you think about the thickness

versus the size of this thing?

Well, you said it's 300 millimeters in diameter.

[Paul] Yeah.

[Joe] It looks to be about one millimeter thick, maybe?

Right.

That's the whole point is a lot of silicon

is two dimensional.

The actual thickness we use for circuits

is actually less than one.

That one millimeter is there for structure.

Right. In a real circuit,

it's actually thinner than this piece of paper.

I'm going to fold this.

And think about if I had a big chip before,

what happens when I stack it, right?

So I was trying to communicate from this end of the paper

to this end of the paper.

It's pretty long distance. Yeah.

What happens now when I fold it together?

The distance of that communication,

that interconnect is much smaller.

And so that means it's much faster,

and it's much more power efficient.

Right.

That's another way to keep scaling the compute

of our system by just adding more and more layers

in the third dimension. That's amazing.

So when you think about everything we talked about, Joe,

what excites you the most about the future?

Well, as an aspiring engineer,

I think the idea that we're really running up

against the laws of physics is very exciting.

Finding ways around kind of the universe that we live in,

in order to make better machines that can enhance our lives

is very interesting,

and I'm really excited to be entering this field.

Really happy to have you join the club,

because we need a lot of smart people like you.

Thank you, Joe. Thank you, Paul.

[bright music]

Welcome, Stephanie. How are you doing today?

I'm doing well. And you? Good, thank you.

So you're a professor at Penn State.

Can you describe what you do there?

Yeah, so I'm a professor of Material Science

and Engineering, and we make new materials

atomic layer by atomic layer.

I know you do a lot of your work on 2D materials.

That's continuing the scaling.

Can you talk a little bit about that?

Absolutely. Right now, your transistors are what?

[Paul] 10 atoms thick? Of that order? Exactly.

It turns out for a material like silicon,

there's a limit to how small you can make

a silicon transistor.

And if we want to keep packing more and more transistors

on a chip to make our chips more and more powerful,

we need to look at new materials.

And one material class

that we can think about are 2D materials.

These are materials that are literally

one or two layers of atoms thick.

So you can imagine if you go from a transistor

made of 10 atoms, to a transistor made of only one atom,

now maybe you can pack 10 times

as many transistors on your chip,

and now your chip can be 10 times as powerful.

So there's a lot of great stuff

that's happening on new material research.

But what about packaging?

How do you see things shifting in packaging these days?

Packaging is an interesting topic.

If you're able to, instead of making one big chip

that has the compute here and the memory and whatever,

if you can make all these different little chiplets,

you can stack them up,

you can package them inside one package.

Now, in addition to being able to go in the third dimension,

you can also design each chiplet to do its own job

as efficiently as possible.

Yeah, I'm so glad you mentioned that,

because here's an example.

Here's one of our CoWoS cell, latest technology.

So I have my main silicon, my HBM on the sides,

and then on the top and the bottom, I have my I/O chips

that communicate with the outside world.

So now I can have each piece of silicon optimized exactly

for what it needs to do.

And that also means that you reduce your energy usage.

And on the consumer side,

it also means that my cell phone battery lasts much longer.

Yes, exactly. You can do so much more.

Exactly.

Going back to Penn State with your students,

what are you going to be working on now?

What are we going to be working on?

I mean, we're going to keep designing new materials

for the future.

We're going to keep working on 2D materials,

trying to understand...

One of the major problems right now in 2D materials

is actually it's called the doping problem.

So how can we, in one material, change its properties

just by adding tiny amounts of doping materials?

So this is actually a real challenge in 2D materials,

and it's a challenge we need to solve

if we want to use these materials for advanced transistors.

It sounds like high-tech cooking, right?

How do you add just the right amount of spice

to make it take?

It exactly is like that, right? So when you want to...

Say you want to make chocolate chip cookies, what do you do?

You go to the store, you buy the bag of chocolate chips,

you follow the recipe on the back.

We can do the same thing with material science.

If we want to make a known material,

we can look in the literature, find its recipe,

follow the recipe, get the material.

But if you want to bake something new, what do you do?

Well, you try some things.

Maybe you start from a recipe you know,

you change the thing here, you change it there,

and eventually, maybe you have a new better recipe

for chocolate chip cookies,

or a totally different kind of cookie.

And that's exactly what we're doing, right?

We start from a known recipe,

we change things a little bit here and there,

and see if we can't make our materials even better.

So what excites you most about where we're headed?

I think the thing I'm most excited about

is actually the ability to do this

really interdisciplinary work,

where people who have different expertise work together

to try to solve a big problem.

And this is the only way that we're going to be able

to solve the big problems that we have nowadays.

Yeah, yeah, beautifully said. We have a name for that.

We call it STCO, system technology co-optimization.

Which basically means that people can't go off

and design everything in their own sandbox.

They have to work together.

I think one of the things that I'm really excited about

is the ability to use AI and machine learning

to accelerate materials development.

Right now, we get so much data from our experiments,

and I'm sure that you have orders of magnitude more data

than we do, and trying to spot that needle in the haystack

for a human is very difficult.

It's too much data for one person to think about.

But artificial intelligence,

it doesn't have those same limitations

at looking at huge data sets, so it can look at all the data

and find the signal that we're really interested in

to solve some of these problems.

Well, thank you, Stephanie.

It's been a real pleasure talking to you,

and I want to wish you and your students a lot of luck,

and hope you find many needles in those haystacks.

Thank you so much.

I hope you enjoyed learning about

how some of the teeniest technology in the world

is powering the biggest advances in artificial intelligence.

It's a great time to be an engineer, or to become one,

because there's plenty of tough problems to solve,

and we need to solve them together.

It's also a great time to be alive, because with AI,

nothing has to be done the way it was done before.

All you need is an idea, and an AI platform

running on chips designed by our customers

that we built with atomic precision.

And here's the part I love.

With AI, there's a virtuous cycle at work.

AI is now helping us design the next generation of chips,

which will power the next generation of AI.

So somewhere out there, there's a chip we haven't built yet,

doing things we haven't imagined yet,

maybe designed by a kid

who's holding a postage stamp right now.

[bright music]