チップ時代へようこそ

Hey, your open finance prototype works. Now, make it production ready with MasterCard APIs. Access permission financial data and scale on infrastructure designed to meet the security and reliability standards that financial services demand. Head to developer.mastercard.com. Hello and welcome to the Stack Overflow podcast, a place to talk all things software and technology. I'm your host, Ryan Donovan, and today we are talking about life in the chip age. And my guest for that topic is Rakesh Kumar, who is a professor of computer engineering at Embraer-Machampaign. And he's author of the book, The Chip Age. So welcome to the show, Rakesh. Thank you. Thanks for having me. So before we get started talking about chips, let's get a little flyover of how you got started in software and technology. Well, I grew up in a small town in India, the kind of town where you know everybody, everybody knows you.

Then for college, I went to this college, which you get into after appearing in this really hard national exam. Yeah. And then I picked computer science, not because I had any experience with computers or any interest in computers, because apparently that's what you did if you did really well at this exam. So I go to college and then I find that I like the computer organization and design interesting. So that was interesting to me. I became a junior in year 2000. March of that year, there was this dot-com bust. So I'm like, hey, this is a good time to apply for graduate school. So I begrudgingly applied for graduate school, and I picked chip design largely because, you know, I was interested in computer organization design. And then the more I learned about chips, the more fascinating I found them, you know, how complex they are, how intricate they are, how dynamic the field is. And I was hooked. And here we are. Well, it's a series of happy accidents, sounds like.

Certainly, yeah. I know a lot of the folks listening probably used to running a lot of software on chips and hear a lot about different chips. But I want to start with the sort of most basic question. What is a chip? I think of a chip as this fingernail-sized or postage stamp-sized device. Think of it as an electronic brain. It consists of a large number of microscopic switches. These switches can manipulate data or perform calculations at lightning speed. So, you know, at its most basic, that's what a chip is. It essentially is a substrate that consists of millions, billions, or in some cases now trillions of these microscopic switches performing, you know, processing data and doing calculations at lightning speed. And because you can map really any piece of software as ultimately a set of manipulations of the data or a set of calculations that you're doing, you are essentially using chips to run all your software. And for most people thinking about computers, you have the marquee chips to get all the attention, the GPUs, the CPUs, memory chips.

But there's chips almost everywhere today, right? That's right. So, in fact, the chips that are the most popular in the world are the ones that you may have heard the least about. So, there are these microcontroller chips, you know, these tiny chips that in the microphone I'm using for this podcast and the one that you are likely using. There are chips that are powering up this laptop that I'm using for this podcast. They're just power switches, right? There are chips that sit in the gateways and the routers and the switches that are essentially transmitting all this information that we are exchanging right now. So often, you know, we think a lot about, you know, these memory chips and these GPUs and these CPUs, but there are so many of these chips we use everywhere for powering systems, for communication, for, you know, basic tasks, chips that make your toys go, right? So indeed, there are just so many different kinds of chips. And, you know, the chip manufacturing has a longer history than I think most people understand, but still a pretty short history in the grand scheme of things.

How did we get to sort of a ubiquitous state of chips in everything? So maybe they go back to when chips were invented and then what happened right afterwards. So these microscopic switches that I talked about, the switch was invented sort of toward the end of 40s. Actually, one of the inventors was John Barbina. And I'm incidentally the John Bardeen sort of scholar at my university. So I take that sort of call it honor distinction very proudly. Toward the end of 50s, the chip was invented. So again, going back to what a chip is, it's a collection of these switches. So right after the switches were invented, people knew that it would be a great idea to be able to put a lot of these switches onto a single substrate. So chips are not some kind of accidental sort of discovery. Everybody was waiting for chips to happen. So end of 50s, chips happened. And then incidentally, early 60s, the United States government was trying to get to the moon and was trying to build intercontinental ballistic missiles. And the folks in the government realized that, well, only if we could have something that makes decisions very quickly and something that's small, then wouldn't that be great?

They decided to funnel in a lot of money into this invention that allowed this thing to sort of explode. And, you know, it's a capital-intensive industry, so government essentially provided some of the initial capital, both as an investor as well as an early customer. Now, the reason I'm mentioning all of this is that during its, let's call it the initial days, you know, when things are hard and you need capital and you need to survive, there were these happy coincidences because of which this sort of happened. But afterwards, people figured out how to keep increasing the number of transistors or these switches on the chip, really at a rate of, you know, they figured out how to double this every two years. So there was this guy, Gordon Moore, who made this observation. And then the entire industry decided that, well, it's not just an observation, we'll make it a goal. So the industry decided that every two years will double the number of these switches on the chip, which led to these breathtaking advances in terms of performance, breathtaking reductions in cost for capability.

And fundamentally, the way I see it, it's this exponential that you got on. This is what allowed us to have chips everywhere. Yeah, that's the Moore's Law, right? Gordon Moore eventually founded Intel. That's right. So, yeah, when he made this observation in 1965, he was already with Intel. I find that this observation was made with three points. So today, if a student comes to me and tries to draw a graph with just three points, I'm like, you know, that's not a very scientific method, right? That's right, G+. Exactly. So he had these three points. He uses these three points to make this observation that the number of transistors is sort of doubling every two years. And then, you know, it changes the course of history. Where we are now after, you know, 60 years almost of Moore's Law, how many transistors do we have in our chip? Broadly speaking, chips that you find in servers are a lot bigger than the chips you find in some electronic toy, right? So it depends on, of course, the size of the chip. But the largest chips today have trillions of transistors. The chips that you and I often mostly talk about, you know, the GPUs and CPUs now have billions of transistors.

So that's the scale that we are talking about. I don't want to make an observation about Moore's law since we are on this topic. You know, some people believe that Moore's law has sort of slowed down, that we are no longer miniaturizing these transistors at the rate that we did, and therefore we can no longer put in twice-as-mean transistors in the same area as before. At some level, it's true, but we as humanity have also been so creative that we have started growing in the Z dimension. Right. So once you start growing in the Z dimension, then when you look top down, then the total number of transistors per millimeter square is still increasing at twice every two years. In fact by some measures it increasing at a faster rate So in that sense Moore law is still continuing in 2026 And another of the tricks that folks have used to increase the speed and power of chips has been overclocking right Speeding up the internal clock. And I think that's run into another factor of chips is the Denard scaling, right?

That's right. So, you know, how do you increase the capability of a chip? Well, one, pack as many transistors or disk switches as you can into an area. So that's one way to increase the capability of a chip. Another way to increase the capability of the chip, as you said, is, well, maybe each switch switches a lot faster. And that's where this overclocking comes in, that if you can make these switches, you know, I'm being a reductionist a little bit, but broadly speaking, if you can have these switches switch faster or do tasks a lot faster, then, well, you have a lot more capability. Overclocking runs into this problem of power. So broadly speaking, you know, power depends on voltage and the frequency at which you are running the chip. So the more you overclock, the more power you sort of dissipate. And in fact, it's not even linear. It's super linear. You know, it goes into a little bit of electrical engineering. But when you want to overclock, often you also have to increase the voltage because otherwise you may get wrong answers. So the power increases sort of super linearly with the overclocking rate, if you will.

And then you start, you know, getting worried about these questions of, is my chip going to melt? So, but yeah, certainly overclocking is a famously infamous hobby of many folks using computers. Yeah, yeah. I mean, I remember when it was just people trying to get their Doom games to run faster, but. That's right. That's right. Yeah. So I've seen, I've actually seen demos of people frying an egg. I've seen demos of people using liquid nitrogen. Yeah, it's pretty creative what people can do with overclocking. Right. Yeah, creative by necessity. So with the chips sort of fitting more and more transistors on it and getting all these new technologies and creativity, the manufacturing process is incredibly complicated now. I think there was a line in there. You said the manufacturing process uses every element on the periodic table except the radioactive ones. Is that true? That's right.

It's pretty, you know, at one level it's funny, but at another level this is astounding. I mean, imagine building something so small having so much impact on the world that is so hard to build. A single switch today is 10 times smaller than a virus, single switch. In one day, we are producing quadrillions of transistors, quadrillions, one day. Then a single chip factory could cost $30 billion. So by far, it's the most expensive factory that humanity builds. A single chip is traversing borders, you know, it's traversing 25,000 plus miles before it ends up into a product. So, you know, the scale, the complexity of doing all of this is just so astounding. We are manipulating these transistors where some of the parts of the transistor are only a few atoms thick. So you're literally performing manufacturing where you're manipulating things at atomic scales. You know, one of the things I say in the book is how this is the hardest undertaking that the humanity has at this point.

So how do you manufacture a transistor that's smaller than a virus? It's magic. I mean, not literally, right? So we'll start with this silicon wafer. So a silicon wafer, essentially, think of it as a disk of crystalline silicon, extremely pure sort of disk. Now, this disk itself is built, at the end of the day, using sand. So you're finding quartz or sand, but you're melting sand to eventually creating a cylinder of this silicon out of it. And then you're slicing the cylinder to create a bunch of these disks. Once you have these disks, now you need to imprint whatever chip design that you have in mind onto this silicon. So what you do is use these masks. Think of them as sort of patterns that you create off the chip design. And then you shine light through these patterns onto your wafers. Before you do that, you have polished the wafer with all these different chemicals that are photosensitive. So when you shine light through these masks onto the wafers, your chip design effectively gets imprinted on these wafers.

So you're following this process that you call photolithography. So it's sort of, you know, it's some kind of mixture of the process that was developed for just regular photography, where you have positives and negatives and you're trying to imprint the reality, if you will, onto a piece of paper using photosensitive material. You're using a similar process here. The amazing thing is that you're taking a very similar process, but then you will have put it on steroids. and you're using the best chemistry known to mankind, the best physics known to mankind, and you are now doing this in a way that the imprint's smallest unit can be so small that the parts are, you know, few atoms thick. So broadly, that's how we manufacture chips today. It has a wild logistics chain where, like, most of the silicon comes from, what was it, Spruce Hill, North Carolina. There's some stuff that comes from, you know, Democratic Republic of Congo. There's, you know, the Dutch company that makes the UV lithography stuff. And then there's the best chip manufacturing plant in the world that's in Taiwan.

Yeah. So, again, you know, this is not only one of the hardest exercises that the humanity takes. It's also the most collaborative exercise that humanity undertakes. Let's go back and maybe think through, you know, why is it that the whole humanity has to collaborate, right? So, starting with just input materials. As you mentioned, you need so many input materials, really almost the entire periodic table for this. and certain things are just found in different parts of the world. So that's one reason why you see concentration of certain parts of the chip supply chain in certain regions of the world. Another reason why some countries dominate certain parts of the chip supply chain is that they have just decided that they want to be in that game. For example, chip manufacturing is extremely expensive. We alluded to it earlier. It's so capital intensive that for a country to be in that game, there needs to be sustained support and there needs to be enormous patient capital over decades. So certain countries in Asia decided that they want to be in that space because they have low input costs, you know, cheaper labor, cheaper materials, and then a lot of state support. So

manufacturing gets dominated by sort of certain countries. There's a combination of these factors, some of them geopolitical, some of them just natural, some of them just happenstance. I'll give you an example before we move to a different thread. You mentioned the Dutch dominating the advanced ship equipment. So you're referring to ASML. There's this company that builds the equipment needed to build the world's most advanced ships. The technology that they use was actually developed in the U.S. A bunch of companies in the U.S. alongside the government were working on this technology called EUV, which is the technology that sort of underlies the most advanced ships today. It was actually being developed here in the U.S. And at some point, these consortium members decided that they don't want to sort of continue putting effort into it and they need to license the technology to someone else who can take this and run with it. They decided to grant the license to, at that point, a small company, ASML, that, hey, take this technology and sort of run with it. And that's why you see ASML now being a leader in it. So there are all these

really interesting factors that have ultimately led to this reality where we have this crazy chip supply chain that you talked about where really the entire humanity sort of participates in it. Right. I think one of the surprising things for me was how geopolitical chip making, chip design, all that is now, but it has always been, right? There's a lot of like supply chain infiltration. There was a great story about like stealing chips and then writing on the chips to be like, steal from the best. Once you have built this microscopic switch, you can just tell the benefits you will have from collecting a lot of the switches on the same device That the chip Jack Kilby was the person who often credited as the inventor of the chip He worked at Texas Instruments So when he and his manager announced the chip the invention of the chip at this press conference in 1957 New York Times covered it as one of the top three inventions of 1957. So people knew that this was going to change the world. Okay. Well, which means that your friends know it's going to change the world. Your adversities know it's

going to change the world. Competitors in future, they know it's going to change the world. So, So you're right. Right after chips were invented, there was this scramble among nations to own this technology or to at least have an unfettered, unrestricted access to this technology. And at that time, you know, the big thing was the Cold War. So there are these fascinating stories about how Russians tried to acquire the chip technology. First, they tried to build their own indigenous chip technology, but it was becoming hard for them to catch up because, you know, Morslaw is a very stubborn thing. And so they tried to run alongside the U.S. for some number of years. It became very hard for them. It became very expensive for them. So then as a state policy, they wanted to sort of clone the chips. And the same was true for a lot of other countries in the Eastern Bloc. Yeah. And there's a lot of knock-on societal effects. I think anybody who's been following the data center conversation understands, like, not only is there the hardware shortages, there's water usage concerns, power usage concerns, there's pollutants.

And that's sort of always been a concern with the chips, right? That's right. So what has changed is the scale. So chips have this significant water, air, et cetera, impact in two stages. One, when they are being built. Another, when they are being used. So during the chip manufacturing itself, you need enormous amounts of electricity. You need enormous amounts of water. Then when you use the chips, of course, you need a lot of electricity to power the chips. But then all this power gets dissipated as heat. So you need a lot of electricity to sort of cool the chips. And you use a lot of water in order to do that cooling. And you're right that fundamentally these factors have always been there. What has changed at the scale? So especially with data centers and now with AI data centers, the scales are astounding. So imagine using 500,000 GPUs to train a single model. Now imagine each GPU consuming a kilowatt or two kilowatts of power. So now we are talking about AI data centers that consume gigawatts of power.

So if a single building or single data center is consuming as much power as an entire city, even though your data centers serve globally, they exist locally, right? So imagine the effect they have locally. Imagine the amount of water they are going to draw from your local water pool to cool the data center. Imagine all the electricity this data center is going to draw from the grid. When you're spinning up the data center, it can cause a lot of local air pollution. So those are the concerns that exist around data centers that primarily sort of trace their origin to the chips that you're using in them. Companies are being very creative with how they are powering data centers. So essentially, imagine that you want to set up a data center that consumes a gigawatt of power. You want really quick connectivity to the grid, but that's not how regulations have been set up in the U.S. It takes years to connect a load, especially a large load, to the grid. similarly there are these permitting delays in a lot of cases people who are building these data centers don't have the patience to wait for the permits or wait for these connections to happen

using a regular queue that will go on for years so yeah there are these creative things that are being done one approach is what is called bring your own power so a data center would try to power self up as much as possible by the power it brings itself now the form it takes you know sometimes stretches the limits of, let's just say, law or certainly a civic sense. Because in some cases, there have been examples where these data centers have used these temporary or mobile diesel plants or natural gas plants. You know, you put them on these things that look like semis, and you can just drive them off across the border if you needed to. A lot of these diesel plants were used and caused a lot of pollution and still causing much consternation in the local community. There's so much effort expended. There's so many hands touching this. There's a lot of knock-on effects. I know this is the Software Engineering podcast, but is it worth it? Is all this investment in chips worth it? Yeah, that's a great question. That's not a million-dollar question. That's a trillion-dollar question. Yeah, yeah.

So let's unpack this a bit. What does it mean by it being worth it? So these AI data centers I already mentioned, some of these data centers have hundreds of thousands of GPUs. They cost billions, if not tens of billions of dollars to set up. And one way to interpret your question is the cost that you're putting in in order to set up these data centers. Are these costs sort of worth it for whatever they're used for? Right. So then the question goes back to the AI question. Right. That what is the return on investment for AI? As you know, there are people who believe that AI can unlock significant productivity in the society. It can cure diseases. It can discover new materials. It can do a lot of those things. Sure. That's the pitch. That's right. That's right. So if you think that way, then you may say that all of it is worth it because, you know, you're putting in all this money for the promise of something that's wonderful and that can turbocharge our society for some definition of turbocharging our society. On the other hand, if you look at all of this as, let's say, a business decision, then there's this question of, well, you're putting in all this money.

Where is all this money coming from? When do you have to return this money? How much revenue do you need to make in order to be able to sustainably continue this process? And then everything looks very questionable because the chips are so expensive. These data centers are very expensive. Most of these AI data centers are currently funded through private debt. So this is all coming from private equity markets. The capex to revenue ratio is high and keeps increasing, which makes some people very nervous. So it also makes, you know, I've written about it recently. It makes me nervous at some level in that there are two things here. One, what if these companies start defaulting? What if you don't have the returns that you expect you'd have? If they start defaulting, then a lot of companies, a lot of people are going to lose a lot of money, especially because at this point, so much of the economy depends on the small number of companies. The second reason is that what we don't know is what is the exposure of these creditors to the public banks. So I know that during the 2008 financial crisis, after that, steps were taken to limit the exposure of these private creditors to public banks.

But people are creative and it's hard to tell what exposure really exists. If there is exposure directly or indirectly to public banks, then there is this wider concern. It's not only about people who are invested in AI losing money. There's a wider concern that it could lead to some kind of recession. And finally, the third way to look at it is, is it worth it? Well, you know, it's a market. People are taking risks, making investments, and hopefully sort of market correct itself. My hope just is that it doesn't lead to some costs that are irreversible and that are so significant that we may regret it. I talk to folks every week that are seeing, you know, really interesting implementations of AI, changing how software works. But to see some of these PO prospectuses that say, like, the addressable market is a third of the world's GDP, it just seems like that's a crazy thing to say. It does. I mean, it is certainly based on promise of a very different world. And I'm as curious as you to see how it goes. You end the book on a sort of look of like, what's the new normal? What's the possibilities for much talked about? Like, what if Taiwan is reintegrated into China?

They have become such strategic assets that they will certainly determine the future world order. It's not only because of the impact they've had on the world. It's also because of two other reasons. One, that there exists different kinds of choke points. So this is a technology where certain countries dominate certain parts of the chip supply chain, which gives them this leverage that they can use in different sort of ways. So that makes it very strategic because somebody or another can turn the faucet off if they wanted to And then another reason is that chips underlie some of the most strategic technologies We were talking about AI. You know, there's this belief that whosoever wins the AI race wins the world order. Chips are key to winning the AI race. A lot of AI is fundamentally based on technologies that were developed in the 60s. The thing that changed was that these technologies required a lot of data and a lot of computational power, neither of which existed. And finally, it became available. So once powerful enough chips became available, then you could use these technologies in order to solve meaningful problems.

Once you started solving meaningful problems, there was a lot more investment into chips that could help you solve meaningful problems. And now it has created this sort of nice feedback loop. Right. So here's the strategic technology that is completely intertwined with chips. Chips become a critical resource for that reason. It's not only just AI. Other strategic technologies such as quantum computing, such as biotechnology, such as renewable energy, a lot of them depend critically on chips. And that's the reason why they have become a critical resource. When we think about the new normal and when we consider it in conjunction with the fact that access security is challenging because different countries are dominating different parts of the chip supply chain, I do think that we will continually see these tussles where the ambitious countries would want to own more and more of the chip supply chain, would want to have as much of the chip sovereignty as possible. On the other hand, the adversaries or their competitors would try to deny this access or use access as sort of a lever in the geopolitics.

So I think we'll continue to see that. Specifically about Taiwan, as you mentioned earlier, Taiwan hosts this company, TSMC. That's the only company in the world that builds the world's most advanced ships. And this is crazy. One company in the world builds the world's most advanced ships, which means that it has this inordinate influence on the world's geopolitics. It has this inordinate influence on the world's economy. So there's this jockeying that has been happening over the last few years, and I think will continue to happen, around who controls DSMC, who has access to DSMC chips. And I think what happens with Taiwan, especially vis-a-vis U.S. and China, I think will have a lot to do with access to these chips. I've heard a lot of companies talking about, for these AI models, creating custom chips for specific models or using AI to create the chips and work the chips themselves. How is the current iteration of software changing how chips are created?

Yes, I think software will have an enormous implication in terms of both what chips went out, how chips will look like, and how chips will be designed. So let's address these one by one. So a great example in my mind is CUDA as a moat for NVIDIA. So for the longest time, we would talk about CUDA as being that moat for NVIDIA. You know, everybody is trained in CUDA. CUDA being the general purpose CPU programming language for NVIDIA. That's right. That's right. Students are trained in CUDA. So much software exists around CUDA. So therefore, that gives NVIDIA this enduring advantage over its competitors. But the coding agents are now so good, I don't know anymore if CUDA is that enduring advantage because you don't have to have that programmability to program new chips. Perhaps AI is the one that's programming your new chips. Perhaps you'll have these tools that will automatically convert CUDA code into something else. So I think the jockeying between different companies in terms of their chips, I think itself will change because the value of software as a moat or as an enduring thing, I think is going to shrink.

To that point, I spoke to folks at AMD, and they are absolutely doing exactly that, doing AI-generated ROCAM code to do their GPU programming. Yeah, no, I can see that, right? Because then either your humans are no longer programming, in which case it doesn't matter whether you have a lot of software framework that exists around CUDA for you to program better and easily and well. And second, because at the end of the day, if your metrics are performance and cost and things like that, then maybe programmability sort of gets thrown to the wayside. So I think that's an interesting development. There are two examples I'll bring up here. One is this announcement by OpenAI of their jalapeno chip. They talk about going from conception to sort of having a prototype in nine months. They talk about using a lot of software tools, these AI models, in order to design their chips. And these chips, even third-party evaluations, look really, really good. This now tells you how much of chip design is now going to be done by these software tools or these AI models versus what used to be sort of hardcore RTL designers or circuit designers, so and so forth.

If AI is going to write future software, then there is this question of, would you want your chips to be programmable? If yes, how programmable? Or would you rather have your chips be ASICs, so, you know, really hard-coded for the specific task at hand? As long as there's a lot of money to be made in the AI world, I see that more and more movement would happen towards these ASICs where we'll say, let's forego programmability in favor of efficiency. I see that change happening as well. That's been a software engineering tradeoff for a long time, the flexibility versus the hard-coded speed. Another thing with AI generating chip designs is that chips already have a pretty high failure rate in the manufacturing process and in actual operation. Do you think with both the sort of financial pressures competition and the AI generation, we're going to see failures increase? So I do think that there is an increased pressure to bring these chips out to the market. And because of that, at one level, you would imagine that your testing, your verification, all of that, because you have less time, you'd have more failures escape.

But on the flip side, testing and verification always were these tasks that involved, let's say, a lot of humans, for example. And even when there were tools, we didn't have good ways to sort of explore the search space. One thing that AI is really good at is exploring the search space. So you see a lot of investment into test and verification companies that are trying to leverage AI for these tasks. So I don't know how it's going to turn out at then. So while at one end, we'll spend less time, likely less time doing testing and verification, but at the other end, I think we'll get a lot more effective at testing and verification. And I won't be surprised if now we'll pump out chips much quicker than before, but then they are also more reliable than before. All right. I love the hope built into that. Well, thank you for listening, everyone. It's that time of the show where we shout out somebody who came on to Stack Overflow, dropped some knowledge, shares, and curiosity, earned themselves a badge. Today, for the start of spooky season of this recording, I'm going to shout out a necromancer badge,

somebody who answered a question more than 60 days later with a score of five or more. So congrats to Faisal Khan for answering difference between Swift Interact and File Act transmissions. If you're curious about that, we'll have it for you in the show notes. I'm Ryan Donovan. I host the podcast, Edit the Blog, here at Stack Overflow. If you like what you heard, leave a rating and review. It really helps us. And if you want to reach out to me directly, you can find me on LinkedIn. And I am Rakesh Kumar. I teach at the University of Illinois. and if you like the discussion that we had today, you can pick up my book, The Chip Age, How Semiconductors Became the World's Most Popular Resource. All right, thank you for listening, everyone, and we'll talk to you next time. Thank you.