Jim Keller: Moore’s Law is Not Dead

There's a good talk by him about how Moore's law isn't dead: [VIDEO] . Obviously, people have different ideas about what they mean when they say "Moore's law is/is not dead", but I appreciate Jim's point that even if the current innovation curves we've been riding are slowing down, there are other innovation curves that we can take advantage of, and that the combination these curves mean that there's still plenty of innovation and improvement to be made in processor design. Of course, the question of whether the x86 architecture will be the foundation for that future improvement remains to be seen. ARM and RISC-V are both eating x86 from below, and the fact that both of the processor architectures allow for greater competition among processor implementations suggests to me that one (or both) may catch up with x86 at some point in the future.

Especially the part at [VIDEO] That Sunny Cove architecture slide shows a massive increase in the number of execution units. It may be that Intel's next architecture, combined with memory speed improvements, is competitive with AMD. But AMD also isn't sitting still. AMD's contributions (from what I can tell) include: * Core counts have blown past 6 cores / 12 threads * CPU prices have just been cut by half * PCIe lanes have blown way past 8 * ECC ram is being offered in consumer PCs

Jim Keller recently spoke at UC, on the topic of Moore's law not being dead. Very impressive guy. [VIDEO]

There is this interesting talk by Jim Keller. Talks about new approaches (different transistor tech, architecture, etc) to keep moores law going for another decade or so. You should defintely check it out if have time. [VIDEO]

Jim Keller giving a talk on Moore's Law at Berkeley, [VIDEO] And another [LINK] And another [LINK] DEC, PA Semi, Apple, AMD, Tesla, Intel. He's been everywhere. [LINK] The internal announcement memo: [LINK]

Here's a random collection of thoughts that I have about the progress of Artificial Intelligence (AI): Machine learning (ML) and Deep Learning (DL) in particular benefit from fast computer chips. The most impressive gains in AI in the 2010s (Computer Vision & Natural Language Processing) were made thanks to DL. There's a famous post by AI researcher Rich Sutton, which summarises this fairly neatly [1]. Now, this connection between DL and fast chips would tie the progress of AI pretty tightly to the progress of Moore's Law [2]. There's some compelling evidence that Moore's Law is at least slowing down [3]. On the other hand, there are industry experts like Jim Keller who pretty strongly disagree with these assessments [4] and even TSMC seems to be bullish on being able to keep up with Moore's Law [5]. Some estimates of GPT-3s training cost put it in the range of ~5-10 million dollars [6]. It's hard to say how big of an impact on the economy GPT-3 will have. It's probably safe to assume though, that OpenAI is already working on GPT-4. The jump in parameter sizes from GPT-2 to 3 was roughly 100x (1.5 billion vs. 175 billion) and I'm assuming the price of training the model increased in roughly the same proportion (I might be wrong here and if anyone can point me to evidence on this, it would be much appreciated). With these assumptions, and provided that GPT-4 won't be affected by diminishing returns of adding more parameters (big if), the price for it would be somewhere between

For quite some time these nanometers is only a name. The only limitation right now is imagination of people working in TSMC and other (also money, I guess). Here's a video of Jim Keller presenting Intel's share of tricks for a few future generations: [VIDEO]

They've been saying Moore's law is dead for 20+ years. Reasons usually quoted for Moore's law's death are quite narrowly focused on a particular technology stack, whereas semiconductor fabrication happens on a complex web of technologies all moving forward with their own S-curves. It is worth watching Jim Keller's lecture at UC Berkeley literally titled "Moore's law is not dead". He talks on a different level alltogether and far more convincing than the pedantic look at the details that enable Moore's law. [VIDEO]

Actually Moores law is dead. His prediction was not transistors would get smaller. His prediction was initial that for 10 years was that density would double every year. And then later that it would double every 18 month. And this held longer then even he thought. But it has ended. While things still get denser, its no longer on that same trend. This interview with Moore explains it: [LINK] History by Cantrill: [LINK] David Patterson perspective: [LINK] Opposing point of view Jim Keller: [VIDEO] Patterson response to Keller: [LINK] There are talks by David Patterson, Jim Keller and

According to Jim Keller, not dead: [VIDEO] This isn’t the best recording on YouTube but it’s late and I couldn’t quickly find the other one.

We are ridiculously far from physical limits in our current artificial computers (both theoretical [1], and practical [2]). For more technical details see Jim Keller: [3] [4] [5]. [1] [LINK] [2] The ~12 watts computer inside each living human adult skull (and perhaps each eukaryote cell [6]) is still the state-of-the-art, for quite some time. [3] 2021, Jim Keller: The Secret to Moore's Law , [LINK] [4] 2019, Jim Keller: Moore’s Law is Not Dead , [VIDEO] [5] 2023, Change w/ Jim Keller , [LINK] [6] Our computers aren't yet able of polycomputation, where the computation topology, data, and functions depend on the observer, instead of computation in a passive implementation, once done forever set in s̶t̶o̶n̶e̶ silicon, 2023, Michael Levin, Agency, Attractors, & Observer-Dependent Computation in Biology & Beyond , [LINK]

Jim Keller himself talks about this a little bit here [VIDEO] I think I've listen to asianometry (YT channel) talk about this too, but I am unable to find any clip now where he explicitly talks about interference lithography...

I used to agree with you until I saw this presentation by Jim Keller, one of the biggest names in chip architecture. It really illustrates how many more options we have to improve performance. [VIDEO]

Jim Keller gave a talk at Berkeley titled "Moore's Law is not dead" (2019) where he summarized a bunch of up and coming technologies that will enable the continued scaling of transistor density - [VIDEO]

Scaling laws are not dead. The number of people predicting death of Moore's law doubles every two years. - Jim Keller [VIDEO]

> Moore's law is dead People have said this for decades. Jim Keller believes otherwise and brought receipts: [VIDEO]

Someone always crawls out of the woodwork to repeat this supposed "fact" which hasn't been true for the entire half-century it's been repeated. Jim Keller (designer of most of the great CPUs of the last couple decades) gave a convincing presentation several years ago about just how not-true it is: [VIDEO] Everything he says in it still applies today. Intel struggled for a decade, and folks think that means Moore's law died. But TSMC and Samsung just kept iterating. And hopefully Intel's 18a process will see them back in the game.