Reiner Pope says Google's TPU v1 was announced in 2016 and helped prompt a wave of AI chip startups.
“TPU v1 was announced in 2016, I think. That was what actually kind of led to the creation of all of those 2016, 2017 startups.”
Person
Reiner Pope
1 episodes · 25 quoted claims
Pope says TPU v1 was built in roughly a year to a year and a half by a team of 20 to 30 people.
“It was done on a very short timeline, maybe, I don't know the full details, but maybe about a year or so, maybe a year and a half, with a skeleton team of 20, 30 people.”
Pope says Google stopped publishing its research around 2022.
“In 2022 was about the time when just Google completely stopped publishing its research.”
Pope says the matrices used in LLM workloads have grown from about 128 in dimension to many thousands.
“The matrices have grown from maybe 128 in dimension into the many thousands.”
MatX raised a Series B round led by Jane Street and Situational Awareness.
“So we've raised a series B round. It's led by Jane Street and Situational Awareness.”
Pope identifies Situational Awareness as the fund of Leopold Aschenbrenner.
“Situational Awareness, that is Leopold Ashenbrenner's fund.”
MatX's Series B round is $500 million, intended to ramp manufacturing and supply chain.
“It's a $500 million round. helps us actually ramp the manufacturing and supply chain for our chip so we can bring our chip to market.”
Pope estimates that producing a chip in small volumes costs about $100 million.
“roughly I would say it costs all about $100 million to produce a chip in small volumes.”
MatX aims to eventually ship chips amounting to multiple gigawatts of capacity per year.
“We would like to be at a place where we're shipping multiple gigawatts a year.”
Pope says reading through all of an HBM-based chip's memory takes about 20 milliseconds, setting the per-token time, versus about one millisecond for SRAM.
“There's just some simple math of, like, how long does it take you to read through all of HBM? It takes about 20 milliseconds”
Pope names SK Hynix, Samsung and Micron as the three big HBM vendors, with logic dies typically from TSMC or Samsung.
“there's dependency see on logic dies from typically TSMC, maybe Samsung, or HBM, which are the big three HBM vendors, Hynix, Samsung, and Micron.”
Pope says the systolic array is the gold standard for matrix multiplication in area and power efficiency.
“There is a gold standard for that that is called the systolic array. Make a really large systolic array. You can't beat that in area or power efficiency.”
Pope says MatX expects 4-bit precision to be its main number format, similar to NVIDIA, within a mix of precisions.
“We think probably the main thing will be similar to where NVIDIA is at, which is 4-bit precision.”
Synopsys and Cadence provide the electronic design automation tools that turn Verilog into gates and then polygons.
“Synopsys and Cadence provide EDA tools.”
Pope says a tape-out costs about $30 million, and the second chip costs about $1,000.
“your first tape-out, tape-out costs like $30 million.”
Pope says a chip fix limited to the metal layers can cost about $100,000 rather than $30 million.
“you can redo just the metal layers, which costs you only like $100 ,000.”
Pope says many chip vendors alternate years between physical technology upgrades and architecture overhauls, a 'tick-tock' model.
“Many chip vendors have this sort of tick-tock model, which is you'll do on one generation, like maybe you're trying to release every year.”
Pope says NVIDIA builds racks of 64 chips with eight spares to tolerate failures.
“NVIDIA has eight spare chips and a rack of 64.”
After tape-out, TSMC creates masks using ASML tools, then builds around 15 metal layers on wafers.
“So they create a mask. So that is where the ASML tools come in.”
Pope says the time from tape-out to receiving chips is roughly four to five months.
“what we see is time from, like, tape out to chips back. Again, depends on Node, but it's ballpark four or five months.”
Pope says MatX will tape out in under a year from late 2026 or early 2027, with chips available later.
“Tape out in under a year. And then that means chips available in sort of...”
Pope describes JAX as Google's more modern alternative to TensorFlow for writing models in Python to run on TPUs.
“The JAX team develops, this is sort of Google's new, more modern version of TensorFlow or competitive PyTorch.”
Google staff, with Pope contributing after leaving, wrote the guide 'How to Scale Your Model' about running LLMs efficiently on TPUs.
“wrote this guide called How to Scale Your Model, how to run an LLM as fast as possible.”
Pope says AI models have had extensive reinforcement learning on Rust and Python but little on Verilog.
“They've done a lot of RL on them. They have not done as much RL on Verilog.”
Pope describes cuckoo hashing as hashing a key to two locations and using the less full bucket, and says he beat top hash tables by adding SIMD instructions on Intel CPUs.
“hash into two different locations and then you use the bucket which is less full.”