The die and the reticle

A scanner prints a chip by shining light through a mask onto the wafer, one rectangle at a time, and that rectangle has a fixed size. ASML publishes it per machine rather than as an industry constant: the TWINSCAN NXT:2050i lists a full 26 x 33 mm field size with 4X reduction, so the pattern drawn on the six-inch mask blank is demagnified four times on its way down to the resist. Multiply the field out and you get 858 mm2. That is the ceiling every chip designer works under, and NVIDIA's own phrase for Blackwell, the largest die possible within the limits of reticle size, is a designer telling you they hit it.

Put those fields on a wafer and the second constraint appears. A 300 mm wafer has 706 cm2 of area, so at most 82 full-field dies fit on it before you account for the round edge, and a die half that size gets you at most 164. The edge is not a rounding detail. Leachman's yield note lists edge loss as a mechanism of its own, films deposited poorly near the rim causing wholesale die yield losses that have nothing to do with a particle landing on the die.

Lithography is not about to relax the limit either. High-NA EUV, the generation after the current one, uses anamorphic optics and exposes a half field of 26 x 16.5 mm, so a design that wants the full width has to be printed as two stitched exposures. Semiconductor Engineering's survey of the question is the readable account. The size limit that shapes the largest AI chips gets tighter before it gets looser.

Defects land on a wafer roughly at random, and one landing in the wrong place kills the die under it. Write D0 for defects per square centimetre and A for die area, and the simplest model of the survivors is Poisson, DY = exp(-D0 * A). Double the area and you square the survival probability. That single fact is why a reticle-sized die costs much more than twice a half-sized one, and it is the pressure behind every packaging trick in this lesson.

Poisson is also the pessimistic model. Leachman is explicit that it tends to underestimate die yield when the expected number of defects per chip is greater than one or when the die area is relatively large, because real defects cluster, and a wafer that dumps six particles onto one die leaves its neighbours clean. Murphy's fix at Bell Labs was to let D0 itself be a distribution and integrate over it. A triangular distribution gives the model that carries his name, DY = ((1 - exp(-A*D0)) / (A*D0))^2. An exponential one gives Seeds, DY = 1 / (1 + A*D0).

One model absorbs the others. Assume a Gamma distribution and the integral collapses to the negative binomial, DY = (1 + A*D0/alpha)^(-alpha), where alpha is a cluster parameter you estimate from defect data as mean squared over variance. Leachman gives the correspondence outright: at alpha >= 10 it is essentially Poisson, at alpha = 5 it closely approximates Murphy, at alpha = 1 it closely approximates Seeds. Choosing alpha is choosing how clustered you believe the defects are, and nothing more.

Foundries do not publish defect density for a leading node, so the D0 = 0.1 below is a teaching value you can vary, not a TSMC figure. The shape of the answer survives whatever you set it to.

from math import exp

D0 = 0.10              # defects/cm2: a teaching value, not a TSMC figure
FULL = 26 * 33 / 100   # 8.58 cm2, one full reticle field
HALF = FULL / 2

poisson = lambda a: exp(-D0 * a)
murphy = lambda a: ((1 - exp(-D0 * a)) / (D0 * a)) ** 2
negbin = lambda a, k: (1 + D0 * a / k) ** -k

for name, f in [("poisson", poisson), ("murphy", murphy),
                ("negbin k=5", lambda a: negbin(a, 5))]:
    print(f"{name:11} full {f(FULL):.3f}  half {f(HALF):.3f}")

# dual-die parts per cm2 of wafer: harvested halves against whole dies
pairs = poisson(HALF) / HALF / 2
whole = poisson(FULL) / FULL
print(f"pairs / whole = {pairs / whole:.2f}")

# poisson     full 0.424  half 0.651
# murphy      full 0.451  half 0.661
# negbin k=5  full 0.453  half 0.663
# pairs / whole = 1.54

The yield argument for splitting a die is easy to state wrongly. Under Poisson, the probability that one specific pair of half-dies are both good is exactly the yield of the undivided die, because the exponents add and nothing has changed. You are never required to use a specific pair. You test every half on the wafer, discard the dead ones, and assemble parts from any two survivors.

So count parts per unit of wafer rather than per die site. At D0 = 0.1 a full-field die yields 0.42 and a half-field die yields 0.65, and pairing harvested halves gives 1.54x as many dual-die parts out of the same silicon area. Murphy, being kinder to large dies, puts the same ratio at 1.47x. The gain is real and it is not enormous, which is why yield on its own rarely decides the question.

Harvesting also happens inside a single die, and that version ships in the highest volume of all. Draw the design with more SMs, more memory controllers and more L2 than any part will be sold with, test what comes off the wafer, then sort each die by how much of it works. A die with a few dead SMs is not scrap, it is a cheaper part. NVIDIA sells no full GH100 at all, and both H100 SKUs run with SMs switched off. The SM lesson prints the whitepaper's three configurations of that one design side by side, and works from them. Designing in redundancy and then selling by what survived is how a reticle-sized die stays economically viable, which means the SM count printed on a datasheet is a yield outcome as much as a design decision.

Two other reasons carry more weight. One is the wall from the first section: once a design wants more transistors than 858 mm2 can hold, there is no single-die version left to compare against, so no yield argument is needed. The other is that different circuits want different processes. TSMC advertises CoWoS as designed for both heterogeneous and homogeneous integration, which is the packaging way of saying the logic can sit on a leading node while I/O and analog blocks stay on an older, cheaper one.

Every cut has a price, and it is paid in the join. A wire that used to run inside a die now crosses a package boundary, costing energy per bit and latency, and it needs a physical interface at each end that occupies area on both dies. That is why the number worth looking up on any multi-die part is the bandwidth of the join, and why the Blackwell worked example below starts there.

The join happens on a slab of silicon underneath. TSMC describes CoWoS as TSMC's proprietary chip-last on interposer process, and ships it as three families, CoWoS-S, CoWoS-L and CoWoS-R. Whatever the process details, a multi-die part only pays off if each die can be tested before it is committed to a package, because one bad die would otherwise take every good die on the same interposer down with it.

The interposer itself runs well past a single reticle field. TSMC says it completed the certification of CoWoS advanced packaging solution for 5.5 times mask/reticle size interposers in 2025 and will start volume production in 2026. Reporting points further, at a nine-reticle carrier with twelve HBM4 stacks around 2027, though that is press rather than a TSMC statement and should be read as such.

Around the dies sit the memory stacks, and their arithmetic is division. An H100 SXM5 carries five HBM3 stacks for 80 GB and 3.35 TB/s, so each stack is 16 GB at 670 GB/s. A B200 carries eight HBM3e stacks for 192 GB and 8 TB/s, so each is 24 GB at 1.0 TB/s. The GPU chip lesson quotes those same totals rounded to 3.4e12 and 8.0e12 bytes per second; this is where the totals come from.

Divide once more and you learn something about headroom. JEDEC's HBM3 update sets 6.4 Gb/s per pin and 819 GB/s per device, which is a 1024-bit interface if you do that division too. The H100's 670 GB/s per stack works out to 5.23 Gb/s per pin, about 82% of what the standard allows, so five stacks at the ceiling would have been 4.1 TB/s. The B200's 1.0 TB/s per stack needs 7.8 Gb/s per pin, past anything HBM3 permitted, which is what the E generation was for.

Blackwell Ultra makes the last point on its own. It keeps eight stacks and the same 8 TB/s but moves to 12-high stacks at 36 GB each, 288 GB in total. Capacity grew by stacking more dies in each tower. Bandwidth did not move at all, because bandwidth is set by the interface width and the pin rate rather than by how tall the stack is.

Every level of the memory hierarchy is one tradeoff applied at a different scale. An SRAM bit is a latch of a few transistors sitting on the same die as the logic, fast to reach and stable as long as the power is on. A DRAM bit is a capacitor whose charge leaks away, so it has to be refreshed, and it is small enough that you can afford billions of them on a die of their own.

The area cost shows up plainly in a published floorplan. TPU v1 put 24 MiB of Unified Buffer on chip, and Jouppi's ISCA paper describes that one block as almost a third of the die, on a 28 nm part whose total area Google withheld beyond a footnote saying The TPU die is <= half the Haswell die size. Twenty-four mebibytes of SRAM cost a third of a chip.

Set that against the DRAM beside it. An H100 carries roughly 50 MB of L2 on its 814 mm2 die and 80 GB of HBM3 in five stacks around it, a capacity ratio near 1600 to one. The hierarchy exists because the cheap memory cannot be on the die and the fast memory cannot be large, and no amount of design cleverness moves either half of that sentence.

The GPU chip lesson walks the ladder from registers down to HBMThe chip’s main memory: large, far, and the resource memory-bound ops spend. 8.2e11 bytes per second on v5e, 1.6e12 on v6e.taught in /l/tpu → with the H100's capacities and bandwidths on it, and stage 0 turns the bottom rung into a ridge. What sits underneath both is the economics above: the fast levels are paid for in die area under a reticle limit, and the large level is paid for in stacks of a different kind of silicon placed on the same interposer.

NVIDIA states the whole design decision in one paragraph of the Blackwell technical brief.

Each of the two dies are the largest die possible within the limits of reticle size, as big as can possibly be built today. The two dies are connected and unified with a single 10 terabyte-per-second (TB/s) chip-to-chip NVIDIA High-Bandwidth Interface (NV-HBI), providing one fully coherent, unified GPU.

Read that against the three reasons for splitting a die and only one of them applies. Both halves sit at the reticle limit, so the harvest argument buys nothing here, and NVIDIA names a single process for the part, TSMC's 4NP, so it is not mixed-node integration either. Blackwell is split because the design wanted more transistors than one exposure can print: 208 billion of them, which the brief notes is more than 2.5x the amount of transistors in NVIDIA Hopper GPUs.

You cannot get a per-die area out of NVIDIA. The brief says only the largest die possible within the limits of reticle size and prints no square millimetres, so a B200 die area quoted anywhere is somebody's estimate. A bound is still available. If both dies sat exactly at 858 mm2 the package would hold 1716 mm2 of silicon, and 208e9 / 1716 is 121 million transistors per mm2. An H100 puts 80 billion on 814 mm2, which is 98 million per mm2, and an A100 put 54.2 billion on 826 mm2, which is 66 million. So Blackwell's density is at least 1.23x Hopper's, and higher still if the dies come in under the full field. That is a floor derived from a limit rather than a measurement, and it is worth more than a guessed area.

The 10 TB/s on NV-HBI has a gap of its own. NVIDIA does not say whether the figure is per direction or the sum of both, and publishes neither the width of the interface nor its signalling rate. Compare it to a number the same brief does decompose. For fifth-generation NVLink it prints the link count, the per-link rate, the direction and the total those multiply out to, so every term of the product is checkable by hand. NV-HBI gets none of that, so quote it as 10 TB/s with the direction unspecified and stop there.

Both gaps are worth keeping as a habit rather than a complaint. A vendor brief is a marketing document that happens to contain engineering, and the parts it decomposes are the parts you can check by hand. Where the decomposition stops, saying so is the honest move, and it is the same discipline the TPU chapters apply to Google's unpublished clocks.