The Write Problem: Which AI Bytes Actually Belong on NAND

AI is pulling NAND into memory tiers it has never served. Third in our free memory series with Rick Xie.

---

Good morning. It’s been a busy week of Hot Chips and NVIDIA earnings and all the action. Pour yourself a coffee and dig in, this is a long one!

This is a free report from Ben and Rick.

Co-authored with Rui “Rick” Xie (rickxie.cn), a memory-systems researcher with a PhD from Rensselaer Polytechnic Institute whose work runs from DRAM to SSDs. This is the third installment in our memory series. Rick leads the engineering and standards analysis; Ben Pouladian leads the investment framing.

The first two installments lived on the DRAM side of Rick’s research. The Bandwidth Tax argued that the real cost of HBM is levied on the system, not the stack. The Tax Dodge tested the first commercial architecture built to owe less of that bill, and it closed with a promise: the next installment crosses to flash, and the honest question there is not whether NAND can deliver more bandwidth. It is which AI bytes actually belong on NAND. This is that piece.

At its August investor day, SanDisk put a number on a slide that deserved more attention than the guidance did. Of the 1.2 zettabytes of data-center NAND demand the company forecasts for 2030, 420 exabytes is KV cache. I was not on the call. I read the deck, and my first reaction was the useful one: that was a big number. That’s crazy how big that is.

It’s always a problem, KV cache, because you just never have enough of it, and it slows things down. It is the live working memory of inference, the scratchpad a model writes to with every token it generates, and constantly writing it and deleting it is exactly what reduces the endurance of a flash drive. So a third of the demand forecast for the storage tier is the one workload flash is worst at serving. The deck prices those bytes. It does not price the writes.

NAND is priced by the byte and worn out by the write, and the biggest new line on SanDisk’s demand slide is a write stream.

Call it the write problem. It decides which part of the zettabyte is real.

What SanDisk Actually Told Investors

SanDisk’s investor-day slide is a forecast of how much paper the AI factory will buy. Three piles by 2030, all inside the data center, adding to 1.2 zettabytes.

480 exabytes is staging. The unread homework piled up for the next training run. 300 exabytes is fast data lakes. The filing cabinet of work already finished. 420 exabytes is KV cache. The notebooks. That last pile is a third of the forecast, and it is the pile flash is worst at holding, because notebooks get written the whole time the session runs.

The rest of this section is their supply story. Same factory, more pages per wafer, they still choose to ship fewer pages than they could. They have already pre-sold about half of next year’s paper and two-thirds of the year after to eight data-center customers. They want to sell it like a scarce good, not like a commodity. Then, near the end of the roadmap, they propose stacking that paper next to the GPU. That product is HBF. This piece is not about whether the slide is big. It is about which pile on the slide is real.

The demand claims first, and they are large. SanDisk expects industry bit shipments to reach 1.2 zettabytes in 2026, up from 960 exabytes in 2025, with data centers now roughly half of bit demand. By 2030 the company forecasts 1.2 zettabytes of data-center demand alone, split three ways: 480 exabytes of staging for training data, 420 exabytes of KV cache, and 300 exabytes of fast data lakes. The mix maps onto cell types: TLC at roughly two-thirds of bits, serving staging and hot KV cache, with QLC at a third, serving fast data lakes and cold KV cache. Zephyr, whose recap on the Citrini Research chat is the best public read of the day, called the forecast conservative, and separately estimated that persistent agent memory could add another 500 to 600 exabytes per year on top of it. That framing is theirs; the deck numbers are SanDisk’s own. My read on the buckets: obviously it’s data centers, but most personal compute devices will probably need bigger amounts of data to run onboard compute and these bigger models, and then you have this whole token explosion of robotics and edge AI cases. People move a lot of their AI close, on premise. You just keep on training bigger and bigger models. Maybe you don’t need that much for fast data lakes. I don’t know about that.

A third of SanDisk’s 2030 data-center forecast is KV cache. KV cache is a write stream, and the deck prices the bytes, not the writes.

The supply strategy is the more disciplined half of the deck. SanDisk plans to grow bits through wafer productivity rather than new fabs: each BiCS generation adds about 54 percent more bits per wafer, roughly 27 percent a year, and the company targets only mid-to-high-teens bit growth. A vendor that can grow bits 27 percent a year through layer count alone, and chooses to ship mid-to-high teens, is telling you it intends to sell scarcity, not volume.

The commercial structure says the same thing louder. SanDisk has signed NAND bit agreements, long-term contracts in all but name, with eight data-center customers, covering about 50 percent of FY2027 bits and about two-thirds of FY2028 bits. On top sits a FY28 through FY30 target of mid-to-high-teens revenue growth and a non-GAAP gross margin of roughly 80 percent, which is less a stretch than a hold: SanDisk printed 84.6 percent non-GAAP in its fiscal fourth quarter. A commodity flash vendor is now writing multi-year contracts that price forward a constraint, and guiding to keep the margin the constraint produces, which is the same behavior we documented in the HBM oligopoly one tier up.

Two-thirds of FY2028 bits are already under contract with eight data-center customers. The constraint is priced at signing, not at shipment.

Then, near the end of the roadmap, High Bandwidth Flash: NAND stacked and interfaced like HBM, the product this series flagged in July.

Weights Read. KV Cache Writes.

When you open Claude, ChatGPT, or Grok and type a question, two different piles of bytes go to work.

The first pile is the model itself. Fable 5, GPT-5.6 Sol, Grok 4.6. That brain got written to storage when the lab shipped it. Then it gets read, over and over, by every session on earth until the next version drops. Think of a textbook. They print a new edition a few times a year. Every student reads the same copy. You do not reprint the book for each class. That is weights. Read mostly. Flash, the cheap chips inside a data-center SSD, is good at this job. It hates being rewritten. It loves being read.

The second pile is this conversation. Every sentence you send, and every sentence the model sends back, has to be remembered so the next answer still knows what you asked. Paste a 10-K and ask fifteen follow-ups. Let Cursor chew a repo for an afternoon. Run a research agent for an hour. That memory grows for as long as the session runs. It is a notebook you keep adding lines to, not a textbook on a shelf. That is KV cache. It is a write stream. And a write stream is exactly what wears flash out.

A one-line haiku barely fills a page. A hundred-thousand-token agent session fills a notebook. SanDisk’s demand slide adds the textbooks and the notebooks together and calls it one market.

Weights change on deployment cadence, not nanosecond cadence, so paying HBM prices to store them is architecturally wasteful; this newsletter has been making that case since April. A read-mostly workload is the best case NAND ever gets. Endurance barely enters the equation, access patterns are large and predictable, and the only hard questions are bandwidth, latency, and heat.

KV cache is the opposite animal. Putting that write stream on NAND means paying costs that never appear on a demand slide, and this is where Rick’s triage begins. What follows is his arithmetic, worked on public sources.

Endurance. Think of endurance as how many times you can fill and toss the notebook before the paper wears through. DWPD, drive writes per day, is that number for a flash drive. The arithmetic turns the write problem from a metaphor into an endurance budget. Take a representative conventional attention configuration with 80 layers, 8 KV heads, a head dimension of 128, and BF16 storage. The KV state created by each token is:

KV bytes per token = 2 × 80 × 8 × 128 × 2 bytes = 320 KiB

A 100,000 token session therefore creates approximately 32.8 GB of logical KV state. For latent attention, quantized KV, or compressed representations, the serialized bytes per token replace this value while the rest of the calculation remains unchanged. The physical write rate is:

Physical DWPD = Tokens per day × KV bytes per token × Persisted fraction × WAF / Provisioned capacity

On a 1 PB flash tier, persisting 10,000 sessions of this size per day produces 328 TB of logical writes, equal to 0.328 logical DWPD. A WAF of 1.5 raises the physical rate to 0.492 DWPD. A WAF of 2.0 raises it to 0.656 DWPD. A WAF of 3.0 raises it to 0.984 DWPD.

The result exposes the variables that control the endurance budget. Session length sets the KV bytes created by each request. Request volume sets the daily write stream. Admission policy determines how much of that stream reaches flash. Placement determines how many physical writes the media ultimately absorbs. SanDisk’s TLC and QLC mapping identifies the intended capacity tiers. The operating economics depend on how quickly each tier consumes its write budget. At the scale of the 420 EB forecast, every 0.1 physical DWPD corresponds to 42 EB of media writes per day. The value of the forecast therefore depends on the lifetime of the bytes as much as their total capacity.

Write amplification. Flash never writes just the notebook you asked for. To add a line, it often has to copy a whole page. That extra copying is write amplification, WAF. Write amplification connects logical KV demand to physical NAND wear: physical writes equal logical writes times WAF. In the 1 PB example, the logical workload writes 328 TB per day, equal to 0.328 logical DWPD. Using WAF values of 1.5, 2.0, and 3.0 as illustrative sensitivity cases, the physical write rate becomes 492 TB, 656 TB, and 984 TB per day, respectively. Each additional 0.5 of WAF adds 164 TB of media writes per day, equal to 0.164 physical DWPD on a 1 PB tier. Moving from WAF 1.5 to WAF 3.0 doubles the daily physical write rate and consumes the same endurance budget twice as quickly.

The investment implication is direct. A capacity forecast becomes an endurance forecast only after logical write intensity and WAF are included. For the same installed capacity, physical DWPD determines replacement cadence, sustained media traffic, and the useful lifetime of the tier. The deck measures how many KV bytes the market may store. WAF determines how many NAND writes those bytes generate.

Reload cost. Reuse is the only reason to keep the notebook. If thirty employees ask “what’s our refund policy” every morning, you might keep yesterday’s notes and reread them instead of re-reading the whole manual. That reread is a reload. Recomputing from scratch is prefill. A one-off haiku never pays. A company bot answering the same prompt ten thousand times a day might. Reuse determines whether persisted KV creates economic value. Live decode KV is appended and reread on the token path, so active state remains in HBM or another upper memory tier. Persisted KV enters flash when its expected reuse value exceeds the cost of the admission write and subsequent reloads. Using time as a first-order resource proxy, persistence is favorable when:

T_write + E[N_hits] × T_reload < E[N_hits] × T_prefill

Here T_prefill is the prefill time avoided by each successful reuse. When T_reload is less than T_prefill, the break-even reuse count is:

E[N_hits] > T_write / (T_prefill − T_reload)

The 100,000 token example contains approximately 32.8 GB of KV state. Its idealized transfer time is 1.31 seconds at 25 GB/s, 0.33 seconds at 100 GB/s, and 0.082 seconds at 400 GB/s. Consider an illustrative case in which the initial write and each reload take 0.33 seconds. If one reuse avoids 5 seconds of prefill, the first hit creates a net time benefit: 0.66 seconds to write and reload against 5.0 seconds of prefill avoided. If one reuse avoids 0.5 seconds of prefill, the break-even reuse count becomes 0.33 divided by 0.17, or 1.94, and the second hit crosses the threshold.

Context length determines how much prefill computation can be avoided. Batch size determines the economic value of the accelerator time recovered. The storage and interconnect path determine the reload cost. The reuse distribution determines how many times that value can be collected before the object expires. Direct GPU I/O reduces CPU involvement, memory copies, and software overhead within T_reload. Effective path bandwidth acts on a separate component of the same term. For a 32.8 GB object, increasing effective bandwidth from 25 GB/s to 100 GB/s reduces idealized transfer time from 1.31 seconds to 0.33 seconds. Lower transfer and software costs move more prefixes across the reuse threshold. Discounted context caching is most valuable for frequently reused prefixes, where avoided prefill accumulates across repeated hits. Its economic value depends on identifying those prefixes before committing the admission write.

The demand slide counts KV bytes. The investable opportunity is the subset that earns its place on flash.

The one-line version of Rick’s boundary, from his framing of this piece, is the line to keep: HBF looks strongest for read-heavy weights, while KV cache may drive much larger NAND demand but remains constrained by writes, endurance, placement, and reload cost. The decisive question is no longer whether NAND can deliver more bandwidth. It is whether the tier actually improves inference once placement, reuse, writes, thermal limits, and the remaining HBM tier are included.

The read-mostly rows are proven NAND business. The 420EB KV row is the one carrying the writes, and it is the row the forecast leans on hardest.

HBF Is the Test Case, Not the Story

High Bandwidth Flash is the product name for a simple idea: take those cheap flash chips and stack them next to the GPU the way HBM, the expensive fast memory, is stacked today. A bigger shelf on the same desk, not a warehouse across town. SanDisk wants that shelf to hold the textbooks, and maybe some of the notebooks. The rest of this section is whether the notebooks survive on paper that wears out when you write.

High Bandwidth Flash has moved from a concept into a standards and sampling story, and the record on that trajectory is partly ours to correct. In January, The Hierarchy Rewrites mapped HBF as the first of Patterson’s four research directions, quoting his paper’s claim of “10× memory capacity with HBM-like bandwidth,” and said the SanDisk and SK Hynix standardization effort would decide whether it became a real product or vaporware. In July, the Bandwidth Tax migration map gave HBF a row on the strength of that standardization and carried the then-current timeline: samples this half, first inference devices in 2027.

That timeline did not hold, and the update goes on the record here: on SanDisk’s own investor-day roadmap, accelerators with HBF now arrive late 2028 or early 2029 at the earliest. We printed 2027 in July. The company’s own deck has since moved the date by 18 months to two years. Adjust the row accordingly. I can’t get the timing right on everything. I was early. I’m more of the direction of the thesis, and the direction held.

The scope narrowed along with the schedule, and two independent reads converged on the same boundary. Zephyr’s read of the investor day is that HBF is a weights tier, because KV-cache write traffic would consume NAND endurance, and that real deployments will be hybrid, HBM for the write-hot state and HBF underneath it. Rick’s read of the public FMS and OCP materials lands in the same place with the skepticism intact: NAND endurance is not solved, HBF may work well for read-mostly data such as model weights, and KV cache is much less clear because the actual write stream depends on placement, eviction, reuse, and write amplification. Zephyr also flagged that SanDisk’s deck misstates the HBM specs it compares against (their X thread), which is worth keeping in frame when reading any HBF-versus-HBM chart from the company.

The spec sheet will throw numbers at you. Stacks 8 or 16 dies high. Up to 512 GB on the shelf. Bandwidth grades that sound like HBM. SanDisk’s 1.6 TB/s is one product target, not a promise the notebooks will keep up. Version 0.7 is a draft interface, not a qualified product. Peak speed on the pin is the highway speed limit. Useful speed is what you get in traffic, with stops, heat, and paper that wears. Read-mostly textbooks keep the useful speed high. Tiny writes, erasing old notes, and shuffling pages are what kill it for KV. The formula a few lines down is just that sentence in symbols.

HBF mechanics. The public OCP v0.7.0 high-level base-die specification describes UCIe-based xPU connectivity, 8-high and 16-high stacks, capacity up to 512 GB, and bandwidth grades of approximately 0.384, 1.536, and 3.072 TB/s. SanDisk’s 1.6 TB/s number is therefore a first-generation product target near one public grade, not the only bandwidth defined by the architecture. Version 0.7.0 is a pre-1.0 high-level interface specification, not a qualified product specification. It establishes an interface and system contract, but it does not establish sustained application throughput or workload qualification.

At system level, the useful quantity is service rate under the actual access mix, not peak pin bandwidth. A compact model is B_useful = B_peak × η_protocol × η_scheduling × η_media, subject to tail latency, power, and thermal constraints. Read-mostly weights are favorable because static placement and parallel reads can keep η_media high. Small writes, invalidation, and reclamation couple KV throughput to wear and latency variance, so the same peak interface number does not imply the same usable bandwidth for the two byte classes.

The qualification questions are therefore workload specific. For weights, test sustained random-read service rate, p99 latency, thermals, retention, read disturb management, and error recovery within the package power envelope. For persisted KV, add program rate, reclamation behavior, endurance, and recovery from bursty admission. Public evidence does not yet identify one universal failure mode or publish enough data to close those gates.

So the test case is narrower than the pitch, and that is the honest shape of it. A weights tier with HBM-class packaging and a 2029 accelerator date is a real product thesis, but it is not 420 exabytes of KV cache. The story that carries the zettabyte is the broader one this piece is actually about: an industry trying to move the working set of inference down the hierarchy, one byte class at a time, and discovering which classes survive the move.

The Room Attacked the Write First

Hot Chips is the chip conference. Sunday’s memory tutorial had a live Slack channel for attendees. They did not let the bandwidth pitch land. They asked the notebook question first.

If you take notes on cheap paper, does the paper wear out?

Flash can be rewritten far fewer times than the fast memory sitting on the GPU. One person in the room said a million times fewer. That is not a lab spec. It is an engineer smelling the problem. Another assumed 100,000 rewrite cycles, the kind of number cheap flash is often rated for, and said a real agent session would burn the drive. His fix: only put the textbooks on it. Never the notebooks.

Someone asked for a tiny expensive notepad on the same chip, so you do not scribble on the cheap paper every second. That is a coalescing cache.

The Oxmiq speaker said new notes per batch are small and can be bundled. He did not give a wear number. Energy, he said, is not in the spec. Samsung and Micron answered a different question, heat on the expensive stack. Nobody answered whether flash survives the notebook.

The rest of this section is that Slack, almost as it happened.

Oxmiq Labs presented “HBF in AI Compute” just after noon. One attendee asked how HBF can serve KV cache if write endurance is more than 1E6 lower than HBM, and whether the simulations assumed a read-only case. That 1E6 figure is an attendee characterization: HBM DRAM has no analogous program-and-erase cycle rating, so it does not establish a like-for-like endurance ratio. A researcher at UC Merced ran the same attack with an SLC assumption of 100K program/erase cycles and said KV writes would burn the flash. The 100K P/E figure is his illustrative SLC assumption, with no corresponding SanDisk product rating or OCP requirement, and P/E count alone cannot establish KV-tier lifetime without admitted writes, physical write amplification, provisioned capacity, temperature, and retention. His read: stripe it for read-only data. Another asked for an SRAM or DRAM coalescing cache on the HBF base die. Others asked whether read and write bandwidth even match, and for pJ/B including the controller.

Anurag Agrawal of Oxmiq answered later Sunday afternoon. On decode writes: “batch size number of new entries. Its little and can be aggregated.” No P/E number. Aggregation can reduce transaction overhead and some physical amplification, while the logical KV write volume remains. On energy: “Its not in the spec.” The simulations, as he described them in the channel, assume the same pJ/bit as HBM, and that equal pJ/bit assumption is a simulation input, not a measurement of system energy including the controller and media path. On a coalescing cache: “I am not sure how it will evolve.”

The speaker who did answer memory questions was Samsung’s Sangwook Han, and he was answering a different problem. He said it would be difficult to have more than about 2K IOs on the interposer. On zHBM, the C-die stack sits on the xPU with no B-die, so HBM PHY and D2D are not necessary; only vertical TSVs exist, and power is “extremely low.” Heat is the catch: “DRAM dies will sit directly on the xPU.” On aHBM: not HBM PIM. Processing elements live in the B-die, not the C-die, and in one worked example, moving some attention units cut D2D bandwidth to “about half,” an example rather than a spec. That is still the HBM wall one tier up, not NAND writes. He did not answer HBF endurance. Micron’s Raghu Sreeramaneni called stacked DRAM important and blocked by thermals, packaging, and architecture. He did not answer HBF either.

The channel is not a 420-exabyte forecast. It is a room that already knows the KV write stream is the part of the slide that has to survive contact with NAND.

The Same Overflow, Contested From Two Sides

The notebooks have to go somewhere when they no longer fit in your hand. NVIDIA already built the overflow system. Dynamo is the software that decides which notes stay on the desk, which go to a nearby shelf, which go down the hall, which get thrown out. BlueField is the clerk who moves them. That stack is already running in production clouds.

SanDisk’s 420-exabyte slide is the same claim, restated by a paper company. NVIDIA also has a huge pile of cheaper memory at the rack, LPDDR, 384 terabytes in a full Vera setup. We asked if that pile becomes addressable notebook space. They would not say. If the overflow lands on that rack memory, SanDisk’s KV forecast deflates. If it lands on flash, the 420 exabytes gets real, and the wear problem becomes the industry’s problem. Both routes go around expensive HBM. They pay different tolls. The write problem is the toll when the route is flash.

The biggest platform in the industry has already built the plumbing for the KV rows. NVIDIA’s Dynamo serving stack manages a four-tier KV hierarchy, with rack-scale storage on BlueField hardware purpose-built for cache overflow. When the market sold memory stocks on a compression paper in March, our response piece read that software roadmap as the tell that the industry is planning for “the future involves more cached state, not less.” The cached-state forecast SanDisk just published is that same claim, restated in exabytes by a flash vendor.

That deferral is on the record in the Vera piece, where we put the question to NVIDIA directly. SanDisk’s investor day is, in effect, a bid for the same bytes one tier further down.

You route around a wall and you pay for the route. The wall here is HBM capacity cost per byte of context; the routes are LPDDR at the rack, NAND in the storage tier, and eventually HBF under the HBM; and the toll changes form at each stop, from interposer and packaging at the top to endurance and placement at the bottom. The write problem is what the tax looks like when it reaches flash.

Two Routes Into the Same Bytes

Days before SanDisk’s investor day, the buyer side published its own version of the slide. At FMS in Santa Clara, NVIDIA argued in a company blog post that storage has to become an active component of the compute pipeline, with microsecond-class access, because the memory-versus-storage tradeoff “now plays out in microseconds.” The post names a product tier for exactly the bytes on SanDisk’s slide: CMX Context Memory Storage, NVIDIA’s storage tier for agentic AI inference context.

The plumbing NVIDIA named is how the library works. cuFile lets the GPU fetch the notebook itself instead of asking a CPU librarian to walk it over. SCADA is the rules for those trips. BlueField-4 STX is the clerk NVIDIA says is faster than a generic x86 clerk at packing and locking the files. Those speedup numbers are NVIDIA’s own, untested by us. Storage-Next is the vendor list trying to fill that library, more than 40 names. The investable split is still the same: library down the hall versus shelf on the desk.

Around it sits the plumbing to make a drive behave like a slow, enormous memory: the cuFile APIs open-sourced so that “GPUs, not just CPUs, read from and write to storage directly” with hundreds of thousands of GPU threads, a data-access framework called SCADA, and a Vera BlueField-4 STX storage processor that NVIDIA claims delivers up to 3.21x the throughput of x86 in compression and encryption pipelines. Those are NVIDIA’s own numbers, vendor-run and untested by third parties, and the Storage-Next initiative behind them lists more than 40 vendors, DDN, KIOXIA, and Micron among them.

In the second installment, NVIDIA’s Vera validated datacenter LPDDR as a tier while Qualcomm pitched HBC at the same wall from inside the accelerator package. Here, NVIDIA’s CMX validates the microsecond context tier while SanDisk pitches HBF into the same bytes. The platform owner keeps building the socket; the challenger keeps proposing a different part to fill it.

But the two vendors reach those bytes by different routes, and the distinction is investable.

Picture two ways to store the notebooks. NVIDIA’s CMX is a shared library down the hall. Any GPU on the floor can check a notebook out, so you buy fewer copies, and you wait a little to walk the hall. SanDisk’s HBF is a bigger shelf on the desk. Faster to grab, still the same paper, still wears out when you write. Neither is the live conversation in your hand. That still lives in HBM, the fast memory on the GPU. These are overflow shelves for notes you might need again.

NVIDIA’s public CMX model adds a network-attached flash tier between local SSD and shared storage and stages reusable KV toward system memory and HBM. SanDisk moves flash into the accelerator package through HBF. CMX can pool capacity and reduce part of T_reload. HBF can reduce physical distance and raise local interface bandwidth. Neither turns flash into a latency-equivalent substitute for active KV.

Both routes are governed by the same admission inequality, but they move different terms. The network route primarily changes reload latency, sharing, and queueing. The packaged route primarily changes locality and bandwidth while retaining NAND write, retention, and endurance constraints. The addressable flash demand is therefore gross KV bytes multiplied by the fraction admitted, the probability of reuse before eviction, and the capacity provisioned for the required service level. Pricing every generated KV byte as durable flash demand overstates the opportunity.

None of this arrived unannounced, and the January record is worth re-reading now. The Hierarchy Rewrites carried NVIDIA’s context-memory platform plans, BlueField-4 DPUs managing KV cache lifecycle across HBM, NVMe, and the rack, and put the mechanism on the page: “Runtime policies that intelligently evict and reload from SSD tiers enable workloads that would otherwise be impossible.” We drew the tier in January. NVIDIA gave it a product name in August.

CMX and Storage-Next were on the same Hot Chips agenda. The tutorial room’s fight over the write is in the section above; the fabric route’s turn comes as the CMX qualification lists fill in.

Who Collects in the Tier Below

If the tax moves down the memory stack, someone still collects it. Not by selling the cheapest paper. China can already print that. YMTC is the proof. What is scarce is everything that turns cheap paper into a shelf the exam room will accept: the contracts, the stacking, the software that decides which notes get kept, and the engineering that keeps the paper from wearing through.

Four collectors.

The contract is already collecting. SanDisk pre-sold the paper. Two-thirds of FY2028 bits are spoken for by eight customers, before a single HBF stack ships.

The stacking layer collects only if HBF becomes a real part. Flash wearing HBM’s clothes: stacked dies, tiny vertical wires, a logic chip, the same footprint as the expensive memory. That work pays the bonding and test shops, the same class of names that got paid when HBM got stacked.

The placement layer is the sleeper. Whoever decides which notes stay on the desk and which get thrown out owns how fast the paper wears. That is NVIDIA’s Dynamo, CMX, and BlueField. NVIDIA gets paid at least once or twice. They own the hallway. SanDisk gets paid once, when it sells the part.

The equipment layer collects on thinner paper. Fifty-four percent more bits per wafer per generation is a tools story. The deposition and etch vendors get paid no matter which flash brand wins share.

The Tax Dodge promised that if the tax moves down the hierarchy, we map who collects there. Rick placed the feasibility; the monetization read is mine.

As we wrote in The Bandwidth Tax: “the investable question is where the next unit of value accrues while qualified DRAM dies and stacks remain scarce.” Swap DRAM for NAND and the question survives intact, because bits are not the scarce thing in flash. YMTC’s rise, covered in the bear case below, is the demonstration that raw bits are the most contestable commodity in memory. What is scarce is everything that turns a cheap bit into a qualified tier: the contracts, the packaging, the placement software, and the endurance engineering.

The contract layer collects first, and it is already collecting. The NAND bit agreements are the receipt. Two-thirds of FY2028 bits sold forward to eight data-center customers is a constraint priced at signing. If the margin structure lands anywhere near the deck’s ambition, the LTA is where the tier-below tax shows up in a financial statement first.

The stacking and packaging layer collects if HBF scales. HBF is NAND in HBM’s clothing: stacked dies, through-silicon vias, a logic interface, an HBM-matching footprint. Every dollar of that integration lands on the same class of suppliers the first installment mapped for DRAM: bonding, assembly, and test. The tax dodging HBM’s interposer at the top of the hierarchy re-arrives as stacking complexity at the bottom.

The placement layer is the sleeper. If bytes tier across HBM, LPDDR, TLC, and QLC, then the software and controllers that decide which byte lives where hold the margin lever for the whole hierarchy. That is NVIDIA’s Dynamo, CMX, and BlueField at rack scale, and controller and firmware players below it. Whoever manages the churn owns the endurance budget, and the endurance budget is the cost of goods of a KV tier. In English: the teacher who decides which notebooks get kept decides how fast you have to buy new paper. And note the asymmetry between the two routes. NVIDIA gets paid at least once or twice; they control the whole stack and data flow. SanDisk collects once, when it sells the part. If NVIDIA’s fabric route wins the KV bytes, the collection point is the platform layer NVIDIA already owns and the flash vendor sells drives. If HBF wins them, the collection moves to qualification and packaging. Either way, the bit stays the cheapest part, and the bit side of this is gonna turn into a huge volume game where whoever can absorb the lowest margins at scale will win.

The equipment layer collects on the layer-count race. A supply strategy built on 54 percent more bits per wafer per generation is a deposition and etch intensity story, and it accrues to the NAND tool makers regardless of which flash vendor wins share.

Bits are the contestable commodity. The tax collects at qualification: contracts, stacking, placement, and tools.

The honest read on timing: I am on the fence about when, and whether, HBF ships in accelerator volume. Late 2028 on the vendor’s own deck means 2029 in practice, and the graveyard of memory-adjacent architectures we walked in The Tax Dodge argues for humility about all of it. But the contract layer does not wait for HBF. The scarcity-forward selling is happening now, on read-mostly workloads that are already proven, and that is the part of this piece that does not depend on a 2029 product date.

And the positioning read, which is mine and not Rick’s: I’ve always been a fan of complex things with compute that have a moat and have a software layer. Obviously that’s NVIDIA, and maybe some of the optics, which are harder to replicate. I wouldn’t try to buy commodity storage, flash, and maybe some memory. It’s just too volatile. Demand is real and healthy and it’s going, but I kinda feel like it’s an extended, bigger super cycle. It’s still a cycle, and people need to understand that.

Where This Breaks

1. Bits drown the tier. NAND overcapacity is structurally easier to create than DRAM overcapacity. Cheap paper is easy to overproduce. DRAM, the expensive memory, needs EUV machines almost nobody else has. Flash does not. China’s YMTC is already the number-three NAND shipper. If the world can print paper this fast, an 80 percent gross-margin target is a claim that contracts beat commodity physics. That bear hits the cell vendors. It actually helps the qualification story. Abundant cheap bits make the accepted shelf the only scarce thing left. Hybrid bonding via Xtacking lowers the barrier further. YMTC is now the number-three NAND player by bit shipments as of Q2 2026 per TrendForce, at 14 percent share, having passed Kioxia, though still fifth by revenue on a consumer-heavy mix, with Fab 4 and Fab 5 online by 2028. Zephyr’s read is that the incumbents’ share loss continues from here, and the history of this industry says supply discipline rarely survives a demand slide this good. This is the bear that attacks the margin ambition directly: an 80 percent gross margin target, held from an 84.6 percent print, published into a rising YMTC supply curve is a claim that contracts beat commodity physics. It changes the slope for every cell vendor. It does not change where the tax collects, and it arguably strengthens the qualification layers, because abundant cheap bits make the qualified tier the only scarce thing left.

Roughly 8x KV compression in about a year. The bear that changes direction, not slope: another step like this deflates the KV rows of every NAND forecast.

2. Compression outruns the pull. Labs got better at taking shorter notes for the same conversation. DeepSeek cut the notebook size about eightfold in a year. If the notes get that much smaller, you need that much less paper. Context windows still got longer, so people are stuffing more into each session even as each page got cheaper. The race is whether shorter notes, or longer sessions, win. SanDisk’s 420-exabyte KV line is a bet that the sessions win. DeepSeek cut KV cache per million tokens from roughly 35 gigabytes in R1 to 4.4 gigabytes in V4, per Zephyr’s recap, and DeepSeek’s own V4 paper corroborates the magnitude, reporting about a tenth of V3.2’s KV cache at million-token context. Every frontier lab is pushing context length up while pushing KV bytes per token down. This is the standing algorithmic-compression short our memory coverage has carried all year, and it lands inside the compression window we flagged as the thesis-breaker. The market is continuously scared of the oversupply of memory and storage, and the fear is that companies just use less and optimize, and that whole bull thesis goes away. SanDisk fell 11 percent in a day on a compression paper in March; we defended memory demand through that selloff, and the defense has held every print since. We’re always gonna Jevons. The cheaper things get, the more you’re gonna use. There’s always gonna be memory optimizations, but it’s not gonna kill stuff. But a 420-exabyte KV forecast is the largest single bet yet that cached state keeps outgrowing compression, and we will not pretend the race is settled. This is the bear that changes direction, not slope: if KV bytes per useful task fall faster than tasks grow, the KV rows of the demand slide deflate and the NAND story shrinks back to staging and data lakes.

3. HBF never ships in volume. HBF slips, ships as a weights-only curiosity, or joins the graveyard of near-memory ideas. The textbooks stay on ordinary SSDs and on LPDDR. The staging pile, the filing cabinet, the bit contracts, and the tool vendors all survive. The 420-exabyte notebook forecast does not, unless some other controller figures out the writes. The timeline has already slipped two years against what we printed in July, the deck’s HBM comparisons have a flagged accuracy problem, and the endurance question forces a hybrid design that keeps HBM in the socket anyway.

Two dated tests. By the end of 2027, a real HBF spec with wear ratings, plus one named chip that will actually use it. That turns the stacked shelf from a slide into a product. By the end of 2028, a hyperscaler or a frontier lab saying out loud that they keep chat notebooks on flash, with the economics attached, and that fetching the notes beats starting the textbook over. Until those two print, the 420-exabyte line is a forecast, not a receipt.

What would change our mind. First, by end-2027: a 1.0-or-later HBF specification carrying endurance and qualification requirements out of the SanDisk and SK Hynix standardization work, plus at least one named accelerator design win, would move HBF from roadmap to product and activate the stacking-layer leg. Second, by end-2028: a hyperscaler or frontier lab disclosing production KV-cache offload on flash with economics attached, reload beating recompute at disclosed context lengths, would validate the KV rows. If neither arrives on schedule, the verdict in plain words is the one I’d give you on a call: things are delayed, it’s not working out exactly how we thought it was gonna happen in theory, and we’re just gonna use more of the same memory. We will write that follow-up, and the verdict will print in the series’ own terms: the tax stayed where it was.

So What?

For two pieces we watched the industry climb around expensive HBM from above: delete the interposer, stack SRAM, promote cheap LPDDR. SanDisk’s investor day is the same move from below. The storage floor reaching up for bytes that used to live in memory. Both sides count bytes. The constraints are not bytes. At the top of the stack the limit is watts. At the bottom it is writes. Which AI bytes belong on NAND has a narrower answer than a zettabyte. The textbooks already belong, and they are already paying. The notebooks are the open question.

Placement, endurance, and compression will settle the notebook question in public over the next two years. The layers that make the tiering decision, contracts, packaging, placement, and tools, are where the tier-below tax collects. The commodity bit is the one thing here we would not underwrite, and YMTC is the reason. That is not a contradiction with the contract layer collecting first: the NBMs price the committed, qualified tier, not the merchant bit.

Long the constraint, not the cell. In the tier below, the constraint is the write.

Like the first two installments, this piece is free and citable on purpose. The thesis is the marketing. The expression, the names and entries across the four collection layers, including the under-covered controller and equipment names, goes to paid subscribers in the follow-up.

Coming Up

The rest of the Hot Chips field notes, beyond the tutorial room reported above, go to paid subscribers this week. The paid follow-up to this piece, the collection map for the tier below with names and entries, follows. Rick and I are also planning a joint Substack chat Q&A on this piece, engineering feasibility and monetization side by side, like the one we ran for The Bandwidth Tax.

One more piece of housekeeping. BEP's principal runs occasional SPVs in private companies across this stack, offered to verified accredited investors under Rule 506(c). If you want to see those deals when they open, join the list at tools.bepresearch.com/syndicate.

What to Watch

Watch the spec, not the slide. A published wear and interface spec is the single most important disclosure in this story. Watch who fills NVIDIA’s named context shelf, ordinary drives over the fabric or HBF in the package. Watch whether a lab admits, in public, that flash is holding live agent notes. Until then, long the constraint, not the cell. In the tier below, the constraint is the write.

• CMX design-ins and the Storage-Next qualification lists. The Hot Chips memory track was the first public round of that argument; the tutorial section above reports it.

• NAND bit agreement expansion beyond the current eight data-center customers, and whether competitors sign equivalents. The contract layer is the tell that collects first.

• The next frontier-model KV disclosure. Another DeepSeek-scale compression step would pressure the 420EB row directly.

• YMTC Fab 4 and Fab 5 progress and quarterly bit share. The overcapacity bear runs through Wuhan.

• SK Hynix’s dual position: the HBM leader co-authoring the flash tier below it is either a hedge or a confession, and its own HBF commentary will say which.

• Answers to the two questions this piece puts on the record: which workload is HBF’s first qualification target, and whether SanDisk or SK Hynix will publish sustained application bandwidth, p99 latency, package power, and the resulting reduction in GPUs required; and from NVIDIA, at what model size or topology near-package flash reduces the minimum GPU count, and which model state must remain in HBM.

Sources

• SanDisk Investor Day materials, August 13, 2026 (bit shipment forecast, 2030 demand breakdown, BiCS productivity, NAND bit agreements with eight data-center customers, non-GAAP gross margin target, HBF roadmap); primary deck and press releases reviewed for the gross margin and customer-coverage figures

• OCP High Bandwidth Flash Architecture Specification v0.7.0 (high-level base-die specification: UCIe connectivity, 8-high and 16-high stacks, capacity and bandwidth grades)

• Zephyr (Citrini Research), investor-day recap and analytical framings: the conservative-forecast call, the persistent-agent-memory estimate, the YMTC share-loss read, and the HBF weights-only read, credited throughout; X thread flagging the deck’s HBM spec comparisons

• NVIDIA blog, FMS 2026 (Jason Hardy, Aug 4, 2026): CMX Context Memory Storage, cuFile open-sourcing, SCADA, Vera BlueField-4 STX claims, Storage-Next vendor list

• SanDisk, “Memory-Centric AI” (company blog)

• SanDisk, “Scaling Beyond the Wall: Inside SanDisk’s High Bandwidth Flash for AI” (company blog)

• SanDisk and SK Hynix, HBF global standardization (Feb 25, 2026)

• DeepSeek-V4 technical report (arXiv, 2026): roughly 10 percent of V3.2 KV cache at million-token context; the R1 35 GB and V4 4.4 GB per-million-token working figures are per Zephyr’s recap

• David Patterson et al., January 2026 paper on memory-hierarchy research directions (HBF characterization)

• TrendForce Q2 2026 NAND data as reported (Caixin, Electronics Weekly): YMTC number three by bit shipments at 14 percent, fifth by revenue

• Hot Chips 2026 memory tutorial Slack, #t1-memory_technology, August 23, 2026 (Oxmiq “HBF in AI Compute” session Q&A; Samsung and Micron speaker responses); attendee assertions marked as such in the text

Related BEP Research

• The Bandwidth Tax

• The Tax Dodge: Qualcomm’s 133 TB/s Isn’t What You Think

• The Hierarchy Rewrites

• TurboQuant and Dynamo: Why the Market Sold the Wrong Story

• NVIDIA Vera: When CPU Latency Becomes GPU Economics

About the Authors

Rui “Rick” Xie is a researcher in computer systems, memory architecture, and AI infrastructure (rickxie.cn). He holds a PhD from Rensselaer Polytechnic Institute, where his research focused on memory architecture and computer systems, with a particular interest in DRAM and SSDs. For this piece he set the framing boundaries, built the endurance, write-amplification, and reload arithmetic through public sources, and read the OCP HBF v0.7.0 specification and SanDisk’s primary investor-day materials. His views and analysis are his own and do not represent his employer.

Ben Pouladian is the publisher of BEP Research and CEO of BEP Holdings. He led the investment framing and the collection-map analysis. Framings are credited to their originators throughout. More at www.bepresearch.com

Disclosure: BEP Research’s principal is long NVDA, NOW, LITE, CRDO, TSEM, ALAB, WOLF, SMCI, BE, NBIS, and ORCL (2027 LEAPS). No position in SNDK or MU. This is investment research, not investment advice. Do your own work.

---