TurboQuant co-developer clarifies the 6x memory savings and 8x speed claims that rattled chip stocks
When Google unveiled TurboQuant on March 24, headlines declared the algorithm could slash AI memory use sixfold with zero accuracy loss and deliver eight times faster processing.
Within days, Samsung Electronics and SK hynix shed billions in market value. Micron plunged nearly 10 percent. The Philadelphia Semiconductor Index dropped 4.8 percent.
Investors read the numbers and reached a simple conclusion: If AI needs far less memory, the companies making that memory are in trouble.
Two weeks on, the panic has largely subsided, with analysts drawing comparisons to the short-lived DeepSeek shock of early 2025. But much of the public understanding of what TurboQuant actually does — and what it does not — remains fuzzy.
Han In-su, an assistant professor of electrical engineering at Korea Advanced Institute of Science and Technology who co-developed two of TurboQuant's three core algorithms, agreed to an email interview with The Korea Herald to separate fact from hype.
Han, who has served as a visiting researcher at Google Research since 2025, is among the few people positioned to speak with direct technical authority on what the algorithm does and does not deliver.
Does TurboQuant really cut AI memory by six times?
Not exactly. TurboQuant compresses the key-value cache, the temporary memory where AI models store the context of a conversation, not the model's entire memory footprint.
How much that matters depends on how long the input is. In a state-of-the-art model, the KV cache accounts for less than 1 percent of total memory during short exchanges, Han said. But during long-context tasks of 200,000 tokens or more, it can exceed 60 percent and become the primary bottleneck.
"If you compress the KV cache by 6x in a long-context environment, overall memory reduction could reach approximately 2x," Han said.
Is it really lossless?
The two headline numbers, "6x compression" and "zero accuracy loss," are both real, but come from different settings in the paper. They cannot be combined into a single claim.
When the compression is dialed to roughly 4.6 times, benchmark scores match the uncompressed model exactly. Push it further to 6.4 times and scores dip by about 0.6 point on a 50-point scale.
"Around 4 to 5x compression shows no observable performance degradation. At 6.4x, there is a slight decrease," Han said, though he added that the gap would likely be imperceptible in everyday use.
Think of it as a dial, not a switch: Operators choose where to set the trade-off.
What about the 8x speed claim?
The eight-times figure applies only to the computation of attention logits — the step where the model determines which parts of its stored context are relevant. It is only one of several steps, and others are unaffected.
For short conversations, this stage is a small fraction of total processing time. For very long inputs, it dominates. Han offered a practical rule of thumb: "If attention computation accounts for half of total inference time and that portion becomes 8x faster, overall speed improvement would be roughly 1.5 to 2x."
The paper intentionally omits end-to-end latency figures because total speed depends heavily on the type of graphics processing unit, batch size and software framework, Han said.
Will it work on the large models companies actually deploy?
The paper's experiments used models of around 8 billion parameters, raising questions about performance at the 70 billion-plus scale used in commercial deployments.
Han pointed out that TurboQuant's mathematical guarantees depend on vector dimension and bit width, not total parameter count.
Vector dimensions do not increase proportionally with model size, so the theory should hold regardless of whether the model has 8 billion or 70 billion parameters.
"It would not be accurate to say performance would break down in larger models," he said.
What is TurboQuant actually for?
Han reframed the technology's purpose.
"It is more accurate to understand this not as 'saving memory' but as 'being able to do more with existing resources,'" he said.
Freed memory lets operators handle longer documents, serve more simultaneous users or run larger models on the same hardware. The primary design goal, Han emphasized, was not to save memory at all — but to speed up inference by reducing the volume of data moving through the memory bus.
What Han did not say
Several Korean outlets attributed market forecasts to Han, including predictions that TurboQuant would ultimately benefit chipmakers.
But Han told The Korea Herald that he did not make these statements. The market-oriented language in the KAIST press release on March 27 reflected the university's framing, not his own opinion.
"I was focused entirely on building a compression technique that reduces the memory bottleneck," Han said. "I did not think about what impact this might have on the hardware market."
He did note, however, that the underlying demand driver was structural: Today's AI models store every step of their reasoning individually, and "memory requirements continue to increase in proportion to reasoning length."
mjh@heraldcorp.com
