How Is zHBM Different from Conventional HBM?
이 포스팅은 쿠팡 파트너스 활동의 일환으로, 이에 따른 일정액의 수수료를 제공받습니다.
Samsung's zHBM targets AI response speeds of 100→1000 tokens per second…Vertical stacking is the key to the 10-fold speed target# Samsung's zHBM targets AI response speeds of 100→1000 tokens per second…Vertical stacking is the key to the 10-fold speed target
On 2026-09-16 (local time), Samsung Electronics announced a goal of increasing AI system response speeds from approximately 100 tokens per second per user to 1,000 tokens per second. The key technology, zHBM, is a next-generation 3D memory architecture that reduces the distance data must travel by stacking HBM vertically directly on top of an AI accelerator rather than placing it beside the accelerator. However, 1,000 tokens per second is a target set by Samsung, not a speed verified in currently available commercial products.
How Is zHBM Different from Conventional HBM?
The biggest difference is the arrangement of HBM and the AI accelerator. In conventional architectures, HBM sits beside the AI accelerator, whereas zHBM stacks HBM directly on top of it. Samsung designed zHBM to shorten the physical distance between the two components and reduce bottlenecks during data transfer.
Even if an AI accelerator can perform calculations quickly, overall system performance is constrained when the necessary data cannot be delivered on time. Large-scale AI models require increasing amounts of data to move between processors and memory, making the speed at which data can be supplied just as important as computational performance. This is why Samsung changed not only the speed of the memory itself but also the physical arrangement of the memory and accelerator in zHBM.
1,000 Tokens per Second and Up to 8 Times the Performance of HBM5 Are Not the Same Metric
The '10-fold' and 'up to 8-fold' figures presented in the announcement refer to different comparisons.
Samsung's proposed increase from 100 tokens per second to 1,000 tokens per second is a target for AI system response speed. The calculation is 1,000÷100=10, meaning the target is 10 times the currently stated level. By contrast, 'up to 8 times' refers to the performance of a next-generation interface system incorporating zHBM compared with HBM5. Because these are different metrics, they should not be interpreted as meaning that zHBM itself makes every AI system respond 8 times or 10 times faster.
In official materials released at FMS 2026, Samsung stated that a next-generation interface system incorporating zHBM could deliver up to 8 times the performance and up to 3 times the performance per watt compared with HBM5. It also presented a reduction in thermal resistance of more than half.
Why Does Placing Memory on Top of an AI Accelerator Make a Difference?
The central design principle of zHBM is to shorten the physical path that data must travel. Samsung presented an architecture using hybrid copper bonding and multi-wafer bonding technologies to increase signal transmission speeds and reduce thermal resistance.
This is different from simply stacking HBM higher. The defining feature of zHBM is its 3D architecture, which goes beyond stacking memory within HBM and changes the relative positioning of HBM and the AI accelerator from side by side to vertically stacked.
Samsung also outlined a design that allows customer-specific IP to be incorporated into the interlayer between the memory and AI accelerator, enabling memory capacity or accelerator characteristics to be optimized for individual customers.
Does This Mean AI Responses Will Immediately Become 10 Times Faster?
The announcements confirmed so far do not support that conclusion. 1,000 tokens per second is Samsung's target for AI system response speed in agentic AI applications, while zHBM remains a next-generation architecture intended for future AI systems. What Samsung unveiled at August 2026 FMS was a zHBM mockup and its technological vision.
Therefore, the statement '1,000 tokens per second' must be distinguished from the claim that 'generative AI services currently used by consumers will soon become 10 times faster.' The announcement did not present final response speeds from commercially deployed AI services.
What can currently be confirmed is that Samsung has introduced a zHBM architecture designed to reduce data-transfer bottlenecks between AI accelerators and HBM, and that it is developing this approach to improve AI system training performance, inference performance, and responsiveness.
Heat Management Is Another Key Challenge for zHBM
Thermal management becomes an important design consideration when HBM is placed directly on top of an AI accelerator. During the September 16 announcement, Samsung representatives described a co-design approach involving collaboration with AI accelerator customers from the earliest design stages to address the transfer of heat generated by the accelerator to the memory.
Samsung's previously released zHBM technical materials also included a target of reducing thermal resistance by more than half compared with HBM5. In other words, assessing zHBM's actual competitiveness requires examining not only its maximum performance figures but also whether performance, power efficiency, and thermal management can be achieved together in a vertically stacked architecture.
The Next Question Is How These Targets Will Be Achieved in Real Products
As of 2026-09-17, Samsung's direction is clear. The company aims to move beyond the conventional arrangement in which HBM sits beside an AI accelerator, connect the two semiconductors vertically, and shorten the distance data travels to improve AI system performance and power efficiency.
The figures must be evaluated according to their respective benchmarks. The AI system response-speed target is a 10-fold increase from approximately 100 tokens per second per user to 1,000 tokens per second. The next-generation interface system incorporating zHBM is projected to deliver up to 8 times the performance and up to 3 times the performance per watt compared with HBM5.
The important question for zHBM's next stage is when and under what conditions these figures can be achieved in actual systems combining customer-specific AI accelerators and memory. At this stage, the announcement is best understood not as 'AI response speeds have increased 10-fold' but as 'a technological target aimed at 1,000 tokens per second.'
Related Articles
이 포스팅은 쿠팡 파트너스 활동의 일환으로, 이에 따른 일정액의 수수료를 제공받습니다.