在2022年年底ChatGPT發佈後，對整個半導體市場產生了衝擊性的影響。從那個時候開始，AI晶片的設計加速了它的變化。

過去，Moore’s Law每2~2.5年翻倍電晶體密度，而基本上電晶體密度的提升就是主要算力提昇的來源。但是，從大語言模型爆發以後，我們可以感受到，整個晶片設計方式產生了很大的變化，算力產生的邏輯有了一個很大的典範轉移。

The release of ChatGPT at the end of 2022 had a disruptive impact on the entire semiconductor market. From that point on, the pace of change in AI chip design accelerated.

In the past, Moore’s Law doubled transistor density every 2 to 2.5 years, and that density gain was essentially the main source of compute growth. Since the rise of large language models, however, the way chips are designed has changed substantially, and the logic by which compute is produced has gone through a major paradigm shift.

你可以發現，這幾年大家對半導體產業的討論，有著很大的變化。從電晶體層級的討論，到更多先進封裝、不同類型記憶體、甚至雷射、光通波導設計……等的討論。

You can see that the industry conversation about semiconductors has changed a great deal over the past few years. It has moved from the transistor level to advanced packaging, different types of memory, and even lasers and optical waveguide design.

那麼，為什麼會這樣呢？原因不是先進製程不再進步了，而是因為運算的硬體單位範圍徹底改變了。

Why is this happening? Not because advanced process nodes have stopped improving, but because the hardware unit of computation has changed completely.

這幾年發生了什麼重大的變化？

So what are the major changes of the past few years?

半導體產業的典範轉移: AI-Silicon Co-Evolution

The Semiconductor Paradigm Shift: AI-Silicon Co-Evolution

首先，我們可以觀察到，AI晶片的設計的趨勢發生了三個主要的變化：

First, we can observe three main shifts in the trend of AI chip design:

1. 晶片設計的單位不再是以單顆晶片為單位，而是以整個Rack，甚至於資料中心為單位。
2. 整個Rack、資料中心所有晶片及軟硬體的共同優化，提供了比電晶體密度提昇更快速的算力成長。
3. AI模型的演進，會是影響硬體設計的關鍵因子。AI模型現在正在持續快速演化，而AI模型的演化會驅動AI晶片設計的演化。

1. The unit of chip design is no longer a single chip, but an entire rack, or even a whole data center.
2. Co-optimizing every chip and all of the software and hardware across the rack and the data center delivers compute growth faster than transistor density scaling.
3. The evolution of AI models is the key factor shaping hardware design. AI models are evolving rapidly and continuously, and that evolution drives the evolution of AI chip design.

因為上述三點的關係，其實我們觀察到，整個半導體產業和供應鏈正在發生劇烈的變化。我們可以預期接下來的半導體硬體設計和供應鏈還會持續有很大的變化。

Because of these three points, we are seeing dramatic change across the entire semiconductor industry and its supply chain. We can expect semiconductor hardware design and the supply chain to keep changing substantially in the years ahead.

具體的現象我們可以觀察到的：包含目前記憶體的設計正在快速演化、AI Rack的設計每一代都還在快速的演進，或是CPO生態系目前的快速成長，都是源自於目前正在發生的典範轉移。

Concrete signs we can observe include the rapid evolution of memory design, the generation-by-generation evolution of AI rack design, and the fast growth of the CPO ecosystem. All of these stem from the paradigm shift now under way.

因此，我們可以看到AI硬體整體技術和供應鏈的瓶頸正在移動當中，就是因為AI模型除了在持續變大外，在演算法上也在持續的變化。

That is why the bottlenecks across AI hardware technology and the supply chain are moving: AI models are not only getting larger, their algorithms are changing as well.

AI模型正在驅動晶片硬體的演進

AI Models Are Driving the Evolution of Chip Hardware

而具體來說，現在AI硬體的設計正在快速特化當中。原因是因為，如果我們將AI模型拆開的話，我們會發現，AI模型每一個Phase所需要的硬體配置並不一樣。

More specifically, AI hardware design is specializing rapidly. The reason is that once you break an AI model apart, each phase of the model needs a different hardware configuration.

如圖：
– Training主要是Compute Bound
– Inference起碼還需要分成Prefill和Decode Phase，Prefill為Compute Bound，Decode為Memory Bound。

As shown in the figure:
– Training is mainly compute bound.
– Inference splits at least into a prefill phase and a decode phase. Prefill is compute bound; decode is memory bound.

而這些Phase還可以再細分，這些演算法還在繼續演化。因此，目前會看到很多新的類型的晶片在專注在解決更多細分的問題。如Groq晶片，NVIDIA 把他定位為解決Decode中的FFN Phase，而Google也首次在TPU v8把晶片設計分成專門給Training的TPU v8t和給Inference的TPU v8i。

These phases can be broken down further, and the algorithms are still evolving. This is why we now see many new types of chips focused on ever more specific sub-problems. NVIDIA positions the Groq chip as the solution for the FFN phase within decode, and Google, for the first time with TPU v8, split its chip design into TPU v8t for training and TPU v8i for inference.

接下來AI模型和晶片設計都還會持續演進。

Both AI models and chip design will keep evolving from here.

介紹：Silicon Constraint Theory

Introducing the Silicon Constraint Theory

目前在AI模型的需求與硬體技術的發展下，會產生一些現行技術無法完全滿足模型運算需求的瓶頸。

Under the current combination of AI model demand and hardware technology development, bottlenecks emerge where existing technology cannot fully meet the compute requirements of the models.

如在Inference Decode的情況下，記憶體頻寬在搬運資料的速度，通常都會比GPU運算慢很多，導致在Inference Decode Phase下，通常GPU都要空著等記憶體把資料搬過來，也就是我們的Memory Bandwidth Bound (記憶體頻寬不足的情況)

In inference decode, for example, memory bandwidth moves data much more slowly than the GPU can compute. During the decode phase the GPU typically sits idle waiting for memory to deliver the data. This is what we call Memory Bandwidth Bound (a shortage of memory bandwidth).

而目前在AI模型和硬體都在持續發展的情況下，瓶頸會在各個運算資源分配不均勻的地方產生。各種限制會導致系統的價值流向瓶頸，比如說我們其實目前就在見證Memory Bandwidth Bound的發展。記憶體的頻寬和量的需求 (更多顆HBM就能湊出更高的頻寬)，導致記憶體公司的毛利從去年幾乎不賺錢，到現在2026年下半年，記憶體大廠的毛利都攀升到80%以上。

With both AI models and hardware still developing, bottlenecks form wherever compute resources are allocated unevenly. These constraints direct the system’s value toward the bottleneck. We are witnessing exactly this in the development of Memory Bandwidth Bound: demand for memory bandwidth and capacity (more HBM stacks add up to more bandwidth) has pushed memory makers from barely profitable last year to gross margins above 80% in the second half of 2026.

這就是我們所謂的：”Silicon Constraint Theory”。我們現在正在見證半導體產業一個很大的變化，AI系統算力的增長需求讓半導體產業需要整體性的看各個系統瓶頸，想辦法解決系統瓶頸或是做更好的資源分配。也就是說，AI系統的算力，取決於系統中的”Silicon Constraint”，而目前這樣的Constraint決定了整個供應鏈的價值會往哪邊移動。

This is what we call the ”Silicon Constraint Theory”. We are witnessing a major change in the semiconductor industry: the growing compute demand of AI systems forces the industry to look at every system bottleneck as a whole, and to either resolve the bottleneck or allocate resources better. In other words, the compute of an AI system is determined by the ”Silicon Constraint” within that system, and that constraint determines where value in the supply chain will move.

從HBM到Custom HBM

From HBM to Custom HBM

而為了解決目前AI系統的Silicon Constraint，我們觀察到整個半導體產業都在針對各種可能的瓶頸去推出新的技術規劃，放到Roadmap上。

To resolve the Silicon Constraint of today’s AI systems, we observe the whole semiconductor industry proposing new technology plans against every possible bottleneck and placing them on the roadmap.

比如三大記憶體廠 (SK Hynix、Samsung、Micron) 就和客戶一起提出”Custom HBM”，以新的設計來試圖減緩問題，提昇HBM頻寬，並且也在 Host Die上面空出更多的空間，讓人能夠放入更多的運算單元。

The three major memory makers (SK Hynix, Samsung, Micron), for example, have proposed ”Custom HBM” together with their customers. The new design attempts to ease the problem by raising HBM bandwidth while also freeing up more area on the host die, so that more compute units can be placed there.

其實為了解決記憶體頻寬的問題，大家也都試圖提出不同的解決方案，如Hynix所提出的AiM，Samsung所提出的PIM，都是希望能夠在記憶體內增加一些運算能力，不過目前最受到市場歡迎，且客戶比較歡迎的方向，還是Custom HBM。也就是，在解決這些瓶頸的研發上，整個生態系也會共同的去發展出一些比較能為大家接受的技術Roadmap。

Other solutions to the memory bandwidth problem have been proposed as well, such as Hynix’s AiM and Samsung’s PIM, both of which add some compute capability inside the memory. The direction the market and customers favor most, however, is still Custom HBM. In other words, as the ecosystem works to resolve these bottlenecks, it converges on a technology roadmap that most participants can accept.

之後我們會再解析一下目前Custom HBM的發展近況。

We will analyze the latest developments in Custom HBM in a later article.

新的典範正在創造不同的瓶頸，也正在改變整個供應鏈

The New Paradigm Creates Different Bottlenecks and Reshapes the Supply Chain

因此，我們會發現，AI模型的演進正在改變AI系統中各種硬體的需求，而硬體技術和產能的需求，又會改變整體平衡，產生出供應鏈的瓶頸，進而影響關鍵瓶頸的價格和獲利，如我們前面所分享的HBM的例子和目前的最新發展。

So we find that the evolution of AI models is changing the demand for every kind of hardware in an AI system. The resulting demand for hardware technology and capacity then shifts the overall balance, creates supply-chain bottlenecks, and in turn affects the pricing and profits at the critical bottleneck, as in the HBM example and the latest developments we shared above.

而我們提出Silicon Constraint Theory，就是為了幫助大家更好的去了解目前半導體硬體的瓶頸遷移和價值流動方向。在未來的文章中，我們會跟大家分享更多相關的分析。

We propose the Silicon Constraint Theory to help readers better understand where the bottlenecks in semiconductor hardware are migrating and where value is flowing. We will share more related analysis in future articles.