If They're Right, Sohu Will Change the World!
Imagine replacing 160 H100 GPUs with just one 8xSohu server!!
The AI industry is on the brink of a revolution, and at the forefront of this change is Etched's groundbreaking Sohu chip. Just as Bitcoin reshaped the financial landscape, Sohu promises to redefine AI capabilities. Etched has embedded the transformer architecture directly into the chip, creating a specialized solution that far outperforms traditional GPUs in both speed and efficiency. This leap is not just incremental; it's a paradigm shift that could revolutionize AI applications across the board. Imagine replacing 160 H100 GPUs with just one 8xSohu server, offering unprecedented efficiency and performance.
500,000 tokens per second!!!
Sohu is the world’s first transformer ASIC, designed specifically for AI workloads. With a memory capacity of 144 GB HBM3E and the ability to process over 500,000 tokens per second, Sohu is set to handle tasks previously thought impossible on GPUs. This kind of specialized hardware is essential as we move into an era where AI models are becoming larger and more complex.
The economic implications are equally significant. The cost-effectiveness of Sohu could democratize access to powerful AI, enabling more startups and smaller companies to harness advanced AI capabilities without the prohibitive costs associated with current GPU-based solutions.
GPUs Are Hitting a Wall
With Moore’s law slowing, the only way to improve performance is to specialize
Santa Clara’s dirty little secret is that GPUs haven’t gotten better, they’ve gotten bigger. The compute (TFLOPS) per area of the chip has been nearly flat for four years. From 2022-2025, AI chips haven’t really gotten better, they’ve just increased in size. NVIDIA’s B200, AMD’s MI300, Intel’s Gaudi 3, and Amazon’s Trainium2 use the trick of counting two chips as one card to "double" performance. All GPU performance gains in this period rely on this method, except for Etched.
With Moore’s law slowing, the only way to improve performance is to specialize. By embedding the transformer architecture directly into the Sohu chip, Etched has avoided the pitfalls of brute-force scaling. Instead, they’ve created a specialized solution that maximizes efficiency and performance.
This graph clearly shows that the compute density (TFLOPS per mm²) of GPUs has barely improved, highlighting the need for smarter, more specialized designs like Sohu.
Why Current GPUs Are Not That Smart and Depend on Scale
Only 3.3% of the transistors in an H100 GPU are used for matrix multiplication!
Current GPUs, while powerful, rely heavily on scaling up their size and power to achieve performance gains. This approach has several limitations:
Brute Force Scaling: GPUs like Nvidia's H100 achieve improvements by increasing the number of cores and power consumption rather than through smarter, more efficient designs. This leads to diminishing returns as adding more power becomes less efficient.
Low Utilization: General-purpose GPUs are designed to handle a variety of tasks, resulting in a lower percentage of their transistors being used for any specific task. For example, only about 3.3% of the transistors in an H100 GPU are used for matrix multiplication, which is critical for AI workloads.
Specialization vs. Generalization: The flexibility of GPUs makes them less efficient for specific tasks. In contrast, Sohu’s specialized design, tailored for transformer models, allows it to achieve over 90% FLOPS utilization, maximizing performance for its intended purpose.
Heat and Energy Constraints: As GPUs grow larger and more powerful, they generate more heat and consume more energy, which presents significant challenges in cooling and operational costs.
Specialization Needed: The variety of architectures that GPUs need to support (CNNs, LSTMs, SSMs, etc.) means they cannot be as efficient as specialized chips like Sohu, which are designed specifically for transformer models. This specialization allows Sohu to achieve higher performance and efficiency by dedicating more resources to the specific tasks required by transformers.
By addressing these limitations, Sohu promises to deliver smarter, more efficient performance that doesn't rely solely on scaling up power and size.
Why I Hope Sohu Succeeds
The success of Sohu is crucial for fostering healthy competition in the AI hardware market. Currently, Nvidia dominates with its powerful but expensive GPUs. A viable competitor like Sohu can drive innovation and lead to better performance at lower costs. This competition will ultimately benefit consumers and businesses by providing more options and preventing monopolistic practices that stifle progress.
The Promise of Sohu
Sohu’s architecture is a game changer. By focusing on a single algorithm—transformers—it removes unnecessary control flow logic and dedicates more resources to essential computations. This design choice makes Sohu an order of magnitude faster and cheaper than even Nvidia’s next-generation Blackwell (B200) GPUs.
The implications are profound. With the ability to run over 500,000 tokens per second, Sohu can enable products and applications that are currently impossible with GPUs. Whether it’s real-time voice agents or dynamic content generation, Sohu’s capabilities open new horizons for AI.
If the transformer architecture evolves or becomes obsolete. Sohu will become useless!
However, there's a significant risk involved. If the transformer architecture evolves or becomes obsolete, Sohu's specialized design could become less relevant. Despite this, betting on Sohu could revolutionize AI technology if it succeeds. Its potential to offer superior performance and efficiency makes it a worthwhile gamble in the fast-evolving AI landscape.