Processing AI frenzy: Nvidia speeds ahead with H100 booster

The breathless artificial intelligence (AI) frenzy is currently the rallying theme on the Wall Street.
Riding on the wave, Nvidia Corp, already the world’s most valuable semiconductor firm, has become the first computer-chip company ever to hit $3tn in market capitalisation.
Nvidia’s H100 processor has enabled a new generation of AI tools that promise to transform entire industries. Demand for the H100 is so great that some customers are having to wait as long as six months to receive it.
The H100 is a beefier version of a graphics processing unit that normally lives in PCs and helps video gamers get the most realistic visual experience.
It includes technology that turns clusters of Nvidia chips into single units that can process vast volumes of data and make computations at high speeds. That makes it a perfect fit for the power-intensive task of training the neural networks that underpin generative AI.
Nvidia says the H100 is four times faster than the chip’s predecessor, the A100, at training so-called large language models (LLMs), and is 30 times faster replying to user prompts.
Since releasing the H100 in 2023, Nvidia has announced versions that it says are even faster — the H200 and the Blackwell B100 and B200.
For companies racing to train LLMs to perform new tasks, that growing performance edge can be critical. Many of Nvidia’s chips are seen as so key to developing AI that the US government has restricted the sale of the H200 and several less capable models to China.
Founded in 1993, the Santa Clara, California-based Nvidia is the world leader in graphics chips.
The most powerful of those are built with hundreds of processing cores that perform multiple simultaneous threads of computation, modelling complex 3D renderings like shadows and reflections.
Nvidia’s engineers realised in the early 2000s that they could retool these graphics accelerators for other applications, by dividing tasks up into smaller lumps and then working on them at the same time.
Nvidia now controls about 92% of the market for data centre GPUs, according to market research firm IDC. Dominant cloud computing providers such as Amazon.com’s AWS, Alphabet’s Google Cloud and Microsoft’s Azure are trying to develop their own chips, as are Nvidia’s rivals Advanced Micro Devices and Intel.
Those efforts haven’t made much headway in the AI accelerator market so far, and Nvidia’s growing dominance has become a concern for industry regulators.
AMD, the second-largest maker of computer graphics chips, unveiled a version of its Instinct line last year aimed at the market that Nvidia’s products dominate.
Intel is now designing chips geared for AI workloads but acknowledged that, for now, demand for data centre graphics chips is growing faster than for the processor units that were traditionally its strength.
Nvidia’s shares have rallied roughly 147% this year, adding about $1.8tn as the insatiable demand for its chips used to power AI tasks skyrockets.
The last time Nvidia was worth more than Apple was in 2002, five years before the first iPhone was released. At the time, both companies were worth less than $10bn each.
The rise of generative AI is a new industrial revolution and Nvidia expects to play a major role as the technology shifts to personal computers, says co-founder Jensen Huang. The company has been arguably the biggest beneficiary of a massive flood of AI spending.
Amid the unabated AI appetite on the Wall Street, Microsoft, Nvidia and Apple are now worth more combined than China’s stock market.
With a total market capitalisation of about $9.2tn, the three most valuable tech firms have overtaken all of the nearly $9tn worth of stocks actively traded on Chinese exchanges excluding Hong Kong, according to data compiled by Bloomberg.