Nvidia sold 900 tons of artificial intelligence chips in the second quarter of the year

According to a new report from research firm Omdia, chip giant Nvidia sold 900 tons of its H100 GPU in the second quarter due to “a strong acceleration in demand.”

According to Omdia analysts, Nvidia shipped 900 tons of its new H100 GPUs in the second quarter. This is due to increased demand for Nvidia’s AI chips, most of which are likely to come from Meta.

Accelerating demand for Nvidia’s AI chips is pushing up average server prices, according to Omdia research.

According to the latest Cloud and Data Center Market Insights report, the number of GPUs being installed in hyperscale data centers has increased significantly. However, this resulted in a 1 million unit or 17% decrease in server shipments compared to last year. Notably, this is the first time in 15 years that server submissions have decreased.

Instead, Omdia reports a “bold rush” into AI GPUs, with AI hardware price tags pushing average server prices up more than 30 percent year-over-year. This “dramatic shift” in the server mix toward high-end servers led Omedia to forecast a market size of $114 billion, up 8 percent from last year.

Nvidia is set to gain the most, with Omdia reporting that 22 percent of its $13.5 billion in second-quarter revenue came from one customer; Probably meta, provided. Omdia also expects Microsoft, Google and other major cloud service providers to become important customers for Nvidia.

Demand for AI hardware will continue through the first half of 2024, especially as TSMC’s chip foundry ramps up the packaging capacity needed to build the H100.

Growing demand for artificial intelligence hardware has led companies and the US government to rent out their infrastructure, letting others use powerful computers like Perlmutter’s supercomputer and the new Hugging Face training cluster at low rates.

However, the demand for AI hardware does not mean that the use of AI has also grown rapidly. “The rapid investment in AI training capabilities that we are currently experiencing should not be confused with rapid adoption of the technology,” Omdia’s report states.

Omdia expects the AI ​​wave to continue until 2027 and predicts that in the next five years the number of large language models will increase as many consumer applications.

Artificial intelligence (AI) is becoming more specialized by focusing on industry-specific models such as Med-PaLM, Bio-GPT, BioBert, and PubMedBert. These models are built upon broader AI models such as ChatGPT or Claude that can handle a wider range of tasks and data. This shift reflects the increasing importance of adapting AI technology to specific industries, while larger, more general models of AI become less prominent.

However, even if the adoption of AI by companies and other organizations increases, the need for AI servers should decrease over time. According to the research firm, this is because AI models are becoming more efficient and require fewer resources such as domains, parameters, data size, tokens and periods.