Elon Musk’s xAI activates “most powerful AI training cluster in the world” with 100k Nvidia H100 GPUs

Elon Musk’s artificial intelligence startup, xAI, has activated its supercomputer cluster in Memphis, Tennessee, which is comprised of 100,000 Nvidia H100 GPUs. Tesla CEO Elon Musk confirmed the milestone in a post on social media platform X. 

xAI had a busy weekend, with the AI startup announcing a hiring ramp on X. Musk highlighted these efforts on Monday morning, stating that xAI was looking to increase its human talent advantage. Interestingly enough, Musk also encouraged those who are interested in AI to apply to Tesla and X. 

But definitely looking to increase our human talent advantage, so please apply at @xAI, as well as @Tesla and @X https://t.co/sf4hMRco9f

— Elon Musk (@elonmusk) July 22, 2024

Musk then mentioned that a collaboration between teams from xAI, X, and Nvidia has enabled the Memphis Supercomputer cluster to start training at 4:20 a.m. local time. As per Musk, the supercomputer cluster features 100,000 Nvidia H100 GPUs, effectively making it the most powerful AI training cluster in the market today. 

“Nice work by xAI team, X team, Nvidia team, and supporting companies getting Memphis Supercluster training started at ~4:20 a.m. local time. With 100k liquid-cooled H100s on a single RDMA fabric, it’s the most powerful AI training cluster in the world! This is a significant advantage in training the world’s most powerful AI by every metric by December this year,” Musk wrote in a post on X. 

This is a significant advantage in training the world’s most powerful AI by every metric by December this year

— Elon Musk (@elonmusk) July 22, 2024

xAI’s supercomputer cluster is expected to be used to train the company’s large language model, Grok, which is currently rolled out as a feature for X Premium subscribers. Earlier this month, Musk noted in a post on X that xAI’s Grok 3 would be trained on 100,000 H100 GPUs so “it should be really something special.” 

Previous reports from Memphis back in early June suggested that xAI would be building a supercomputer cluster informally dubbed the “Gigafactory of Compute” at the 785,000-square-foot former Electrolux Memphis facility. In a press release, the Greater Memphis Chamber stated that xAI’s supercomputer project represents the city’s largest capital investment by a new-to-market company in Memphis history. 

xAI’s investment is substantial. As per a Benzinga report, each Nvidia H100 GPU is estimated to cost between $30,000 to $40,000. Considering that xAI is using 100,000 Nvidia H100 units, Elon Musk’s AI startup appears to have spent about $3 billion to $4 billion for the project.

Don’t hesitate to contact us with news tips. Just send a message to simon@teslarati.com to give us a heads up.

The post Elon Musk’s xAI activates “most powerful AI training cluster in the world” with 100k Nvidia H100 GPUs appeared first on TESLARATI.