Meta’s 24k H100 Cluster Pods for Llama3 training – Meta AI

  • Editor
  • March 13, 2024

In a significant leap forward for artificial intelligence, Meta has announced the launch of two colossal 24k GPU clusters, underscoring its deep investment in the AI-driven future.

This development marks a pivotal moment in Meta’s ambitious infrastructure roadmap, aiming to harness unprecedented compute power for diverse AI workloads, including the training of its advanced Llama 3 models.

Cofounded and lead PyTorchat at Meta said in the official X post:

Meta’s commitment to fostering open compute and source initiatives shines through this latest endeavor. By building these clusters atop industry pillars such as Grand Teton, OpenRack, and PyTorch, Meta reinforces its pledge to drive open innovation across the tech landscape. This approach not only democratizes artificial intelligence development but also sets a new benchmark for collaborative progress in the field.

Here is how people are responding to this news:

With its eyes set on the horizon, Meta’s ultimate goal is to construct artificial general intelligence (AGI) that is both open and responsibly built. This vision ensures AGI’s broad availability, enabling widespread benefits.

As Meta strides toward this future, it continues to scale its clusters, significantly enhancing its AI offerings. These advancements pave the way for innovative products, features, and computing devices centered around AI.
As always there are some controversial remarks:

Reflecting on its journey, Meta highlights the instrumental role of its AI Research SuperCluster (RSC) in accelerating AI research. The RSC, equipped with 16,000 NVIDIA A100 GPUs, has been vital in the development of Llama and Llama 2 models, among others.

Looking ahead, Meta’s roadmap includes an expansive infrastructure build-out featuring 350,000 NVIDIA H100 GPUs, representing compute power akin to nearly 600,000 H100s by the end of 2024.

People seemed to be really excited about the news:

As Meta embarks on this ambitious journey, it remains steadfast in its commitment to building a responsible AI ecosystem. By advancing open and responsible AI research, Meta not only pushes the boundaries of technology but also ensures its advancements benefit society at large.
For more of such latest AI news, visit AI news

Was this article helpful?
Generic placeholder image

Dave Andre


Digital marketing enthusiast by day, nature wanderer by dusk. Dave Andre blends two decades of AI and SaaS expertise into impactful strategies for SMEs. His weekends? Lost in books on tech trends and rejuvenating on scenic trails.

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *