Facebook’s owner wants extraordinary computing power to develop AI models to recognise speech, translate languages and power 3D worlds
24 January 2022
Facebook’s parent company, Meta, is building the world’s most powerful AI-specific supercomputer to develop better speech-recognition tools, automatically translate between different languages and help build its 3D virtual metaverse.
Although far from complete, the AI Research SuperCluster (RSC) is up and running and has already overtaken Meta’s previous fastest supercomputer. That machine was designed in 2017 and ran on 22,000 powerful graphics processing units (GPUs) which, despite being designed for playing games, are highly effective tools to train artificial intelligence models with.
RSC currently has only 6080 GPUs, but they are more powerful than those in the older machine and it is already three times faster at training large AI models than its predecessor. Its current performance is on a par with the Perlmutter supercomputer at the National Energy Research Scientific Computing Center in California, which is currently placed at number five in the TOP500 global supercomputer rankings.
When RSC is complete, it will consist of 16,000 GPUs and be almost three times more powerful than it is now. Meta says that at this point, it will be the fastest AI-optimised supercomputer in the world, performing at nearly 5 exaflops.
Supercomputers can be designed to excel at certain tasks. Meta’s machine is specialised to train and run large AI models. There will be more powerful computers in the world when it is complete, but only a few, and none that shares its exact architecture or intended use.
The cutting edge of AI research has relied on scale in recent years, and ever more powerful machines to train models with. One of the largest neural networks, the Megatron-Turing Natural Language Generation model, has 530 billion parameters, which are roughly equivalent to the connections between brain cells. Meta says its machine will eventually run models with trillions of parameters.
James Knight at the University of Sussex, UK, says the proposed computer is “huge” in scale, but may not overcome some of the challenges in AI research. “A system this large is definitely going to let them build larger models,” he says. “However, I don’t think that merely increasing the size of language models will address the well-documented problems of existing models repeating sexist and racist language or failing basic tests of logical reasoning.”
More on these topics: