The system is developed in collaboration with OpenAI. It consists of 285,000 CPU cores and 10,000 GPUs. The goal is to learn large distributed AI models to improve the AI capabilities of Microsoft products.
Microsoft announced further details of the collaboration with the research institute OpenAI at Build 2020. Together with OpenAI, the software company is developing a supercomputer to teach large, distributed AI models. One of the goals is to improve the artificial intelligence capabilities of our own software and services.
According to the company, the world’s fifth fastest supercomputer is to be created according to the TOP500 ranking. The two partners are aligning the system with Massive Distributed AI Models, as researchers believe that they offer better performance than the small and separate AI models that were previously used.
Microsoft already has its own large AI models, called Microsoft Turing. They are mainly used to improve understanding of natural language in Bing, Office and Dynamics. The Turing model for natural language is considered the largest publicly available AI language model worldwide.
Redmond also announced that it would place the Turing model under an open-source license. The same applies to the recipes for training Turing using Azure Machine Learning. In addition, the ONNX Runtime, an open-source library for the transferability of AI models, is to receive support for distributed training.
The OpenAI supercomputer is designed as a single system with 285,000 CPU cores, 10,000 GPUs and a network connection of 400 GBit / s for each GPU server. The supercomputer is hosted in Azure with access to Azure Services.
Turing is part of a broader Microsoft initiative called AI at Scale. It is intended to make it possible to train huge neural networks on a high-performance infrastructure in order to then reuse the same model in many other scenarios. This is to help improve artificial intelligence in various products. Microsoft has trained a model for language understanding, which is called Turing NLR. The model is also adapted for Bing, Word, SharePoint and Outlook.
Microsoft customers will not have direct access to the OpenAI supercomputer. However, you can use the updated Azure Compute infrastructure. Its DeepSeed software, which is also available as open source, can train large models. According to Microsoft, the ONNX Runtime should also speed up the setup of models. However, Microsoft also offers the option of accessing Microsoft’s Turing model instead of learning its own models.