Introduction
Every few decades, computing is reinvented to address new workloads. Mainframes gave way to personal computers, which in turn were succeeded by the mobile and cloud era – and now we are entering the age of AI computing. This blog explains what AI computing is, the different layers of AI infrastructure, and why it is one of the most sought-after technologies today.
What is AI Computing?
AI computing is the area of computing which studies the infrastructure needed to perform AI operations, including training and inference. It comprises both the necessary specialized hardware and the associated software needed to operate it.
The reason why AI computing is a distinct area is that AI models learn from data through a process called training and then utilize that knowledge to perform inference. Both of these steps require performing matrix multiplication and other linear algebra operations, which are too data-intensive for traditional computers.
Why does AI Computing Matter
It is a bottleneck for most of the research into AI, as the improvement of models is often limited not by innovations in algorithms, but by the amount of available computing power.
- It is costly: the training of a single model can require millions of dollars in hardware and energy, making it one of the largest expenditures for AI development.
- It dictates the infrastructure: to meet the needs of AI, data centers are now often built near cheap electricity.
- It is political: cutting-edge chips are now subject to export restrictions, and AI is seen as a strategic resource similar to oil or natural gas.
- It is democratizing: with cloud platforms and new techniques such as quantization, even small organizations or cell phones can now perform sophisticated AI computations.
How AI Computing Differs From Traditional Computing
Traditional computers execute instructions given to them by the user. Such instructions can be anything from calculations to logical statements, but they are always programmed directly by the user. This makes traditional computers extremely versatile, but also limits their performance on any particular task.
In contrast, AI models are not programmed directly. Instead, they are trained to perform certain tasks, which requires immense amounts of linear algebra operations. These operations are massively parallel, meaning that they can be executed simultaneously on large sets of data. This is too resource-heavy for traditional computers, which is why AI computing requires specialized hardware.
The Hardware Behind AI Computing
Due to the nature of these intensive parallel operations, AI computing requires different hardware than traditional computing.
GPUs
Graphics Processing Units, or GPUs, were initially developed to render images. However, it was realized that the same parallel operations needed for graphics can be utilized for other purposes, including training AI models. The GPUs we use for AI computing are largely the same as those used in graphics cards, although there are also specialized alternatives. Companies such as NVIDIA make GPUs which can be used for a variety of purposes, including AI computing.
TPUs and Accelerators
Some companies specialize in making application-specific integrated circuits, or ASICs. These are microprocessors designed for a particular task, which is in contrast to CPUs which are designed to perform a variety of tasks. For AI computing, companies such as Google, Amazon, and others make TPUs, which are specialized for AI operations.
High-Bandwidth Memory and Interconnects
It is not enough to simply perform calculations quickly – it is also important to move data quickly between memory and processors. This is especially true for AI computing, where extremely large amounts of data need to be processed. This is why AI data centers often utilize specialized interconnects and memory.
Distributed and Cluster Computing
The amount of data which AI models need to process is so large that it is often distributed across thousands of GPUs or accelerators. This is necessary to reduce the amount of time needed for training, but also makes AI computing extremely dependent on distributed and cluster computing.
Benefits of AI Computing
Here are some of the benefits of AI Computing are:
Faster time-to-training and innovation
Specialized AI infrastructure enables faster training of foundational models, allowing researchers and innovators to bring products to market sooner.
Ingestion of large datasets
AI computing infrastructure is generally designed to consume large amounts of data as input to the model training process, and this represents a significant advantage for foundational model training.
Higher accuracy and capability
The opportunity to leverage more compute for model training enables researchers to train more capable models compared to what would be possible with more limited compute budgets.
More accessible and affordable via cloud-based AI
The ability to provide cloud-based AI compute enables economic access to critical graphics processing units (GPUs) and application-specific integrated circuits (ASICs) for a wider set of users and developers versus requiring them to directly purchase such equipment.
Efficiency gains enable AI everywhere including on the edge
Recent advances in model optimization, such as quantization and distillation, enable AI to run on low power, low performance chips and other devices at the edge of the network, such as cell phones, cameras, and sensors.
Economic and strategic opportunity
The confluence of computers and AI represents an opportunity to advance national security, economic competitiveness, and innovation leadership in the years ahead. Foundational models are enabling a new wave of innovation and productivity gains, and the compute infrastructure underpinning them represents a strategic asset for years to come.
Conclusion
AI computing has become an integral part of modern AI progress. In other words, almost any AI technology one encounters every day involves it in some capacity. The importance of this concept is difficult to overestimate, as it enables the creation of sophisticated systems, which, in turn, can perform a multitude of tasks. From language modeling and image generation to prediction algorithms and other tasks, AI computing makes them possible. On the other hand, it is also necessary to acknowledge that the process requires an array of advanced technological equipment, without which it would not be possible to produce functional AI systems. Such tools and mechanisms are no less important than the intelligence algorithms themselves.
FAQ’s
What makes it different from traditional computing?
Traditional computing processes instructions sequentially and is usually driven by central processing units (CPUs). On the other hand, AI computing uses accelerators that can perform mathematical operations in parallel, making it much faster and more efficient for training and operating AI models.
Is cloud computing different from AI computing?
The terms cloud computing and AI computing are not mutually exclusive, but they are not the same either. Cloud computing is a broad term that refers to the delivery of computing services through the internet. These services can include AI-specific accelerators or other tools. On the other hand, AI computing refers to the use of specialized tools and hardware required to operate AI systems.
Can AI computing be run on personal devices?
AI models can be run on personal devices like phones and laptops, but it is not usually cost-effective to operate large AI systems on such devices. This is because large AI systems require high-end processors that are usually found on high-end devices, and thus incur high costs. Additionally, the devices have to have high processing capabilities to effectively process the AI data.
