America has dominated the computer chip market for years — but with the advent of AI, maintaining this dominance and investing in next-gen AI chips has taken on a new urgency. Amazon, Google, Nvidia, and Microsoft have been joined by DARPA and a slew of smaller startups in the race to create a flexible, efficient, and powerful chip architecture that can power deep learning processes.
The chip development field is currently incredibly dynamic due to the explosion of interest in new chip development. New chip architecture emerges on a monthly basis at least, with some innovations more promising than others. To survey this complex and rapidly evolving market, NewtonX consulted with 25 executives at major US chip-making companies (or companies investing heavily in chips), including Amazon, Graphcore, AMD, and Nvidia. The data and insights in this article are informed by the results of these consultations.
Achieving AI Software and Hardware Symbiosis: The AI Chip Developments
Around 2015, interest in chips began to explode: by 2017, VCs had tripled investment in the space, and by 2019 investment in the space hit close to $1B. Just last year, Graphcore secured $200M in funding led by BMW and Microsoft while elsewhere companies such as Wave Computing and Mythic Inc. have likewise raised tens of millions in funding. This only paints a small part of the enormous picture, though: public companies including Nvidia, Qualcomm, and AMD, not to mention Amazon and Google, have also entered the fray.
So what are these chips that everyone is after?
Until recently, AI software has been mostly run on graphical chips (GPUs). These chips have high capacity for parallel processing as well as a higher capacity than CPUs for transistors. However, as Moore’s Law (the idea that the number of transistors on a chip will double roughly every 18 months) has begun to slow, more and more researchers have started positing that microchips designed specifically for deep learning computations could be even more powerful than traditional GPUs. Many researchers believe that we’ve reached the lowest voltage possible for GPUs, meaning that while it’s possible still to ad mre transistors, we’ve reached the limit for how fast they can go. Laptops, for instance, still run at 2Ghz, they just have more cores in them. For machine learning, we need thousands of cores — meaning that depending on old GPU architecture may be insufficient.
At this year’s MARS conference, a researcher with MIT, funded by DARPA and working with researchers from Nvidia, presented Eyeriss, a new deep learning chip that allows for edge computing and 10-1,000x higher efficiency than existing hardware. The chip is flexible enough to be adapted for numerous applications, but is also efficient — two features that had previously been difficult to balance in a single architecture.
The Eyeriss chip is hardly the only player in this game: IBM is developing specific AI processor, Google’s Tensor Processing Unit, or TPU, is a special AI chip for neural networks (Google still uses both CPUs and GPUs for other AI applications), and Microsoft started working on its own AI processor for Azure in late 2018. The aforementioned chip company Graphcore has developed what it calls Intelligent Processor Units (IPUs) that rely on a wholly different architecture than GPUs and CPUs.
The newest iPhones and other high-end competitors use simpler AI chips already; more efficient edge computing chips would allow for more powerful code with more complex capabilities — a necessity for self-driving cars, which currently rely on multiple large computers, and other complex AI computations.
Who Will Win the AI Chips Arms Race?
There is no clear indication of which approach — building on GPUs, TPUs, IPUs, or any number of other architectures — will prove the most effective at processing AI code. Because of this, the market is currently fiercely competitive and highly fractured. Chips experts predict that by 2025 the new chip architecture will have solidified, and competition will switch directions to focus on cost per capabilities rather than approach.