MIT Technology Review's What's Next series provides a look into the future, covering industries, trends, and technologies. You can read the rest here.
Thanks to the boom in artificial intelligence, the world of chips is on the verge of a major tide change. Chips that can train AI models faster and allow devices like smartphones and satellites to send pings and use those models without exposing private data are in high demand. Governments, tech giants and startups alike are competing to carve out a slice of the growing semiconductor pie.
Here are four trends to watch over the coming year that will define what future chips will look like, who will make them, and what new technologies will be unlocked.
This story is available to subscribers only.
Don't be satisfied with only half the story.
Access the latest technology news without a paywall.
Subscribe now Already a subscriber? Sign in
CHIPS operates all over the world
Outside Phoenix, the world's largest chipmakers, TSMC and Intel, are competing to build desert campuses in hopes of becoming a hub for U.S. chip manufacturing capacity. One thing these efforts have in common is their funding. In March, President Joe Biden announced his $8.5 billion in direct federal funding and his $11 billion in loans for Intel's national expansion. A few weeks later, another $6.6 billion investment in TSMC was announced.
The award is just part of the U.S. subsidies funneled to the chip industry through the $280 billion CHIPS and Science Act signed into law in 2022. The funding means companies entering the semiconductor ecosystem are analyzing how to restructure their supply chains for profit. From cash. Much of the funding is aimed at boosting U.S. chip manufacturing, but there is room for other players to apply, from equipment manufacturers to niche materials startups.
But the United States isn't the only country looking to bring parts of its chip manufacturing supply chain domestically. Japan will spend $13 billion in its own CHIPS Act costs, Europe will spend more than $47 billion, and India announced a $15 billion commitment to build local chip factories earlier this year. The roots of this trend date back to 2014, said Chris Miller, a professor at Tufts University and author of “Chip Wars: The Battle for the World's Most Critical Technology.'' It was at this time that China began offering huge subsidies to its chip makers.
simon & schuster
“This created the momentum for other governments to decide they had no choice but to offer incentives or watch companies move manufacturing to China,” he says. This threat, combined with the proliferation of AI, is prompting Western governments to fund alternatives. Next year, this could snowball into more countries starting their own programs for fear of being left behind.
Miller said the funding is unlikely to result in the emergence of entirely new chip competitors or fundamentally reshape who the biggest chip players are. Instead, it will encourage powerful players like TSMC to establish roots in multiple countries. But funding alone will not be enough to make that happen quickly. TSMC's efforts to build a factory in Arizona were bogged down by missed deadlines and labor disputes, and Intel similarly missed its promised deadlines. And whenever the factory opens, it's unclear whether its equipment and workforce will be capable of the same level of advanced chip manufacturing it maintains overseas.
“Supply chains only change slowly over years and decades,” Miller said. “But things are changing.”
More AI on the edge
Currently, most interactions with AI models such as ChatGPT occur via the cloud. In other words, when you ask GPT to choose an outfit (or be your boyfriend), the request is pinged to OpenAI's servers, and the model stored there processes it before returning a response ( You will be prompted to draw inferences (referred to as “inferences”). to you. There are some drawbacks to relying on the cloud. For example, it requires internet access and also means that some of your data is shared with the model maker.
That's why there's so much interest and investment in edge computing for AI. With edge computing, the process of pinging an AI model occurs directly on a device such as a laptop or smartphone. The industry is increasingly working towards a future where AI models will know more about us (Sam Altman says of his killer AI app, “My life, every email I've ever exchanged, They want faster “edge” chips that can run models without sharing private data. These chips face different constraints than those in data centers. Typically, chips must be smaller, cheaper, and more energy efficient.
The U.S. Department of Defense is heavily funding research into high-speed private edge computing. In March, its research arm, the Defense Advanced Research Projects Agency (DARPA), announced a partnership with chipmaker EnCharge AI to develop ultra-powerful edge computing chips used for AI inference. EnCharge AI is working to develop chips that can operate on significantly less power while providing greater privacy. This makes it suitable for military applications such as satellites and off-grid surveillance equipment. The company plans to ship the chip in 2025.
AI models will always rely on the cloud for some applications, but new investment and interest in improving edge computing is introducing faster chips and, as a result, more AI into everyday devices. may be. If edge chips become small enough and cheap enough, there could be even more AI-powered “smart devices” in our homes and workplaces. Currently, AI models are primarily limited to data centers.
“Many of the challenges seen in data centers will be overcome,” said Naveen Verma, co-founder of EnCharge AI. “I expect there to be a big focus on the edge. I think that will be important for deploying AI at scale.”
Big Tech enters the chip manufacturing race
In industries ranging from fast fashion to lawn care, companies pay exorbitant computing costs to create and train AI models for their businesses. Examples include external-facing technologies such as models that employees can use to scan and summarize documents or virtual agents that can explain how to fix a broken refrigerator. This means that the demand for cloud computing to train these models is exploding.
The companies that provide most of that computing power are Amazon, Microsoft, and Google. These tech giants have long dreamed of boosting profit margins by making data center chips themselves rather than buying them from companies like Nvidia. Nvidia is a giant with a near monopoly on cutting-edge AI training chips and is worth more than GDP. Out of 183 countries.
Amazon started working on it in 2015, acquiring startup Annapurna Labs. Google made the next move in 2018 by introducing its own chip called the TPU. Microsoft launched its first AI chip in November, and Meta announced a new version of its AI training chip in April.
AP Photo/Eric Risberg
This trend could tip the scales away from NVIDIA. But NVIDIA isn't just playing the role of a rival in the eyes of Big Tech companies. Regardless of their efforts, cloud giants still need his NVIDIA chips in their data centers. Part of that is because its own chip manufacturing efforts can't meet all of its needs, but it's also because customers expect to be able to use his top-of-the-line Nvidia chips.
“This is really about giving customers choice,” says Rani Borkar, who leads hardware efforts at Microsoft Azure. She said she cannot imagine a future where Microsoft supplies all the chips for its cloud services, adding, “We will continue our strong partnerships and deploy chips from all of our silicon partners. ” states.
As cloud computing giants try to wrest some market share from chipmakers, Nvidia is trying to do the opposite. Last year, the company launched its own cloud service, allowing customers to bypass Amazon, Google and Microsoft and get computing time directly on his Nvidia chips. As this dramatic battle for market share unfolds, next year will see whether customers see Big Tech's chips as more similar to Nvidia's most advanced chips, or more like their own smaller cousins. It will be decided.
Nvidia fights startups
Despite Nvidia's dominance, a wave of investment is flowing into startups that aim to beat Nvidia in certain parts of the future chip market. All of these startups promise to speed up AI training, but each has a different idea of which fancy computing technologies, from quantum to photonics to reversible computing, will get us there. There is.
But Murat Onen, 28, the founder of Eva, one such semiconductor startup spun out of his doctoral work at MIT, says he can't start a chip company at this point. Be honest about what it's like.
“The king of the hill is Nvidia, and that's the world we live in,” he says.
Many of these companies, such as SambaNova, Cerebras, and Graphcore, are looking to change the underlying architecture of their chips. Imagine an AI accelerator chip that needs to constantly shuffle data between different regions. Some of the information is stored in the memory zone, but must be moved to the processing zone for calculations and then returned to memory. Zone for storage. All of this takes time and energy.
Making that process more efficient means AI training can be delivered to customers more quickly and cheaply, but it's also important to note that chip manufacturers have to ensure that AI training companies have enough software to seamlessly transition to new chips. Only in case. If the software transition is too clumsy, model makers like OpenAI, Anthropic, and Mistral are likely to stick with the big chip makers. This means that companies like SambaNova that take this approach are spending more time on more than just chip design. But so is software design.
Onen proposes another step deeper change. Instead of traditional transistors, which have been miniaturized for decades to achieve high efficiency, he is using a new component called a proton-gate transistor. Eva is said to be designed specifically for the mathematical needs of AI training. This allows devices to store and process data in the same location, saving time and computational energy. The idea of using such a component for AI inference dates back to his 1960s, but researchers have never quite figured out how to use it for AI training, partly due to material obstacles. i didn't understand. This requires materials that can operate accurately, among other qualities. Controls conductivity at room temperature.
One day in the lab, Onen says, “we optimized these numbers and, very fortunately, we had the material we wanted.” “Suddenly, this device is not a science fair project.” This raises the possibility of using such components on a large scale. After months of working to make sure the data was correct, he founded his Eva and the research was published in his Science journal.
But in an area where many founders have promised – and failed – to break the dominance of big chip makers, there is no question whether the design will work as intended and whether the manufacturers will agree to produce it. It will take years to know, Onnen freely admits. . Leading a company through that uncertainty requires flexibility and a willingness to accept skepticism from others, he says.
“I think sometimes people get so attached to their ideas that they worry that if this goes away, there will be nothing next,” he says. “I don't think I feel that way. I'm still looking for people to challenge us and say this is wrong.”
