Qualcomm is taking another stab at the server, throwing its hat into the market for AI inference processing for the data center.
SAN FRANCISCO — Qualcomm is taking another stab at the server, throwing its hat into what is rapidly becoming a crowded ring — AI inference processing for the data center.
Qualcomm said in an event here Tuesday that it plans to begin sampling a 7nm AI inference accelerator for the cloud that emphasizes power efficiency and boasts peak performance of more than 350 trillion operations per second (TOPS). The device is expected to be in production next year.
Details about the chip — known as the Qualcomm Cloud AI 100 — were limited. But Keith Kressin, Qualcomm's senior vice president of product management, said it would have more than 10 times the performance-per-watt of anything deployed today.
While providing few details, Qualcomm executives stressed the power efficiency of the Cloud AI 100, noting that the company's heritage as a smartphone chip designer has ingrained power efficiency into its engineers' DNA.
"I really think the way to get the most power-efficient processor is that you have to start with a mobile mindset," Kressin said.
The market for inference accelerators is currently dominated by Nvidia GPUs, though more specialized solutions such as Google's Tensor and Amazon's Inferentia are available. In all, more than 30 companies are known to be developing chips purpose-built for AI inferencing, ranging from heavyweights such as Intel and Xilinx to a host of startups led by the likes of Graphcore, Wave Computing and Mythic.
"Qualcomm will certainly have to do something impressive to differentiate from these other vendors," said Linley Gwennap, president and principal analyst at the Linley Group. "Qualcomm executives like to wave their hand and say, 'power efficiency.' But what you can do in a smartphone is very different from what you can do in a server."
Qualcomm says the peak AI performance of the Cloud AI 100 is more than 50 times that of the Snapdragon 845 mobile applications processor. (Source: Qualcomm)
Gwennap added that the lack of detail shared by Qualcomm makes it difficult to gauge the Cloud AI 100's prospects for success. He noted that the chip is not likely to be in production before late 2020 and that the rapidly evolving AI inference market could look much different by then.
But Patrick Moorhead, president and principal analyst at consulting firm Moor Insights and Strategy, called the introduction of the Cloud AI 100 "promising" and "a good start." He added that the devices' target power envelope — which he estimated to 2-5 watts — is likely to differentiate its target market it from the high-octane servers that use Nvidia's high-end Tesla T4 GPUs in inferencing, limiting overlap.
Data Center Power Consumption Growing Rapidly
To underscore the importance of power consumption in the data center, Joe Spisak, product manager for Facebook AI, presented data showing that Facebook's power consumption doubles each year — with AI being a major culprit. "This is unsustainable for us," Spisak said. "We can't build data centers fast enough."
"I always like to look for a reason to believe that a company is taking a differentiated approach, and [Qualcomm] is," Moorhead said. He agreed with Kressin's point that coming from a low-power mindset and building a larger chip is easier than the other way around.
Moorhead added that most of the startups chasing the AI inference processing market are likely to be out of business or acquired within the next five years. He said many might be hit hard by the Cloud AI 100 announcement.
"After this announcement, there's likely to be a sigh of angst in the VC market," Moorhead quipped.
Facebook's data center power consumption is doubling each year, according to the company. (Source: Facebook)
To date, Qualcomm has had a difficult time penetrating the server. The company pulled the plug on its ARM-based server processor, Centriq, last year — about a year after launching it to great fanfare.
Also Tuesday, Qualcomm expanded its Snapdragon mobile processor roadmap, adding the Snapdragon 730, 730G and 665 mobile platforms. The devices include features that emphasize AI, gaming and advanced camera capabilities.