From a strategy perspective, there is a longer term enterprise play here, noted Moor’s Kimball; Humain is Qualcomm’s first customer, and a cloud service provider (CSP) or hyperscaler will likely be customer number two. However, at some point, these rack-scale systems will find their way into the enterprise.
“If I were the AI200 product marketing lead, I would be thinking about how I demonstrate this as a viable platform for those enterprise workloads that will be getting ‘agentified’ over the next several years,” said Kimball.
It seems a natural step, as Qualcomm saw success with its AI100 accelerator, a strong inference chip, he noted. Right now, Nvidia and AMD dominate the training market, with CUDA and ROCm enjoying a “stickiness” with customers.
“If I am a semiconductor giant like Qualcomm that is so good at understanding the performance-power balance, this inference market makes perfect sense to really lean in on,” said Kimball.
He also pointed to the company’s plans to re-enter the datacenter CPU space with its Oryon CPU, which is featured in Snapdragon and loosely based on technology it acquired with its $1.4 billion Nuvia acquisition.
Ultimately, Qualcomm’s move demonstrates how wide open the inference market is, said Kimball. The company, he noted, has been very good at choosing target markets and has seen success when entering those markets. “That the company would decide to go more ‘in’ on the inference market makes sense,” said Kimball. He added that, from an ROI perspective, inferencing will “dwarf” training in terms of volume and dollars.
