The Nvidia processors, he explains, are for processing massive, large language models (LLMs), while the Google TPU is used for inferencing, the next step after processing the LLM. So the two chips don’t compete with each other, they complement each other, according to Gold.
Selling and supporting processors may not be Google’s core competence, but they have the skills and experience to do this, said Alvin Nguyen, senior analyst with Forrester Research. “They have had their TPUs available, from what I understand, to some outside companies already, mainly startups from ex-Googlers or Google-sponsored startups,” he said.
As to the rumor of the Meta purchase, the question is, what does Meta want to do with them, said Gold. “If they’ve already built out a model and they’re running inference workloads, then Nvidia B100s and B200s are overkill,” he said. “And so what are the options there are now? There are a number of startups that are trying to do inference-based chips as well, and Intel and AMD are moving in that direction as well. So it really is a function of getting a chip that’s optimized for their environment, and again, Google’s TPUs that are optimized for a hyperscaler cloud type of environment.”
Nguyen says it’s one thing to make their own chips for their own use, it’s another thing to be selling them that’s an infrastructure and a competency that they don’t have, and Intel and AMD and Nvidia are way ahead of them in that regard.
“Yes, they know how to do it for themselves, long as you were talking about as a service or as a cloud service. For on premises, or for people who want to take it for themselves, that’s a muscle memory they have to develop,” he said.
For that reason, he doubts that other hyperscalers with their own custom silicon will go into the chip selling business for themselves as well. “There’s nothing to stop them, but each of them has their own challenges,” said Nguyen. Microsoft, AWS and OpenAI all have multiple partnerships and would inevitably end up in competition with somebody.
