Has the hunt for AI compute uncovered the subsequent Cerebras?

The raging demand for computer systems to run AI fashions has solely accelerated, however there are two main obstacles that anybody within the enterprise wants to beat: getting the fitting chips, and getting them into information facilities the place they’ll begin producing income.

General Compute, a brand new inference neocloud — an organization that rents out AI processing energy, specializing within the part when fashions are operating and responding to customers reasonably than being skilled — has solutions to these questions that illuminate the place the AI ecosystem is headed. These solutions helped it increase a $15 million seed spherical at a $60 million post-money valuation, led by FUSE VC with participation from Carya Enterprise Companions and Village World Ventures.

First, what’s the proper chip? The demand for GPUs has gone by means of the roof, but it surely’s turning into typical knowledge that they aren’t the best-suited chips for operating AI fashions as soon as they’ve been skilled. The part of AI the place a mannequin is actively producing responses has completely different computational necessities than coaching, and a brand new class of chips is being designed particularly for it. Nvidia’s $20 billion Groq transaction in December and Cerebras’ $57 billion IPO final week level the way in which.

With capability strained at each these corporations, the co-founders of Common Compute, CEO Finn Puklowski and CTO Jason Goodison, discovered another choice. They’re turning to specialised chips constructed by SambaNova, an Intel-backed chipmaker centered on inference that has fallen a bit out of the Silicon Valley dialog.

That will change when SambaNova releases its new chips this yr. The structure is extra versatile and makes use of extra reminiscence to retailer context throughout inference calculations, and SambaNova claims that it outperforms not simply GPUs but additionally different specialised chips constructed by the likes of Groq or Cerebras. Puklowski says the brand new chips will generate 600 to 700 tokens per second, versus about 250 tokens per second for GPUs.

Common Compute has $300 million of the corporate’s SN50 chips on order and says will probably be the primary neocloud deploying them.

These chips additionally assist remedy the second large drawback — the place to place them — for Common Compute: They’re air-cooled, not water-cooled, and devour much less energy, to allow them to be put in in present information heart amenities with out new infrastructure investments.

Puklowski is pursuing colocation offers — preparations the place Common Compute installs its {hardware} in another person’s facility — not simply with information heart suppliers, but additionally with crypto miners trying to repurpose their infrastructure as the price of producing a bitcoin has typically exceeded its worth.

Common Compute launched its cloud providing final week, claiming it’s already the quickest at operating MiniMax 2.7, a strong open-source LLM.

Joe Hasselmann is a enterprise investor who received in on the bottom ground of the inference growth when he invested in Groq in 2021. This yr, he launched a brand new fund, Evercrest Capital Companions, centered on the AI house, and made Common Compute his first funding. Hassleman sees in SambaNova’s partnership with Common Compute parallels to Coreweave’s relationship with Nvidia — and to the pairing of Groq’s chip-making with its former cloud providing.

“They do want a wholesome combine of consumers which might be going to place their chips in environments which might be going to have excessive development to them,” Hassleman mentioned. “As a lot as Common Compute is betting on SambaNova, SambaNova is betting on Common Compute.”

The query is what sort of pc structure will seize essentially the most worth within the AI future. Inference clouds are implicit bets on a world of a number of fashions and brokers, one the place no single supplier dominates and pace and value of inference change into the important thing aggressive variables. Think about the $113 million Sequence B raised for OpenRouter this week, reflecting the corporate’s means to supply clients entry to a number of fashions to be able to optimize their token spend.

Velocity issues in that calculation, for worth, and for functionality. Puklowski needs to show hour-long workloads for coding brokers into five- or ten-minute duties, and make audio brokers for customer support, which require quicker inference to converse successfully, extra economical.

“Should you use ChatGPT and it offers you 50 tokens per second, that’s nonetheless a heck of rather a lot quicker than we are able to learn,” Puklowski advised TechCrunch, “Now that issues have moved to agent-to-agent, the place brokers are on the market studying on our behalf or pinging databases, they should go quicker.”