Following SambaNova in a massive round for a small semiconductor business

American AI chip startup Groq has raised $300m in a funding round co-led by Tiger Global Management and D1 Capital, valuing the company at $1bn.

The startup was founded in 2016 by two Google employees who helped create the tech giant's tensor processing unit (TPU) family of machine learning chips.

Groq and roll

“AI is limited by existing systems, many of which are being followed or incrementally improved upon by new entrants," Groq co-founder and CEO Jonathan Ross said.

"No matter how much money you throw at the problem, legacy architectures like GPUs and CPUs struggle to keep up with the growing demands of artificial intelligence and machine learning.”

The company's tensor streaming processor eschews the multicore approach of most modern architectures, instead focusing on a giant single core.

"At the system level, we avoid some of the complexity that goes along with global adaptive routing and load balancing these network links that connect the TSPs," Dennis Abts said at ISC2020.

"In general, modern supercomputers use these scalable networks to be able to efficiently connect up and cooperate between the processors, and there's a lot of techniques that get exposed at a system level that introduce latency variants, complexity, and reordering that is difficult to reason."

This makes the TSP the fastest single-core chip and the first to process 1 quadrillion operations per second, the company claims. Actual tests by customers showed the chip performing below those claims, the Financial Times reported.

Groq sells a consumer version of the chip as a bundled 5U Groq node with 8 TSPs, and also offers the platform on Nimbix Cloud.

“We are more than doubling-down on our investment in Groq,” said Nicolas Sauvage, managing director of TDK Ventures, an existing investor that also took part in the recent round.

“Their elegant AI chipset architecture is impressive and will have a powerful impact for our planet. Scaling traditional compute consumes enormous energy. Groq’s solution delivers order-of-magnitude more efficient compute-per-energy performance at scale, thereby improving the carbon footprint of hyperscale data centers.”

Including the $300m raised, Groq has brought in a total of $367m – a significant sum for a small company, but half of what its rival SambaNova raised earlier this week.

The AI chip and services business brought in $676m in a Series D round, valuing the company at $5bn. We spoke to the company’s CEO earlier this year to learn how SambaNova plans to dominate the AI semiconductor space.

But it is far from alone: back in December, British competitor Graphcore raised $222m, at a value of $2.77bn. Then there’s Lightmatter, which spun out of MIT to develop photonic AI chips.

All operate in the long shadow of Nvidia, currently worth $400bn. The GPU giant this week announced that it would expand into the CPU market, with a new Arm chip specifically for AI workloads.