April 13, 2021
“Nvidia is now a three-chip company”
Nvidia plans to release its own server CPU, designed for artificial intelligence workloads.
Set for launch in 2023, ‘Grace’ is based on the Arm architecture – developed by the eponymous company that Nvidia is trying to acquire for $40bn. Named after American programming pioneer Grace Hopper, the CPU is designed for "for giant-scale AI and HPC," Nvidia's founder and CEO Jensen Huang said.
Nvidia won’t stop
"Coupled with the GPU and DPU, Grace gives us the third foundational technology for computing, and the ability to re-architect the data center to advance AI. Nvidia is now a three-chip company,” Huang proclaimed.
He added that the chip will target training workloads for next-generation natural language processing models with more than one trillion parameters.
To pull this off, the CPU will be sold deeply integrated with Nvidia GPUs – it is not clear whether the chip will be available standalone.
Grace will connect to Nvidia's GPUs via the company's proprietary NVLink 4 interconnect, giving at least 900GBps of cumulative bandwidth, a considerable improvement over the 64GBps offered by PCIe 4 x16, which Nvidia uses to connect its GPUs to AMD Epyc CPUs in its DGX systems.
This emphasis on memory, necessary for large AI models, is also found elsewhere on the server. Along with the CPU and GPU, Nvidia plans to place an undisclosed amount of LPDDR5x memory next to the chips.
Few details about performance were revealed, although Nvidia claimed that Grace and a GPU was 10x as fast as Grace and a current-gen x86 CPU in certain AI workloads. The chip is based on a future generation of Arm’s Neoverse CPU cores.
Grace will join a growing number of Arm-based hopefuls who look to break Intel and AMD's stranglehold over the server market. Among them are Ampere, Fujitsu, Marvell, and Nuvia – which was recently acquired by Qualcomm for $1.4bn.
But with Arm available for licensing, unlike x86 (at least for now), and potentially offering significant energy efficiency improvements, others are interested in developing Arm-based CPUs.
Amazon Web Services already offers its homegrown chip on its cloud platform, and Microsoft is thought to be building an Arm chip. Earlier this year, Google said it would build its own server CPU, but it's not clear which architecture it will use.
With so many businesses flocking to the vendor-neutral architecture, Nvidia’s attempted acquisition of Arm has raised some concerns. Google, Microsoft, and Graphcore are among those that have written to the UK government asking for the deal to be investigated.