Nvidia Grace CPU - the key catalyst for ARM in the cloud datacenter?

I was sat listening to Jensen Huang's keynote the other day, and I kept hearing "Grace" and "Hopper". I knew these names were familiar!

Was I thinking about Hopper from Stranger Things, the famous local policeman in the Netflix series? No......I was hearing my own 8 year old daughter, talking about Grace Hopper, the inventor of COBOL, in a project she had done 6 months earlier for school (with a little assistance from her Dad)! A great choice of names from the company that has highlighted many scientific luminaries achievements in the past with their naming conventions.

To recap, Nvidia has made an interesting turn towards ARM. Not by purchasing them, it looks like that might never happen. What Nvidia have done is pre-release details of the "Grace" CPU, which is planned to revolutionize the datacenter in 2023, particularly AI, running side by side with the next generation "Hopper" GPU.

Now ARM is not a new thing. In fact if you are old like me and remember a computer called the BBC Micro. It was based on ARM all those years ago and enabled us to even play games in 1980s computing classes, like Chuckie Egg.......

ARM is everywhere nowadays, most notably its low power architecture has meant it is the incumbent CPU architecture found in almost every mobile phone, from Samsung to Oppo through Huawei and thousands of others, they have ARM tech inside.

Whilst ARM is a CPU company. ARMs business model is very different to other CPU companies you may be familiar with, such as Intel. ARM based CPUs are not manufactured by ARM. They don't have expensive fabrication plants like other CPU companies. ARM is manufactured by many companies, and I mean many, like hundreds! Effectively ARM license the designs and IP associated with the CPU to anyone with the money. Intel on the other hand, uses an architecture called x86, which is closed, and has been licensed to very few companies. AMD being one of them.

So now you know the history, but is ARM relevant in the datacenter? As a server expert, historically, I would have had to say it is relevant to a degree, especially with partners like Ampere and their Altra CPU becoming more available - but it is not the norm. Overall, in the DC, ARM has been a bit of an oddity. Why?

The main reason for the lack of ARMs success thus far has been twofold, in my view.

Firstly, it has struggled to gain significant share because 90% of off the shelf PC and Server software is written for x86, the architecture that underlies Intel and AMD processors. As we shifted to the cloud, continued use of products like Windows, VMWare and x86 based Linux has been the norm, and re-architecting those for another CPU architecture was seen as un-necessary and complex.

The other reason was a real lack of ARM supporting server platforms and high core count ARM CPUs, but this has slowly changed over the years - resulting in products like the Grace CPU we see Nvidia announcing, with high core counts, massive bandwidth, fast interconnects and enough IP inside the cores to drive real performance across a broad spectrum of applications in the datacenter.

This is a brave, new, AI fueled world we live in, which is typified by use of open source software, and is reliant on massively parallel co-processors such as GPUs and ASICs.

This new world has became far less reliant on both the x86 architecture AND x86 CPU horsepower. So why do I need an x86 based CPU alongside this AI accelerator? More importantly for Nvidia, why do customers need to spend money on Intel or AMD CPUs to use Nvidias GPUs?

That is exactly the question Nvidia will have asked, high in the ivory towers, and more importantly to Nvidia and its shareholders - how can Nvidia increase it's share of wallet in these AI led infrastructures?

The answer could be Grace............