Is OpenAI Working on a Secret Model that’s Getting Close to AGI?

OpenAI is rumored to be working on a model more powerful than its flagship GPT-4 that can handle text, video, audio and images.

Popular leaker Jimmy_Apples said the Microsoft-backed company was working on a model codenamed ‘Arrakis’ to rival Google’s upcoming Gemini model, a multimodal foundation model and challenger to ChatGPT.

The leaker said that Arrakis is “an everything-everything model, meaning you can input any combination of text, image, audio and video … and get an output of any combination of these modalities.”

Arrakis “far exceeds GPT-4 and is really close to AGI,” the leaker said. The reference to AGI, or artificial general intelligence, sent Reddit ablaze, with Jimmy_Apples’ X (Twitter) account suddenly shutting down after the rumor arose. AGI is achieved when an AI system can perform general cognitive tasks like a human instead of narrowly defined ones. It is considered the Holy Grail of AI.

Hallucinations are down, but not out on the model and half of the model was trained on synthetic data, according to reports. It is also supposedly good at operating as an autonomous agent.

Jimmy_Apples said Arrakis was not meant to launch in the next 12 months but could be brought forward to 2024. OpenAI could even release Arrakis in various parts – slowly opening access to the varying modalities, which means the public could have already seen it in the form of GPT-4V, released earlier this week.

Related:ChatGPT Gets Eyes, Ears and a Voice

OpenAI did not respond to AI Business’ request for comment.

However, CEO Sam Altman quashed rumors on Reddit, saying: “Y’all have no chill!” He also wrote that “agi has been achieved internally” but backtracked and said he was “just memeing” and then “when agi is achieved it will not be announced with a reddit comment…”

The Arrakis name comes from the Frank Herbert novel Dune. Arrakis is the name of the fictional desert planet, and it would have been an apt choice by OpenAI considering in the book's universe, AI took over and humanity was banned from constructing anything akin to the technology.

A full breakdown of Jimmy_Apples’ post on Arrakis can be found here.

Is Arrakis one model? Or lots of smaller models?

Jimmy_Apples had been touting that OpenAI had been working on a 125 trillion parameter model since January. The company did release GPT-4 in March but did not say exactly how big it was.

But could Arrakis be one giant model or several smaller models?

In June, the co-founder of PyTorch, Soumith Chintala leaked that GPT-4 is not one big model, but is made up of several smaller systems – a Mixture of Experts (MoE) architecture.

If OpenAI were to use Arrakis – a supposedly massively multimodal system, it would make more sense to use an MoE approach, according to Alexander Harrowell, Omdia principal analyst in advanced computing for AI.

Harrowell explained that MoE architectures are not limiting as “not all of them need to be active at any one time.”

“Now, of course, if you were planning to add multimodal features, adding new experts to a MoE model might be a really smart way to do that, taking advantage of the inherent modular nature of the thing, and I wouldn’t bet against them having done just that.

“Getting to 125 trillion parameters on this route, without changing the size of the experts, would mean 27 heads/experts or 3.3x the rumored GPT-4.”

By Harrowell’s calculations, it would take some 74,000 Nvidia H100s to train such a model, and while unlikely, he said it’s “perhaps achievable with Microsoft’s money.”

Have we reached AGI?

X (Twitter) and Reddit were ablaze with talk of AGI – even the likes of Nvidia software engineer Bojan Tunguz jumped on the bandwagon.

Eden Zoller, chief analyst, Applied AI at Omdia, noted that although AGI is not inherently bad, without robust alignment and restraints it has the capacity to magnify risks and harm that could potentially lead to extreme risks. This is intensifying scrutiny on highly advanced foundation models and the firms that build them.

Claims around AGI are often rooted in the argument that ‘robots will take over’ – perpetuating apocalyptic fears seen in stereotypical films like the Terminator franchise. But Zoller said these fears of existential risk could distract from the current harms AI systems can cause.