A startup has raised $124 million in a Series A round, on a promise of building reliable and steerable AI systems.

Anthropic was founded by OpenAI's former VP of research Dario Amodei, and his sister, Daniela, who previously worked as OpenAI's VP of safety and policy; at least nine other OpenAI employees joined the young company.

The funding round was led by Jaan Tallinn, co-founder of Skype. Also taking part was Infotech's James McClave, Facebook co-founder Dustin Moskovitz, the Center for Emerging Risk Research, and former Google CEO Eric Schmidt.

Schism

Anthropic claims it will research the means to increase the safety of AI systems, focusing in particular on the reliability of large-scale AI models, making them more interpretable, and on allowing human feedback to influence the development and deployment of the models.

“Anthropic’s goal is to make the fundamental research advances that will let us build more capable, general, and reliable AI systems, then deploy these systems in a way that benefits people,” Dario Amodei said.

“We’re thrilled to be working with investors that support us in this mission and expect to concentrate on research in the immediate term."

Anthropic said it will approach AI as an empirical science, adding that it was important to critically evaluate the potential societal impacts of the technology.

In a job listing, Anthropic said: “We work as a single cohesive team on just a few large-scale research efforts. And we value impact – advancing our long-term goals of steerable, trustworthy AI – rather than work on smaller and more specific puzzles.”

The company counts multiple former OpenAI employees as staff members, including research lead Sam McCandlish, lead GPT-3 engineer Tom Brown, research scientist Amanda Askell, software engineer Catherine Olsson, policy director [and former technology journalist] Jack Clark, technical staffers Tom Henighan, Kamal Ndousse, Benjamin Mann, and Nicholas Joseph.

Cartograph.ai founder Andrew Jones is Anthropic’s only publicly listed employee who did not work at OpenAI. Most left their previous workplace between December 2020 and January 2021; public records show Anthropic registered in California on 3 February.

Initially promoted as an effort to build safe artificial general intelligence, OpenAI has gradually transitioned from a non-profit to a revenue generator.

Today, it can be seen as something of a Microsoft vassal, raising a billion from the tech giant, using an Azure supercomputer, licensing its tech exclusively to Microsoft, and raising Microsoft funds for its own investment spree.

Anthropic is a for-profit company, but claims to be a "public benefit corporation." In some states, this can mean a company is non-profit, but "Anthropic, PBC" was initially registered in Delaware.

There, a for-profit corporation can call itself a PBC as long as it believes it is acting for the public benefit. Every other year, the company has to tell stockholders that it is indeed working for the public benefit, but there is no audit or certification of its impact.

Had Anthropic been incorporated as a PBC in California, the company would not have been able to distribute profits, gains, or dividends to any person.