Devin: AI Software Engineer that Codes Entire Projects from Single Prompt

An AI startup has launched what it claims to be the world’s first AI software engineer.

Cognition AI has unveiled Devin, an autonomous agent that can plan and execute complex software engineering tasks from a single prompt.

Housed in its own sandbox environment, Devin can solve tasks using its own code editor and web browser. It can even recall relevant context, learn over time and fix mistakes. For example, Devin can benchmark an AI model on different APIs.

Cognition showcased the model testing Meta’s Llama 2 on Replicate, Perplexity and Together. The system was able to build the entire project, even fixing errors.

Businesses could use Devin to build and deploy web apps, fix bugs in codebases and even train and finetune AI models.

Cognition is not marketing Devin as a replacement for human software engineers, describing it as a “teammate.”

Devin reports its progress in real time and works with human engineers, accepting feedback on projects.

“With Devin, engineers can focus on more interesting problems and engineering teams can strive for more ambitious goals,” Scott Wu, Cognition’s co-founder and CEO, wrote in a blog post.

Outperforms Major Models

Cognition evaluated Devin on SWE-bench, a benchmark that tasks agents to solve issues on software engineers would typically find in open-source projects.

Related:CISOs’ Most Common Concerns with Generative AI

Devin correctly resolved 13.86% of issues end-to-end, outperforming specialized coding models like SWE-Llama and large language models like OpenAI’s GPT-4 and Anthropic’s Claude 2.

Devin was unassisted during the benchmark tests, meaning it was told exactly which files needed to be edited. All other models were assisted.

Credit: Cognition AI

A dedicated technical report will launch “soon,” Wu confirmed in his blog post.

How to Access Devin

Devin is not currently publicly available. Wu said in his blog post that it is currently in early access as Cognition “ramp up capacity.”

For those interested in using Devin for engineering work, you will have to reach out to Cognition directly via email or through the contact form.

Who is Cognition AI?

Cognition unveiled Devin and blew up overnight. Its announcement post on X (Twitter) has amassed 24 million views and counting.

The startup was founded in November 2023 and has already secured $21 million in funding led by Peter Thiel’s Founders Fund.

The likes of Fred Ehrsam, founder of crypto platform Coinbase, DoorDash CEO Tony Xu and serial tech investor Elad Gil are among those backing Cognition.

Upon its reveal, Devin caught the attention of some big names in AI. Former Tesla AI director AI Andrej Karpathy described Devin as “an impressive demo” and Perplexity founder Aravind Srinivas said it “seems to cross the threshold of what is human level and works reliably.”

Related:3 Most Common Problems with Small Language Models

Though not everyone was impressed, one user pointed out that the startup used Google Forms for its onboarding, instead of using Devin to create one.