Secure Your Open Source AI: Meta Launches 'Purple Llama'

Facebook parent Meta is introducing new tools to protect and secure its open source AI models.

Dubbed Purple Llama, the project provides developers with a series of tools to test their models to ensure safety before release - with plans to add more tools over time.

Meta said it is launching the security toolset to “build trust in the developers driving this new wave of innovation” and its investment in the project “will be comprehensive.”

Among the first tools available in Purple Llama is Llama Guard, an open pre-trained model that developers can use to “help defend against generating potentially risky outputs.”

Llama Guard was trained on a mix of publicly available datasets and is designed to detect common types of risky content. Meta wants developers to customize Llama Guard to support use cases and allow for best practices to be adopted more easily.

Another new Purple Llama offering is CyberSec Eval, a set of cybersecurity safety evaluation benchmarks for large language models.

CyberSec Eval contains tests for quantifying LLM cybersecurity risks as well as tools to evaluate the frequency of insecure code suggestions and to evaluate models to make it harder for them to generate malicious code.

The Purple Llama project itself is named after Meta's popular open source AI model, Llama, and a concept from the cybersecurity world where red teams attack systems and blue teams defend. When the two colors combine, they create the color purple, said Sy Choudhury, Meta's director of business development, AI partnerships, during a panel at the AI Summit New York 2023.

Related:OpenAI, Cloud Giants: In 10 Years, All Companies Will Use Gen AI, AI Summit NY 2023

The Purple Llama project's tools will be licensed permissively so they can be used for commercial use, as well as research. Meta said that members of the recently formed AI Alliance, of which the company was a founding member, will work to improve the tools and make them more widely available to the open source community.

“We believe this is a major step in enabling collaboration among developers and standardizing trust and safety tools for generative AI,” according to a Meta blog post.