Code Llama is a family of code generation models fine-tuned on Llama 2

Deborah Yao, Editor

August 25, 2023

1 Min Read
Llama
Getty Images/AI Business

At a Glance

  • Code Llama is a family of code generation models fine-tuned on Llama 2.
  • It has three variations: Code Llama, Code Llama-Python and Code Llama-Instruct. They come in 7, 13 and 34 parameters.
  • Most users, including companies, can access Code Llama for free.

Meta has unveiled Code Llama, a family of code generation models fine-tuned on its open-source Llama 2 large language model (LLM).

Code Llama generates code from text or code prompts. It also can generate natural language about code. It has infilling capabilities and zero-shot instruction-following ability for programming tasks, meaning it can do tasks it was not trained on, according to a Meta paper.

Code Llama is free to most users, including companies, unless their products or services have more than 700 million monthly active users. In the latter case, they must get a license from Meta. These rules follow the ones Meta set for Llama 2.

Users can request access here. Code Llama is on GitHub and Meta also released its model weights.

Code Llama is built on top of Meta’s Llama 2 and comes in three sizes: 7 billion, 13 billion and 34 billion parameters. It supports popular languages including Python, C++, Java, PHP, Typescript (Javascript), C# and Bash.

Meta also fine-tuned two variations of Code Llama – Code Llama-Python, which is geared for the Python language and Code Llama-Instruct, fine-tuned to understand natural language prompts.

Each model size was trained on 500 billion tokens of code and code-related data. The Code Llama models can generate up to 100,000 tokens of content, or roughly 75,000 words. Inputs can be up to 100,000 tokens as well.

Related:AI Code Generation Models: The Big List

Meta said Code Llama beat other publicly available LLMs on coding tasks.

Read more about:

ChatGPT / Generative AI

About the Author(s)

Deborah Yao

Editor

Deborah Yao runs the day-to-day operations of AI Business. She is a Stanford grad who has worked at Amazon, Wharton School and Associated Press.

Keep up with the ever-evolving AI landscape
Unlock exclusive AI content by subscribing to our newsletter!!

You May Also Like