With Code Llama, Meta Joins the Code Generation Party

Meta has unveiled Code Llama, a family of code generation models fine-tuned on its open-source Llama 2 large language model (LLM).

Code Llama generates code from text or code prompts. It also can generate natural language about code. It has infilling capabilities and zero-shot instruction-following ability for programming tasks, meaning it can do tasks it was not trained on, according to a Meta paper.

Code Llama is free to most users, including companies, unless their products or services have more than 700 million monthly active users. In the latter case, they must get a license from Meta. These rules follow the ones Meta set for Llama 2.

Users can request access here. Code Llama is on GitHub and Meta also released its model weights.

Code Llama is built on top of Meta’s Llama 2 and comes in three sizes: 7 billion, 13 billion and 34 billion parameters. It supports popular languages including Python, C++, Java, PHP, Typescript (Javascript), C# and Bash.

Meta also fine-tuned two variations of Code Llama – Code Llama-Python, which is geared for the Python language and Code Llama-Instruct, fine-tuned to understand natural language prompts.

Each model size was trained on 500 billion tokens of code and code-related data. The Code Llama models can generate up to 100,000 tokens of content, or roughly 75,000 words. Inputs can be up to 100,000 tokens as well.

Related:AI Code Generation Models: The Big List

Meta said Code Llama beat other publicly available LLMs on coding tasks.