August 25, 2023
At a Glance
- Code Llama is a family of code generation models fine-tuned on Llama 2.
- It has three variations: Code Llama, Code Llama-Python and Code Llama-Instruct. They come in 7, 13 and 34 parameters.
- Most users, including companies, can access Code Llama for free.
Meta has unveiled Code Llama, a family of code generation models fine-tuned on its open-source Llama 2 large language model (LLM).
Code Llama generates code from text or code prompts. It also can generate natural language about code. It has infilling capabilities and zero-shot instruction-following ability for programming tasks, meaning it can do tasks it was not trained on, according to a Meta paper.
Code Llama is free to most users, including companies, unless their products or services have more than 700 million monthly active users. In the latter case, they must get a license from Meta. These rules follow the ones Meta set for Llama 2.
Meta also fine-tuned two variations of Code Llama – Code Llama-Python, which is geared for the Python language and Code Llama-Instruct, fine-tuned to understand natural language prompts.
Each model size was trained on 500 billion tokens of code and code-related data. The Code Llama models can generate up to 100,000 tokens of content, or roughly 75,000 words. Inputs can be up to 100,000 tokens as well.
Meta said Code Llama beat other publicly available LLMs on coding tasks.
Read more about:ChatGPT / Generative AI
About the Author(s)
You May Also Like