LLaMa (Large Language Model Meta AI)
Meta’s LLaMA (Large Language Model Meta AI) is a family of autoregressive large language models developed by Meta. The first version of LLaMA, released in February 2023, included models with 7, 13, 33, and 65 billion parameters. [3].
LLaMA-2, introduced in partnership with Microsoft in July 2023, is the next generation of LLaMA, featuring models with 7, 13, and 70 billion parameters. The architecture of LLaMA-2 remains largely unchanged from the first version, but it was trained with 40% more data. The models also include improvements such as increased context length from 2,000 to 4,000 tokens and the use of rotary positional embeddings[3].
Code Llama is a specialized version of LLaMA-2, designed for coding tasks. It can generate and discuss code based on text prompts, supporting many popular programming languages. Code Llama is available in three sizes with 7B, 13B, and 34B parameters, and it is released for both research and commercial use under the same community license as Llama 2. The models are trained with 500 billion tokens of code and code-related data, and there are also fine-tuned variations such as Code Llama – Python and Code Llama – Instruct[4].
Meta plans to release Llama 3 in July 2024, which aims to provide better responses to contentious questions by understanding the context more effectively. This version is part of Meta’s efforts to make the model’s responses more nuanced and useful[5].
Citations:
[2] https://ai.meta.com/blog/large-language-model-llama-meta-ai/
[3] https://en.wikipedia.org/wiki/LLaMA
[4] https://about.fb.com/news/2023/08/code-llama-ai-for-coding/
[6] https://huggingface.co/meta-llama