Meta just announced Code Llama, a new artificial ielligence model built on top of Llama 2’s large language model, which aims to help programmers generate and debug code. The use of this model is completely free.
Code Llama can code like platforms like ChatGPT and GitHub Copilot Chat. This model can even debug a problematic code sample. Meta has trained the Code Llama model based on source code data, which can support various programming languages such as Python, Java, C++, PHP, TypeScript, C#, and Bash scripting.
The Code Llama model can also handle 100,000 text tokens. Meanwhile, ChatGPT can only manage 4000 to 8000 tokens. Code Llama is preseed in three versions of 7, 13 and 34 billion parameters. These parameters are actually the numerical compones of the neural network that are adjusted during the training process. The more number of parameters means more complexity and better capabilities and they need higher computing power.
Meta has also released two specialized versions, which include Code Llama – Python and Code Llama – Instruct. The Python version is optimized for Python programming, which is an importa language in the field of artificial ielligence. The Instruct version is also designed to better ierpret the user’s ie when natural languages are requested.
Meta has always supported an open approach to artificial ielligence technologies, and what the company offers is much more open than OpenAI technology, which usually does not make the code of its advanced models publicly available.
Of course, the meta did not reveal the exact source of Code Llama’s tutorial data, but apparely it was extracted from the StackOverflow website. Code Llama’s performance review by HumanEval benchmark tests shows that Code Llama’s specialized Python version scored 53.7%, which is the highest score among similar models. But in the same tests, the GPT-4 model managed to score 67%, which shows that it has much more parameters.
Code Llama basic models with 7B, 13B and 34B parameters scored 33.5, 36 and 48.8% respectively in HumanEval. These scores are lower than the GPT-4 score. The Instruct version could not perform better than GPT-4 and the Instruct 34B version scored 41.5%.
To use Code Llama you need specialized experience in setting up such software, but it is expected that more user-friendly ierfaces will be provided for it soon. Meta hopes to pave the way for more specialized platform developme on Llama 2 by iroducing the Code Llama model.
You can submit your request for access to Code Llama by filling out a form on the Meta website.
Source: Arstechnica
