Meta presents Code Llama: A New AI Model for programmers

Code Llama

Facebook’s parent company, Meta, today presented the new version of the Code Llama language model together with Microsoft. The Llama 2 models are available for both research and commercial use and are available through Microsoft Azure and Windows, Amazon Web Services, and Hugging Face.

The pre-trained and fine-tuned models with 7 billion, 13 billion, and 70 billion parameters were presented by Meta boss Mark Zuckerberg and Microsoft CEO Satya Nadella. The number of parameters describes the knowledge of a model. In a Facebook post, Zuckerberg explains that Llama 2 has been trained with 40 percent more data than its predecessor and also offers an improved architecture. In total, it was trained with two trillion tokens.

Code Llama openness of meta

The openness that Meta shows with the key data alone is remarkable. Neither OpenAI for GPT-4 nor Google for PaLM 2 recently published any information on parameters or training data – a development that the AI ​​industry is critically observing. Meta, on the other hand, provides further information and offers the model, including weights, for download. Inquiries can be accessed via the corresponding meta website.

Llama 1 was a success for Meta, the group claims to have received more than 100,000 inquiries from researchers. In addition, the full model was circulating on the web shortly after it was published, becoming a kind of blueprint for a large number of open-source models that have appeared in recent months.

Microsoft expands AI ecosystem

In addition to local download, Llama 2 is also available through Microsoft’s Azure cloud, Amazon’s AWS services and Hugging Face. However, the focus is on the cooperation with Microsoft. Both groups emphasize how successfully they have worked together in the past in the field of AI. This applies, for example, to the integration of a PyTorch environment into the Azure Cloud.

Microsoft is thus continuing to knit its AI ecosystem. So far, the structure has been primarily based on OpenAIs GPT language models, which Microsoft has integrated into the Bing search and as a co-pilot into various applications and Windows 11. With Llama 2, the group is now expanding the product range for AI developers in Azure with another powerful language model. Developers should also benefit from AI tools for model training or fine-tuning when handling.

Other companies are also on board. Qualcomm plans to offer an AI implementation based on Llama 2 in 2024 so that the generative AI models run better on smartphones or VR devices.

ChatGPT competitor for disk

So it becomes clear: Llama should find its way directly to the devices of the users. A strategy that contrasts with OpenAI and Google, which only offer AI solutions like ChatGPT and Bard online. It thus continues on the course set with the release of Llama 1. ChatGPT competitors that run directly on the home computer are conceivable.

Meta extends the Llama models with variants that are specifically designed for programming. The basis is the Llama 2 language model, which has been adapted through training. As with the base model, it is freely available, so it can be used for both research and commercial use.

Meta offers the model known as “Code Llama” in three variants: The general code base model, a model specialized for Python and an “Instruct” version, which should be fine-tuned to deal particularly well with natural language inputs. Details about the models can be found in the blog post and in the paper, The code is available on GitHub and it can be downloaded from Meta.

Programming support is considered to be one of the most used use cases for AI assistants based on Large Language Models (LLM). Microsoft offers this with GitHub Copilot, for example, but generative chatbots such as ChatGPT and Bard also have corresponding functions. However, as is usual with AI chatbots, users must expect answers that range from misleading to incorrect. According to a study, around 52 percent of ChatGPT results receive inaccuracies. Read, the features of ChatGPT 4.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top