Alibaba’s first claim with Qwen2-Math mathematical model

If you haven’t heard the name “Qwen2” before, it’s not surprising, but with the release of a new and amazing model from Alibaba company, we should hear this name more and more, a model that with its advantages can soon create a big revolution in engineering and other technical applications.

Qwen2 what is it

The pace at which new AI models are emerging from startups and tech companies is so fast that even those who follow the space closely can find it difficult to keep up.

Qwen2 is an open-text large language model (LLM) that competes with OpenAI’s GPTs, Meta-Llama, and Entropic’s Claude family, but is provided by Alibaba Cloud, the cloud storage division of Chinese e-commerce giant Alibaba.

In August 2023, Alibaba Cloud started releasing its LLMs under the sub-name “Tongyi Qianwen” or Qwen for short, including the opentext models Qwen-7B, Qwen-72B and Qwen-1.8B with parameters of 72 billion and 1.8 billion, respectively. It was followed by multi-sensory variants including Qwen-Audio and Qwen-VL (for visual inputs) and finally Qwen2 in early June 2024 with five different variants: 0.5B, 1.5B, 7B, 14B and 72B. It has released more than 100 AI models of different sizes and functions in the Qwen family.

And customers, especially in China, are taking notice, with more than 90,000 companies reportedly using Qwen models in their operations in the first year of availability.

While many of these models were at or near state-of-the-art performance when they were released, the broader competition of LLM and AI models around the world moved so quickly that they were quickly eclipsed by other open-text and closed-source competitors.

Qwen2-Math what is it

Alibaba Cloud’s Qwen team recently unveiled Qwen2-Math, a “series of large language models for mathematics” designed for the English language. The most powerful models in this series have beaten all their competitors in the world, including the acclaimed OpenAI GPT-4, Anthropic Claude 3.5 Sonnet and even Google’s Math-Gemini Specialized 1.5 Pro.

In particular, the Qwen2-Math-72B-Instruct model with 72 billion parameters has reached a score of 84% in the MATH benchmark for LLMs. The benchmark includes 12,500 challenging math and word problems that are too difficult for LLMs to solve (for example, determining whether 9.9 or 9.11 is greater). To be honest, I couldn’t answer this question myself, let alone answer it in seconds, but apparently Qwen2-Math can most of the time.

Perhaps unsurprisingly, Qwen2-Math-72B Instruct also outperformed the competition on the GSM8K elementary math benchmark (8,500 questions) by 96.7% and on the college math benchmark by 47.8%. However, it is notable that Alibaba did not compare Microsoft’s new Orca-Math model released in February 2024 in its benchmark charts. This model with 7 billion parameters (a variant of Mistral-7B which itself is a variant of Llama) achieved a score of 86.81% for Orca-Math versus 89.9% for Qwen2-Math-7B-Instruct.

However, even the smallest version of Qwen2-Math, the 1.5 billion parameter version, performs remarkably well and is close to a model more than 4 times its size, with 84.2% in GSM8K and 44% in Academic Math. It has reached 2 percent.

Applications of mathematical artificial intelligence models

While the initial use of LLMs has focused on their use in chatbots and in the case of companies, to answer employee or customer questions or to prepare documents and analyze information more quickly, mathematics-focused LLMs seek to provide more reliable tools for those seeking Solving equations and working with numbers are regular.

Although all the code is based on mathematical principles, so far LLMs have not been as reliable in solving mathematical problems as previous AI or machine learning courses, or even older software.

The Alibaba researchers behind Qwen2-Math say they hope Qwen2-Math can help society solve complex math problems.

Custom license terms for companies and individuals who want to use Qwen2-Math are less than pure opentext and require that any commercial use with more than 100 million monthly active users obtain an additional license from the developers. But this is still a very lenient upper limit, allowing many startups, small and medium businesses, and even some large corporations to use Qwen-2 Math commercially (to make money) for free.

source