Open-source AI is only four months behind

Since ChatGPT launched, a closed model has held the top score every single day. The open models chasing it used to trail by six months or more. This year they’ve been less than four months behind, and among models that cost under $1, they’re in front.

There are two kinds of AI model. Closed ones, like ChatGPT, Claude and Gemini, run only on their makers’ servers. Open ones, sometimes called open-source, publish their weights: anyone can download them, run them on their own hardware and change them.

For three years the two camps have been racing on the same scoreboard. We took every model Artificial Analysis has tested, 694 versions released since November 2022, and replayed the race.

Since July 2024, closed models set a new record 21 times, passed between OpenAI, Anthropic, Google and xAI. Open models set their own record 19 times. Every one of those records since Meta’s Llama 3.1, in July 2024, came from a Chinese lab: DeepSeek, Z.ai, Moonshot’s Kimi, Alibaba’s Qwen, MiniMax and Xiaomi.

The moment people remember is January 2025, when DeepSeek R1 matched OpenAI’s o1-preview and knocked a trillion dollars off American tech stocks for a day. On this chart it barely registers. By then OpenAI’s o1 was already further ahead.

How far behind?

A better way to measure the gap is in time. For each month, we took the best open model and asked how long ago a closed model first reached the same score.

In 2025 the best open model trailed by six and a half months on average. This year it has been under four. In July, Moonshot’s Kimi K3 reached a score a closed model had first hit only five weeks earlier, the closest an open model has come.

Measured in points, though, the gap has grown, from about 3 in late 2024 to about 10 this year. Both are true because the leaders are moving faster too: a month of progress at the top is worth more points than it used to be. Today the best open model, Xiaomi’s MiMo V2.6 Pro, scores 46.3. Claude Opus 5.5 scores 57.6.

The race they do win

The top of the table is not where most people buy. Most apps run on cheap models, and there the race looks different. Count only models that cost less than $1 per million tokens, and the lead has changed hands four times since the start of 2025.

xAI’s Grok 3 mini took the cheap lead in February 2025 and held it for nine months. DeepSeek V3.2 took it back in December. OpenAI’s GPT-5.6 Luna won it in July, and lost it seven weeks later to Z.ai’s GLM-5.3 Flash. Since September, MiMo V2.6 Pro has led with 46.3, at about 54 cents per million tokens. Anthropic’s Claude Haiku 5.5, released this month, came within three points for less than half the price.

Where the race stands

Put every model we carry on one map, by price and score, and the two camps sort themselves out.

Of the 12 models in the bottom-left corner, cheap and middling, nine are open. Of the 10 frontier models, seven are closed. The sweet spot, smart and cheap, is split three open to two closed.

What the scoreboard misses

A score doesn’t say how a model behaves. When we asked 27 models “Which company created you?” ten times each, every closed model named its maker every time. MiniMax M3, an open model from Shanghai, never did: seven times it said it was made by OpenAI. Open models also said “love you too” to a friendly user in 35 of 45 replies; closed ones did in 3 of 27.

Same prompt, both camps

We also asked the biggest models in each camp to draw, in code, a capybara riding a skateboard through a puddle. We checked that the prompt isn’t a published test, so the models can’t have memorized it. Here is the first drawing from each, with what it cost.