Illustration: chili pepper mascot illustrating open vs. closed
Remember the good old days? Like, last year? When "open source" meant scrappy underdogs fighting the corporate giants? Well, October 2026 has arrived, and the open vs. closed model war is now a battle fought in tenths of a percentage point, with "open" increasingly meaning "Chinese labs' supply chain."
The latest benchmarks from benchlm.ai, updated October 12, 2026, paint a picture that's less David and Goliath, more like two corporate behemoths duking it out with slightly different marketing strategies. And the biggest irony? Meta, once the loudest champion of open weights, is now topping an agentic leaderboard with a closed model. You just love to see it.
Toolathlon: the closed sweep
If you're building AI agents, you're probably paying for a closed model. The Toolathlon leaderboard for agentic capabilities is a bloodbath for open weights. Out of 26 models, the top eight are all closed.
Meta's Muse Spark 1.1, a closed model, sits pretty at #1 with a commanding 75.6%. Anthropic's Claude Opus 4.8 follows at 59.9%, and OpenAI's GPT-5.6 Sol clocks in at 58%. Google's Gemini 3.5 Flash holds 56.5%, with OpenAI's GPT-5.5, GPT-5.4, GPT-5.6 Luna, and GPT-5.6 Terra rounding out the top eight. It's a clean sweep, folks.
The first open model to even sniff the top tier is DeepSeek V4 Pro 0813, way down at #9 with 51.8%. Moonshot's Kimi K2.6 barely makes the top ten at 50%. This isn't a gap; it's a chasm. It's like bringing a butter knife to a laser sword fight.
This leaderboard proves one thing: if you want your AI to actually do things in the real world, you're probably going to fork over cash to a big tech company. The "open" dream of agentic supremacy? That's a fantasy.
OSWorld-Verified: the open strike-back
But don't count the open models out entirely. When it comes to grounded reasoning and vision tasks, the tables turn faster than a VC pivoting to the next big thing. The OSWorld-Verified leaderboard shows a different story.
Here, Alibaba's Qwen3.8 Max, an open model, takes the crown at #1 with 86.1%. That's a solid win against Anthropic's Claude Fable 5 and Claude Mythos 5, both closed models, tied at 85%. Alibaba's Qwen3.8-27B also makes a strong showing at 84.3%, just ahead of Claude Opus 4.8 at 83.4%.
This is where the "open" narrative gets a little complicated. While it's great to see open models leading, the top performers are often backed by massive corporations like Alibaba. This isn't your garage startup anymore; it's a state-backed supply chain.
RealWorldQA: open takes the top eight
Forget the closed-source hype. The benchlm.ai RealWorldQA leaderboard, updated October 12, 2026, just dropped a truth bomb: open models are running laps around the big guys. Out of 17 systems, open models snagged the top eight spots.
Alibaba's Qwen3.8-Flash-Next, an open model, leads the pack at 88.5%. Right behind it? Qwen3.8 Max, also Alibaba, also open, at 88.0%.
Meanwhile, Google's Gemini 3 Pro (closed) and OpenAI's GPT-5.2 (closed) are tied at #9 and #10, both hitting 83.3%. That's a 5.2 percentage point gap between the #1 open model and the #9 closed contender. Ouch.
The 4B upset
But wait, there's more! The closed-source giants just got schooled by a featherweight. H2O-Lightning-4B, a 4-billion-parameter open-weights decision model, just crushed the JevBench Composite Score leaderboard.
Announced October 12, 2026, this open model scored 72.5/100, beating Jev, the benchmark's own closed API reference system, which only managed 71.5. It ranked first among 104 open-weight systems as of October 7, 2026.
And the kicker? It costs an estimated $0.021 per 1,000 decisions. H2O.ai CEO Sri Ambati put it best: "Better decisions don't need bigger models." This isn't just about performance; it's about making AI accessible. Closed models, take notes.
The autopsy: what "open vs closed" even means now
The verdict, the angriest most-entertaining one the evidence supports: the scoreboard is a tie fought in tenths of a point. Forget the scrappy underdog narrative; "open" in 2026 mostly means Alibaba/Qwen, DeepSeek, Moonshot — Chinese labs as the open-weights supply chain, backed by balance sheets no garage startup can match. The promised-but-unshipped weights (Reflection Beam, Mistral Large 4, per the AI Week YouTube roundup) are the "coming soon" page of the open world, perpetually loading. And Meta? The company that made open weights a movement now leads an agentic board with a closed model, Muse Spark 1.1, while reportedly scaling back its own internal Claude use. Money talks, even in AI. The underdog story is dead; openness is infrastructure now — and infrastructure has owners.
Sources & further reading
- Toolathlon leaderboard
- OSWorld-Verified leaderboard
- RealWorldQA leaderboard
- H2O-Lightning-4B tops JevBench
- AI Week YouTube roundup
Enjoyed this? Get the next one first.
The Spicy Dispatch: one email a week, zero slop, unsubscribe anytime.
One email a week. Zero slop. Unsubscribe anytime.