Open vs closed
What a task costs
Each mark is one model on ARC-AGI-2: higher scores are better, marks further left cost less per task. Blue squares are open-weight models you can download and run; orange rings are closed models you can only rent.
| Model | Vendor | Weights | ARC-AGI-2 score | Cost per task |
|---|---|---|---|---|
| GPT-6 Astra | OpenAI | Closed | 95% | $1.12 |
| GPT-6.1 Sol | OpenAI | Closed | 94.2% | $0.254 |
| Claude Opus 5.5 | Anthropic | Closed | 93.3% | $0.408 |
| Gemini 4 Argon | Closed | 91.2% | $0.325 | |
| Gemini 3.8 Flash | Closed | 89.2% | $0.4 | |
| GPT-5.5 | OpenAI | Closed | 85% | $1.87 |
| Claude Sonnet 5.5 | Anthropic | Closed | 83.9% | $0.546 |
| DeepSeek V4.1 Flash 1017 | DeepSeek | Open | 78.1% | $0.163 |
| Grok 4.6 | xAI | Closed | 67.1% | $0.757 |
| GLM-5.3-Flash | Z.ai (Zhipu) | Open | 65.8% | $0.093 |
| Grok 4.7 | xAI | Closed | 61.4% | $2.01 |
| DeepSeek V4 Flash 0731 | DeepSeek | Open | 61.4% | $0.042 |
| Claude Haiku 5.5 | Anthropic | Closed | 61.2% | $0.085 |
| DeepSeek V4 Pro 0813 | DeepSeek | Open | 61.2% | $0.598 |
| Kimi K3 | Moonshot AI | Open | 60.4% | $1.59 |
| GPT-6 Luna | OpenAI | Closed | 59.3% | $0.062 |
| Muse Spark Open | Meta | Open | 57.5% | $0.728 |
| Qwen3.8-27B | Alibaba (Qwen) | Open | 42.4% | $0.447 |
| Inkling Small | Thinking Machines Lab | Open | 40.1% | $0.233 |
Source: ARC Prize verified leaderboard, results as of 28 Oct 2026. Cost per task is what the benchmark run paid at API list prices; running an open model on your own hardware costs electricity and the machine instead.
Local performance
Rigs that run it
Measured results from people running open models on their own hardware. Decode is how fast the answer streams; prefill is how fast the model reads your prompt. The top three in each tier lead; scroll a row to see up to ten.
Local mini
One consumer device: a single GPU with up to 32 GB, an Apple Silicon Mac with up to 64 GB, or a small PC. Models up to about 35B parameters.
| # | Model | Hardware | Engine | Decode tok/s | Prefill tok/s |
|---|---|---|---|---|---|
| 1 | Qwen3.6-35B-A3B UD-Q4_K_M (20.61 GiB) | 1x NVIDIA RTX 5090 32 GB | llama.cpp | 271 | 9,240 |
| 2 | gpt-oss-20b MXFP4 (11.27 GiB) | 1x NVIDIA RTX 4090 24 GB | llama.cpp | 221.9 | 8,022 |
| 3 | Qwen3-30B-A3B Q4_K_M | 1x AMD Radeon AI PRO R9700 32 GB (RDNA4) | llama.cpp | 183.5 | 3,033 |
| 4 | Gemma 4 12B Q6_K (9.1 GiB) | 1x NVIDIA RTX 5090 32 GB | llama.cpp | 122.3 | 7,160 |
| 5 | Qwen3-30B-A3B-Instruct-2507 4-bit MLX | Apple M4 Max, 64 GB unified memory | MLX | 113.3 | 1,754 |
| 6 | Qwen3.8-27B NVFP4 (Digisensus build) | 1x NVIDIA RTX 5090 32 GB (power limit 480 W) | vLLM (patched) | 113 | 4,475 |
| 7 | Qwen3.5-35B-A3B Q4_0 (ubergarm, 19.78 GiB) | 1x NVIDIA RTX 3090 Ti FE 24 GB (450 W, tuned with LACT) | ik_llama.cpp | 107.3 | 2,936 |
| 8 | Qwen3.5-35B-A3B Q4_0 (ubergarm, 19.78 GiB) | 1x AMD Radeon RX 7900 XTX 24 GB | llama.cpp | 104.8 | 2,105 |
| 9 | Qwen3.6-27B Q4_K_M (15.66 GiB) | 1x NVIDIA RTX 5090 32 GB | llama.cpp | 77.1 | 3,826 |
| 10 | Qwen3-Coder-30B-A3B-Instruct 4-bit MLX | Apple M4 Pro (20-core GPU), 48 GB unified memory | oMLX | 72.6 | 750 |
Local medium
A workstation: two to four GPUs, one 48–96 GB card, a DGX Spark, a 128 GB Strix Halo box or a 96–256 GB Mac. Models of about 70B to 150B parameters.
| # | Model | Hardware | Engine | Decode tok/s | Prefill tok/s |
|---|---|---|---|---|---|
| 1 | gpt-oss-120b MXFP4 (GGUF, 59.02 GiB) | 1x NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | llama.cpp (CUDA) | 196.3 | 5,518 |
| 2 | Qwen3.8-Flash-Next NVFP4 (Local Inference Lab build) | 1× NVIDIA RTX PRO 6000 Blackwell Max-Q (325 W limit) | vLLM (Local Inference Lab image) | 173.2 | 12,073 |
| 3 | gpt-oss-120b MXFP4 (GGUF, 59.02 GiB) | Apple M2 Ultra, 192GB, 76 GPU cores (Mac Studio) | llama.cpp (Metal) | 79.7 | 1,245 |
| 4 | gpt-oss-120b MXFP4 (GGUF, 59.02 GiB) | NVIDIA DGX Spark (GB10 Grace Blackwell, 128GB unified LPDDR5X) | llama.cpp (CUDA) | 58.7 | 2,444 |
| 5 | gpt-oss-120b MXFP4 (GGUF, 59.02 GiB) | AMD Ryzen AI Max+ 395 'Strix Halo', 128GB unified RAM (Framework Desktop), Fedora 43 | llama.cpp (Vulkan) | 56.6 | 720 |
| 6 | Qwen3.5-122B-A10B 4-bit (MLX) | Apple M3 Ultra, 60-core GPU, 96GB unified memory (Mac Studio) | oMLX | 55.6 | 651 |
| 7 | Qwen3.5-122B-A10B INT4 AutoRound + FP8 shared experts | NVIDIA DGX Spark (GB10, SM121, 128GB LPDDR5X unified, ~273 GB/s) | vLLM (sm121 build) | 36 | — |
| 8 | MiniMax-M2.7 UD-Q3_K_S (GGUF, 87.2 GiB) | AMD Ryzen AI Max+ 395 'Strix Halo', 128GB unified RAM (Framework Desktop), Fedora 43 | llama.cpp (Vulkan) | 31.1 | 242 |
| 9 | Qwen3.5-122B-A10B UD-Q5_K_XL (GGUF, 85.6 GiB) | AMD Ryzen AI Max+ 395 'Strix Halo', 128GB unified RAM (Framework Desktop), Fedora 43 | llama.cpp (Vulkan) | 22.3 | 309 |
| 10 | Qwen3.5-122B-A10B NVFP4 (W4A4) | NVIDIA DGX Spark (GB10, SM121), single node | vLLM nightly | 15.1 | 1,602 |
Local extreme
Frontier-size open models on hardware you own: 512 GB Macs and Mac clusters, 4–8 GPU servers, CPU and GPU hybrids. Models of 200B parameters and up.
| # | Model | Hardware | Engine | Decode tok/s | Prefill tok/s |
|---|---|---|---|---|---|
| 1 | DeepSeek-V4-Flash-0731 FP8 + MXFP4 experts | 8× RTX PRO 6000 Blackwell Server Edition (768 GB GDDR7) | vLLM + DSpark speculative decoding | 204 | — |
| 2 | MiniMax M2.7 UD-Q5_K_XL (157.8 GiB) | 8× RTX 3090 (24 GB each) | ik_llama.cpp | 61.1 | 788 |
| 3 | Qwen3.5-397B-A17B 4-bit MLX | 1× Mac Studio M3 Ultra (80-core GPU) 512 GB | oMLX | 35.3 | 481 |
| 4 | GLM-5.1 NVFP4 (lukealonso build, 434 GB) | 6× RTX PRO 6000 Blackwell Workstation Edition (96 GB each) | vLLM (TP 2 × PP 3) | 33.3 | — |
| 5 | DeepSeek-V4-Flash-0731 MXFP4 routed experts (CPU+GPU split) | 8× RTX 5090 + x86 CPU host (CPU model not stated) | KTransformers + SGLang | 32.7 | — |
| 6 | Kimi K2 Thinking INT4 (native weights) | 4× Mac Studio M3 Ultra (2× 512 GB + 2× 256 GB, 1.5 TB total) over TB5 RDMA | exo (MLX, RDMA over Thunderbolt 5) | 25 | — |
| 7 | Kimi K2.6 Q4_X (GGUF, ubergarm) | 1× RTX PRO 6000 (96 GB) + AMD EPYC 9355, DDR5-6400 (experts in system RAM) | ik_llama.cpp | 20.2 | 729 |
| 8 | GLM-5.2 UD-Q4_K_XL (unsloth GGUF) | 2× RTX PRO 6000 + AMD EPYC 9B45, 12-channel DDR5-6400 | llama.cpp (fork) | 20 | — |
| 9 | DeepSeek-V3.2 4-bit MLX | 1× Mac Studio M3 Ultra (80-core GPU) 512 GB | oMLX | 19.3 | 210 |
| 10 | GLM-5.2 4-bit MLX (oQ4e), MTP on | 1× Mac Studio M3 Ultra 512 GB | oMLX | 19 | 194 |
Single-user numbers unless a tile says otherwise. Figures come from the linked source and use its prompt length, quantization and engine version, so compare tiles as reports rather than a controlled benchmark. We link to Hugging Face where a result is published there, then GitHub, then the original post.
Ranked daily
News
Releases, engines, hardware and benchmarks from the people and projects building local inference. Ranked by impact and freshness, not by volume.
Model
Mistral Large 4 weights released: 1T-parameter MoE with GGUF and NVFP4 builds huggingface.co
Mistral opened weights for Large 4 (about 1T total, 49-52B active, 1M context, modified-MIT license with a model-as-a-service carve-out); the Q2_K GGUF runs on a 512 GB M3 Ultra at about 15 tok/s.
Model
MiniMax publishes M3.1 Flash weights: 428B MoE with 23B active huggingface.co
MiniMax released quantization-aware-trained weights for its multimodal M3.1 Flash four days after the hosted launch, under a community license with branding and commercial-authorization conditions.
Model
Recursive releases Flint, a 40B-A2.5B coding model under Apache 2.0 recursive.com
A small-active-parameter MoE coding model that the vendor says runs at over 125 tok/s on an M4 Pro MacBook Pro.
Model
DeepSeek ships V4.1-Flash-1017 update under MIT license huggingface.co
A minor update to the 763B-parameter V4.1-Flash that DeepSeek says improves reasoning, coding and agentic scores (HLE no-tools 36.8 to 42.6) while producing shorter outputs.
Model
Meta releases Muse Spark Open, an 816B-parameter open-weight MoE model huggingface.co
Meta published weights for an 816B-total, 42B-active MoE model with 1M context under its Muse Model License, with a stated target of two M5 Ultra Mac Studios.
Hardware
RTX Spark laptops with up to 128GB unified memory go on sale unite.ai
NVIDIA's Grace CPU plus Blackwell RTX laptop chip shipped in machines from ASUS, Dell, HP, Lenovo, Microsoft and MSI; NVIDIA claims up to 1 petaflop of FP4.
Benchmark
NVIDIA runs full GLM 5.3 on an 8-node DGX Spark cluster at 31 tok/s developer.nvidia.com
With 1 TB of pooled memory and an NVFP4 checkpoint, eight Sparks decode GLM 5.3 at 31 tok/s single-stream with MTP speculative decoding, using a new RoCE all-to-all backend merged into FlashInfer.
Engine
vLLM 0.32.0 adds NVFP4 KV cache on SM8x and SM12x GPUs github.com
The release brings 4-bit KV cache to Ampere and consumer Blackwell (and to ROCm), adds kernel-download and preload commands for faster startup, and supports EmbeddingGemma 2 and MiniCPM-V 4.7.
Quantization
vLLM and Ant Group release Humming kernels for quantized inference vllm.ai
An open JIT-compiled kernel library (in vLLM since v0.29) for low-bit integer, FP4-FP8, MXFP4 and NVFP4 weights on Turing and newer GPUs, including SM120/121 workstation parts; it reports about 2x over Marlin for W4A16 at larger batch sizes.
Model
NVIDIA releases Nemotron 3.5 Super VL, a 120B-A12B hybrid Mamba MoE huggingface.co
An open vision-language model with 1M context under the OpenMDW-1.1 license.
Model
Liquid AI releases LFM2.5-24B-A2B for on-device agents huggingface.co
A 24B-total, 2B-active reasoning MoE with day-one llama.cpp and MLX support; Liquid reports 184 tok/s on an M5 Max and 114 tok/s on a Ryzen AI Max+ 395.
Model
Liquid AI releases LFM3 family of 2B-class text, vision, audio and omni models huggingface.co
The LFM3-1.7B text model fits in about 1 GB at 4-bit with 256K context, and the vision and audio variants target laptops and edge devices.
Engine
Ollama 0.40.12 adds LFM3 and speeds up prompt processing on Apple silicon github.com
The release adds the LFM3 family (Apple-only for now), faster MLX prefill for Gemma 4, Flint and Laguna, and DFlash2 drafts for MLX speculative decoding.
Engine
Transformers can now load and run llama.cpp GGUF quants on Apple Silicon huggingface.co
Transformers keeps GGUF weights packed on the GPU using ggml Metal kernels, with throughput close to llama.cpp; the path is MPS-only and initially covers the Qwen3.5 and Qwen3.8 architectures.
Quantization
llama.cpp team publishes GLM-5.3-Flash GGUF quants huggingface.co
Official ggml-org GGUFs of the 320B-A18B model start at 110 GB (Q2_K_S), with MTP sidecars for speculative decoding and a vision mmproj.
Engine
LM Studio adds Splash engine for Qwen3.8-27B on Apple Silicon lmstudio.ai
Inco AI's open-source engine, paired with DFlash 2 draft models, reports 74 tok/s for Qwen3.8-27B on a 48 GB M5 Pro, about 2x the next-fastest engine Inco tested.
Hardware
NVIDIA announces a 64GB DGX Spark at $4,999 blogs.nvidia.com
The cheaper-memory GB10 configuration ships October 23 from OEM partners, and two units can be cabled together to pool 128 GB.
Engine
AnyLanguageModel 1.0 brings local MLX, llama.cpp and Core ML backends to Swift apps huggingface.co
A drop-in replacement for Apple's Foundation Models API; version 1.0 adds tool calling, guided generation, image input and Linux support across local and hosted providers.
Engine
mlx-lm 0.32.0 adds NVFP4 support for quantized MoE models github.com
The first mlx-lm release since April adds an NVFP4 global scale for quantized MoEs, a MiniCPM5 tool parser and a Muse Spark NVFP4 model.
Engine
Ollama 0.40.5 adds Nemotron 3.5 Super with up to 1M context github.com
Ollama added NVIDIA's 120B-A12B vision-language model on release day, along with llama.cpp and MLX backend updates.
Quantization
MLX 0.32.4 adds 1-bit affine quantization on Metal github.com
Apple's array framework adds 1-bit Metal quantization, gradients for SDPA and gated delta nets, and CUDA gather_qmm speedups.
Engine
vLLM 0.31.0 adds fast restart and makes NVFP4 KV the default for DeepSeek-V4.1-Flash github.com
A new 'vllm preload' command keeps quantized weights in GPU memory across restarts, and the release supports GLM-5.3-Flash MXFP4 checkpoints.
Engine
SGLang 0.5.21 adds DeepSeek-V4.1 Flash support and a Rust prefix cache github.com
The release adds DeepSeek-V4.1 Flash, MiMo-V2.6 and Qwen-Image 2.1, prefill/decode role switching without restart, and new Decisions and Score APIs.
Quantization
ExLlamaV3 1.5.4 adds token-embedding quantization and disk streaming github.com
The release lowers RAM and VRAM use, improves quantization accuracy, integrates Transformers 5 and fixes MiMo-V2.6.
Benchmark
Community recipe runs full GLM-5.3 on four DGX Sparks with a 3-bit EXL3 quant forums.developer.nvidia.com
A 292 GB, about 3.04 bpw EXL3 quant served at TP4 reports about 43 tok/s on code and 28 tok/s on prose (self-reported).
Run it yourself. Check it yourself.
Results and corrections
Send a resultBenchmarks to add, numbers to correct, sources we should follow.
Private inference research
Discuss researchThreat models, benchmarks and private-inference prototypes.