Leaderboard

Qwen 3.8 Flash Next

One DGX Spark

On one DGX Spark it decodes 66.7 tok/s and reads a 48,000-token codebase at 2,065 tok/s. It answers 80.7% of MMLU-Pro and gets 2 of 7 one-shot builds working. Ray tested it on 2026-10-19.

Machine
One DGX Spark: NVIDIA GB10, 128 GB unified memory
Engine
TensorFold
Weights
MLX 4-bit of the official weights (Vontra)
Recipe
MiaAI-Lab recipe, MTP 6 drafts, 220K context
Made by
Alibaba Qwen
Size
MoE
Tested
2026-10-19
Asked who made it
“I am Qwen (Tongyi Qianwen), a large language model independently developed by Alibaba Group’s Tongyi Lab.”
Overall
550
Builds
286
Knowledge
807
Speed
558

Scores out of 1,000. Overall is the mean of the other three; how each is scored.

Speed

Decode speed
66.7 tok/s
Decode, prose
66.8 tok/s
Decode, Snake
66.5 tok/s
First token
282 ms
Prefill, 48k prompt
2,065 tok/s
Tokens for the seven builds
131,829
Time for the seven builds
35m 54s

Decode speed is the median of a prose prompt and a Snake game, one request at a time. Prefill is how fast it reads a 48,000-token codebase before it answers.

Accuracy

SetCorrectScoreMinutes
MMLU-Pro, 140 questions, thinking low113 of 14080.7%18.8
GSM8K, 100 problems, thinking off99 of 10099.0%3.6
No page written
SnakeNo page
66.5 tok/s8m 12s32,768 tokens
No page: at the template’s default effort (xhigh) it thought through the whole 32,768-token budget. At effort low it wrote a working game in 60 s (extra run).
Rocket launch by Qwen 3.8 Flash Next
Rocket launchBroken
61.0 tok/s4m 10s15,273 tokens
Shows the rocket on the pad, then a script error (dt is not defined) stops it; it never launches.
Heptagon balls by Qwen 3.8 Flash Next
Heptagon ballsBroken
64.8 tok/s2m 02s7,927 tokens
Blank: a duplicate variable name (t) stops the script before it draws.
Solar system by Qwen 3.8 Flash Next
Solar systemWorks
75.6 tok/s1m 34s7,140 tokens
Pelican on a bicycle by Qwen 3.8 Flash Next
Pelican on a bicycleBroken
52.0 tok/s8m 19s25,992 tokens
Background only: a duplicate variable name (frame) stops the script before the pelican is drawn.
City fireworks by Qwen 3.8 Flash Next
City fireworksWorks
67.9 tok/s2m 01s8,254 tokens
Gummy bears by Qwen 3.8 Flash Next
Gummy bearsBroken
60.4 tok/s9m 31s34,475 tokens
Empty scene: its import map blocks the OrbitControls module.

Run local models in Ownlight

Ownlight connects the model servers you already run, like Ollama, LM Studio, Unsloth Studio, llama-swap or Model Deck, and starts Claude Code or Codex on a local model beside your cloud agents. Local is part of Free.

Get OwnlightDownload for MacHow Local works

Every step

Decode speed, time to first token, prefill speed, output tokens and total time for each prompt in the suite.

TestDecode tok/sFirst tokenPrefill tok/sOutput tokensTotal timeRun
Prose (speed)66.8282 ms —2,12832.1 s851
Snake66.5360 ms —32,7688m 12s853
Read a 48k-token codebase59.822.2 s 2,06592337.6 s854
Who made you?78.4248 ms —841.3 s855
Rocket launch61.0291 ms —15,2734m 10s856
Heptagon balls64.8304 ms —7,9272m 02s857
Solar system75.6302 ms —7,1401m 34s858
Pelican on a bicycle52.0273 ms —25,9928m 19s862
City fireworks67.9289 ms —8,2542m 01s863
Gummy bears60.4440 ms —34,4759m 31s870

Other runs of Qwen 3.8 Flash Next

MachineDecode tok/sProseSnakeEngine and weights
Two RTX 5090s 218.5164.8272.2Strata, IQ2_XS
Mac Studio M3 Ultra 100.978.8123.0MLX 4.7 bpw
Two RTX 5090s 69.669.6—llama.cpp, IQ4_XS
Two DGX Sparks 56.846.966.8vLLM NVFP4, 47K draft vocabulary

More models on the bench

Updated 2026-10-20 16:39.