Leaderboard

DeepSeek V4.1 Flash 1017

Two DGX Sparks

On two DGX Sparks it decodes 78.6 tok/s and reads a 48,000-token codebase at 1,919 tok/s. It answers 87.9% of MMLU-Pro and gets 7 of 7 one-shot builds working. Ray tested it on 2026-10-17.

Machine
Two DGX Sparks: 2 × NVIDIA GB10, 2 × 128 GB unified memory, linked at 200 Gb/s
Engine
TensorFold Zig (Bertholomus AI)
Weights
EXL3 2.9 bpw community quant (LibertAIDAI)
Recipe
Tensor parallel over both Sparks, 1M context
Made by
DeepSeek
Size
MoE
Tested
2026-10-17
Asked who made it
“I was made by DeepSeek.”
Note
The gummy-bear page is run 822 (2026-10-18), the fair rerun at the model’s own sampling with the 262,144-token budget; the first suite run hit a 65,536 cap.
Overall
828
Builds
1,000
Knowledge
879
Speed
606

Scores out of 1,000. Overall is the mean of the other three; how each is scored.

Speed

Decode speed
78.6 tok/s
Decode, prose
60.6 tok/s
Decode, Snake
96.5 tok/s
First token
365 ms
Prefill, 48k prompt
1,919 tok/s
Tokens for the seven builds
204,621
Time for the seven builds
42m 43s

Decode speed is the median of a prose prompt and a Snake game, one request at a time. Prefill is how fast it reads a 48,000-token codebase before it answers.

Accuracy

SetCorrectScoreMinutes
MMLU-Pro, 140 questions, thinking low123 of 14087.9%11.7
GSM8K, 100 problems, thinking low99 of 10099.0%2.5
Snake by DeepSeek V4.1 Flash 1017
SnakeWorks
96.5 tok/s1m 12s6,967 tokens
Rocket launch by DeepSeek V4.1 Flash 1017
Rocket launchWorks
81.0 tok/s10m 35s51,372 tokens
Starts black for its first seconds, then launches.
Heptagon balls by DeepSeek V4.1 Flash 1017
Heptagon ballsWorks
74.7 tok/s4m 52s21,800 tokens
Solar system by DeepSeek V4.1 Flash 1017
Solar systemWorks
82.3 tok/s4m 08s20,284 tokens
Pelican on a bicycle by DeepSeek V4.1 Flash 1017
Pelican on a bicycleWorks
81.8 tok/s4m 59s24,441 tokens
City fireworks by DeepSeek V4.1 Flash 1017
City fireworksWorks
79.3 tok/s8m 13s39,096 tokens
Gummy bears by DeepSeek V4.1 Flash 1017
Gummy bearsWorks
77.9 tok/s8m 42s40,661 tokens
The bears are clusters of spheres, but all of them pour and settle in the bowl.
Capybara onsen by DeepSeek V4.1 Flash 1017
Capybara onsenRuns with faults
79.4 tok/s15m 22s73,183 tokens
Every control works, but the capybara, tub and boat are one brown block with a lantern.

Run local models in Ownlight

Ownlight connects the model servers you already run, like Ollama, LM Studio, Unsloth Studio, llama-swap or Model Deck, and starts Claude Code or Codex on a local model beside your cloud agents. Local is part of Free.

Get OwnlightDownload for MacHow Local works

Every step

Decode speed, time to first token, prefill speed, output tokens and total time for each prompt in the suite.

TestDecode tok/sFirst tokenPrefill tok/sOutput tokensTotal timeRun
Prose (speed)60.6365 ms —1,78629.8 s798
Snake96.5662 ms —6,9671m 12s800
Read a 48k-token codebase76.323.7 s 1,91993335.9 s801
Who made you?80.7256 ms —51890 ms802
Rocket launch81.0424 ms —51,37210m 35s805
Heptagon balls74.7416 ms —21,8004m 52s806
Solar system82.32.1 s —20,2844m 08s807
Pelican on a bicycle81.8362 ms —24,4414m 59s808
City fireworks79.3461 ms —39,0968m 13s809
Gummy bears77.9583 ms —40,6618m 42s822
Capybara onsen79.41.0 s —73,18315m 22s915

More models on the bench

Updated 2026-10-20 16:39.