Compare configurations
Choose up to 3 tested recipes to see their benchmark results side by side.
Choose at least two configurations to see the comparison.
Benchmark results
Single-stream speed and time to first token use the same short-prompt decode test (~256 input / 512 output tokens per request). Sustained results show the duration, concurrency, and errors from each soak. Compare tool scores across complete runs with the same protocol ID.
Scroll the table to see every configuration.
| Measurement |
|---|
| DGX Spark count |
| Serving runtime |
| Tested |
| Single-stream decode |
| Time to first token · p95, c1 |
| Peak decode |
| Sustained throughput |
| Validated retrieval |
| 240K passkey TTFT p95 |
| Tool evaluation |
| Safety gate |
1× DGX Spark
vLLM + DSpark7
41.59 output tok/s
0.28 s
229.47 output tok/sOffered concurrency 16
145.04 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
155.99 s
complete88.30 / 100 · tool-eval-standard-v1-b60e2f0
failed
2× DGX Spark
vLLM 0.34.0
44.35 output tok/s
0.36 s
103.83 output tok/sOffered concurrency 4
140.54 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
99.43 s
complete92.00 / 100 · tool-eval-standard-v1-b60e2f0
failed
2× DGX Spark
TensorFold
41.85 output tok/s
0.51 s
77.02 output tok/sOffered concurrency 8
65.01 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
199.29 s
complete91.00 / 100 · tool-eval-standard-v1-b60e2f0
failed
1× DGX Spark
TensorFold
41.54 output tok/s
0.35 s
111.73 output tok/sOffered concurrency 4
77.79 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
126.64 s
complete89.00 / 100 · tool-eval-standard-v1-b60e2f0
failed
1× DGX Spark
SGLang
4.29 output tok/s
0.61 s
32.65 output tok/sOffered concurrency 16
31.28 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
452.84 s
complete95.00 / 100 · tool-eval-standard-v1-b60e2f0
passed
2× DGX Spark
TensorFold
42.67 output tok/s
0.37 s
121.04 output tok/sOffered concurrency 16
62.29 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
391.26 s
complete93.00 / 100 · tool-eval-standard-v1-b60e2f0
failed
2× DGX Spark
TensorFold
67.50 output tok/s
0.51 s
125.87 output tok/sOffered concurrency 16
108.73 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
120.40 s
incomplete90.00 / 100 · tool-eval-standard-v1-b60e2f0
passed
2× DGX Spark
TensorFold
54.53 output tok/s
0.44 s
93.20 output tok/sOffered concurrency 4
80.18 output tok/s20 min · c8 · 0 errors
Passkey not run
Unavailable
complete94.00 / 100 · tool-eval-standard-v1-b60e2f0
failed
1× DGX Spark
llama.cpp
81.73 output tok/s
0.32 s
83.40 output tok/sOffered concurrency 2
74.62 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
220.19 s
complete86.00 / 100 · tool-eval-standard-v1-b60e2f0
passed
1× DGX Spark
vLLM
77.46 output tok/s
0.20 s
429.21 output tok/sOffered concurrency 16
286.66 output tok/s20 min · c8 · 0 errors
Passkey not run
Unavailable
complete88.30 / 100 · tool-eval-standard-v1-b60e2f0
failed
2× DGX Spark
TensorFold
65.62 output tok/s
0.48 s
123.18 output tok/sOffered concurrency 16
103.17 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
125.17 s
incomplete90.00 / 100 · tool-eval-standard-v1-b60e2f0
passed
2× DGX Spark
vLLM (Anemll DSpark)
42.12 output tok/s
4.12 s
104.82 output tok/sOffered concurrency 16
111.10 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
175.40 s
complete88.30 / 100 · tool-eval-standard-v1-b60e2f0
passed
1× DGX Spark
vLLM
34.15 output tok/s
0.58 s
34.95 output tok/sOffered concurrency 12
33.90 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
155.41 s
complete92.70 / 100 · tool-eval-standard-v1-b60e2f0
failed
1× DGX Spark
TensorFold
34.29 output tok/s
0.48 s
100.43 output tok/sOffered concurrency 12
85.72 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
133.79 s
incomplete86.00 / 100 · tool-eval-standard-v1-b60e2f0
passed
2× DGX Spark
vLLM (Anemll DSpark 0.1.1)
25.93 output tok/s
0.69 s
79.91 output tok/sOffered concurrency 8
84.53 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
185.15 s
complete91.70 / 100 · tool-eval-standard-v1-b60e2f0
failed
1× DGX Spark
ExLlamaV3
47.74 output tok/s
2.74 s
52.74 output tok/sOffered concurrency 16
51.35 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
226.64 s
incomplete86.00 / 100 · tool-eval-standard-v1-b60e2f0
failed
1× DGX Spark
TensorFold
50.25 output tok/s
0.44 s
201.49 output tok/sOffered concurrency 16
160.59 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
311.47 s
incomplete90.00 / 100 · tool-eval-standard-v1-b60e2f0
failed
1× DGX Spark
TensorFold
79.23 output tok/s
0.44 s
234.45 output tok/sOffered concurrency 16
145.52 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
294.65 s
complete90.30 / 100 · tool-eval-standard-v1-b60e2f0
failed
2× DGX Spark
TensorFold
83.45 output tok/s
0.23 s
417.63 output tok/sOffered concurrency 16
432.90 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
98.98 s
incomplete88.00 / 100 · tool-eval-standard-v1-b60e2f0
passed
2× DGX Spark
vLLM
44.62 output tok/s
0.42 s
205.06 output tok/sOffered concurrency 12
196.70 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
102.36 s
unavailable
unavailable
2× DGX Spark
vLLM
14.20 output tok/s
0.45 s
66.18 output tok/sOffered concurrency 16
60.90 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
188.82 s
unavailable
unavailable
1× DGX Spark
SGLang
31.83 output tok/s
0.18 s
187.63 output tok/sOffered concurrency 16
157.80 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
395.05 s
unavailable
unavailable
1× DGX Spark
SGLang
18.60 output tok/s
0.28 s
72.86 output tok/sOffered concurrency 4
63.89 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
557.96 s
unavailable
unavailable
1× DGX Spark
vLLM + SparkInfer
29.62 output tok/s
0.54 s
31.44 output tok/sOffered concurrency 4
26.89 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
259.00 s
unavailable
unavailable
2× DGX Spark
TensorFold
51.49 output tok/s
0.41 s
96.72 output tok/sOffered concurrency 12
84.90 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
163.89 s
unavailable
unavailable
2× DGX Spark
vLLM
13.96 output tok/s
1.27 s
30.68 output tok/sOffered concurrency 4
21.88 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
287.63 s
unavailable
unavailable
2× DGX Spark
vLLM
27.42 output tok/s
0.85 s
44.49 output tok/sOffered concurrency 2
35.31 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
408.37 s
unavailable
unavailable
2× DGX Spark
vLLM
47.34 output tok/s
0.40 s
207.77 output tok/sOffered concurrency 16
197.42 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
99.22 s
unavailable
unavailable
2× DGX Spark
vLLM
47.51 output tok/s
0.36 s
213.72 output tok/sOffered concurrency 8
201.58 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
113.18 s
unavailable
unavailable
2× DGX Spark
vLLM
48.93 output tok/s
0.43 s
209.12 output tok/sOffered concurrency 12
197.71 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
117.30 s
unavailable
unavailable
2× DGX Spark
SGLang
38.43 output tok/s
0.41 s
215.06 output tok/sOffered concurrency 8
157.61 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
598.56 s
unavailable
unavailable
2× DGX Spark
vLLM
35.13 output tok/s
0.95 s
79.51 output tok/sOffered concurrency 16
38.53 output tok/s20 min · c8 · 0 errors
245,760 tokens · passkey passed
279.80 s
unavailable
unavailable