Spark Bench

Inkling 2 Mini NVFP4 (vLLM, DSpark7, 1 Spark, 262K)

1× DGX Spark · vLLM + DSpark7 ·

Single-stream decode
41.59 output tok/s
Time to first token · p95, c1
0.28 s
Tool eval completeSafety gate failed

This configuration failed a critical tool-use check. Review the findings before using it for consequential automation. See the tool evaluation

Tested configuration

Hardware
1x NVIDIA DGX Spark (128GB)
Serving runtime
vLLM + DSpark7
Runtime version or build
Unavailable
Weight quantization or precision
NVFP4
Benchmark date
Recipe credit
vLLM's one-Spark Inkling serving recipeBy thinkingmachines

Decode sweep

~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.

ConcurrencyOutput tok/sTTFT p95 (s)
141.590.28
267.660.61
489.736.86
8146.433.13
12199.833.38
16229.472.85

Prefill and context

Configured request window
262,144 tokens
Longest tested prompt
245,760 target input tokens
Validated retrieval
≈240K input tokens · passkey passed

Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.

Workload (c1)Input tok/sTTFT p95 (s)
4,096 input tokens1,117.463.01
32,768 input tokens2,363.6012.75
131,072 input tokens2,007.4663.84
245,760 input tokensPasskey prompt1,548.23155.99

Sustained load

20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.

Aggregate throughput
145.04 output tok/s
TTFT p95
1.21 s
Completed requests
347
Request errors
0

Serving regression checks 10/13 passed

Tool-use evaluation

Complete · 100% completion Safety gate failed

Compare only against complete results with 100% completion under the same protocol ID.

Protocol ID
tool-eval-standard-v1-b60e2f0
Workload
69 mock-tool scenarios per trial · 3 trials
Mean score
88.3 / 100
Pass@3 · passed at least once
85.5%
Pass³ · passed every trial
76.8%

Read the tool-eval report and scenario findings

External benchmark pilots

Read the pilot report

RULER v1

4K / 32K / 128K context

Standard
3.85% / 0.00% / 7.69%
2,048-token cap
84.62% / 80.77% / 76.92%

Same 78 prompts. Empty answers decreased from 75 to 15. The rerun had zero request errors.

LongBench v2

Standard
0/24
2,048-token cap
4/24

Same 24 questions and context cap. Unparseable answers changed from 24 to 17.

τ²-Bench

6/9 full reward · 8/9 graded

The same local endpoint handled agent, user, and judge roles. Retail task 100 reached the 1,024-token cap without a usable assistant message and remained ungraded.

Standard and 2,048-token runs for sampled RULER v1 and LongBench v2. The local τ²-Bench row comes from the original run.

Standard tool-use benchmark

tool-eval-bench by SeraphimSerapis powers Spark Bench’s Tool eval and Safety gate badges.

Install model

Copy this prompt and paste it into an AI assistant with access to your DGX Spark.