Spark Bench

Nemotron 3.5 Super VL 120B-A12B NVFP4 on vLLM with DFlash, one Spark, thinking off

1× DGX Spark · vLLM ·

Single-stream decode
34.15 output tok/s
Time to first token · p95, c1
0.58 s
Tool eval completeSafety gate failed

This configuration failed a critical tool-use check. Review the findings before using it for consequential automation. See the tool evaluation

Tested configuration

Hardware
1x NVIDIA DGX Spark (128GB)
Serving runtime
vLLM
Runtime version or build
vLLM 0.32.0rc1.dev190+g3f1bcfd82
Weight quantization or precision
NVFP4
Benchmark date
Recipe credit
Kindling Nemotron 3.5 Super VL DGX Spark recipeBy kindlingai452acc1d74e76cfee4b45ad7a16998ee57f0a423

Decode sweep

~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.

ConcurrencyOutput tok/sTTFT p95 (s)
134.150.58
234.0116.58
434.5647.22
834.17105.30
1234.95167.49
1634.85227.70

Prefill and context

Configured request window
1,048,576 tokens
Longest tested prompt
245,760 target input tokens
Validated retrieval
≈240K input tokens · passkey passed

Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.

Workload (c1)Input tok/sTTFT p95 (s)
4,096 input tokens980.801.86
32,768 input tokens1,934.0714.85
131,072 input tokens1,839.1069.19
245,760 input tokensPasskey prompt1,577.68155.41

Sustained load

20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.

Aggregate throughput
33.90 output tok/s
TTFT p95
113.03 s
Completed requests
87
Request errors
0

Serving regression checks 13/13 passed

Tool-use evaluation

Complete · 100% completion Safety gate failed

Compare only against complete results with 100% completion under the same protocol ID.

Protocol ID
tool-eval-standard-v1-b60e2f0
Workload
69 mock-tool scenarios per trial · 3 trials
Mean score
92.7 / 100
Pass@3 · passed at least once
97.1%
Pass³ · passed every trial
81.2%

Read the tool-eval report and scenario findings

Install model

Copy this prompt and paste it into an AI assistant with access to your DGX Spark.