Spark Bench

DeepSeek V4 Flash Vision-Exp on the Anemll DSpark vLLM runtime, two Sparks (TP=2), thinking off

2× DGX Spark · vLLM (Anemll DSpark 0.1.1) ·

Single-stream decode
25.93 output tok/s
Time to first token · p95, c1
0.69 s
Tool eval completeSafety gate failed

This configuration failed a critical tool-use check. Review the findings before using it for consequential automation. See the tool evaluation

Tested configuration

Hardware
2x NVIDIA DGX Spark (128GB each)
Serving runtime
vLLM (Anemll DSpark 0.1.1)
Runtime version or build
the recipe's Anemll dspark-vllm-gx10 0.1.1 runtime, built locally from Anemll/dspark-vllm-gx10 v0.1.1 (47503f8e38da): upstream vLLM 752a3a504485 (docker/Dockerfile --target vllm-openai, torch_cuda_arch_list=12.1a), then the Anemll runtime layer (FlashInfer 0472b9b3f2fb, b12x 7dc6fb8fcc64, CUTLASS DSL 4.5.2). The prebuilt ghcr.io/anemll/dspark-vllm-gx10:0.1.1 image was not used.
Weight quantization or precision
Unavailable
Benchmark date
Recipe credit
MiaAI-Lab DeepSeek-v4-Flash DSpark 2x recipeBy MiaAI-Laba8b4636710068fd30edb98e1365d567086cc208e

Decode sweep

~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.

ConcurrencyOutput tok/sTTFT p95 (s)
125.930.69
247.040.76
465.241.01
879.9126.29
1274.8345.51
1678.1581.35

Prefill and context

Configured request window
1,048,576 tokens
Longest tested prompt
245,760 target input tokens
Validated retrieval
≈240K input tokens · passkey passed

Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.

Workload (c1)Input tok/sTTFT p95 (s)
4,096 input tokens722.553.78
32,768 input tokens1,334.3023.04
131,072 input tokens1,150.22111.94
245,760 input tokensPasskey prompt1,325.83185.15

Sustained load

20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.

Aggregate throughput
84.53 output tok/s
TTFT p95
23.27 s
Completed requests
205
Request errors
0

Serving regression checks 13/13 passed

Tool-use evaluation

Complete · 100% completion Safety gate failed

Compare only against complete results with 100% completion under the same protocol ID.

Protocol ID
tool-eval-standard-v1-b60e2f0
Workload
69 mock-tool scenarios per trial · 3 trials
Mean score
91.7 / 100
Pass@3 · passed at least once
94.2%
Pass³ · passed every trial
84.1%

Read the tool-eval report and scenario findings

Install model

Copy this prompt and paste it into an AI assistant with access to your DGX Spark.