MundoKit

Tokens per Second Calculator: Latency and Throughput

Enter measured output tokens and elapsed time for one streamed request or a whole run. Compare end-to-end throughput with decode rate and request throughput.

Inputs
seconds
Single: request sent to last token. Run: first request sent to last response.
seconds
Required only when output contains tokens; leave blank for an empty response.

Results

End-to-end output throughput
33.667 tokens/s
Mean inter-token latency (excludes first token)
20 ms
Decode rate (reciprocal of ITL)
50 tokens/s
  • Use measured output token counts, not words or input tokens. Aggregate throughput does not predict individual latency.

Results update as you type. The share link stores your inputs after the # in the address, which is not sent to our server, but anyone you send it to can read them.

How it works

Following NVIDIA’s metric definitions, output throughput is output tokens divided by measured elapsed seconds. In single-request mode, mean inter-token latency excludes the first token. Its reciprocal is the decode rate, which differs from end-to-end throughput.

ITL = (elapsed − TTFT) / (output tokens − 1) decode rate = 1 / ITL run throughput = total output tokens / wall time request throughput = completed requests / wall time

For a run, measure wall time from the first request to the last response; do not add overlapping request latencies. Zero or one token has no inter-token interval. TTFT cannot be inferred from aggregate throughput. Results use up to three decimals with no intermediate rounding.

Worked examples

One streamed request

101 output tokens in 3 seconds, with TTFT of 1 second, give 33.667 tokens/s end-to-end. ITL is (3 − 1) / 100 = 20 ms and decode rate is 50 tokens/s.

A concurrent benchmark run

1,000 output tokens and 10 completed requests in a 5-second wall-time window give 200 tokens/s and 2 requests/s.

Frequently asked questions

Why subtract one output token for ITL?
There are N − 1 intervals after the first token in an N-token response. TTFT accounts for the wait before the first token.
Is decode speed the same as tokens per second?
The decode rate here excludes TTFT. End-to-end output throughput includes it. Use the same definitions when comparing reports.
Can I calculate TTFT from total time alone?
No. Record when the first nonempty token arrives. A total duration does not identify that moment.
Should I include prompt tokens?
No. This calculator measures output throughput. Adding input tokens produces a different metric.
What if no output tokens are produced?
Output throughput is zero and TTFT may be left blank. ITL and decode rate are not defined. In run mode, request throughput can still be positive for completed empty responses.

Sources

Last reviewed 2026-11-14. The method is written out above so you can check our math.