LLM API latency dataset

Where the probes run

🇭🇰 Hong Kong, 🇺🇸 US Central (Chicago). The two locations are listed separately and never averaged, because the same endpoint routinely differs by several times between them and the average would describe nobody. The Hong Kong node measures a cross-border international route, which is not the same as a mainland China direct connection. The US node reaching Chinese endpoints crosses the Pacific. There is no mainland node yet; if one is added it will be labelled separately.

How often it measures

TTFT is probed every 10 minutes by default and output speed every hour. A few endpoints are throttled to one round every 30 or 60 minutes to conserve provider quota (currently Kimi K3, Claude Sonnet 5 and Gemini 2.5 Flash), so rows differ widely in how many samples an hour holds. Read the sample count on the row before trusting a figure rather than assuming a uniform cadence. Results are aggregated into one row per endpoint per location per hour, and this page reads those hourly rows. A separate cost probe runs a standard task with reasoning deliberately left on; it uses a different setup and is not part of this dataset.

Reasoning mode is switched off

Reasoning mode is explicitly disabled on every probe request, so these figures describe time to first answer token, not time to finish thinking. Models that cannot disable reasoning are out of scope. Chain of thought tokens count towards completion_tokens and ignore the max_tokens cap (one model returned 896 tokens for a 256 token request), so leaving reasoning on would mix thinking speed into answering speed. The cost is coverage: Gemini 3.x will not disable reasoning, so the Gemini entry here is 2.5 Flash. Newer flagships lean towards forced reasoning, so this comparable set narrows over time.

Window percentiles are second-order aggregates

Window-level p50 and p95 are second-order aggregates of the hourly percentiles, not exact percentiles over every request in the window: raw per-request logs are retained for 14 days and are not publicly readable. ttft_p50_ms is the successful-sample-weighted median of the hourly p50 values. ttft_p95_typical_ms is the same weighted median of the hourly p95 values, i.e. the tail latency of a typical hour. ttft_p95_worst_ms is the highest hourly p95 in the window. The true window p95 lies between the typical and worst figures.

• ttft_p50_ms — successful-sample-weighted median of the hourly p50 values.
• ttft_p95_typical_ms — the same weighted median over the hourly p95 values: the tail latency of a typical hour.
• ttft_p95_worst_ms — the highest hourly p95 in the window.
• For endpoints measured only once or twice an hour, the typical-hour p95 collapses onto that single measurement and can equal the p50. That is a shortage of samples in the hour, not a flat tail. Divide the successful sample count by the hour coverage on the same row to see how many measurements an hour actually holds.
• tps_avg is the successful-sample-weighted mean of the hourly average tokens per second.
• error_rate is exact: the sum of error_rate x sample_count over the window divided by total samples. Hours in which every request failed carry no percentiles but still count towards samples and error_rate.

Why p95 matters, not just p50

The median tells you what a good moment feels like. The 95th percentile is the slowest of every 20 requests, and it is what decides when users start calling your app laggy, where your timeout has to sit, and how often your retry path fires. Two endpoints can both sit at 500 ms p50 while one has an 800 ms p95 and the other a 6 second p95; for anything interactive those are different products. Both the typical hour and the worst hour are listed here because "fine most of the time, awful occasionally" is the common failure shape.

How thin samples are flagged

Rows with measurements in under 50% of the window's hours, or fewer than 100 successful samples, are marked "few samples". The numbers are still shown, but they move around more than the rest. Rows where every hour failed are marked "no valid samples" and the latency columns stay blank rather than showing zero. Failed hours are not dropped from the error rate denominator: an endpoint unreachable half the time would otherwise look as fast as a stable one.

Limits

Two probe locations do not represent global experience, and they do not represent the route from your own servers. The probes send a fixed short prompt, so these figures do not describe long context, high concurrency or tool-calling workloads. Latency is one dimension of endpoint choice and this page does not rank vendors. To judge whether a number is trustworthy, read the sample count and hour coverage on the same row.