time to first token (TTFT)

advancedwidespread

The first token time measures the duration between sending your query and the appearance of the first text segment of the AI response.

In plain terms

When you type a question into an AI chatbot, the answer does not arrive all at once but trickles out in small pieces, called tokens (words or word fragments), onto the screen. The first token time is the wait until the very first piece appears.

It is like in a restaurant: what matters first is not when you finish dinner, but when the waiter brings the first course after ordering. If something arrives quickly, you wait more patiently for the rest.

This is why this metric is primarily about the experience: it shows how fast you feel the system is. The total response generation time and the text's 'typing' speed are measured separately.

It is especially important for voice assistants and phone customer service agents, as even one or two seconds of silence in a conversation can feel awkward.

How it works

You send the question: the message travels across the network to the provider's server, which in itself takes time.
The request queues: if many people use the system simultaneously, it must wait until computing capacity becomes available.
The model processes the entire input: the longer the question and the attached material, the longer this step takes.
The model generates the first token, the server sends it to you immediately, and it appears on the screen. Here the TTFT stopwatch stops.
With "thinking" models and agents, internal reasoning or tool use occurs before the visible response, so this time can be much longer for them.

An everyday example

You ask the chatbot what to cook for dinner and press Enter. That half-to-one second where you only see the blinking dots before the first word appears is the time for the first token. If you attach a long document to the same question, it takes noticeably longer.

Who is affected

developerThis is one of the most important metrics to monitor and improve when building an AI application, as it directly determines user experience. → Code

entrepreneur, executiveWhen choosing a model or provider, slow startup can deter customers, especially in customer service and voice-based solutions.

everyoneYou feel this when one AI tool responds instantly while another takes a long time to "think".

A common misconception

MythIf the first word appears quickly, the model is fast and the response will be ready soon.

In realityThe first token time only measures the start. A fast-starting model can still generate slowly; the total response time also depends on text length and generation speed.

Limits and risks

It says nothing about the quality of the answer: a fast-starting response can still be inaccurate or wrong.
It varies strongly depending on the time of day, load, question length, and your network connection, so it is not advisable to draw conclusions from a single measurement.
Provider measurements are not always comparable because it may differ whether network latency, queuing, or internal "thinking" time is included.
For background tasks without human supervision (such as overnight document processing), it matters little; there, total runtime and cost are what count.

Similar, but not the same

token speed (tokens/second) – It measures how fast the text streams after the response has started, whereas TTFT only covers the wait time until the start.
total response time – It refers to the time from sending the request to the last token, meaning TTFT is only the first phase of this.
time to first byte (TTFB) – A related metric from the web world measures time up to the server's first data byte, not up to the model's first meaningful text segment.

Related concepts

Broader

latency, response time, performance measurement

Narrower

time to first audio

On the edge

token, response streaming, token throughput, inference, prompt caching, cold start

A brief history

The name of the concept follows the pattern of "time to first byte" (TTFB), long used in web performance measurement. It became widespread after ChatGPT was released at the end of 2022, when continuous, chunk-by-chunk response delivery became the standard operation of chat AI.

Where things stand

At the end of 2026, TTFT is one of the basic metrics for comparing language models and providers, and benchmarks and provider documentation report it regularly. With the spread of reasoning models and agents, the time until the first visible response is increasingly measured separately as well.

Status: 2026 November

Also known as

TTFT · time to first token · time-to-first-token · time to first token · time to first token · time to first token · first token latency · first token latency

Sources

NVIDIA NIM – LLM Inference Benchmarking: Metrics

This entry was machine-edited and has undergone machine expert review; the “In the news” section updates automatically from Agent Age stories.