Understanding LLM Inference Performance: Latency, Throughput, and GoodputA guide to core LLM serving metrics—including TTFT, TPOT, throughput, SLOs, and goodput—from prefill and decode onward.
#llm-inference#ai-infra#performance