YOLOv8 Inference Speed Benchmarks: Performance Across Models and Hardware

YOLOv8 inference speed benchmarks show how quickly different YOLOv8 model sizes can process images on CPUs, GPUs, edge devices, and optimized inference runtimes. Speed is usually reported as milliseconds per image (ms/image) or converted into frames per second (FPS). However, benchmark numbers only make sense when the model, image size, hardware, precision, batch size, and runtime are clearly specified.

Ultralytics’ official YOLOv8 detection benchmark at 640-pixel input reports CPU ONNX inference times ranging from 80.4 ms/image for YOLOv8n to 479.1 ms/image for YOLOv8x. On an NVIDIA A100 using TensorRT, the same official table reports 0.99 ms/image for YOLOv8n and 3.53 ms/image for YOLOv8x. These figures demonstrate the large impact of both model size and deployment hardware.

The official YOLOv8 COCO detection benchmark is:

These values should be treated as reference benchmarks rather than guaranteed deployment performance.

Introduction to YOLOv8 Inference Speed

Inference speed describes how quickly a trained YOLOv8 model can process new inputs after training is complete. It directly affects whether a detector is suitable for offline image processing, live webcams, surveillance systems, robotics, edge devices, or high-throughput server workloads.

A typical inference pipeline is:

Input Image
    ↓
Preprocessing
    ↓
YOLOv8 Forward Pass
    ↓
Post-Processing / NMS
    ↓
Final Detections

The total user-visible processing time may include all of these stages, while some published benchmark figures focus primarily on model inference. This distinction matters when comparing numbers from different sources.

A detector reporting:

5 ms model inference

does not necessarily mean the complete application processes each frame in exactly 5 ms. Video decoding, image resizing, transfers between CPU and GPU, post-processing, tracking, visualization, and file writing can add additional latency.

Ultralytics provides Benchmark mode specifically to compare model accuracy and inference time across export formats and hardware environments.

What Is Inference Speed in YOLOv8?

Inference speed is the time required for a trained YOLOv8 model to generate predictions from new input data.

Unlike training, inference does not normally perform:

loss calculation
backpropagation
gradient computation
optimizer updates

The model performs a forward pass and prediction post-processing.

For deployment, inference speed is usually evaluated using latency, throughput, or FPS.

Latency vs Throughput

Latency measures how long one request takes.

For example:

Latency = 10 ms/image

means one image requires approximately 10 milliseconds under the tested conditions.

Throughput measures how many inputs can be processed per unit of time.

For example:

Throughput = 200 images/second

Batching may improve throughput without reducing the latency of each individual request.

For a live camera, low latency may be more important than maximum throughput.

For a server processing millions of stored images, throughput may matter more.

Frames Per Second and Milliseconds per Image

FPS can be approximately calculated from inference time using:

FPS ≈ 1000 / milliseconds per frame

For example:

10 ms/image
≈ 100 FPS

and:

40 ms/image
≈ 25 FPS

Using the official YOLOv8 CPU ONNX benchmark:

YOLOv8n
80.4 ms/image
≈ 12.4 images/sec

while the official A100 TensorRT figure:

0.99 ms/image
≈ 1010 images/sec

is a theoretical conversion of model benchmark latency, not a guarantee of 1,010 FPS in a complete camera application.

YOLOv8 Model Sizes and Speed Differences

YOLOv8 is available in five primary detection sizes:

YOLOv8n
YOLOv8s
YOLOv8m
YOLOv8l
YOLOv8x

The larger models generally provide higher accuracy but require more computation.

The official benchmark clearly demonstrates this trade-off.

YOLOv8n Inference Speed

YOLOv8n is the smallest and fastest standard YOLOv8 detection model.

Official figures at imgsz=640 are:

CPU ONNX:
80.4 ms/image

A100 TensorRT:
0.99 ms/image

Parameters:
3.2 million

FLOPs:
8.7 billion

Its COCO mAP50-95 is:

37.3

YOLOv8n is therefore a strong starting point for:

edge devices
live camera systems
high-FPS applications
CPU deployment
multi-camera servers

when speed is more important than maximizing accuracy.

YOLOv8s and YOLOv8m Performance

YOLOv8s increases model capacity substantially.

Official YOLOv8s figures:

CPU ONNX:
128.4 ms/image

A100 TensorRT:
1.20 ms/image

mAP50-95:
44.9

Parameters:
11.2M

YOLOv8m moves further toward accuracy:

CPU ONNX:
234.7 ms/image

A100 TensorRT:
1.83 ms/image

mAP50-95:
50.2

Parameters:
25.9M

YOLOv8m gains approximately 5.3 mAP50-95 points over YOLOv8s in the official COCO benchmark, but requires considerably more computation.

YOLOv8l and YOLOv8x Performance

YOLOv8l and YOLOv8x prioritize detection accuracy over minimal inference cost.

YOLOv8l:

CPU ONNX:
375.2 ms/image

A100 TensorRT:
2.39 ms/image

mAP50-95:
52.9

YOLOv8x:

CPU ONNX:
479.1 ms/image

A100 TensorRT:
3.53 ms/image

mAP50-95:
53.9

YOLOv8x reaches the highest COCO mAP50-95 among the standard YOLOv8 detection variants, but it contains 68.2 million parameters and requires about 257.8 billion FLOPs at the benchmark input size.

This illustrates diminishing returns: going from YOLOv8l to YOLOv8x increases computation considerably for a smaller accuracy gain than earlier model-size jumps.

YOLOv8 Inference Speed on Different Hardware

Hardware can change YOLOv8 speed by orders of magnitude.

A model that struggles to reach real-time speed on a general-purpose CPU may run extremely quickly on a modern GPU with TensorRT.

You should therefore never quote a YOLOv8 FPS number without specifying the hardware.

CPU Performance

The official YOLOv8 benchmark uses ONNX CPU measurements at 640-pixel input.

The reported values are:

YOLOv8n → 80.4 ms
YOLOv8s → 128.4 ms
YOLOv8m → 234.7 ms
YOLOv8l → 375.2 ms
YOLOv8x → 479.1 ms

Converted approximately:

YOLOv8n → 12.4 FPS
YOLOv8s → 7.8 FPS
YOLOv8m → 4.3 FPS
YOLOv8l → 2.7 FPS
YOLOv8x → 2.1 FPS

Again, these conversions refer to benchmark inference time and not necessarily complete end-to-end video throughput.

CPU performance varies significantly by processor architecture, instruction support, thread count, runtime, and power limits.

NVIDIA GPU Performance

NVIDIA GPUs can accelerate YOLOv8 dramatically, particularly when TensorRT is used.

Ultralytics’ official YOLOv8 table reports A100 TensorRT speeds of:

YOLOv8n → 0.99 ms
YOLOv8s → 1.20 ms
YOLOv8m → 1.83 ms
YOLOv8l → 2.39 ms
YOLOv8x → 3.53 ms

A separate current Ultralytics TensorRT guide contains YOLOv8n measurements across an A100, RTX 3080, RTX 3060, RTX 2060, and Jetson Orin NX. It shows that FP16 and INT8 can significantly reduce latency compared with FP32, although INT8 may reduce accuracy depending on calibration and model.

For example, the guide reports YOLOv8n TensorRT prediction latency on an RTX 3080 at 640 pixels, batch 8, of approximately:

FP32 → 1.06 ms
FP16 → 0.62 ms
INT8 → 0.52 ms

under the documented benchmark environment.

Edge Device Performance

Edge devices usually have tighter constraints around:

power consumption
memory
thermal limits
GPU capability
CPU performance

Ultralytics’ TensorRT benchmark includes a Jetson Orin NX 16 GB test using YOLOv8n at 640 pixels. Under the documented setup, prediction latency was approximately:

FP32 → 6.11 ms
FP16 → 3.18 ms
INT8 → 2.30 ms

at batch 8.

These results demonstrate why optimized runtimes and reduced precision are particularly important for embedded systems.

Real deployment speed may still be lower once camera decoding, tracking, visualization, and application logic are included.

Factors That Affect YOLOv8 Inference Speed

YOLOv8 speed is not determined by the model name alone.

Several factors interact:

model size
input resolution
batch size
hardware
precision
runtime
preprocessing
post-processing

A benchmark is only fair when these conditions remain consistent.

Image Resolution

Image resolution strongly affects computation.

A model using:

imgsz=640

processes far fewer pixels than one using:

imgsz=1280

Doubling width and height approximately quadruples the pixel count:

640 × 640
= 409,600 pixels

1280 × 1280
= 1,638,400 pixels

Therefore, higher resolution usually increases latency significantly.

Higher resolution may still be necessary for:

small objects
distant objects
fine defects
aerial imagery

so speed should not be optimized without measuring detection quality.

Batch Size

Batching allows multiple images to be processed together.

Example:

batch=1

focuses on single-request latency.

A larger batch such as:

batch=8

may improve GPU throughput.

However:

batch size ↑
→ VRAM usage ↑
→ potential throughput ↑
→ individual batch latency ↑

A large batch is usually better for offline processing than low-latency camera applications.

Model Size

The official YOLOv8 benchmark shows the clearest model-size trend.

At 640 pixels:

YOLOv8n
8.7B FLOPs

YOLOv8x
257.8B FLOPs

That roughly 30-times difference in FLOPs explains why YOLOv8x requires significantly more computation.

Larger models also use more:

VRAM
RAM
memory bandwidth
storage

Precision Mode and Device

Inference can often run with:

FP32
FP16
INT8

depending on runtime and hardware.

FP16 can improve speed and reduce memory use on compatible GPUs.

INT8 can improve performance further but generally requires quantization and may introduce accuracy loss.

Ultralytics’ TensorRT YOLOv8n benchmarks show substantial latency reductions when moving from FP32 to FP16 or INT8 on tested NVIDIA hardware.

YOLOv8 Inference Speed by Export Format

The original YOLOv8 checkpoint uses PyTorch, but deployment runtimes can provide better performance on specific hardware.

Ultralytics Benchmark mode supports testing formats such as ONNX, TensorRT, OpenVINO, CoreML, and others.

The correct runtime depends on the device.

PyTorch Inference

PyTorch .pt models are convenient for:

development
experimentation
training
validation
general Python inference

Example:

from ultralytics import YOLO

model = YOLO("yolov8n.pt")
results = model("image.jpg")

PyTorch is flexible, but it may not be the fastest deployment runtime for a fixed hardware target.

Exporting can enable hardware-specific optimization.

ONNX Runtime Performance

Export:

from ultralytics import YOLO

model = YOLO("yolov8n.pt")

model.export(
    format="onnx"
)

ONNX is commonly used for cross-platform deployment and CPU inference.

Ultralytics Benchmark documentation specifically identifies ONNX as a useful option for CPU performance and recommends benchmarking it against other formats on the target machine.

The official YOLOv8 CPU numbers in the main model table are measured using ONNX.

TensorRT Inference

Export:

model.export(
    format="engine"
)

TensorRT is optimized for NVIDIA GPUs.

Ultralytics Benchmark documentation describes TensorRT as the format aimed at maximum GPU efficiency, and the current benchmark guide notes that TensorRT can provide major GPU speedups depending on the model and hardware.

TensorRT is especially attractive for:

NVIDIA server GPUs
RTX GPUs
Jetson devices
high-throughput inference

FP16 and INT8 modes can further improve performance on compatible hardware.

OpenVINO Performance

Export:

model.export(
    format="openvino"
)

OpenVINO is particularly useful for Intel hardware.

Ultralytics currently describes OpenVINO as supporting YOLOv8 detection, segmentation, classification, pose, and OBB. Its current integration guide demonstrates substantial CPU acceleration on recent Intel processors for newer YOLO families, reinforcing the benefit of hardware-specific optimization.

OpenVINO can target:

Intel CPU
Intel GPU
Intel NPU

on compatible systems.

For YOLOv8 specifically, benchmark it on your own machine rather than assuming the speed of a newer YOLO family will transfer directly.

How to Benchmark YOLOv8 Inference Speed

The most reliable benchmark is one performed on the hardware where the model will actually run.

Ultralytics provides built-in benchmarking utilities to compare speed and accuracy across export formats.

Measure Latency with the Ultralytics Tools

Ultralytics provides:

from ultralytics.utils.benchmarks import benchmark

benchmark(
    model="yolov8n.pt",
    data="coco8.yaml",
    imgsz=640,
    device=0
)

This exact YOLOv8 benchmark workflow is shown in the current YOLOv8 documentation.

For export dependencies:

pip install "ultralytics[export]"

can be used before running comprehensive export benchmarks.

Benchmark mode reports metrics including inference time and accuracy, allowing you to compare speed without ignoring model quality.

Benchmark with the Python API

You can also perform controlled application-level measurements.

Example:

import time
from ultralytics import YOLO

model = YOLO("yolov8n.pt")

image = "image.jpg"

for _ in range(10):
    model(image)

runs = 100

start = time.perf_counter()

for _ in range(runs):
    model(image)

elapsed = time.perf_counter() - start

average_ms = (
    elapsed / runs
) * 1000

print(
    f"Average: {average_ms:.2f} ms"
)

This measures a more complete high-level call than a pure kernel-only benchmark.

For GPU timing, synchronization is important because CUDA operations can execute asynchronously. A dedicated benchmark tool is generally safer when precise device-level timing is required.

Run Repeated Tests for Reliable Results

One inference is not enough.

A better benchmark performs:

warm-up runs
+
many measured runs
+
average / median
+
optional percentile analysis

For example:

10 warm-up runs
100 measured runs

Repeated measurement reduces noise from:

model initialization
memory allocation
caching
CPU scheduling
GPU clock changes
background processes

Ultralytics’ benchmark utilities are designed specifically for repeated performance profiling rather than relying on one prediction call.

YOLOv8 Speed vs Accuracy Trade-Off

YOLOv8 model selection is fundamentally an accuracy-speed decision.

Official COCO figures make the progression clear:

YOLOv8n
37.3 mAP
0.99 ms A100 TensorRT

YOLOv8x
53.9 mAP
3.53 ms A100 TensorRT

YOLOv8x improves accuracy significantly over Nano, but Nano is substantially cheaper to run.

Choosing a Faster Model

Choose YOLOv8n or YOLOv8s when:

latency matters
many cameras must share one GPU
hardware is limited
power consumption matters
objects are relatively easy

YOLOv8n is especially useful for proving whether the application can meet its latency target before moving to larger variants.

Choosing a More Accurate Model

YOLOv8l or YOLOv8x may be preferable when:

accuracy is more important than FPS
objects are difficult
classes are visually similar
small-object detection is important
powerful GPU hardware is available

The largest model should not automatically be chosen.

If YOLOv8m already meets the required accuracy, using YOLOv8x may add latency without enough practical benefit.

Finding the Best Balance for Your Project

A practical comparison is:

Model
↓
Measure mAP
↓
Measure recall
↓
Measure latency
↓
Measure memory
↓
Test real deployment data

Then identify the smallest model that satisfies the required quality.

This is often more efficient than selecting the largest checkpoint and trying to optimize it afterward.

How to Improve YOLOv8 Inference Speed

Several optimization strategies can improve speed without retraining the model from scratch.

The largest gains usually come from reducing compute or using a runtime optimized for the hardware.

Reduce Input Image Size

Example:

results = model.predict(
    source="image.jpg",
    imgsz=512
)

instead of:

imgsz=640

or:

imgsz=1280

Lower image size reduces computation and often increases FPS.

The trade-off is reduced spatial detail.

Always compare validation performance after changing resolution.

Use FP16 or INT8 Precision

FP16 example:

results = model.predict(
    source="image.jpg",
    device=0,
    half=True
)

On compatible GPUs, FP16 can improve speed and memory use.

For TensorRT deployment, FP16 and INT8 optimization can provide larger improvements. Ultralytics’ YOLOv8n TensorRT benchmarks show FP16 and INT8 outperforming FP32 across tested NVIDIA hardware.

INT8 may reduce accuracy, so calibration and validation are essential.

Export to an Optimized Runtime

Potential choices include:

ONNX → general/CPU
OpenVINO → Intel
TensorRT → NVIDIA GPU
CoreML → Apple platforms

Benchmark mode exists specifically to identify which export format performs best on your target environment.

Do not assume the fastest format on one machine will be fastest everywhere.

Use Batch Inference

Batching multiple stored images can increase throughput.

For example:

batch=1
→ low single-image latency

batch=16
→ potentially higher total throughput

This is particularly useful on GPUs where a single image may not fully utilize compute resources.

Batching is less useful for ultra-low-latency live cameras where frames must be handled immediately.

Real-Time YOLOv8 Performance

The term real-time does not have one universal FPS requirement.

The required speed depends on the application.

For some industrial monitoring:

5 FPS

may be sufficient.

For smooth surveillance:

20–30 FPS

may be desirable.

For high-speed sports or robotics, even higher rates may be required.

Inference Speed for Video

Video processing requires more than model inference.

End-to-end time includes:

decode frame
↓
preprocess
↓
YOLO inference
↓
post-process
↓
draw / track / save

A model capable of:

2 ms inference

may still produce only:

100 FPS end-to-end

or much less if decoding, tracking, and rendering dominate.

Therefore, benchmark the complete application when determining video capacity.

Webcam and RTSP Stream Performance

Live streams add:

camera encoding latency
network latency
RTSP buffering
video decoding
model processing

A model may be extremely fast but still show delayed detections because the RTSP buffer contains old frames.

For live deployment, measure:

source timestamp
→ detection timestamp

rather than only YOLO’s printed inference time.

This provides true application latency.

Minimum FPS for Real-Time Detection

There is no official universal minimum.

Some useful practical categories are:

1–5 FPS
slow monitoring

10–15 FPS
responsive monitoring

20–30 FPS
smooth video analytics

30+ FPS
high-speed applications

These should be treated as application guidelines rather than YOLO requirements.

The correct target is determined by how quickly objects move and how quickly the system must respond.

Common Benchmarking Mistakes

Incorrect benchmark design can make one model appear faster even when the comparison is not fair.

You should control the test conditions carefully.

Ignoring Warm-Up Runs

The first few inferences may include:

model initialization
CUDA kernel setup
memory allocation
runtime optimization
cache population

Therefore, the first result can be slower than steady-state performance.

Use warm-up iterations before recording measurements.

Comparing Different Image Sizes

This is not a fair comparison:

YOLOv8n
imgsz=640

YOLOv8m
imgsz=320

because both model size and input resolution changed.

A meaningful model comparison should use the same:

imgsz
batch
precision
device
runtime
dataset

The official YOLOv8 table standardizes detection comparisons at 640 pixels.

Mixing Preprocessing with Model Latency

Suppose:

preprocessing = 8 ms
inference = 5 ms
post-processing = 2 ms

Total latency is:

15 ms

If another benchmark reports only:

5 ms

for model inference, comparing the two values directly is misleading.

Always state whether measurements represent:

model inference only

or:

end-to-end processing

Comparing Different Hardware Without Context

A statement such as:

YOLOv8 runs at 1000 FPS

is incomplete.

It should specify something like:

YOLOv8n
A100
TensorRT
640 pixels
specific batch/precision conditions

Ultralytics’ official tables explicitly report hardware and runtime because those variables fundamentally determine speed.

FAQs About YOLOv8 Inference Speed Benchmarks

Which YOLOv8 model is the fastest?

Among the five standard YOLOv8 detection sizes, YOLOv8n is the fastest.

Official 640-pixel benchmark:

CPU ONNX:
80.4 ms/image

A100 TensorRT:
0.99 ms/image

It also has the fewest parameters and FLOPs among the standard variants.

How many FPS can YOLOv8 achieve?

There is no single YOLOv8 FPS number.

Speed depends on:

model size
GPU or CPU
input resolution
batch size
runtime
precision

For example, the official YOLOv8n A100 TensorRT benchmark reports 0.99 ms/image, while CPU ONNX reports 80.4 ms/image.

Those correspond roughly to:

A100 benchmark:
~1010 images/sec

CPU benchmark:
~12.4 images/sec

but actual end-to-end video FPS will usually differ.

Is YOLOv8 faster on GPU than CPU?

Usually, yes, especially on modern NVIDIA GPUs.

The official YOLOv8n table reports:

CPU ONNX:
80.4 ms

A100 TensorRT:
0.99 ms

at the documented 640-pixel benchmark configuration.

The magnitude of the difference depends on the CPU, GPU, runtime, and workload.

Does TensorRT make YOLOv8 faster?

TensorRT can significantly accelerate YOLOv8 on compatible NVIDIA hardware.

Ultralytics Benchmark documentation recommends TensorRT for GPU optimization, and its TensorRT guide shows strong YOLOv8n latency improvements with FP16 and INT8 on NVIDIA GPUs and Jetson hardware.

The exact gain must be benchmarked on the target GPU.

How does image size affect inference speed?

Larger image size increases computational work.

For example:

640×640
→ 409,600 pixels

1280×1280
→ 1,638,400 pixels

The second input contains four times as many pixels.

Inference therefore normally becomes slower as resolution increases.

However, larger input may improve detection of small objects.

What is a good inference time for real-time detection?

There is no universal value.

For a 30 FPS target:

1 second / 30
≈ 33.3 ms/frame

so the complete pipeline must average below approximately 33 ms per frame to sustain 30 FPS.

For 60 FPS:

~16.7 ms/frame

is the full-frame time budget.

Remember that YOLO inference is only part of this total.

How can I benchmark YOLOv8 on my own hardware?

Use the Ultralytics benchmark utility:

from ultralytics.utils.benchmarks import benchmark

benchmark(
    model="yolov8n.pt",
    data="coco8.yaml",
    imgsz=640,
    device=0
)

This YOLOv8 benchmarking approach is documented by Ultralytics and can compare speed and accuracy across supported export formats.

Use a representative dataset and repeat tests under consistent conditions.

Conclusion

YOLOv8 inference speed benchmarks demonstrate that model size, hardware, runtime, precision, and input resolution all strongly influence performance.

The current official YOLOv8 COCO detection benchmark at 640 pixels is:

             CPU ONNX      A100 TensorRT

YOLOv8n       80.4 ms          0.99 ms
YOLOv8s      128.4 ms          1.20 ms
YOLOv8m      234.7 ms          1.83 ms
YOLOv8l      375.2 ms          2.39 ms
YOLOv8x      479.1 ms          3.53 ms

The associated accuracy values are:

YOLOv8n → 37.3 mAP50-95
YOLOv8s → 44.9
YOLOv8m → 50.2
YOLOv8l → 52.9
YOLOv8x → 53.9

which illustrates the central YOLOv8 trade-off:

Smaller Model
     ↓
Lower Latency
Lower Compute
Lower Memory
     ↓
Usually Lower Accuracy

versus:

Larger Model
     ↓
Higher Compute
Higher Latency
Higher Memory
     ↓
Usually Higher Accuracy

For NVIDIA deployment, TensorRT can provide major acceleration, and Ultralytics’ YOLOv8n benchmarks show additional gains from FP16 and INT8 on A100, consumer RTX GPUs, and Jetson Orin NX hardware.

For CPUs, ONNX and OpenVINO should be evaluated. For Intel-specific systems, OpenVINO is specifically designed to optimize inference across compatible CPUs, GPUs, and NPUs.

A practical benchmark workflow is:

Choose YOLOv8 Model
        ↓
Fix Image Size
        ↓
Fix Dataset
        ↓
Fix Batch Size
        ↓
Warm Up Model
        ↓
Measure Repeated Runs
        ↓
Compare Latency
        ↓
Compare mAP
        ↓
Export to Optimized Runtime
        ↓
Benchmark Again

For reproducible testing, Ultralytics provides:

from ultralytics.utils.benchmarks import benchmark

benchmark(
    model="yolov8n.pt",
    data="coco8.yaml",
    imgsz=640,
    device=0
)

and Benchmark mode reports both inference time and task accuracy so speed improvements can be evaluated without ignoring model quality.

The best YOLOv8 model is therefore not simply the fastest or the most accurate one. It is the smallest and fastest configuration that still satisfies the accuracy, latency, throughput, memory, and deployment requirements of the specific project.

Jane Torres

I’m Jane Austen, a skilled content writer with the ability to simplify any complex topic. I focus on delivering valuable tips and strategies throughout my articles.