The YOLOv8 confusion matrix is a validation tool that shows how predicted object classes compare with the actual ground-truth classes. It helps identify correct detections, class-to-class confusion, false positives, and false negatives in a way that overall metrics such as mAP cannot fully show. In current Ultralytics detection validation, the confusion matrix uses True classes on the horizontal axis and Predicted classes on the vertical axis, with an additional background row and column for unmatched objects and detections.
A strong YOLOv8 model should generally have high values along the main diagonal and relatively low values outside it. However, the matrix should not be interpreted alone. Precision, recall, F1, mAP, per-class AP, and visual inspection of false positives and false negatives should be considered together when evaluating detection quality.
Introduction to the YOLOv8 Confusion Matrix
YOLOv8 validation generates several metrics and plots that summarize model performance. Precision, recall, and mAP provide numerical summaries, while the confusion matrix makes specific error patterns easier to see. Ultralytics describes the confusion matrix as one of the visual aids saved during validation to help identify areas where the model is performing poorly.
For example, suppose a detector contains three classes:
car
truck
bus
The overall mAP may appear strong, but the confusion matrix might show that many trucks are being predicted as cars. This is important information because simply looking at overall mAP would not immediately reveal which classes are being confused.
A confusion matrix can therefore answer questions such as:
Which classes are detected correctly?
Which classes are confused with each other?
How many real objects are missed?
Which classes generate false detections?
Is one class much weaker than the others?
This makes it particularly useful for debugging custom YOLOv8 datasets.
What Is a Confusion Matrix in YOLOv8?
A confusion matrix is a table that records relationships between model predictions and ground-truth objects.
For YOLOv8 object detection, this process requires more than comparing class names. Predictions first have to be matched with ground-truth bounding boxes according to overlap and confidence conditions. Current Ultralytics’ detection confusion-matrix implementation accepts a confidence threshold and an IoU threshold for this matching process.
Once matching has been performed, detections are categorized as correct matches, incorrect class matches, unmatched predictions, or unmatched ground-truth objects.
Purpose of the Confusion Matrix
The main purpose is to make error patterns visible at the class level.
For example:
Actual: truck
Predicted: truck
→ correct detection
but:
Actual: truck
Predicted: car
→ class confusion
or:
Actual: truck
Prediction: nothing
→ missed object
These outcomes are difficult to understand from one average metric.
The confusion matrix allows developers to see where the detector fails and then decide whether the solution involves more training data, better labels, class balancing, hard-negative examples, or hyperparameter changes.
How It Measures Detection Performance
Current Ultralytics first filters detections by confidence and then computes IoU between predictions and ground-truth boxes. Matches above the configured IoU threshold are deduplicated so each ground-truth object and each detection can participate in at most one match.
The resulting outcomes are recorded in the matrix.
Conceptually:
Prediction overlaps GT sufficiently
↓
Compare class IDs
↓
Same class?
↙ ↘
Yes No
↓ ↓
TP Class confusion
If no prediction matches a ground-truth object, that object becomes a false negative.
If a prediction remains unmatched, it becomes a false positive.
Structure of the YOLOv8 Confusion Matrix
For detection, Ultralytics creates a matrix with:
number_of_classes + 1
rows and columns.
The extra entry represents background. Current source code allocates (nc + 1, nc + 1) for detection and OBB confusion matrices.
A three-class example conceptually looks like:
| Predicted ↓ / True → | Car | Truck | Bus | Background |
|---|---|---|---|---|
| Car | 90 | 8 | 1 | 12 |
| Truck | 5 | 75 | 4 | 7 |
| Bus | 2 | 3 | 82 | 4 |
| Background | 10 | 14 | 6 | — |
The exact values are only illustrative.
Predicted Classes
In the current Ultralytics plot, the vertical Y-axis is labeled Predicted.
This means each row represents what the model predicted.
For example, the row:
Predicted = car
contains counts showing how often true cars, trucks, buses, or background were predicted as cars.
This axis orientation is important because confusion matrices from other libraries sometimes use the opposite arrangement.
Actual Ground Truth Classes
The horizontal X-axis is labeled True in current Ultralytics confusion-matrix plots.
Each column therefore represents the real ground-truth category.
For example:
True = truck
allows you to inspect what happened to all ground-truth truck objects.
Ideally, most of those values should appear in:
Predicted truck
on the diagonal.
Background Row and Column
Object detection requires an additional background entry because some predictions have no matching real object and some real objects have no matching prediction.
In the current Ultralytics matrix:
Predicted class row
+
True background column
represents false-positive detections. The source code records unmatched detections as:
matrix[predicted_class, background] += 1
Conversely:
Predicted background row
+
True class column
represents false negatives, meaning real objects that the detector failed to match. Ultralytics records these as:
matrix[background, true_class] += 1
This background row and column are one of the most important differences between an object-detection confusion matrix and a simple image-classification confusion matrix.
How to Read a YOLOv8 Confusion Matrix
A useful reading strategy is to first inspect the diagonal, then examine the off-diagonal class values, and finally inspect the background row and column.
This separates the model’s mistakes into three major categories:
correct detections
class confusion
missing or extra detections
Correct Predictions on the Diagonal
The main diagonal contains matches where:
Predicted class = True class
For example:
True: car
Predicted: car
is placed in the car-to-car diagonal cell.
Current Ultralytics increments the matrix at [predicted_class, true_class], meaning same-class matches naturally appear on the main diagonal.
Higher diagonal values generally indicate better class-level recognition.
Conceptually:
True
Car Truck
Pred Car 90 6
Truck 4 82
The values 90 and 82 are correct class matches.
Misclassifications Outside the Diagonal
Off-diagonal cells between real classes represent class confusion.
For example:
True class = truck
Predicted class = car
appears at:
row = car
column = truck
This means the detector found the object but assigned the wrong class.
If one pair of classes is frequently confused, investigate whether they:
- look visually similar,
- have inconsistent labels,
- lack enough training examples,
- appear at very small sizes,
- have overlapping class definitions.
False Positives and False Negatives
False positives and false negatives appear through the background entries.
A false positive is an unmatched prediction:
Predicted object
but no matching ground-truth object
Ultralytics places this in:
predicted class row
background column
A false negative is an unmatched real object:
Ground-truth object exists
but no prediction matches it
This is placed in:
background row
true class column
True Positives, False Positives, and False Negatives
These three concepts form the foundation of many validation metrics.
Ultralytics’ confusion-matrix implementation can also retain matched examples categorized as TP, FP, FN, and ground truth for visualization when match saving is enabled.
True Positive Predictions
A true positive occurs when a predicted object:
- sufficiently overlaps a ground-truth object,
- is successfully matched,
- has the correct class.
For example:
Ground truth:
car
Prediction:
car
IoU:
sufficient
Result:
True Positive
This contributes to the diagonal of the confusion matrix.
False Positive Predictions
A false positive occurs when a detection cannot be correctly associated with a real object.
Examples include:
background texture detected as car
or:
unlabeled empty area detected as person
In current Ultralytics, unmatched detections are added to the background column for their predicted class.
False positives reduce precision.
False Negative Predictions
A false negative occurs when a real labeled object is not successfully detected.
For example:
Ground truth:
helmet
Prediction:
nothing
This contributes to the background predicted row under the helmet true-class column.
False negatives reduce recall.
Normalized vs Raw Confusion Matrix
Ultralytics can represent the confusion matrix using either raw counts or normalized values.
The current ConfusionMatrix.plot() method supports a normalize argument, which defaults to True, while its summary method can also produce normalized or non-normalized representations.
Both views are useful but answer different questions.
Raw Detection Counts
A raw matrix contains actual counts.
For example:
car correctly detected = 1,820
truck predicted as car = 140
car missed = 95
Raw counts are useful when you want to understand the absolute number of errors.
They also make severe dataset imbalance obvious.
For example:
car = 20,000 objects
ambulance = 250 objects
A raw matrix will visually be dominated by car counts.
Normalized Class Performance
Current Ultralytics normalizes the plotted matrix by columns, meaning values are divided by the total for each true class column.
This makes each true class easier to compare even when class frequencies differ.
For example:
True truck column:
Predicted truck 0.80
Predicted car 0.10
Predicted bus 0.03
Predicted background 0.07
This means approximately 80% of the relevant true truck instances represented in that column were correctly assigned as trucks under the matrix’s matching settings.
Normalization is particularly useful for imbalanced datasets.
When to Use Each View
Use the raw matrix when you want to answer:
How many mistakes occurred?
How many false positives are there?
How large is each class?
Use the normalized matrix when you want to answer:
Which classes perform relatively poorly?
What fraction of one class is confused?
Which minority class has weak recall?
For a complete analysis, inspect both.
Confusion Matrix and YOLOv8 Metrics
The confusion matrix is closely related to precision, recall, and F1 because all of these depend on true positives, false positives, and false negatives.
However, mAP is calculated using precision-recall behavior across detection confidence and IoU criteria and should not be treated as simply one number read directly from a single confusion matrix. Ultralytics reports mAP50, mAP75, and mAP50-95 separately during validation.
Relationship with Precision
Precision is:
Precision =
TP
──────
TP + FP
A confusion matrix with many predictions in the:
True background column
indicates many unmatched predictions.
This can contribute to low precision.
For example:
Predicted car:
900 true cars
250 background false positives
suggests the car detector produces many unnecessary detections.
Relationship with Recall
Recall is:
Recall =
TP
──────
TP + FN
A high value in:
Predicted background row
for a particular true class indicates many missed objects.
For example:
True pedestrian
Predicted background = high
suggests poor pedestrian recall.
Relationship with F1 Score and mAP
F1 combines precision and recall:
F1 =
2 × Precision × Recall
──────────────────────
Precision + Recall
A class with many background false positives or false negatives will generally have weaker F1 behavior.
mAP goes further by evaluating precision-recall performance and localization quality. Current Ultralytics validation exposes map50, map75, map for mAP50-95, and per-category maps through the returned metrics object.
Therefore:
Confusion Matrix
→ explains types of errors
Precision / Recall / F1
→ summarize detection tradeoffs
mAP
→ summarizes precision-recall and localization performance
How to Generate a Confusion Matrix in YOLOv8
YOLOv8 can generate confusion-matrix plots during model validation.
Ultralytics validation mode is designed to compare predictions against ground-truth labels and save visual aids such as confusion matrices and precision-recall curves.
Generate It During Validation
Using CLI:
yolo detect val model=best.pt data=data.yaml plots=True
Using Python:
from ultralytics import YOLO
model = YOLO("best.pt")
metrics = model.val(
data="data.yaml",
plots=True
)
Current Ultralytics’ confusion-matrix API is accessible from validation results, and its documentation shows examples using:
results.confusion_matrix.summary()
for programmatic analysis.
Locate the Saved Confusion Matrix Files
Current Ultralytics’ plotting code names the normalized matrix:
confusion_matrix_normalized.png
and the non-normalized version:
confusion_matrix.png
because file names are constructed directly from the plot title.
They are saved inside the validation run’s output directory.
A typical structure may look like:
runs/
└── detect/
└── val/
├── confusion_matrix.png
├── confusion_matrix_normalized.png
└── other validation plots
The exact run number or folder name can differ depending on previous experiments and project or name settings.
Analyze Per-Class Results
You can also access confusion-matrix information programmatically.
For example:
from ultralytics import YOLO
model = YOLO("best.pt")
results = model.val(data="data.yaml", plots=True)
cm = results.confusion_matrix
summary = cm.summary(
normalize=True,
decimals=4
)
print(summary)
Current Ultralytics documents summary() as returning one dictionary per predicted class with values for all true classes.
This is useful when you want to export or automatically analyze class confusion instead of manually reading the image.
How to Improve a Poor YOLOv8 Confusion Matrix
A poor confusion matrix is usually a symptom rather than the root problem.
The pattern of errors tells you where to investigate. Heavy class-to-class confusion suggests a different issue from large numbers of background false positives or missed objects.
Fix Class Imbalance
Suppose the dataset contains:
car: 20,000
truck: 5,000
ambulance: 200
If ambulance has weak diagonal performance, the model may simply lack enough diverse examples.
Possible improvements include:
- collect more rare-class examples,
- oversample minority classes,
- use realistic augmentation,
- evaluate class weighting where supported.
Do not focus only on equal image counts. Count actual labeled object instances as well.
Improve Annotation Quality
Incorrect annotations can directly create confusing validation patterns.
For example:
same object sometimes labeled truck
sometimes labeled van
can produce persistent class-to-class confusion.
Check:
class consistency
bounding-box quality
missing labels
duplicate labels
wrong class IDs
Ground truth is the standard against which validation is calculated, so incorrect validation labels can also make correct predictions appear wrong.
Add More Difficult Training Examples
If the model repeatedly fails in specific situations, add those situations to the training data.
For example:
small pedestrians
partially hidden vehicles
night images
crowded scenes
objects near image edges
visually similar non-target objects
For false positives, difficult negative images are particularly valuable.
If a model repeatedly identifies round signs as helmets, training examples containing round signs with no helmet annotation can teach the detector the difference.
Tune Confidence and IoU Thresholds
Confidence and IoU settings influence which detections are considered and how matching is performed.
Current Ultralytics ConfusionMatrix.process_batch() exposes:
conf = 0.25
iou_thres = 0.45
as method defaults for confusion-matrix processing.
Changing thresholds can alter the apparent balance of false positives and missed objects.
However, threshold tuning should not be used to hide fundamental dataset problems.
For model comparison, use the same evaluation configuration for every model.
Common YOLOv8 Confusion Matrix Problems
Certain patterns appear frequently in custom detection projects.
Learning to recognize them can make model debugging much faster.
One Class Is Frequently Confused with Another
Example:
True truck
↓
Predicted car frequently
Possible reasons include:
- classes are visually similar,
- class definitions overlap,
- labels are inconsistent,
- one class has too few examples,
- objects are too small to show distinguishing features.
Inspect real images behind the confusion instead of relying only on the matrix.
Too Many Background False Positives
If a predicted-class row contains a large value in the background column, the model is detecting objects where no matching ground truth exists.
Possible causes include:
- background patterns resemble the object,
- insufficient negative images,
- confidence threshold behavior,
- missing ground-truth labels,
- overfitting.
Remember that a prediction can appear as a false positive if the object actually exists but was accidentally left unlabeled.
Too Many Missed Objects
A large value in:
Predicted background
True target class
means many real objects are being missed.
Possible causes include:
- objects are too small,
- poor contrast,
- insufficient examples,
- class imbalance,
- occlusion,
- inadequate training resolution.
This pattern is strongly associated with low recall.
Strong Overall Metrics but Weak Individual Classes
Overall results may look good when majority classes perform strongly.
For example:
car AP = 0.90
person AP = 0.86
truck AP = 0.82
ambulance AP = 0.25
The overall mAP may still appear acceptable.
A normalized confusion matrix makes this type of minority-class weakness easier to notice because each true-class column is scaled independently in the current Ultralytics plot.
FAQs About the YOLOv8 Confusion Matrix
What does a YOLOv8 confusion matrix show?
It shows how predicted classes compare with ground-truth classes and highlights correct detections, class confusion, false positives, and false negatives.
Current Ultralytics detection matrices include an extra background row and column in addition to the dataset classes.
How do I read a YOLOv8 confusion matrix?
In current Ultralytics plots:
X-axis = True class
Y-axis = Predicted class
Start with the diagonal for correct predictions, then inspect off-diagonal cells for class confusion.
Finally inspect:
background column
→ false positives
background row
→ false negatives
What does the background class mean in the confusion matrix?
Background is not a normal trained object category in this context.
It represents unmatched detections or unmatched ground-truth objects.
In current Ultralytics detection matrices:
Predicted class + True background
→ False Positive
Predicted background + True class
→ False Negative
What do diagonal values represent?
Diagonal values represent matches where the predicted class equals the ground-truth class.
For example:
True car
Predicted car
contributes to the car diagonal cell.
A strong model generally has a high concentration of values along the diagonal.
What is the difference between normalized and raw confusion matrices?
A raw matrix contains actual detection counts.
A normalized matrix displays relative proportions.
Current Ultralytics normalized plots divide each column by the sum of that true-class column, making class-level comparisons easier even when dataset frequencies differ.
How are false positives shown in the YOLOv8 confusion matrix?
False positives are unmatched model predictions.
Current Ultralytics records them at:
row = predicted class
column = background
A large value in a class’s background-column cell indicates that the model frequently predicts that class where no matching ground-truth object exists.
How can I improve my YOLOv8 confusion matrix results?
Start by identifying the specific error pattern.
For class confusion:
improve class definitions
add more examples
fix labels
For false positives:
add difficult negative images
check missing labels
review confidence behavior
For false negatives:
add difficult target examples
increase useful image resolution
improve class balance
check annotation quality
Then retrain and compare the normalized and raw matrices using the same validation dataset.
Conclusion
The YOLOv8 confusion matrix is one of the most useful tools for understanding why a detector succeeds or fails.
Current Ultralytics object-detection confusion matrices use:
Horizontal axis → True classes
Vertical axis → Predicted classes
and include a background row and column.
The matrix can be interpreted as:
Main diagonal
→ correct class detections
Off-diagonal class cells
→ class confusion
Predicted class + True background
→ false positive
Predicted background + True class
→ false negative
A strong evaluation workflow is:
Run Validation
↓
Open Normalized Confusion Matrix
↓
Check Diagonal
↓
Inspect Class Confusion
↓
Check Background Column for FP
↓
Check Background Row for FN
↓
Review Per-Class Precision and Recall
↓
Inspect Failed Images
↓
Improve Dataset or Training
Ultralytics validation also reports mAP50, mAP75, mAP50-95, and per-category performance, so the confusion matrix should be used together with those metrics rather than as a replacement for them.
The normalized matrix is particularly useful for identifying weak classes in imbalanced datasets, while the raw matrix helps determine the absolute number of mistakes. By combining both views with precision, recall, mAP, and visual inspection of false detections and missed objects, you can obtain a much clearer understanding of YOLOv8 model performance.
I’m Jane Austen, a skilled content writer with the ability to simplify any complex topic. I focus on delivering valuable tips and strategies throughout my articles.