YOLOv8 Segmentation Label Format: Complete Annotation Guide

The YOLOv8 segmentation label format represents each object with a class ID followed by a sequence of normalized polygon coordinates. Unlike normal object detection labels, segmentation labels do not describe an object only with a rectangular bounding box. Instead, multiple (x, y) points trace the object’s shape, allowing YOLOv8 segmentation models to learn pixel-level object boundaries. Ultralytics uses one text label file per image and one annotation row for each segmented object.

Introduction to YOLOv8 Segmentation Label Format

YOLOv8 instance segmentation requires more detailed annotations than standard object detection.

A detection label tells the model approximately where an object is located using a rectangular bounding box. A segmentation label describes the actual visible shape of that object using polygon points.

This distinction is important for applications where knowing the exact object area matters, such as defect segmentation, medical imagery, vehicle outlines, product segmentation, agriculture, or background removal.

Ultralytics supports a YOLO-style instance segmentation format in which polygon coordinates are stored directly inside .txt annotation files.

What Is the YOLOv8 Segmentation Label Format?

A YOLOv8 segmentation label consists of a class ID followed by normalized polygon coordinates.

The general structure is:

<class_id> <x1> <y1> <x2> <y2> <x3> <y3> ... <xn> <yn>

Each (x, y) pair represents one vertex along the boundary of the segmented object.

For example:

0 0.245 0.310 0.390 0.210 0.550 0.260 0.630 0.470 0.520 0.650 0.300 0.610

Here, 0 is the class ID and the remaining values describe the object’s polygon. Ultralytics specifies that segmentation coordinates are normalized and that a polygon must contain at least three (x, y) points.

Difference Between Detection and Segmentation Labels

Standard YOLO detection labels usually follow:

class_id x_center y_center width height

For example:

0 0.50 0.45 0.30 0.60

This creates a rectangular bounding box.

A segmentation label instead follows:

class_id x1 y1 x2 y2 x3 y3 ... xn yn

For example:

0 0.31 0.25 0.46 0.19 0.62 0.34 0.58 0.61 0.35 0.65

The polygon can follow an irregular boundary more closely than a rectangular box.

Detection is therefore suitable when approximate object location is enough, while instance segmentation is useful when the object’s shape or occupied pixels matter.

How Polygon Annotations Represent Object Masks

A polygon is created by placing points around the boundary of an object.

Imagine a six-point polygon:

P1 ───── P2
 \        \
  P6      P3
   \      /
    P5──P4

Each point is stored as an x and y coordinate.

When these vertices are connected in sequence, they form a closed shape describing the object’s region.

The segmentation model learns from these polygon-defined masks and predicts masks for new objects during inference.

More complex shapes can use additional points to approximate their boundaries more accurately.

Structure of a YOLOv8 Segmentation Label

Every object annotation begins with its class and continues with polygon coordinates.

Conceptually:

Class ID | Polygon Point 1 | Polygon Point 2 | ... | Polygon Point N

Or:

0 x1 y1 x2 y2 x3 y3 ... xn yn

One important difference from detection labels is that segmentation rows can have different lengths because different objects may require different numbers of polygon points. Ultralytics explicitly allows varying polygon lengths.

Class ID in Segmentation Labels

The first value represents the object’s class.

For example:

0

might mean:

person

while:

1

could mean:

car

The class mapping is normally defined in the dataset YAML file.

For example:

names:
  0: person
  1: car
  2: bicycle

Every annotation class ID must correspond correctly to this mapping.

X and Y Polygon Coordinates

After the class ID, each pair of numbers represents one polygon vertex:

x1 y1
x2 y2
x3 y3
...
xn yn

The x coordinate represents horizontal position.

The y coordinate represents vertical position.

For example:

0.25 0.40

represents one point located at 25% of the image width and 40% of its height.

A complete object polygon might contain:

0.25 0.40
0.40 0.25
0.62 0.30
0.70 0.55
0.48 0.70
0.28 0.62

Normalized Coordinate Values

YOLO segmentation coordinates are normalized relative to the image dimensions.

The conversion is:

normalized_x = pixel_x / image_width
normalized_y = pixel_y / image_height

Suppose an image is:

1000 × 500 pixels

and one polygon point is located at:

x = 250
y = 100

The normalized coordinates are:

x = 250 / 1000 = 0.25
y = 100 / 500 = 0.20

So the label stores:

0.25 0.20

This normalization makes annotations independent of the original image resolution.

YOLOv8 Segmentation Label Example

Consider an image containing one car.

A simplified segmentation label might be:

0 0.20 0.42 0.30 0.30 0.60 0.29 0.75 0.45 0.69 0.70 0.27 0.68

The first value identifies the class.

The remaining coordinate pairs define the car’s polygon boundary.

Understanding a Single Annotation Line

Break this example into parts:

0

Class ID.

Then:

0.20 0.42
0.30 0.30
0.60 0.29
0.75 0.45
0.69 0.70
0.27 0.68

These are six polygon points.

When connected in order, they form the segmentation boundary for that object.

A polygon must contain at least three coordinate pairs, but it can contain many more when a more detailed boundary is needed.

Multiple Objects in One Label File

One label file can contain multiple object instances.

Each object gets its own line.

For example:

0 0.10 0.20 0.20 0.15 0.28 0.30 0.25 0.52 0.12 0.48
1 0.50 0.30 0.68 0.25 0.78 0.40 0.74 0.65 0.55 0.67
1 0.22 0.60 0.35 0.55 0.40 0.76 0.24 0.82

This image contains three segmented instances.

The first is class 0.

The next two are class 1.

Each row has its own polygon, and the number of polygon coordinates does not need to be identical between objects.

YOLOv8 Segmentation Dataset Folder Structure

YOLO segmentation datasets commonly use separate directories for images and labels.

A typical layout is:

dataset/
├── images/
│   ├── train/
│   └── val/
├── labels/
│   ├── train/
│   └── val/
└── data.yaml

Ultralytics’ segmentation dataset guidance follows the standard YOLO organization of corresponding images and text annotations.

Images and Labels Directories

Images are stored under:

images/

Annotations are stored under:

labels/

For example:

images/train/car001.jpg
labels/train/car001.txt

The base filename should match so that YOLO can associate the label with the correct image.

Training and Validation Folder Layout

A more complete example is:

custom_seg/
├── images/
│   ├── train/
│   │   ├── img001.jpg
│   │   ├── img002.jpg
│   │   └── img003.jpg
│   └── val/
│       ├── img101.jpg
│       └── img102.jpg
│
├── labels/
│   ├── train/
│   │   ├── img001.txt
│   │   ├── img002.txt
│   │   └── img003.txt
│   └── val/
│       ├── img101.txt
│       └── img102.txt
│
└── data.yaml

The dataset YAML then points to the appropriate image directories.

How to Create YOLOv8 Segmentation Labels

Creating segmentation labels generally involves drawing polygons around objects and exporting those annotations into the YOLO segmentation format.

Annotate Objects with Polygons

For each object, trace its visible outline using a polygon annotation tool.

A polygon should follow meaningful boundaries rather than simply creating a loose rectangle around the object.

For example, when annotating a car, the polygon should follow the visible body of the vehicle as closely as is practical.

Each separately visible object instance should receive its own annotation.

Export Annotations in YOLO Format

If your annotation tool supports YOLO segmentation export, select the Ultralytics or YOLO segmentation format.

The resulting file should contain rows like:

class x1 y1 x2 y2 ... xn yn

Check that the exported coordinates are normalized rather than raw image pixel coordinates.

Some annotation platforms may use their own format internally and convert it automatically when exporting.

Check Labels Before Training

Before starting a large training run, inspect some annotations manually and visualize them.

Verify:

• polygon points follow object boundaries,
• class IDs are correct,
• coordinates are normalized,
• objects have enough polygon points,
• filenames match images,
• no polygons are accidentally assigned to the wrong image.

A small validation step can prevent hours of training on incorrectly formatted labels.

YOLOv8 Segmentation Format vs Detection Format

YOLOv8 detection and segmentation datasets use similar file organization but different annotation structures.

Bounding Box Labels vs Polygon Labels

Detection:

class x_center y_center width height

Segmentation:

class x1 y1 x2 y2 x3 y3 ... xn yn

Detection describes a rectangle.

Segmentation describes a polygon.

For example:

Detection:
┌───────────┐
│  Object   │
│           │
└───────────┘

Segmentation can instead follow the object’s actual outline.

This provides considerably more detailed spatial information.

When to Use Segmentation Labels

Use segmentation when your application requires information about the object’s exact visible region.

Examples include:

• road damage segmentation,
• manufacturing defects,
• medical objects,
• crop and plant areas,
• vehicle outlines,
• product extraction,
• object area measurements.

If only location and class are needed, standard bounding-box detection may be simpler and faster to annotate.

Converting Other Annotation Formats to YOLOv8 Segmentation

Many existing datasets are stored in formats such as COCO JSON.

They can be converted instead of being annotated again manually.

Ultralytics provides official dataset conversion utilities for this purpose.

COCO to YOLOv8 Segmentation Format

COCO instance segmentation commonly stores image and annotation data in JSON files.

The annotations may contain:

• category IDs,
• bounding boxes,
• polygon segmentation data,
• image IDs.

Ultralytics provides the convert_coco() function, which includes a use_segments parameter specifically for segmentation conversion.

For example:

from ultralytics.data.converter import convert_coco

convert_coco(
    labels_dir="path/to/coco/annotations",
    save_dir="converted_dataset",
    use_segments=True
)

This converts compatible COCO annotations into YOLO-format segmentation labels.

The official COCO-to-YOLO guide covers conversion for detection, segmentation, and pose annotations.

JSON Annotations to YOLO Segmentation Labels

Not every JSON annotation file automatically follows the COCO specification.

If your JSON is custom, you first need to identify:

• the object class,
• image width and height,
• polygon point coordinates,
• which polygon belongs to which image.

Each raw point should then be normalized:

normalized_x = x / image_width
normalized_y = y / image_height

and written in YOLO format:

class x1 y1 x2 y2 ... xn yn

If the JSON is already COCO-compatible, using Ultralytics’ convert_coco() utility is usually easier than writing a custom conversion script.

Ultralytics also provides utilities for converting segmentation mask images into YOLO segmentation polygons when the source data is stored as masks rather than polygon JSON.

Common YOLOv8 Segmentation Label Errors

Segmentation training can fail or produce poor masks when annotation files contain structural or coordinate errors.

Incorrect Coordinate Normalization

One common mistake is placing raw pixel coordinates directly inside a YOLO label.

For example:

0 300 220 410 180 520 320

would be incorrect if normalized YOLO coordinates are expected.

Instead, convert the coordinates relative to image dimensions.

For a width of 640:

x_normalized = x / 640

For a height of 480:

y_normalized = y / 480

The resulting values should normally fall between 0 and 1.

Invalid or Incomplete Polygon Points

A polygon needs enough vertices to represent an area.

Ultralytics requires a minimum of three (x, y) coordinate pairs for a valid segmentation row.

For example:

0 0.2 0.3 0.4 0.5

contains only two points and does not form a valid polygon.

A minimum valid structure would look like:

0 0.2 0.3 0.4 0.5 0.6 0.3

Although three points technically form a polygon, real objects often require more vertices for accurate boundaries.

Wrong Class IDs

Class IDs must correspond to the YAML class mapping.

If:

names:
  0: person
  1: car

then:

1 ...

represents a car.

If labels accidentally use:

2 ...

without class 2 being defined, the dataset configuration is inconsistent.

Class numbering should also normally be zero-based.

Mismatched Image and Label Files

Check that filenames match.

Correct:

images/train/object17.jpg
labels/train/object17.txt

Incorrect:

images/train/object17.jpg
labels/train/object18.txt

Also ensure labels are placed in the corresponding train or validation directory.

Incorrect folder organization can cause valid annotations to be ignored.

FAQs About YOLOv8 Segmentation Label Format

What is the YOLOv8 segmentation label format?

A standard YOLO instance segmentation annotation row contains a class ID followed by normalized polygon coordinates:

<class-index> <x1> <y1> <x2> <y2> ... <xn> <yn>

Each row describes one segmented object instance.

Are YOLOv8 segmentation coordinates normalized?

Yes. Polygon x and y coordinates are normalized relative to image width and height.

This means coordinates are normally represented between 0 and 1.

How many points can a YOLOv8 segmentation polygon contain?

A segmentation polygon must contain at least three (x, y) points. Ultralytics allows polygons with additional vertices as needed, and different objects may have different numbers of points.

There is no requirement that every polygon in the dataset contain the same number of vertices.

Can one label file contain multiple objects?

Yes. One text file can contain multiple annotation rows.

Each row represents one segmented object instance.

For example:

0 ...
1 ...
1 ...
2 ...

represents four objects inside the corresponding image.

What is the difference between YOLO detection and segmentation labels?

Detection labels use:

class x_center y_center width height

Segmentation labels use:

class x1 y1 x2 y2 ... xn yn

Detection represents an object with a rectangle, while segmentation represents its boundary with polygon vertices.

Can COCO annotations be converted to YOLOv8 segmentation format?

Yes. Ultralytics provides a convert_coco() conversion utility. Setting:

use_segments=True

enables conversion of compatible COCO segmentation annotations into YOLO segmentation labels.

Does every image need a segmentation label file?

Images containing annotated segmentation objects should have corresponding label information.

For images intentionally used as background examples with no target objects, Ultralytics’ dataset guidance notes that a .txt file is not required when no objects exist in that image.

The important requirement is that all target objects present in labeled training images are annotated consistently.

Conclusion

The YOLOv8 segmentation label format represents individual objects using a class ID followed by normalized polygon coordinates:

class x1 y1 x2 y2 ... xn yn

This structure differs from standard object detection because it describes the actual object boundary rather than only a rectangular bounding box. Each object receives its own annotation row, polygons can contain different numbers of points, and every valid polygon must contain at least three coordinate pairs.

For custom datasets, the most important steps are accurate polygon annotation, correct coordinate normalization, valid class IDs, and matching image and label filenames.

Existing COCO segmentation datasets can also be converted using Ultralytics’ official convert_coco() utility with use_segments=True, making it possible to reuse JSON-based polygon annotations without manually recreating every label.

Jane Torres

I’m Jane Austen, a skilled content writer with the ability to simplify any complex topic. I focus on delivering valuable tips and strategies throughout my articles.