How to Train YOLOv8 from Scratch on a Custom Dataset

Training YOLOv8 from scratch on a custom dataset means building the YOLOv8 architecture with randomly initialized weights instead of starting from pretrained .pt weights. In Ultralytics, the clearest way to do this is to create the model from a YOLOv8 architecture YAML file, such as yolov8n.yaml, rather than loading yolov8n.pt. Ultralytics also exposes pretrained=False as a training setting for random initialization.

Table of Contents

Introduction to Training YOLOv8 from Scratch

Most YOLOv8 custom training projects use transfer learning. In that workflow, a pretrained model such as:

yolov8n.pt

already contains weights learned from a large dataset.

Training from scratch is different. The model architecture is created without loading those learned weights, so the network must learn useful visual features entirely from your own dataset.

Conceptually:

Transfer Learning
Pretrained Weights
      ↓
Custom Dataset
      ↓
Fine-Tuning

versus:

Scratch Training
Random Weights
      ↓
Custom Dataset
      ↓
Learn Everything

Ultralytics supports creating a new model directly from a YAML architecture and then training it on a custom dataset.

What Does Training YOLOv8 from Scratch Mean?

Training from scratch means the neural network starts without previously learned YOLOv8 weights.

The backbone, neck, and detection head must learn their parameters from the custom training data.

Training from Scratch vs Using Pretrained Weights

With pretrained weights:

from ultralytics import YOLO

model = YOLO("yolov8n.pt")

the model starts from previously learned weights.

With scratch training:

from ultralytics import YOLO

model = YOLO("yolov8n.yaml")

the architecture is built from its YAML definition without loading the .pt checkpoint. Ultralytics documentation supports creating a new model from YAML, while its configuration documentation states that pretrained=False trains from random initialization.

The practical difference is:

.pt file
→ architecture + pretrained weights

.yaml file
→ architecture definition
→ fresh initialization

When You Should Train YOLOv8 from Scratch

Scratch training can make sense when:

  • you have a very large custom dataset,
  • your image domain is very different from common photographic datasets,
  • pretrained features may not transfer well,
  • you are researching architecture behavior,
  • you need to avoid pretrained weights for experimental reasons.

Examples of unusual domains may include:

  • specialized scientific imagery,
  • unusual sensor data,
  • synthetic imagery,
  • highly domain-specific industrial images.

For ordinary custom object detection projects with limited data, transfer learning is usually more efficient because the model begins with useful visual features.

Requirements for Training YOLOv8 from Scratch

Scratch training generally requires more data, compute, and training time than transfer learning.

Install Python and Ultralytics

Install Ultralytics:

pip install ultralytics

Then verify the package:

from ultralytics import YOLO

Ultralytics provides the same Train mode for pretrained and newly initialized models.

GPU and Hardware Requirements

A GPU is strongly recommended for scratch training.

Training from random initialization often requires:

  • more epochs,
  • more optimization steps,
  • more experimentation,
  • larger datasets.

Hardware requirements depend on:

  • YOLOv8 model size,
  • batch size,
  • image resolution,
  • dataset size,
  • augmentation settings.

CPU training is possible, but a large scratch-training experiment can become extremely slow.

Choose the Right YOLOv8 Model Size

YOLOv8 model sizes include:

YOLOv8n
YOLOv8s
YOLOv8m
YOLOv8l
YOLOv8x

For a first scratch experiment, a smaller model such as YOLOv8n can be easier to train because it requires fewer computational resources.

A large architecture generally requires more data to train effectively from random initialization.

Prepare a Dataset for YOLOv8 Training

The dataset is especially important when training from scratch because there are no pretrained weights to compensate for limited or weak data.

Collect and Annotate Images

Collect images representing the full range of conditions expected during inference.

Include variation in:

  • lighting,
  • background,
  • scale,
  • object position,
  • orientation,
  • occlusion,
  • image quality.

For standard detection, YOLO labels use normalized bounding-box annotations. Ultralytics’ detection dataset documentation describes the supported YOLO dataset organization and label structure.

Split Data into Training and Validation Sets

Create separate training and validation data.

For example:

dataset/
├── images/
│   ├── train/
│   └── val/
├── labels/
│   ├── train/
│   └── val/
└── data.yaml

The training split is used to update model weights.

The validation split measures how well the model performs on unseen images.

Organize Images and Labels

Image and label filenames should correspond.

For example:

images/train/image001.jpg
labels/train/image001.txt

and:

images/val/image101.jpg
labels/val/image101.txt

Incorrect folder structure or mismatched labels can prevent successful training regardless of whether the model starts from scratch or pretrained weights.

Create the YOLOv8 Data YAML File

The data YAML tells Ultralytics where your custom dataset is stored and what classes it contains.

Define Train and Validation Paths

For example:

path: /datasets/custom

train: images/train
val: images/val

Add Class Names

Add classes:

names:
  0: person
  1: vehicle
  2: bicycle

The numerical IDs must match the IDs used in your label files.

Verify Dataset Configuration

Before starting scratch training, verify:

✓ dataset path exists
✓ training images exist
✓ validation images exist
✓ labels match images
✓ class IDs are valid
✓ class names are correct

A dataset problem can easily be mistaken for poor scratch-training performance.

Choose a YOLOv8 Model YAML Configuration

Scratch training requires the architecture itself rather than only a pretrained checkpoint.

Use a YOLOv8 Architecture YAML File

Instead of:

model = YOLO("yolov8n.pt")

use:

model = YOLO("yolov8n.yaml")

The YAML defines the neural network architecture, including backbone, head, layer connections, and model scaling. Ultralytics describes model YAML files as architectural blueprints.

Initialize the Model Without Pretrained Weights

A clean Python scratch-training setup is:

from ultralytics import YOLO

model = YOLO("yolov8n.yaml")

model.train(
    data="data.yaml",
    epochs=300,
    pretrained=False
)

Ultralytics configuration documentation states that:

pretrained=False

starts training from random initialization.

Using a YAML architecture is the clearest way to avoid accidentally loading a pretrained YOLOv8 checkpoint. Ultralytics guidance also distinguishes using yolov8n.yaml for a newly initialized model from using yolov8n.pt for pretrained weights.

Select Model Depth and Width

YOLOv8 architectures are available at different scales.

Conceptually:

Nano
↓
Small
↓
Medium
↓
Large
↓
Extra Large

As model size increases:

parameter count ↑
compute ↑
memory ↑
training data requirements ↑

For scratch training, avoid selecting a model much larger than your dataset can realistically support.

How to Train YOLOv8 from Scratch

Once the dataset and architecture configuration are ready, training can begin.

Train from Scratch Using the YOLO CLI

A scratch-training command can use the architecture YAML:

yolo detect train model=yolov8n.yaml data=data.yaml epochs=300 pretrained=False

This tells Ultralytics to construct the YOLOv8 Nano architecture rather than load the .pt checkpoint.

Ultralytics supports training new models from YAML files through its Train mode workflow.

Train from Scratch Using Python

Using Python:

from ultralytics import YOLO

model = YOLO("yolov8n.yaml")

results = model.train(
    data="data.yaml",
    epochs=300,
    imgsz=640,
    batch=16,
    pretrained=False
)

This gives explicit control over the scratch-training configuration.

Set Epochs, Batch Size, and Image Size

A more complete CLI command might be:

yolo detect train \
model=yolov8n.yaml \
data=data.yaml \
epochs=300 \
imgsz=640 \
batch=16 \
pretrained=False

Important settings include:

epochs
batch
imgsz
device
optimizer
lr0
patience

Scratch training often needs more epochs than transfer learning because the network must learn useful features from random initialization.

Important Training Settings for Scratch Training

The training configuration becomes especially important when no pretrained weights are available.

Learning Rate and Optimizer

The optimizer controls how weights are updated.

Important settings include:

optimizer
lr0
momentum
weight_decay

A poorly selected learning rate can make scratch training unstable.

If the loss becomes erratic immediately, test a smaller learning rate and verify the labels before changing the architecture.

Data Augmentation

Data augmentation can improve generalization by exposing the model to more variation.

Useful augmentation types may include:

  • scaling,
  • translation,
  • flipping,
  • color variation,
  • Mosaic,
  • MixUp.

However, augmentation should remain realistic for the target domain.

Scratch training does not mean that stronger augmentation is always better.

Early Stopping and Patience

You can use:

patience

to stop training if validation performance fails to improve for a specified number of epochs.

For example:

yolo detect train model=yolov8n.yaml data=data.yaml epochs=500 patience=100 pretrained=False

This can prevent an experiment from continuing indefinitely after performance has plateaued.

Weight Decay and Regularization

Weight decay helps regularize the model.

This can be important during scratch training because the model has no pretrained prior and must learn all weights from the custom dataset.

Regularization should be strong enough to reduce overfitting but not so aggressive that the model cannot learn.

Monitor YOLOv8 Training Performance

Scratch training should be monitored more carefully than a standard fine-tuning run.

Training and Validation Loss

Watch how training loss changes.

A healthy trend is generally:

training loss ↓
validation performance ↑

If training loss stays almost unchanged for many epochs, the model may not be learning effectively.

Precision, Recall, and mAP

Useful validation metrics include:

Precision
Recall
mAP50
mAP50-95

These metrics help determine whether lower training loss actually translates into better object detection.

Detecting Overfitting or Underfitting

Possible underfitting:

training accuracy low
validation accuracy low
loss remains high

Possible overfitting:

training continues improving
validation performance stops improving

If the model underfits, you may need:

  • more epochs,
  • larger model capacity,
  • better optimization settings.

If it overfits, consider:

  • more data,
  • stronger regularization,
  • better augmentation,
  • smaller model.

How to Improve YOLOv8 Training from Scratch

Scratch training usually improves more from better data than from random hyperparameter changes.

Increase Dataset Size and Diversity

A larger and more varied dataset gives the network more information from which to learn general visual features.

Include examples covering:

different backgrounds
different object scales
different lighting
different orientations
different cameras
different environments

This is especially important without pretrained features.

Improve Label Quality

Bad labels can seriously damage scratch training.

Check for:

  • missing objects,
  • inaccurate boxes,
  • incorrect classes,
  • duplicate annotations,
  • mislabeled images.

When starting from random initialization, the model learns exactly from the supervision it receives.

Tune Hyperparameters

Useful parameters to experiment with include:

lr0
batch
imgsz
optimizer
weight_decay
mosaic
mixup

Change parameters systematically and compare runs using the same validation split.

Train for More Epochs When Needed

Scratch training frequently converges more slowly than transfer learning.

A model that performs poorly at 50 epochs may improve substantially by 200 or 300 epochs.

However, do not assume that simply increasing epochs will always fix poor performance.

If validation metrics are not improving, inspect the dataset and optimization configuration first.

Training from Scratch vs Transfer Learning

These approaches differ mainly in initialization and training efficiency.

Accuracy and Convergence Differences

Transfer learning begins with features already learned from previous data.

Scratch training begins with random weights.

Therefore:

Transfer Learning
→ useful initial features
→ faster convergence

Scratch Training
→ random initialization
→ slower feature learning

For limited custom datasets, pretrained models often achieve useful performance much faster.

Training Time and Hardware Requirements

Scratch training usually requires:

more epochs
more GPU time
more training data
more tuning

Transfer learning can often reach useful performance with fewer epochs because many low-level features are already available.

Which Approach Should You Choose?

Use transfer learning when:

  • your dataset is small or medium-sized,
  • your images resemble normal photographic data,
  • training resources are limited,
  • fast convergence matters.

Consider scratch training when:

  • your dataset is very large,
  • the visual domain is very different,
  • pretrained weights are not allowed,
  • you are performing architecture research.

For most ordinary custom YOLOv8 projects, pretrained weights are the more practical starting point.

Common Problems When Training YOLOv8 from Scratch

Scratch training can expose problems that are less obvious during transfer learning.

Slow Model Convergence

This is expected to some degree.

The network must learn everything from random initialization.

If convergence is excessively slow, check:

learning rate
optimizer
dataset size
label quality
batch size

Low Detection Accuracy

Possible causes include:

  • insufficient training data,
  • too few epochs,
  • poor labels,
  • too large a model,
  • class imbalance,
  • weak hyperparameters.

Scratch training usually needs significantly more information than fine-tuning.

Unstable Training Loss

Loss spikes can result from:

  • excessive learning rate,
  • bad annotations,
  • corrupted images,
  • unstable augmentation,
  • inappropriate optimizer settings.

Always verify the dataset before assuming the model architecture is faulty.

GPU Out-of-Memory Errors

If CUDA memory is exhausted, reduce:

batch
imgsz
model size

For example:

batch=32
↓
batch=16
↓
batch=8

or change from a larger architecture to:

yolov8n.yaml

for testing.

FAQs About Training YOLOv8 from Scratch

Can YOLOv8 be trained completely from scratch?

Yes.

You can construct the architecture from a YAML file:

model = YOLO("yolov8n.yaml")

and train it without pretrained weights:

model.train(
    data="data.yaml",
    pretrained=False
)

Ultralytics supports creating new models from YAML, and its configuration documentation states that pretrained=False uses random initialization.

How much data is needed to train YOLOv8 from scratch?

There is no fixed minimum.

However, scratch training generally requires substantially more data than transfer learning.

Hundreds of images may be enough for a narrow experiment, but robust scratch-trained models often benefit from thousands or much larger datasets depending on:

  • number of classes,
  • visual diversity,
  • model size,
  • object complexity.

The more complex the problem, the more data is usually required.

How do I disable pretrained weights in YOLOv8?

Use:

pretrained=False

and preferably construct the model from its YAML architecture:

from ultralytics import YOLO

model = YOLO("yolov8n.yaml")

model.train(
    data="data.yaml",
    pretrained=False
)

Ultralytics configuration documentation explicitly states that pretrained=False trains from randomly initialized weights.

Avoid starting with:

YOLO("yolov8n.pt")

if your goal is a clearly scratch-initialized model, because .pt is a pretrained checkpoint.

Is training from scratch better than using pretrained weights?

Not usually for small or medium custom datasets.

Pretrained weights typically offer:

  • faster convergence,
  • lower data requirements,
  • better initial features,
  • reduced training cost.

Scratch training is more useful when transfer learning is unsuitable or when experimental constraints require random initialization.

How many epochs are needed to train YOLOv8 from scratch?

There is no universal number.

Scratch training often needs more epochs than pretrained fine-tuning.

Depending on the dataset, this might mean:

200
300
500+

epochs.

Monitor validation metrics and early stopping rather than choosing a number blindly.

Can I train YOLOv8 from scratch without a GPU?

Yes, CPU training is technically possible.

For example:

yolo detect train model=yolov8n.yaml data=data.yaml device=cpu pretrained=False

However, scratch training can require many epochs, so CPU training may be impractically slow for large datasets.

Why is YOLOv8 training from scratch slower?

A scratch model begins with random weights.

It must learn:

basic visual features
intermediate patterns
object representations
class-specific features
detection behavior

from your custom dataset.

A pretrained model already contains much of this information, so transfer learning starts much closer to a useful solution.

Conclusion

Training YOLOv8 from scratch on a custom dataset means creating the model architecture without loading pretrained .pt weights and allowing all useful representations to be learned from your own data.

The clean Python workflow is:

from ultralytics import YOLO

model = YOLO("yolov8n.yaml")

model.train(
    data="data.yaml",
    epochs=300,
    imgsz=640,
    batch=16,
    pretrained=False
)

Ultralytics officially supports creating models from YAML architecture files, while pretrained=False specifies random initialization rather than pretrained weights.

The basic workflow is:

Prepare Dataset
      ↓
Create data.yaml
      ↓
Load yolov8*.yaml
      ↓
Use Random Initialization
      ↓
Train for Sufficient Epochs
      ↓
Monitor Validation Metrics
      ↓
Tune Hyperparameters

Scratch training offers complete control over model learning, but it generally requires more data, more epochs, more compute, and more careful tuning than transfer learning.

For most ordinary custom-detection projects, pretrained YOLOv8 weights are more efficient. Scratch training is most useful when the dataset is large, the target domain differs strongly from common pretrained data, or experimental requirements specifically call for random initialization.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top