What Is an Epoch in Machine Learning? Epoch vs Batch vs Iteration

Updated · By Viviai

An epoch in machine learning is one complete pass over the entire training dataset, during which every example is processed once and contributes to the model's updates. A single pass is rarely enough, so models are trained for many epochs, with the data split into smaller batches. The number of iterations in an epoch equals the dataset size divided by the batch size, rounded up.

Epoch, batch, and iteration defined

These three terms describe the same training run at different scales, and each nests inside the one before it.

The Google Machine Learning Glossary defines an epoch as a full training pass in which each example is processed once, and a batch as the set of examples used in one training iteration.

How they connect

The relationship follows a simple formula. If a dataset has 1,000 examples and the batch size is 100:

[ \text{iterations per epoch} = \left\lceil \frac{1000}{100} \right\rceil = 10 ]

So one epoch contains 10 iterations, and each iteration updates the weights once. After all 10 batches are processed, the model has seen every example and the epoch is complete.

Dataset size Batch size Iterations per epoch
200 32 7
1,000 100 10
1,000 50 20
10,000 64 157

The final batch can be smaller than the rest when the dataset size does not divide evenly by the batch size.

What happens inside one epoch

During a pass, the model repeats the same cycle for each batch.

  1. A batch of examples is fed forward through the neural network to produce predictions.
  2. A loss function measures the gap between those predictions and the correct answers.
  3. Backpropagation computes the gradients, and an optimizer such as gradient descent adjusts the weights.
  4. The model moves to the next batch, using the updated weights.

Training data is usually shuffled before each epoch so the model does not learn patterns from the order of the examples.

Why models need more than one epoch

In one pass the model only makes a first rough adjustment. Repeating the pass lets it refine the weights as the training loss generally falls. Think of reading a textbook: one pass gives a rough idea, while several passes with review help the material settle.

The number of epochs is a setting you tune, and both extremes carry a cost.

A common safeguard is early stopping, which monitors validation loss and ends training when that loss stops improving, rather than running a fixed number of passes.

How the number of epochs relates to other settings

The epoch count does not work in isolation; it interacts with batch size and the learning rate.

There is no single correct number for every problem. Practitioners typically watch training and validation curves and stop around the point where validation performance levels off.

FAQ

Is one epoch the same as one iteration?

No. An epoch is a full pass over the entire dataset, while an iteration is one update on a single batch. An epoch contains as many iterations as there are batches.

How many epochs should I use?

It depends on the dataset and model. A practical approach is to start with a modest number, track training and validation loss, and use early stopping so training ends when validation loss no longer improves.

What happens if I set too many epochs?

The model may begin to overfit, memorizing quirks of the training data instead of generalizing, and you spend extra compute. Validation curves and early stopping help avoid this.

Does the batch size change the number of epochs?

Batch size changes how many iterations happen inside each epoch, not the definition of an epoch. It can still affect training dynamics, so the two settings are usually adjusted together.

What is the difference between an epoch and a full training run?

An epoch is one pass through the data. A full training run consists of many epochs repeated until the model converges or a stopping condition is reached.

Key takeaways

An epoch is one complete pass over the training data, built from batches, with each batch producing one iteration and weight update. The number of iterations per epoch is the dataset size divided by batch size. Choosing the right number of epochs balances underfitting against overfitting and wasted compute, and early stopping is a common way to make that decision based on validation performance.

Sources

  1. https://ocw-preview.odl.mit.edu/courses/15-773-hands-on-deep-learning-spring-2024/mit15_773_s24_lec03a.pdf
  2. https://developers.google.com/machine-learning/glossary
  3. https://www.ultralytics.com/glossary/epoch
  4. https://dagshub.com/glossary/epoch-in-machine-learning/

Learn by doing

Read a guide, then practice with bite-sized AI lessons you can take offline.

Start learning free