Machine Learning Glossary: Plain Definitions of Common Terms
This machine learning glossary collects the terms that show up most often when beginners read about ML and AI, with short, plain definitions grouped by topic. Use it as a reference page: each entry states what the term means in one or two sentences, and linked words lead to full explainers. Definitions follow the usage in the Google Machine Learning Glossary, ISO/IEC 22989, and Stanford HAI.
Core concepts
| Term | Definition |
|---|---|
| Artificial intelligence (AI) | The field of building systems that perform tasks normally associated with human intelligence, such as perception, reasoning, and language use. |
| Machine learning (ML) | A branch of AI where models learn patterns from data instead of being given explicit rules. |
| Model | A mathematical structure trained on data to make predictions or generate outputs. |
| Algorithm | The procedure that learns a model's parameters from data, such as a decision tree or gradient descent. |
| Parameter | A value the model learns on its own, such as a weight or bias in a neural network. |
| Hyperparameter | A setting chosen before training, such as the learning rate or number of layers. |
| Deep learning | Machine learning that uses neural networks with several layers. |
Data and features
| Term | Definition |
|---|---|
| Dataset | The collection of examples used for training, validation, and testing. |
| Training data | The examples the model fits to during learning. |
| Validation data | A held-out set used to tune settings and compare options. |
| Test data | A separate set used only for a final estimate of performance. |
| Feature | A measurable input property, such as age or pixel value. |
| Label | The target answer the model learns to predict. |
| Instance | A single data point or row, also called an observation. |
| Classification | Predicting a category, such as spam or not spam. |
| Regression | Predicting a continuous number, such as a price. |
| Supervised learning | Learning from examples that include labels. |
| Unsupervised learning | Finding structure in data without labels, such as clustering. |
Training and optimization
| Term | Definition |
|---|---|
| Loss function | A measure of how far a model's predictions are from the correct answers. |
| Gradient | The derivative of the loss, indicating the direction that would increase it. |
| Gradient descent | The iterative method that updates weights downhill to reduce loss. |
| Learning rate | The multiplier that controls the size of each weight update. |
| Epoch | One full pass over the entire training dataset. |
| Batch | A subset of examples processed together before an update. |
| Backpropagation | The algorithm that applies the chain rule to compute gradients across layers. |
| Epoch vs. iteration | An epoch is a full dataset pass; an iteration is one single batch update. |
| Overfitting | When a model fits the training data closely but does not generalize to new data. |
| Regularization | Techniques that penalize complexity to reduce overfitting. |
| Gradient clipping | Capping gradients at a maximum norm to prevent exploding updates. |
Neural networks
| Term | Definition |
|---|---|
| Neural network | A layered model of connected units loosely inspired by neurons. |
| Weight | A parameter that scales an input as it passes between units. |
| Bias | A parameter added to shift a unit's output up or down. |
| Activation function | A nonlinear transform applied to a unit's output, such as ReLU. |
| Hidden layer | An intermediate layer between input and output. |
| Batch normalization | A technique that rescales layer activations during training. |
| Softmax | A function that turns scores into probabilities that add to one. |
| Vanishing gradient | When the backward signal shrinks so much that early layers stop learning. |
| Exploding gradient | When the backward signal grows so large that training becomes unstable. |
Generative AI and language
| Term | Definition |
|---|---|
| Generative AI | Systems that produce new text, images, audio, or other content. |
| LLM (Large Language Model) | A model trained on large text corpora to understand and generate language. |
| Token | A unit of text the model reads, such as a word or part of a word. |
| Prompt | The input instruction given to a generative model. |
| Hallucination | Output that looks credible but is not grounded in fact or evidence. |
| Context window | The maximum number of tokens a model can consider at once. |
| Fine-tuning | Taking a pretrained model and training it further on a narrower dataset. |
| RAG (Retrieval-Augmented Generation) | A method that fetches external documents before generating an answer. |
| Embedding | A numeric vector that represents the meaning of text or other data. |
| AI agent | A system that plans and takes actions toward a goal, often using tools. |
| Multimodal | Working with more than one input type, such as text and images. |
Evaluation
| Term | Definition |
|---|---|
| Accuracy | The share of predictions that are correct. |
| Precision | Among items predicted positive, the share that truly are. |
| Recall | Among truly positive items, the share the model catches. |
| Confusion matrix | A table showing correct and incorrect predictions across classes. |
| Generalization | How well a model performs on data it has not seen before. |
| Cross-validation | Splitting the data into rotating train and test sets for a steadier estimate. |
How to use this glossary
- Read the core concepts first if you are new; the rest of the table builds on those words.
- When a term appears in an article, return here for a one-line reminder rather than restarting the topic.
- Follow the linked full explainers for the terms that need examples, formulas, or code.
Frequently asked questions
What is the difference between a parameter and a hyperparameter? The model learns parameters from data, such as weights. You choose hyperparameters in advance, such as the learning rate or the number of layers.
Are these definitions fixed across every source? The core meanings are stable, but wording differs between sources, and a few terms are used loosely in industry writing. When precision matters, check the linked standards and official documentation rather than relying on a single sentence.
Does this glossary cover every ML term? No. It focuses on the words beginners meet first. Narrow research terms and product-specific names are added as they become common enough to matter.
Key takeaways
- The vocabulary clusters into data, training, networks, generative AI, and evaluation.
- Parameters are learned; hyperparameters are set ahead of time.
- The same concept often has a short name and a longer formal name, such as LLM and Large Language Model.
- Keep this page as a reference and move to the linked explainers for depth.
Explore the full guides on machine learning, gradient descent, the learning rate, and neural networks.
Sources
Learn by doing
Read a guide, then practice with bite-sized AI lessons you can take offline.
Start learning free