data cleaning

basicwidespread

Data cleaning is the process of finding and correcting faulty, incomplete, duplicate or inconsistent data before it is used for analysis or artificial intelligence training.

In plain terms

Imagine that before cooking you sort through lentils: you remove the stones, the broken grains, then wash them. Data cleaning is the same with data. Raw data is almost never usable as is: it contains typos, empty fields, duplicate rows, and contradictory values.

It is important because artificial intelligence learns from what you feed it. If the training material is flawed, the model learns the flaw and confidently repeats it. The industry calls this: garbage in, garbage out.

It is not flashy work, yet it takes up a significant portion of the time in data projects. A modest model on clean data often yields better results than a more advanced model on messy data.

How it works

Survey: first, they examine what is in the data, meaning how many fields are empty, what values occur, and where suspicious outliers are located.
Unification: dates, units of measurement, names, and addresses are brought to a common form so that "Bp.", "Budapest", and "BUDAPEST" mean the same thing.
Deduplication: identifying and merging rows that refer to the same person, product, or event.
Handling errors and gaps: obviously bad values (for example, a 250-year-old customer) are corrected or deleted, while missing ones are filled in, flagged, or omitted.
Verification and documentation: results are re-examined using rules, and changes are recorded so that decisions can be traced back later.

An everyday example

An online store wants to send a newsletter, but the same customer appears three times in the list: once as "Kovács Anna", once as "kovacs anna", and once with a typo in the email address. After data cleaning, only one row remains, so the customer does not receive the same email three times, and the company knows exactly how many customers it has.

Who is affected

employeeIf you work with Excel spreadsheets or customer lists, much of your work is actually data cleaning, a task increasingly assisted by AI tools. → Work & HR

entrepreneur, executiveAI solutions built on messy corporate data lead to poor decisions, so the cost and time of cleaning must be planned in advance.

developerThe model's quality depends largely on the cleanliness of the training and test data, not just on the chosen algorithm. → Code

patient, doctorIn healthcare data, a typo in a unit of measurement or a mixed-up patient record can lead to incorrect conclusions. → Health

A common misconception

MythData cleaning is a one-time, mechanical cleanup that a program performs with a single button press.

In realityContinuous work full of professional decisions: someone must decide what counts as an error and what is a rare but real case. Tools speed up the process significantly, but responsibility remains with the human.

Limits and risks

Overzealous cleaning can delete valuable information: unusual data may be an error, but it could also be the most important signal, such as a sign of fraud.
Cleaning does not replace what is missing from the start: if little data was collected about a group, the dataset remains biased.
Filling in missing values is an estimate, not a fact, so the resulting outcome must be handled with caution.
If changes are not documented, no one can later explain why the cleaned data differs from the original.

Similar, but not the same

data labeling – Labeling tells what is visible in the data (for example, 'cat' in an image), while cleaning fixes the data itself.
data preprocessing – Preprocessing is broader: in addition to cleaning, it involves transforming, scaling, and formatting data into a form digestible by the model.
anonymization – Anonymization deletes or transforms data to prevent identification of individuals for privacy reasons; it does not fix errors.

Related concepts

Broader

data preprocessing, data quality, data processing, data science

Narrower

deduplication, Imputing missing data, outlier handling, data unification

On the edge

training data, data labeling, bias, synthetic data, machine learning

A brief history

Data cleaning is a practice as old as statistics and databases. The saying 'garbage in, garbage out' became widespread in the early decades of computing, around the turn of the 1950s and 1960s. With the rise of machine learning, it became a critical issue because model quality largely depends on the quality of training data.

Where things stand

At the end of 2026, this is the basic step of every serious data and AI project. Language models and AI agents carry out a growing share of routine tasks, but deciding disputed cases and oversight remain human tasks.

Status: 2026 November

Also known as

data cleaning · data cleaning · data scrubbing · data cleaning · data cleansing · data scrubbing

This entry was machine-edited and has undergone machine expert review; the “In the news” section updates automatically from Agent Age stories.