What are the types of errors in data?

[rifat28dddd]

Imagine that you need to find an answer to a question online. But half of the articles in the search are on a different topic, and many of the information is outdated or incorrect. In such circumstances, it will be difficult to find the correct answer.

Models in Data Science and Machine Learning face the same problem. The data they are trained on can contain a lot of “garbage”: incorrect values, errors, duplicates. This occurs because information is usually collected from many different sources, each with its own representation of the data. Because of this, the data in the sample is heterogeneous and sometimes incorrect.

The model can learn on dirty data, but this can bolivia telegram data greatly reduce its accuracy. If you don’t clean the data before loading it into the model, there’s a high risk that it will end up producing incorrect results — say, forecasts that are far from the truth.

Therefore, in order for the model to work accurately, the data needs to be cleared of “garbage” before training it:

remove errors and inconsistencies that occur in the data sample;
bring data to a unified form, for example, combine identical features;
fill in missing values, remove duplicates;
get rid of noise and outliers - random values ​​that differ sharply from the majority.
Read also
Alisa Radchenko: “I worked as an accountant, and now I analyze data at MTS”
Usually, information is stored in special storage facilities - databases . They can be arranged in different ways, but most often, entities in databases can be divided into two categories:

records - rows in a table, some objects that consist of a set of features;