The first step in exploratory data analysis (EDA) is "making data clean." Domain 2 (Exploratory Data Analysis) in the MLS-C01 exam accounts for about 24% of the test, with data cleaning at its core. No matter how excellent a model architecture is, "garbage in, garbage out" results when data containing missing values, outliers, and duplicates is fed into it.
Today, we address three types of "dirt"—missing values, outliers, and duplicates and errors. We examine methods to detect each, treatment strategies, and the impact these choices have on models.
Before treating missing values, we must first understand why they are missing