Data Cleaning

Data cleaning, data cleansing, or data scrubbing is the process of improving the quality of data by correcting inaccurate records from a record set. The term specifically refers to detecting and modifying, replacing, or deleting incomplete, incorrect, improperly formatted, duplicated, or irrelevant records, otherwise referred to as “dirty data,” within a database. Data cleaning also includes removing duplicated data within a database.

Data provided for communication research often rely on manual data entry, performed by humans, and therefore are subject to error introduction. Because of this manual process, the data require cleaning. The need for such cleaning increases when data come from multiple sources and a standard schema was not used across sources. The goal of data cleaning is to provide a data set that is consistent enough to allow for accurate analysis. The original intent and meaning of the information provided by the participant are not altered but rather inconsistencies caused by data transmission problems, the use of different definitions in different data stores, and user entry errors are addressed to remove or manage the inconsistent data. This entry introduces the data cleansing process, including its manual and computer-assisted approaches, and further discusses the difference between data cleansing and data validation.