Data Cleaning

Lori Carter, Randall Pruim


Student Materials

For Instructors


Main Point: Choices made in data cleaning have ethical implications.

Contents: instructor guide, student handout, suggested assessment questions

Summary: The goal of this lab is to show that data cleaning, while necessary, can be conducted with differing degrees of ethicality. The process of correcting data (changing location, imputing a new value, removing typos etc.), removing incomplete records, or leaving questionable data as is can affect the results of the analysis. Consequently, methods must be scrutinized. Students will evaluate the ethical ramifications of data cleaning choices on a fictional data set description.

Ethics background required: It is helpful if students have learned about the virtue ethics and utilitarian frameworks for making ethical decisions, though not absolutely required. These frameworks are covered in the first year curriculum.

Subject matter referred to in this lab: data cleaning, simple statistics

Placement in overall ethics curriculum:

Time required:

Learning objectives:

Ethical issues to be considered: Transparency, Diversity, Dignity, Privacy, Data Integrity, Reliability

Preparation required

Read through the entire lab. Print a student handout for each student. The handout includes a dataset description, questions for the students to consider, and definitions of potential ethical issues.

Exercise flow

Lesson plan


Depending on your class, you may wish to read/summarise this introductory information.

Bad/incorrect data leads to bad/incorrect decisions. Data can be “bad” (also called dirty) for a variety of reasons. Here are few:

One way to detect dirty data is to look for outliers – values that stick out because they do not fit the pattern of the rest of the data. (But note that not all outliers are incorrect and not all incorrect values will be outliers.)

GPAs of 5 (on a 4-point scale), human ages of 150 years, or other values that are beyond the feasible bounds for a measurement are clear indicators of a problem. Other outliers may be large (or small) but not impossible. And some incorrect values may not “stand out in the crowd” at all.

Suppose student GPA is recorded as 5.0 This would be unusual (impossible, really) at an institution where students are graded on a 4 point scale. But what should the data analyst working on checking the validity of the data do with such a value?

(Van den Broeck et al. 2005) defines data cleaning as

[the] process of detecting, diagnosing, and editing faulty data.

It is often said that data analysts spend 80% of their time cleaning and 20% analyzing (Browne-Anderson 2018). If the time is not spent to clean the data properly, the results could be incorrect, and the process fraught with ethical issues.

In our GPA scenario, the first step of cleaning would be to check to see if the university posting the GPA did, in fact, adhere to a 4 point scale. If so, the data point is erroneous and the issue needs to be addressed. But how?

Whatever approach is taken, it is important that a clear record is kept of the decisions made, in such a way that the entire analysis (including the data cleaning) could be reproduced by another person. Furthermore, these decisions and their potential impact on the conclusions drawn need to be clearly communicated to those relying on the analysis. Repeating the analysis usign a different set of decisions is a form of sensitivity analysis. If the results depend greatly on the data cleaning and analysis decisions, they it is crucial to know how and why those decisions were made. If several competing analyses yeild very similar results, then the stakes are lower.


Pass out the student handout that describes a dataset, the type of analysis for which it will be used, and some potential data cleaning transformations. The definitions of various ethical issues is also included.

Have the students consider the questions by themselves first and then with the class or in small groups.


  1. Regarding confendiality and personally identifiable data: In a large university with many students of all types in each program, it may not be possible to identify individuals with these data. However, for a small university, a small program, or for students with unusual data values, it may be much easier. If the major was computer science, for example, and the gender was female, the student could be easily recognized. Similarly, if there were few veterans at the school, or few non-traditional (older) students, they could be easily recognized.

  2. Since this survey also asks about things like mental illness and the possibility of pregnancy, dignity could be an ethical issue at stake here.

  3. The data must be kept secure and only those with legitimate purposes should have access. Any summaries or analyses of the data should be sure to protect the individuals involved. Additionally, researchers may be in a position to advocate on behalf of marginalized groups based on data like these.

    This could be a place to start a conversation about things like IRB approval, informed consent, etc.

  4. Regarding missingness for age

    1. Fill with the mean of the student population

      Bad option as this is skewed data, so the mean age is likely an over-estimate for most students.

    2. Fill with the median of the student population

      Only slightly better. A median is often a better summary of a skewed distribution, but that doesn’t make the median a good estimate for an individual with missing data. Furthermore, treating “filled in” data as if it were correct can lead to incorrect conclusions.

      Note: this is somewhat like some imputation methods, but those methods also take into account the greater uncertainty associated with imputed values.

    3. Remove records where age is missing and make a note of this in the data cleaning documentation

      If your sample size is sufficiently large and the amount of missing data small, you might go with this option.

      But if individuals for whom age is missing are systematically different from those with age recorded (perhaps older students are reluctant to reveal their age), then our analysis won’t accurately reflect this difference (since we aren’t using any data for one of the groups). This could create bias and be a diversity issue.

  5. Regarding missingness for parental college experience:

    More likely to happen for parents with no college experience or students who are adopted. Removing these records could affect diversity.

  6. Regarding missingness for physical disability:

    Some female students might not want to indicate that they are pregnant or suffer from mental illness. Again, a diversity issue.

  7. Regarding reliability and hospitality:

    If there is no standardized way to fix errors in the study, it could not be replicated with other data (reliability). Even if the data cleaner documented every change, this would be hard to apply to a new case and would make a very cumbersome document for someone trying to find errors after cleaning or to replicate the study (hospitality).

  8. Regarding the GPA outlier, this is simply a data integrity issue. If a rough semester caused a student to leave, the GPA could indeed have been 1.35. You cannot honestly make another assumption.


  1. Ask the class to reflect on this additional example based on what they learned from the exercise:

    A survey-taker has indicated a major that does not exist at the university. What possible choices are there to handle this, and what ethical issues would have to be considered with the choice?

  2. Ask students to share anything that surprised them from this exercise, or changed how they might make a data cleaning decision.

Possible ideas for professor if students are struggling with reflection

If the analyst replaced the major with one that he/she thought was closest (perhaps replacing exercise science (doesn’t exist) with kinesiology (does exist)) but doesn’t record this change, this could be a violation of transparency or reliability. While this approach is reasonable, the cleaner should record the cleaning method so the results could be reproduced. It might also be wise to produce the final analysis both with the change made, and with the offending record removed and see if the results differed significantly.



Browne-Anderson, Hugo. 2018. “What Data Scientists Really Do, According to 35 Data Scientists.” Harvard Business Review.

Van den Broeck, Jan, Solveig Argeseanu Cunningham, Roger Eeckels, and Kobus Herbst. 2005. “Data Cleaning: Detecting, Diagnosing, and Editing Data Abnormalities.” PLoS Med 2 (10): e267.