From the course: 15 Mistakes to Avoid in Data Science

Unlock the full course today

Join today to access over 22,600 courses taught by industry experts or purchase this course individually.

Not considering the level of variation

Not considering the level of variation

From the course: 15 Mistakes to Avoid in Data Science

Start my 1-month free trial

Not considering the level of variation

“

- A common mistake is also to not give enough weight to the amount of variation in your data set when you're trying to model or predict an outcome. For example, I recently worked on a project where I was using a pretty limited data set to predict how schools may perform on a summit of assessment, but the underlying data in that data set had a lot of variation across schools. Students could perform really well as a group or really poorly as a group, but it was really hard to use that data to make any predictions, because the amount of error of those predictions was very high. And that information is really difficult to convey to stakeholders in a way that's meaningful, because they would like to use the data to make decisions. And keeping in mind variation when making decisions is really difficult to do, and so it's often dismissed as unimportant because the decision has to be made, but the variation within the dataset…

Contents