From the course: Lessons from Data Scientists

Unlock the full course today

Join today to access over 22,600 courses taught by industry experts or purchase this course individually.

Understanding the data you work with

Understanding the data you work with

From the course: Lessons from Data Scientists

Start my 1-month free trial

Understanding the data you work with

“

- Typically, when you think of data scientists, you imagine a person taking this beautiful, complete dataset, creating models, and outputting some kind of prediction or a model that will be useful to the stakeholder. A lot of times, that actually isn't the case. In fact, in my 10 years of working with large and different types of datasets, I've never come across one that didn't need some kind of cleaning, or at least some deep exploration into what it may contain that could be problematic. And so a majority of my time is either spent cleaning up data, deep diving into, is this a complete dataset? What might be wrong with this dataset? What might this dataset be missing that might be important to have, or that might be important to have to answer a specific question? And then also on the other side of that, spending a significant amount of time, actually spot-checking the resulting either outputs of the data or the…

Contents