Learn how to remove duplicates. This video covers dropping records with Pandas.
- [Instructor] It's really important to remove duplicates … from your dataset … in order to preserve the dataset's accuracy … and to avoid producing incorrect and misleading statistics. … For example, imagine you're analyzing retail sales data … and shopaholic Sally came in three times … and used three different credit cards to make purchases. … Now imagine that she provided the cashier the same zip code. … 32803 for each sale. … Just based on the card number, … Sally looks like three different customers … all from the same zip code. … If you fail to examine other attributes of the customer … so that you could identify and remove duplicates, … shopaholic Sally's transactions would skew the results … of a customer demographic analysis … because Sally would be counted as three people … rather than one. … To market to 32803 customers effectively, … you need to understand their characteristics. … Don't let duplicate records skew your analysis. … I've already imported numpy and pandas, … so let's start off by just creating a data frame object. …
AuthorLillian Pierson, P.E.
- Why use Python for working with data
- Filtering and selecting data
- Concatenating and transforming data
- Data visualization best practices
- Visualizing data
- Creating a plot
- Creating statistical data graphics
- Performing basic math and linear algebra
- Correlation analysis
- Multivariate analysis
- Data sourcing via web scraping
- Introduction to natural language processing
- Collaborative analytics with Plotly
Skill Level Intermediate
Python Statistics Essential Trainingwith Michele Vallisneri2h 58m Intermediate
1. Introduction to the Data Professions
High-level course road map1m 28s
2. Data Preparation Basics
3. Data Visualization 101
4. Practical Data Visualization
5. Basic Math and Statistics
6. Data Sourcing via Web Scraping
7. Collaborative Analytics with Plotly
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.Cancel
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.