From the course: Faster pandas
Join today to access over 22,600 courses taught by industry experts or purchase this course individually.
Using Vaex
- [Narrator] Vaex is an alternative implementation of a DataFrame. It tries to be API compatible with Pandas or providing better performance with several optimizations. Vaex can work with data sizes that might challenge pandas and can scale up to billions of rows. You first need to install vaex. So python dash m pip install vaex and you can see it going to install a lot of dependencies and now we can use it. I'm going to import both pandas and vaex and compare them. So import pandas as pd and we're going to import vaex and our CS_file is 'taxi.cs.xz' and the time columns we have are 'tpep_pickup_datetime' and 'tpep_dropoff_datetime' and now we're going to look so the regular data frame is pd.read_csv of the csv_file and we're going to tell it parse_dates equal time cols. Next, we're going to use vaex. So vdf is vaex dot read csv of Csv_ file and parse_dates equal time_cols. And if you look at the length, we see that both have the same length. Let's run a calculation of how many trips,…
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.