From the course: Faster pandas
Join today to access over 22,600 courses taught by industry experts or purchase this course individually.
Why memory is important?
- [Instructor] Apart from CPU, we also care about memory allocations. Here are the main reasons. The first is virtual memory. The operating system gives you an illusion of infinite memory called virtual memory. The memory is composed of pages and each page individual memory is mapped to a page in the physical memory. When you access a virtual page that's not in the physical memory, the operating system will take a page from the physical memory, save it to disc and load the requested page from the disc to physical memory. This process is known as a page fault and is very slow compared to accessing data that's already in physical memory. Several hundred times slower. What you will see is your application working okay for a while and then suddenly everything is very slow. You should try to keep the data you're working with smaller than the size of physical memory. The second reason is that memory allocation and deallocation are not free. Every time you allocate memory for your data, it…
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.