Move from Excel to Python with Pandas Transcripts
Chapter: Data I/O (input and output)
Lecture: Pandas loves large files
0:00 so we saw how Excel failed to read in this customer transaction file.
0:05 So let's see how Pandas does.
0:06 I'm gonna rerun my notebook and let's try reading in that CSB file,
0:15 and it's done. Now let's take a look at the data and make sure it's
0:19 all there. Looks like the heads there.
0:21 And this is nice so we can see how Maney Rose we have.
0:25 Looks like all the data is there,
0:27 and we can see that it's 83 84 megabytes of data.
0:32 Look at all the rows that we have and the data type.
0:35 So everything seemed to be read in really quickly.
0:38 And one of the things you can do in Jupiter notebooks if you want to see
0:43 how long something takes, you can use the time it magic command and let's run
0:49 that. And what it does is it actually runs through the command multiple times and
0:53 kind of averages. The time it takes so in this example takes longer than normal
0:58 What's really nice about this is it takes just a little bit over one second
1:05 to read that really large file that Excel couldn't handle,
1:08 and that really brings home. I hope for you how powerful pandas is and how
1:14 it enables you to work with things that excel can't do.
1:18 So let's even continue. Continue this.
1:20 What if we wanted to do just some really quick analysis?
1:25 Maybe we just want a quick group by and we can see how much we sold
1:31 books, pencils and pens. And since it's a little hard to read,
1:36 lifts a plier styling to it just and now we've got nice dollar signs on it
1:42 We can see $52 million and books $13 million pencils in $20 million in pens
1:47 all with a handful of pandas commands very quickly done in a way that we