Python Jumpstart by Building 10 Apps Transcripts
Chapter: App 9: Real Estate Analysis App
Lecture: Concept: generator pipelines
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
For our final concept in this app, let's look at building data pipelines out of generators.
0:06
Now, these generators could be created from generator methods or generator expressions. So let's see how we might use this to answer
0:15
some questions about the housing market. Our first task is let's find all the houses that match my requirements,
0:21
and let's say personally I am looking for at least three bedrooms, New York city, any city will do for a first pass and so on.
0:28
Next, maybe we want to find all the houses that are for sale that I might in theory buy, so that houses that come out on the first set,
0:38
maybe those are some are for sale some aren't. Here we want the ones that are for sale,
0:42
and then, I'd like to take the ones that are for sale and then look around me. Ok, so I am going to start out with this the results of task one
0:50
and filter them further down and process them more in task two, and then maybe I'll process them still further in task three, so how do we do that?
0:59
Here we can get all the real estate transactions and use this data to start answering these questions, so I've got this is interesting method
1:06
and that was kind of like check for three bedrooms at least and so on, so some sort of criteria that I am looking for.
1:11
Now there is no guarantee that these transactions represent houses nearby, or that they are even for sale.
1:17
But then I can take the results of this generator, interesting tx, and pass them to another generator,
1:24
in this generator we look for the houses that are for sale, we don't care about all the houses that are for sale,
1:29
only the ones that I would be interested in. And because they are both generators,
1:33
they do this in a progressive as you consume the data they pull the data, they don't process it all at once, they don't load it in memory all at once,
1:41
all those sorts of things so we saw in application eight. Now, we can ask further questions,
1:46
we can say I would like the ones that are potentially saleable, which when I pull from that set it of course does its test
1:53
but to get those items it pulls back from all the transactions so there is kind of this pipeline or chain
1:58
depending on what which direction you think of it from, and we'll loop through that using yet another generator
2:03
and we'll say for all the ones that are for sale that are interesting to me, I would like to reduce that to ones just nearby,
2:10
maybe I don't want to move cities I just want to look at hey let's find all the houses I'd be interested in Portland Oregon, or wherever.
2:17
So, we can use these generators to build pipelines, you can take one generator and pass it to the next
2:23
and then take that generator and pass it to the next and it creates this really cool pipeline of processing that's extremely efficient
2:30
and it doesn't even require us to write methods.