RESTful and HTTP APIs in Pyramid Transcripts
Chapter: Customer Renderers
Lecture: Adding a CSV renderer
0:01 All right let's start adding some custom renderers to our web api.
0:05 Now, notice over here, I made a copy of this web app
0:09 and moved it into content negotiation,
0:12 so we're going to come up with different contents
0:14 and then we're actually going to add content negotiation
0:16 and I try to structure all of that together
0:18 so if you want to follow along you can start from here,
0:20 there's a starter copy of the site just as we're starting from right now in there.
0:24 So, what do we want to do? The goal of this first part would be
0:28 to change that to csv, and have csv comma separated values
0:33 come out of this method, now we don't want to really lose json
0:36 I'm going to do this here, we can actually if we really wanted to
0:41 we could come over here and set the accept type I believe,
0:44 equal to something like /text/ csv and then only apply that renderer
0:52 in the case that this type matches, but where we're going
0:56 is actually going to be a better place than having separate methods
0:59 so we're not going to do that,
1:01 and also it turns out that the accept type is more complex,
1:04 we need some logic not just a string match there, okay.
1:07 So for now, I'm just going to switch this to csv,
1:09 and then we'll switch it back to enable both json and csv as well.
1:14 Now, if we run this, how well is it going to work?
1:19 It turns out probably not so well, let's find out.
1:22 So we come over here, all right, are we ready to get all the autos, we are—
1:25 oops, we're not, no such renderer factory csv, sad face.
1:29 Okay, so our job is going to be to create such a renderer scv
1:33 so let's go over here and because we're going to make many of these
1:36 and we care about having a well structured web application, let's make a folder,
1:40 so come over here and I'll call this renderers
1:44 and we'll come down here, for now we won't worry about base classes
1:47 or inheritance or type checking sort of like do an abstract type or anything like that
1:52 we're just going to come down here, I'll call this csv renderer,
1:59 let's just do it like this for the file name,
2:02 and in here, we're going to add a class called csv renderer factory.
2:07 Now remember, the api that we needed to follow here is
2:11 we had to have a call and it just took an info here
2:15 and then the call has to return a function, and we don't want this to be—
2:19 we do want to be an instance,
2:23 and then it will return self._render, something like that,
2:28 now not calling it, but just return the function, and then we're going to define this
2:32 and this is going to take the value and the system,
2:36 remember the system gives us access to things like the request and so on.
2:40 Okay, so we'll say something like if not value return nothing
2:45 let's just say we'll return empty string for our csv
2:47 if it's the case that nothing is going on there.
2:51 It says it may be static but let's just say don't bother me for now,
2:56 I don't think it can remain static for long.
2:59 Alright, so what do we need to do? In the end, our goal is
3:02 to come up with a csv string equals something
3:05 and in the end it will return the csv string, right
3:09 so that's going to be our implementation, but the question is
3:11 how do we take this value and convert it over?
3:16 So, we need to be a little bit careful here,
3:19 because we may be given a single object, we may be given a list of objects
3:22 and it turns out doing this in a super general way is not easy
3:26 but let's write this function here, assuming initially that we're given
3:32 a list of dictionaries or a single dictionary and we got to turn that to a csv,
3:38 that's pretty general, and then we'll see that we can do a little bit better.
3:41 So we'll say something like this, if is instance value
3:45 and it's a dictionary, then we're going to say the value is going to come
3:49 a list of one item containing that, we'll say if value is not, if not is instance
3:59 value list, we'll raise value error, something like this;
4:05 can only process lists, something to that effect,
4:09 and then we should check, maybe inside there's only dictionaries
4:12 there's a little bit of annoying tests that we have to do here
4:15 so what we're going to assume is,
4:17 we're going to assume that this is either just a dictionary or a list of dictionaries
4:23 and those are the only two types that we can handle.
4:26 Like I said, there should be probably more error handling here.
4:29 So the first thing that we're going to do
4:31 is we're going to also assume that all the types are the same right
4:33 that the list is homogeneous in terms of the types it contains
4:38 so the first one is representative of all of them.
4:40 Alright, great, so how are we going to do this
4:43 let's come down, we'll get the headers, and this is going to be- let's save first,
4:47 now we want to call this here like the value of zero
4:52 how do we know that this is ok— because if it's a list
4:55 it's going to be false here and will return nothing,
4:58 if it's not a list, it's going to be a list of one item, right;
5:02 so either way this should be okay, there should be a first
5:06 and in here, we're going assume this is a dictionary
5:09 so we are going to go and say give me all the keys,
5:12 this is going to be the header, so we'll say
5:14 this is going to be the comma separated value of header—
5:18 now I probably should be using the csv module,
5:20 but I'm just going to just type it out of here
5:22 just for simplicity sake, I want to introduce more things.
5:25 So what we want to do is come over here and join this on all the headers
5:30 so we'll put commas between all the items and make a single string out of it,
5:33 great, then what's next— we'll say for let's call them row in value
5:39 all right so remember value is a list at this point we've adapted it to that pretty much
5:44 and each row is going to be a dictionary,
5:47 so let's come up with the line data so we're going to store all the values
5:52 and we need to of course store them in the same order, alright,
5:56 so we'll say something like this, for k in header
5:59 so we're going to get the key out of the header,
6:02 and then we'll say line data append, row of key right
6:07 this is one of the parts we're assuming
6:10 every entry in this list is going to have all the keys right
6:13 otherwise this is going to crash, and then we'll say something like this
6:17 the line is going to be again join this on the line data
6:21 and then let's store up the rows here, let's go like this,
6:25 so we'll create this list here and then we'll just store each line
6:28 and then finally we'll like generate the lines with having the header up here,
6:33 as well as all of these, so maybe the first thing we want to do is response rows
6:38 csv string, and let's go ahead and just inline that,
6:42 all right, so we'll go like this and then we're going to come down here
6:46 and we don't want this anymore, we want to put new lines after each row
6:50 so again we'll join and then response rows, there we go.
6:54 So I think this is going to do it for a basic version here
6:57 so we're going to come through and just like this,
7:02 so we're going to come though and we want to create this list of rows
7:06 the first row is going to be comma separated values of the headers
7:10 and then we're going to go through each row
7:12 and make sure we put in the same order,
7:14 because dictionaries are not ordered by default
7:16 so we're going to go through the headers and pull out the value
7:19 for each one of those and then we'll get to join that to a line,
7:21 add it to our response generate the whole response here.
7:24 Now, the final thing is we're not using this yet,
7:28 so let's go ahead and let's do that down, I guess we'll do it at the top.
7:32 So we'll get the request, we can go to system, say get request
7:39 then to the request, we can say response—
7:42 the request always has the response objects on it
7:46 and then we want to set the content type
7:50 text/ csv, let's try this again; now it's going to turn out to still not work
7:57 well, first I guess we have to install it, but let's go ahead and just verify
8:00 it still didn't do, there is as if no such thing as csv.
8:04 So our next step will be to install this in our dunder init,
8:08 just like where we configured our json,
8:12 we also want to do another one of these add renderers but not for json
8:16 but this time for csv, okay, let's make that csv renderer,
8:21 we're just going to have to import this, allocate instance of it,
8:24 now that should almost work, it's not going to work
8:27 you're going to see it's still not going to work but we'll get a different error.
8:30 Car has no object attribute keys,
8:34 remember our data layers now returning rich objects
8:37 that's good, that's how we wanted to work but it's not so happy about this car thing
8:41 okay so let me just hack something in here for a minute
8:44 it's kind of not great, we'll come down here and say
8:47 if this is car, we're not going to leave this here so don't worry, we'll go like this
8:53 if it's a car, then it's not to dict, yeah
8:57 down here we'll go through and we'll say if it's a list,
9:02 go through each item and convert it, not great, but that's what we're going to do,
9:09 let's do it like this idx, and we want to enumerate this we can put it back.
9:13 So, we can ask the same question, if it's a car set back in that same location
9:18 we want to set item.to dict this better be item,
9:23 okay so that's not amazing, so if it's a car, we'll turn it to a list of dictionaries,
9:30 if it's a list and it happens to contain cars,
9:34 we're going to turn those into their dictionaries
9:37 and we know how to do that right there, okay let's see if it works now.
9:41 Ready, set, go!
9:45 We've chosen to download a csv file, amazing
9:48 what do you want to do, let's save that, come over here
9:52 and let's change that to . csv, we're good
9:57 now let's see it in excel, you can always see in the preview,
10:00 it looks like it's going to work; oh would you guys like to see what's new in excel?
10:04 Look at this, all right so this is totally working here
10:08 we've got our price, we've got a year, we've got a little bit of damage
10:11 yes it has damage, ja, ok so we've got all the pieces that we need
10:15 represented as a csv, we've got our header at the top and everything
10:20 this looks great, even when there was no item
10:22 when it was like null basically,
10:24 we still generated that correctly so this looks really good.
10:27 Now, of course if commas are in our values that's not so great
10:30 like I said we should really use the csv module
10:33 but the point is just like keep it simple, right.
10:35 That's great, but what if we want to go back, what if we wanted json again?
10:41 Well we could, let's go and run postmen against that same url.
10:46 We could get this to work with both, but we're going to need to do a few things first
10:49 we're going to need to fix this car thing
10:52 because obviously hard coding car into the renderer
10:55 like what's the point of the renderer then
10:58 so there's a few options on how we can deal with that,
11:01 and the other is how do we keep json around; those are up next.