Consuming HTTP Services in Python Transcripts
Chapter: Reading JSON data with requests
Lecture: Consuming GitHub JSON API
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
I bet by now you are ready to actually consume some real services, we've made some requests and we played with Json,
0:08
but we haven't really put it all together and we are going to start by doing that with github,
0:13
so github is a public API and we can use request issue a get to it and then what we'll come back is some kind of Json document
0:19
that we can work with in our code, so let's switch over to PyCharm and do that now. So, I've made a new file here first to work with and let's go ahead
0:28
and just set that as he run configuration, notice we are running out of our virtual environment and we'll just start by adding a little structure,
0:35
so we are going to import requests and then let's have a main method, and let's conditionally call this, we can use PyCharm's little template
0:43
to help us out there, there are live template. Okay, so we are going to need to get the url here for the API
0:50
and then of course we are going to do something like this, maybe we are going to need to add some headers,
0:56
maybe we are going to need to do some other things, but we are going to need to start by knowing what is the API we are going to actually work with.
1:03
So let's drop in on developer.github,com and check out through API so if you scroll down there is a little bit of a getting started guide,
1:08
authentication, and so on, we are just going to go over here and let's suppose we want to work with repositories,
1:14
so if we want to get information about like say repository here, we could go to twitter bootstrap/bootstrap,
1:20
we could also go to the github repository for this project, so we are going to access https://api.github.com/repos/ let's get the repo for this course,
1:32
so it would be mikeyckennedy that's the username and then it's going to be the repo name, so over here, let me paste that,
1:39
okay so now we have our url, we are going to go to the repo which basically contains well, literally this source code, that is kind of meta isn't it,
1:46
and we are going to go get our own thing here, now the question is what are we going to get back,
1:51
we can start by exploring this in a real simple way, we already know how to do this request here
1:56
and we can just print out the response.text okay, so this is kind of the lowest level, let's go ahead and run that and see if this works,
2:04
cool, it looks like we've got something here, now I don't know about you, this is a little hard to read, if you look at this scroll bar it's huge okay,
2:11
so that's not great, we technically could do this, import json.loads and we can give it this, and we could then say json.dumps,
2:23
this might seem a little bizzare but then of course we could say indent=true, run this and we get at least formated Json,
2:30
but, let's have a better way to explore this, shall we? So let's jump over here and run postman.
2:36
Postman is great for exploring APIs, see I've got some saved ones, that we are going to work with later for more complex interaction
2:43
when we get to modifying data on interesting APIs and so on, but we can just put this url in here and notice, we can do a get put post
2:51
but this is just a get and we could set like headers, for authentication if we needed it but we'll just go and send this off,
2:58
here we get what came back, so you can see we have information about the full name of the repository, its id, the number of teams,
3:05
the number of forks, all sorts of stuff whether or not it has issues and so on,
3:09
okay, so we could print out the get url, the clone url, so let's do this, let's say what we want to do is given the name of a repository and the user,
3:18
we would like to go and actually get the clone command in order to clone it, or maybe get the url to open its documentation or homepage, okay?
3:27
So we are going to be doing that, by going and downloading this, converting to Json and then let's say access the clone url.
3:34
Okay, so we've seen we've already gotten the text and we can also parse it like this, but of course we should really check that this works, right,
3:42
and let's also get some input here so we'll do this. So we'll write a little function here called get_repo_info
3:54
and we'll write that down here above the dunder main check, and this is going to get two things, this is going to get the user, it's going to be input,
4:03
we'll just ask whoever is typing here, what is the username, and then we'll do the same for the repo, and then we'll return user, repo.
4:15
Okay, great, so I am going to put this in a comment up here so we always have it, in case you forget what the name especially this right here.
4:24
Alright, so we are going to do a little bit of formatting like so, okay so that again is going to get what we need,
4:36
let's format that a little bit and tell PyCharm that no, my name is not misspelled.
4:41
Alright, so let's run it make sure this little step is working, what is username, what is a repo name, this is going to be a copy paste,
4:48
and it looks like it still works, great. So the next step is to actually check, because it could have gotten a 404 all sorts of things.
4:57
So, we'll do this, we'll say if resp.status_code != 200 then we'll print error accessing repo and then we'll just bale,
5:12
so if we put in stuff that's wrong, error accessing repo 404, if we put in stuff that's right, everything is working, okay.
5:23
So this is great, we've got our information, and maybe we should check that the format is Json,
5:30
but we are more or less guaranteed through the API that we are going to get Json, so we could avoid this whole step and we can actually avoid
5:38
even using the Json module whatsoever, because requests actually already supports this, so what we are going to do is we are going to have a repo data
5:49
and we are just going to go to the response and if we know that the inbound data is Json,
5:55
we can just say Json and that will actually convert this to a Python dictionary so if I just print the type of repo data and we run this,
6:08
and we put in something legitimate, we get a dictionary. So the final thing, what we were trying to do the whole time is we were going to say this,
6:20
print to clone this person's repo named let's just put a little bit info here, so we'll say user repo print the command is get clone
6:35
and then whatever the clone url is here, so we'll say format and let's just store this for a moment, so what we need to do to get this
6:42
if we go back over to postman, is we need to go to the dictionary and get the thing called clone url, so we can just go to the repo data
6:53
and say either like this, this is a little bit dangerous because who knows what you are going to get, you might get a key error,
7:01
so we could do a get and this will just return nothing, if it doesn't exist, we'll say error no data, something like that,
7:09
so we are going to get this clone url, we can get rid of this and we are going to come down here and just say here is the clone url to go.
7:15
Alright, so let's put this all together, we are going to run this username is mine,
7:19
repo name is the repo from this class consuming services Python demos and if I hit go, the clone command is get clone such and such.
7:30
Well, let's see if that is going to work. Get clone that, oh it looks like it works.
7:44
I guess our service is working perfectly, so that is how we consume services that are Json based,
7:50
HTTP services at least read only get pay services with requests, we just do basically construct the url we do a get
7:58
we make sure the status code is whatever we expect, 200 is not always what is expected, sometimes if we were doing a post and creating thing,
8:08
201 would be expected but you check for what is the success code if you will, and then we can call just the Json function right on the response
8:16
and it converts that straight to a dictionary, it's a not a huge step to save, that we are skipping over using the Json module,
8:22
but it's just one less thing to think about how do you work with Json from this other thing, we just work with it straight from requests and then,
8:29
once you have it as a dictionary in memory, well, it's no longer a web service it's a problem, it's just data in Python, go.