RESTful and HTTP APIs in Pyramid Transcripts
Chapter: Logging and monitoring
Lecture: Exploring Rollbar for real-time monitoring
Login or
purchase this course
to watch this video and the rest of the course contents.
0:01
The last section in this chapter about monitoring logging has to do with real time monitoring,
0:08
so what I'm using currently at Talk Python and the various podcasts and the training site is rollbar, and of the various choices,
0:15
rollbar is a really good one, let me take you on a quick tour and show you how you integrate it into your app,
0:22
it turns out installing this is your app is like super easy there's a free version, so why not try it on your api, right
0:27
So the idea is basically you connect your github project, even if it is private, here and then you can tell rollbar
0:36
when you deploy here's a particular deployment, it'll look back at github and understand the code changes that have been made,
0:43
and then it will wait for any errors, it'll send you messages about those and then correspond that back to the source code that comes from github,
0:49
so a really really nice system, let's first talk about how to install it, then I'll show you like what we've got up here for Talk Python actually.
0:56
So if you go down to the docs, they actually supported a couple of different libraries and languages and whatnot, so we go to Python,
1:04
and it talks about how you could obviously pip install it, there's a couple of things you can do, but if you're working with Pyramid
1:10
you just click here and it says this is super easy, they already have one of these handler type things that listens to every request,
1:17
before and after a little bit like our cors, the thing that we did, so you just have to pip install the Rollbar itself,
1:23
and then the contrib sub module or sub package is already in there, so you just basically tell pyramid to use this,
1:30
in production by the way don't do this, you just have this alone this is only for dev, very important there, and you have to get your api key
1:38
and stick it in here, and there is a little bit more stuff to do, to set up the logging and the environment, make sure dev says dev,
1:46
production says production, things like that. And that's it, you just basically configured develop.ini and production.ini,
1:52
and you're golden, right, so really easy to get going, let's go and look around the Talk Python back end.
1:59
So you can see there's not too many errors, like these blue ones, what is that five or six different types of errors,
2:05
some of them may actually be repeated the same ones, over two months I'm pretty happy that given how many requests that site gets.
2:10
So notice, in the last 24 hours no errors, that's great, nothing has been reactivated or anything like that,
2:18
there are some that I haven't gone and said you know this is okay I kind of approve this or I acknowledge this error has occurred,
2:24
so Talk Python talks to a MongoDb server which is on a separate machine at Digital Ocean,
2:31
internally it's known as Mongo server, although that's not a public name of course,
2:35
and there was some security patch for Linux so I rebooted the server, and I had not run it in like multiple machine replica mode,
2:42
so basically it failed to connect right, I mean the machine was only offline for like ten seconds over the last month, month and a half,
2:51
but still, Talk Python gets so many requests per second that they still generate errors, right, you can see a couple of errors here,
2:57
so we can go look at this, this is pretty interesting, let's see what we got, I could resolve it and say fix this here,
3:04
but we can do that in a minute, so here's the mistake that we got right, we couldn't connect to the Mongo server and let's see,
3:10
we come down here and here's the trace back, so what we were trying to do is we were going to the episodes controller
3:16
and then this feed view model, and then into the Mongo repository, I am calling a thing called git episodes,
3:21
and then we were trying to evaluate the query which came back from the database and notice I can click right here
3:26
and it will actually take me to where we need to go in github to view particular line, right, so this is pretty cool,
3:33
I can see how often this has occurred for what browsers and so on, we can register people and it can grab that, what's the breakdown by platforms,
3:45
so these are errors happening when podcast players are trying to grab the rss feed, which they're just pounding away at,
3:53
see the ip addresses, here is the suspected deploy, you can see all the stuff that I've put out here,
3:58
it says you know, I think this one right here is what happened, we could go look on github and are there similar items, yes,
4:04
are there community solutions, sadly no, especially since it involves very specific machine name, so things like that.
4:12
Okay so this is pretty cool, and you can actually see the call stack down through Mongo engine, down into PyMongo, and then finally all the way down,
4:19
So this is cool, let's go and also see the deploys, you can do your own queries, things like that;
4:25
and let's go look at another one that has a slightly different information. So if we go to Talk Python, down here, right here at this section
4:34
what do listeners think, when I first deployed this, I had in my local database populated a couple of these,
4:41
just test ones for the design and then the plan was I would deploy the site, and go to the back end which also came with those changes,
4:49
where I could enter these, but initially, right when I pushed go there was nothing in the database, right,
4:55
because my next step was to go and add something to the database, well it turns out before I could go and do that, I got this error,
5:01
the home page is not working, what is wrong with it, it's broken, and of course in production it just says 502, or 500, boom, no dice so that's bad,
5:12
but then I got a quick notification about this, and I came over here and said what is going on, so I looked at this and said ok,
5:18
see home control or home view model, doing this oh chosen reviews get random reviews count=3, get chosen reviews random.choice on the reviews,
5:28
and guess what— that was empty, so it crashes, right, the crash was trying to run this line right here,
5:34
which is pretty cool, and check this out, so you could expose the arguments that were passed to this function, it happens to be passed
5:40
because of the default, but or maybe I set that, I don't know, but it's passing three for the count, right it's also pass itself
5:48
and has some local variables down here as well, we can see the various pieces here, so at the time this was running on sqlalchemy
5:56
now it's running on Mongo, but it doesn't matter, the problem is that with the database the problem is this line,
6:02
doing choice on a list that's empty basically and so instead of trying to debug this figure out what's going wrong
6:07
it fails in production but it works on my machine like why is this happening, I would just pull this page up and go oh,
6:13
I better quickly go enter some stuff into the database at least one thing really quickly, so this doesn't crash,
6:19
boom, problem solved, it wasn't actually a coding problem, but it was a configuration data problem;
6:24
so this is the kind of stuff that you get with rollbar, and you saw it's super easy to integrate it,
6:29
you just basically set up your configuration any file and you're good to go.