RESTful and HTTP APIs in Pyramid Transcripts
Chapter: Logging and monitoring
Lecture: Setting up logbook
Login or
purchase this course
to watch this video and the rest of the course contents.
0:01
Now let's add logbook logging to our api. The first thing we have to do is actually install the logbook package and of course set it as a dependency
0:11
so down here we're going to want to put it there and let's go ahead and this time we're just going to pip install it
0:17
so we'll say go to the terminal make sure that our virtual environment is active and we'll pip install logbook,
0:25
great, it looks like that's going to work and we come down here and we're going to say loogbook, right,
0:31
so when we go to the deployment or we check this out on another machine we will run the setup, it automatically has all the dependencies, right,
0:37
we don't want to have to chase those down as run time errors. So we have logbook here, the next thing we need to do is
0:43
do a little bit of configuration, logbook is very flexible, and it can send the log messages in all sorts of locations,
0:49
I think what we're going to do is when we're running in development mode we want to send it to the console,
0:54
so we can just see it come out in the terminal there and in production we want to send it to a particular file
0:59
so let's go here and give us a key that we can use, or a value that we can look up about where this file is if we should use it
1:07
so we'll say logbook log file, something like that, now let's just leave it blank here in the development version,
1:14
and in the production version, let's go over here and we'll just say this is going to be the logfile.txt, something like that
1:23
remember to not put quotes on these, the green is already like the string value so the quotes will cause trouble.
1:30
Okay, so we have this, and let's go and then import this, and now we've got the little init db
1:35
that is very similar to what we're doing, so let's do an init login, like so,
1:39
so we're going to get the settings, and this time we're going to have log file and we wanted to just grab that value, which may or may not be empty
1:48
but we're going to go grab this value here, and then what are we going to do next well, let's create a special single module, whose job is basically
1:57
to set up all the logging for us, so we don't have to bloat our set of file here keep this nice and clean and readable,
2:04
so over in infrastructure, let's just create a logging bit and let's import it, in a nice and easy way
2:11
let's say import restful infrastructure logging as logging so we just say logging, that will keep it nice and simple,
2:19
so down here we'll say logging. let's say global this is kind what we did before, just global init, log file,
2:29
so this global init really means like you call it once at the beginning and that's it, we can use PyCharm to write this function, thank you PyCharm.
2:36
So now, over here, we're going to actually work with logbook, so let's begin by importing logbook, like that,
2:43
so what we need to do is we need to make two basic choices, one, what level are we going to log at,
2:50
and this might be something entirely reasonable to put into the configuration files and debug, we're logging a trace and in production
2:58
we're only doing warnings and above or something to that effect; so let's go over here and just set the level
3:04
so let's say logbook. we have notice, info, trace, notice in production, probably you would want to run info or notice
3:11
these are kind of higher level, and sort of the most broad one is trace,
3:15
so we're going to default the trace if it happens to be that we have a log file here and that would indicate we're kind of in production,
3:22
then we'll make a different choice for the level. Okay, so let's actually use the log files presence
3:27
to indicate whether it's in development mode of production mode, so we'll say if the log file exists, then we want to send this to a file somewhere
3:36
we'll come back to that, we'll do the simpler one first so if there's no log file, we're going to use this trace level
3:42
and what we want to do is we want to set up what's called a stream handler so we can say logbook and the stream handler will write to any stream
3:51
well, what stream do you think we want to use, how about standard out, right, so we're just going to log to where all the other messages go,
3:58
wherever print goes, so go our messages, we can set the level and let's just actually, let's move this in over here
4:04
different values in two locations we can just inline this like so, okay, not sure why we have the parenthesis, thank you PyCharm,
4:15
okay, so this stream handler we're going to need to create it and now what we can do is we can say I would like this particular handler
4:22
to capture messages from where, and so we use this push thing, we can push it to the thread local storage,
4:28
greenlets or globally within our application. So that's what we want to hear is to just say for the application run this
4:36
and we're going to do something similar up here, but we're not going to use a stream handler, we are going to use a different kind of handler
4:43
so look at all these different handlers, we got a gmail handler, a file handler, a rotating file handler, a timed rotating file handler and so on.
4:51
Even a fingers crossed, which is kind of an interesting one, so rotating would be once the file hits a certain size, make another copy of it,
5:00
like switch to a new file name, timed is I think a little more predictable like once a week or once a day switch to a new file,
5:08
so you have a log file for that day, that's what I use and I think it's really important because if you have a popular web service
5:14
you'll see that these log files can pile up really quickly, like on Talk Python we have gigabytes of text in these log files
5:21
and periodically we'll go and take the old ones and compress them and get rid of them, right, so having them based on time is really handy.
5:28
So we want this, and of course, it's going to have different values here
5:31
we're not going to send it there, we're going to pass in the log files, the first thing and then we're going to set the level, just like before
5:39
and this one, let's do this one at info, right, so trace and those kinds of things like really verbose stuff is out,
5:46
so either info or notice seems like a decent option here. And then, for the date format, we want this to go something like
5:54
%Y-%m-%d, something like that, so this is the format of the file, ok, so a timed rotating file handler for, if we have a log file
6:07
and if we don't, we're going to assume that it's just production. Ok, so this looks pretty good, let's just make sure app runs
6:14
and everything we've done so far is hanging together and we'll also do a little print here, print configuring logging with
6:25
and we'll do the log file, and let's do it like this, okay, so run it, are we actually calling this, that is the question,
6:34
so we created the function, but no, of course we're not calling it, unfortunately, there we go, now we'll be calling it, great, configure logging with
6:44
well nothing, right, notice up here that I actually have created a second run configuration,
6:49
let me pull that up, we have a dev which points at development.ini, and a production, which points at you guessed it— production.ini.
6:56
This is super easy to set up, you can go up here and add as many of these as you like, and you just pick the configuration file,
7:02
the working directory, and the Python interpreter to run it. So if we switch to production we'll be using the other configuration file
7:09
and now we're going to be logging with the text file. So it looks like our grabbing the value is working correctly,
7:15
the pushing, the stream handlers to the app worked great, now all that is left is to actually do some logging.