RESTful and HTTP APIs in Pyramid Transcripts
Chapter: Deploying your REST service to DigitalOcean on Ubuntu
Lecture: Create and configuring the droplet (server)
Login or
purchase this course
to watch this video and the rest of the course contents.
0:01
We have our ssh keys generated, we don't actually have them registered in MacOS for now so that is something we'll have to take care of,
0:09
but let's go ahead and create this Digital Ocean droplet as they call them, so we are going to use the five dollar a month server,
0:14
just think about that, five dollars a month, that is crazy affordable, we don't need any of the extras, I'm going to put this in San Francisco
0:19
if you want to hear my thinking on this, like obviously, San Francisco is near Portland Oregon where I live
0:24
so why not do this one locally; most of my machines live in New York because a majority of my traffic is split between the US, Canada and Europe
0:31
and so I figured east coast US is as good as I can get to still serve the west coast of the US well as well as Europe.
0:40
So I typically would pick New York if that was your traffic patterns as well, but I'm going San Francisco.
0:46
So it looks like I've already set this Digital Ocean key, and I've already selected it and I want one droplet, I've set the name,
0:52
so we click create and how long is this going to take— you'll see it takes almost no time at all.
1:10
I think I might leave this in here as real time for you, so you actually see, there's no speeding up. Done. That's got to be less than thirty seconds,
1:22
all right, so now we can go and talk to it, so let's go here and copy the ip address and we'll go over and we can ssh as root that's the account there,
1:36
and first of all, we've never done this before, and so ssh says are you sure, and we say yes, we're sure.
1:41
However, permission denied, we don't have the right public key, we don't have any key really, so what do we need to do?
1:48
Well, this is because we did not use the standard name for our key, so we created a custom one, so I can run ssh add,
2:00
let me just delete this, we can run ssh-add -K to add it to our key chain and what we want to give it is the private key.
2:12
Now if we try again, ta-da, we're here, what should we do first— oh you bet, the very, very, very, very first thing we should do is apt update
2:24
and install any security patches that might still somehow not gotten taken care of, so see if there's anything to upgrade here—
2:33
there are so, so let's do a quick upgrade, see what this is all about,
2:36
bunch of built in stuff, let it go, and just to be sure, let's just do a quick reboot,
2:41
we'll wait about ten seconds, so we can log back in, and then we can get started,
2:45
while that's rebooting, I want to copy some files, let's see if it's back yet, boom, that was pretty quick right, very very fast.
2:54
So over here into our project, we're going to move this up to Digital Ocean on our server, but there's a few things we need to do first,
2:59
let me put some files here, and I'll explain what we get in a second, I put this etc folder here, and it has the configuration
3:07
we are going to need for μwsgi, a configuration we're going to need for nginx, and the set of steps that proceed the existence of that
3:15
so this basically is the set up of the server. Now, the very very first thing we did if we did our update and upgrade
3:20
so that in case there is any security vulnerabilities, we're not getting smacked on.
3:25
The next thing we want to do, I am just going to copy and paste these there is no sense for you to watch me type them,
3:30
is I'm going to set up the firewall, so this is uncomplicated firewall in Ubuntu, and we're going to allow ssh traffic,
3:37
we're going to allow standard unencrypted web traffic and encrypted web traffic and nothing else so we're going to run those over here and it says
3:46
you could disrupt your ssh connection, basically if you mess this up, we want to proceed because now we're still letting 22 go through.
3:55
We can check just by exiting and coming back, yeah everything's fine, we didn't break it, probably you want to check that early.
4:02
Also, we can install fail2ban in case there was, if you wanted to log in with the username and password instead you can install fail2ban,
4:09
I'm not going to do it because we don't have that, but this would prevent like brute force attacks against logging in, I'll go and leave that there.
4:16
The next thing we need to do is use aptitude to install, like build essentials, Python 3, git, so we can get some stuff and just a few other things
4:25
like nload is a handy little thing to have as well, so let's just run through that, and I'll speed up some of these
4:29
as we go through and just shorten them. All right so we successfully installed build essentials,
4:36
it took a little bit, but we'll go and install these next— excellent, now we want to get the latest nginx, and right now
4:46
I think you get 1.10 if you just apt install nginx, but we can go and register what they call their development branch
4:54
which gives you a better version, I actually recommend doing this I believe so, go add that, now we want to make sure that we do an apt update
5:04
so we actually pull those changes before the next thing. Come back over here, this is Python software properties, we need this as well,
5:19
then we can actually install nginx, so I said a bunch of stuff's going to be installed
5:26
and so on, that sounds fine to me, it's what it needs, let's it do it. All right, that looks like it works, now we're ready to start working on μwsgi,
5:35
so let's make sure we install pip for Python 3, so right now if I type pip, we don't have it at all, right, if I do what it suggests,
5:43
it's going to install that for Python 2 I believe, I want it for Python 3, the alternative would be to use the get-pip.py thing
5:50
and not let Linux manage that at all, this does lag a little bit behind so it might be with considering but we'll just do it this way for simplicity.
6:00
All right, pip is set up, now μwsgi takes a long time to install so let's let it go on its own, notice I'm now using pip 3,
6:10
okay, great, looks like we have μwsgi, let's also check nginx, check its version, that's 1.11.9, that's newer than you get
6:19
if you don't register the extra aptitude link there at the repository, and here we go, got it right, so we also have 2.0.15 μwsgi, alright,
6:31
the latest of both, that's good, now glances is just another tool like in load that lets you manage and view your server
6:39
this is like super, super helpful so I am going to go install this now we type glances and you'll see it gives you
6:47
this really cool sort of dashboardy view of your server and notice, so far we're only using 18 percent of our memory,
6:55
and it's the smallest server right, we're doing really well, this little character is using 3 percent,
7:00
this glances thing so that's what this cpu is about. I will come back and use that again in a little bit, to make sure things are running okay.
7:07
Alright, now let's go and create some log files, or actually rather places for log files to go, so we're going to create some directories, web apps,
7:17
web apps log, auto service, app log and so on, so in here is where we're going to put our time rotating file logs,
7:23
whereas we'll just drop the μwsgi log there for example. Okay, so we got our logs, the next thing to do is we're going to use git
7:31
and let's set this credential cash so that we don't have to type our username and password in every time, which will make us crazy
7:37
Alright, so now if we enter our credentials, this is member for what is that, that's a 100 hours,
7:44
all right, now let's, this is not what we want to get, let me change that url, so over here what we're going to do is we're going to go
7:54
and download the repository for this, so we're going to do this git clone here and we want to do it in this folder, right,
8:03
so we're going to have web app/the repository and web app/logs; now in this case because that's a public repo
8:10
I didn't actually have to type in my username and password, but I think most of the time your service would not be a public repo
8:16
so that credential helper will help you. Okay, so now we have this restful thing that's great
8:21
and let's see what we got, let's go into source, go into deployment, and we're going to need to see where some of these things live,
8:28
so the restful service, this directory here, this is the root of our project, ok, so we're going to be using that in a couple of places;
8:38
now what's next, so we want to cd into this location and this right here is that location, so we want to go there, to the root of our thing
8:49
and run Python 3 set up develop, okay, so now we're there, so we want to say Python 3 set up, auto complete shows it, so that's a good sign,
8:58
and this will run and install all the stuff that needs to install all the dependencies including logbook and sqlalchemy like right there,
9:05
as we said, it needs for it to run. Alright, everything worked well, it might be worth giving us a test
9:16
so we can do pserve great, and we could give it the development.ini and it's working, check it out, configured logbook with nothing
9:24
our pyramid cors is working, here is our database and we can actually create another thing that we're going to ssh
9:31
and we could do w get, actually let's do this, this is more fun, pip 3 the install httpie, I always mess up saying that, httpie, and now we can do
9:40
http local host:6543, so what do we get, hey that looks like our website, right
9:49
it looks like our web app is running, so the one is running for development,
9:53
but it turns out we probably have a little more work to do on the production.ini. Okay, so first of all let's look at production,
10:03
here we have the log file, we probably want to put that log file somewhere else, so we'll have a look at that over here, as we had web apps here
10:14
this right there is the place that we want to put that, so now we got our server configured, let's put this as this
10:20
and let's just call this auto service.log, something like that ok, so it's going to go there and of course it will put the time on the end, and so on.
10:31
So, this is looking good, I guess we can go ahead and run this so we're going to push this back up,
10:39
ok, so I like source tree, you can use whatever you want, but we're going to push that along with the various configurations up there,
10:45
so we'll have that to run, now let's see if we can get this to work with production.ini.
10:53
Boom, it's working, great, now if we exit and we do go look at the logs, oh it did run, but it didn't put it in the right place, why— I forgot this,
11:04
now let's try it again, so now you can see we have this auto service log, and let's just go and see what it's about,
11:14
perfect, so it looks like our log file is working, you can see where it has the log set so we're very very close, so what this means basically,
11:22
I didn't do http request, I am pretty sure if we get this far everything's going to be fine,
11:27
what this means is we have our Python code running perfectly, now we have two more sort of outer shells to configure before things run.
11:34
The next thing we need to configure is μwsgi in its emperor mode, and then we're going to have to configure nginx
11:42
to pass the things along to μwsgi, to pass along to our code, and then our web server and our web api will be up and running.