Building data-driven web apps with Pyramid and SQLAlchemy Transcripts
Chapter: Course conclusion
Lecture: Core concepts reviewed
0:00 The first core concept that we talk about
0:02 were just the building blocks Pyramid.
0:05 We saw that Pyramid was built upon routes
0:08 which map URLs to a view method
0:10 sometimes called action methods.
0:12 We have templates, like Chameleon templates
0:15 that we take and pass models from our view methods
0:19 down to the templates. And the templates then take that
0:21 and generate HTML, the templates are basically dynamic HTML.
0:26 And then we added a bunch of our own layers.
0:28 We added the concept of view models and data services.
0:32 And all of this organization that allowed us
0:34 to evolve our web application super, super easy.
0:37 These are just some of the core building blocks
0:39 of Pyramid and our web application.
0:42 We saw there are two powerful ways
0:44 to create your Pyramid app, at least two right.
0:47 There's always more than, more than you might expect.
0:50 One really simple out of the box was
0:53 is to use PyCharm and its tools
0:55 to create Pyramid web applications.
0:57 The other one is that we could use CookieCutter
1:00 and the command line to create them.
1:02 I guess another, you could just start from empty files
1:04 and start typing but that seems
1:06 like you're wasting your time.
1:07 So when I was new I found I used
1:10 the PyCharm way a lot.
1:11 It made me feel real comfortable.
1:12 And it was like super easy to get started.
1:15 And the more I get into it
1:17 and the, as the years have gone by
1:19 I much prefer to do the command line stuff
1:21 and then finally edit in PyCharm.
1:22 But not create in PyCharm.
1:25 You will find maybe that right now
1:27 one thing is the right solution.
1:28 And then later as you get more experience
1:30 there might be something else that is a different
1:33 better solution.
1:34 There are many different types of templates.
1:36 We have at a minimum Jinga 2, Maco, and Chameleon.
1:40 We saw that Chameleon is the most web like.
1:42 With the least extra syntax.
1:45 And so I awarded it this little blue trophy
1:48 and said, this is the best way
1:50 because this is still proper HTML.
1:54 Where all of the other template languages
1:55 are madly broken HTML with lots of open
1:58 begin this, end that, end if, end for sort of stuff.
2:02 And to me that's just, I just don't like typing that stuff.
2:05 And I don't like the fact it's not proper HTML.
2:08 Pyramid supports all three.
2:10 We use Chameleon and I encourage
2:12 you to make those decisions for yourself
2:14 and think through the trade offs.
2:17 Closely related to Chameleon was this idea
2:20 of a core layout template.
2:22 Now this exists in Jinga, Maco, and Chameleon.
2:25 And whichever template language you use
2:27 it's really important that you create
2:28 one of these layout templates that's going to drive
2:31 the overall look and feel of your site
2:33 your navigation
2:37 and then your pages punch out the holes
2:39 where the template allows it to basically.
2:42 Here you can see the Talk Python to Me podcast website.
2:44 Although there are many pages here
2:46 the home page, the episodes page
2:48 and the individual episode details
2:50 they all have this very structured common look and feel
2:53 and that's done by the layout template in Chameleon.
2:57 One of the really important things
2:59 we focused on in the beginning was routes.
3:01 Typically we have some kind of static route
3:06 And then we add routes that map conceptual ideas
3:10 URLs, to their implementation in our web app.
3:13 So we might want to have / go somewhere.
3:15 And that's pretty much required for a website.
3:18 And then we might want to have /help.
3:20 or /project/package.
3:22 These can be static like the first two.
3:24 Or it can pass data in the URL like the package one.
3:27 And we can even put predicates and constraints.
3:31 So we said, we'd like to be able to just say
3:34 /7, and have that look up
3:36 which is the seventh most popular package.
3:38 It's kind of a silly idea.
3:39 But that's what we did.
3:40 We can do that by having this little
3:42 lambda function which is a test.
3:44 Is that thing a number?
3:46 Well this route is only going to match then.
3:48 Now remember, there's an order of operations here.
3:51 We must have the most specific first
3:53 and the most general stuff at the end.
3:56 The way this works, think of Pyramid just taking a URL
3:59 and running it from top to bottom through this list
4:01 the first one that matches, that's it.
4:03 So if you have some super broad thing at the beginning
4:06 well the stuff below it will never hit.
4:08 So very very specific stuff at the beginning
4:11 working your way down to more general.
4:15 We saw that Bootstrap allows us
4:16 to make our design much better.
4:19 The CookieCutter templates already come with Bootstrap
4:22 which would take us from the really bad page
4:24 to the middle page, which is already better.
4:27 And then we can use themes and some of the other Bootstrap
4:30 elements like buttons and nav bars and so on
4:32 to get a really nice page.
4:34 Be sure to take advantage of what Bootstrap
4:37 and its theme ecosystem has to offer for you.
4:41 With our design in place, it was time to write
4:42 some database access code.
4:45 We wanted to work with a relational database
4:47 for most of this course, so we were using SQLAlchemy.
4:50 SQLAlchemy has this concept of a unit of work.
4:53 And that is embodied in this thing here
4:55 we're calling s, a session.
4:57 So we use the session to create the queries.
4:59 Also to do the inserts and deletes and commit the changes.
5:03 Here we're not committing any changes so that's fine.
5:05 But we write this query against our models
5:07 in this case an account that has an email
5:09 and a password hash.
5:10 And then SQLAlchemy's job is to generate
5:13 the SQL targeted at the database we're talking to.
5:17 If you're talking to Oracle, generate it one way.
5:19 If you're talking to SQLite
5:21 generate maybe different SQL statements.
5:24 Which would still be different than say
5:25 Microsoft SQL Server.
5:27 And you can see down on the bottom
5:28 it writes more or less the SQL code that you would write.
5:31 Select star from account where these values
5:33 are equal to this.
5:34 And it uses a parametrized query
5:36 which is really really nice.
5:37 Relational databases are nice, until their not.
5:40 And one of the times they're not nice
5:42 is when you have to evolve from one shape to another.
5:47 We saw SQLAlchemy really hates that.
5:49 Queries themselves
5:50 Don't work very well unless you evolve
5:52 those types of things.
5:53 If we're already using SQLAlchemy
5:54 we might as well use Alembic, which can automatically
5:57 generate the sequel statements
5:59 and sequel operations to transform
6:01 from an old database with an old set of models
6:04 to a new database matching the current models.
6:07 So Alembic allows us to do what are called migrations.
6:09 And this is super important for evolving
6:12 our application and our database structure
6:15 once SQLAlchemy has created it.
6:18 When we are accepting user input
6:21 we use the get post redirect pattern.
6:24 The idea is the user gets some form.
6:26 They edit it locally.
6:28 They post that data back.
6:29 We save it to the database.
6:31 And then we redirect them somewhere else.
6:33 The redirect is really nice because now they're
6:36 in this new location and if they refresh the page
6:38 that's fine, it just keeps showing them that new stuff.
6:40 As opposed to, this form has been submitted.
6:43 Are you sure you want to submit it again?
6:45 And those kind of weird things.
6:46 So it's a really nice and clean workflow
6:48 for accepting user input in web applications.
6:52 As we accept user input, we have to do things
6:55 like validation, and data exchange
6:56 and all of that.
6:57 And we came up with this concept of the view model
7:00 which handles mapping the data from the HTML
7:03 into Python's memory and validating it.
7:06 That doesn't sound like a big deal.
7:07 But that turns out to be a massive amount
7:09 of what we would otherwise put inside these action methods.
7:12 And it makes it real easy to test
7:14 this validation and data exchange.
7:16 And it really simplified our action methods
7:18 and lets it focus on orchestrating the steps
7:20 and doing the high level stuff.
7:22 And our view model can do all of the nitty gritty
7:24 data exchange and conversion.
7:26 Speaking of testing
7:27 we focused on three types of tests
7:29 that I consider to be somewhat unique to the web.
7:33 And we talked about how to do this in Pyramid.
7:35 Of course there's many types of tests.
7:37 One of the tests that we might want to do
7:38 is this data exchange and validation.
7:40 And in our world that means testing the view model.
7:43 And that means passing a real true request object
7:46 from Pyramid on to the view model
7:48 so it can interact with all of the various dictionaries
7:51 and other data exchange mechanisms.
7:53 We can use webtest to create these dummy requests
7:56 and even pre-populate them with data
7:58 to test various scenarios.
8:00 We could do that for the view model.
8:02 We could do that for the view method
8:04 which may or may not use a view model.
8:05 Or we could entirely start up the entire web application
8:09 and pass fake web requests to it.
8:12 So we talked about the three levels of tests.
8:15 And then like I said
8:16 there are others in testing in general.
8:18 But these are pretty relevant to the tools and techniques
8:21 that come along with Pyramid and WebTest
8:23 and things like that.
8:25 Once we wrote our app, we tested our app.
8:27 It was up and working.
8:28 Then we decided to deploy it to a Linux server in the cloud.
8:32 So we used Nginx, which was the front end web server.
8:36 The request came into Nginx.
8:38 That's probably HTPS and get static files
8:40 and stuff like that.
8:41 But when there was a logic based request
8:44 like show me this package
8:45 we need to run our Python code.
8:47 So those requests would flow on through uWSGI
8:50 which is running in this sort of master emperor mode.
8:53 And there's a bunch of worker processes
8:54 where our Python code actually runs.
8:56 And uWSGI figures out which one
8:58 to pass off the request to.
9:00 And then flows it back in reverse through this whole thing.
9:03 So you have a bunch of scripts that set this up.
9:04 And we went through it.
9:05 And it should be pretty straight forward
9:07 to get it working with that all in place.
9:10 The last thing we did is said
9:11 let's change databases.
9:13 That seems like a good idea
9:14 now that we've done all this work.
9:16 Well, there was two ideas here.
9:18 One, we wanted to say well
9:19 maybe some people want to use MongoDB, and NoSQL.
9:22 And actually our relational database
9:24 and our NoSQL database were much closer
9:26 than people might realize
9:28 'cause we could use SQLAlchemy and Mongo Engine
9:30 and those are quite similar to those things.
9:32 Not the same. One is active record.
9:34 One is unit of work. Things like that.
9:36 But not terribly different.
9:37 It was a pretty small amount of effort
9:39 to considering the type of changes we made.
9:41 So we switched to MongoDB.
9:43 And that was maybe interesting in its own right.
9:45 It's also interesting in showing the power
9:48 of what we had built up to this point.
9:50 We only had to change the data services
9:53 obviously new entities, and a few very minor things.
9:56 Like we had to change the type of value exchange
9:59 for the user cookie 'cause one was a BSON object ID
10:02 and the other was an integer.
10:03 And those are obviously not the same thing.
10:05 So there's a little bit of work in the cookie level
10:07 to make that change.
10:08 Other than that, our design patterns
10:11 really isolated the types of changes
10:13 to the locations that we, well want them to be.
10:16 Data changes and the data services for example.
10:18 So hopefully you found that really interesting and powerful.
10:22 And the main takeaway I wanted you to get
10:24 was we don't have to be stuck with how
10:27 we started building our app.
10:28 It's actually if we build it correctly
10:31 quite easy to transform and change
10:32 from one style to another.
10:34 This database change is just one of those types of changes
10:38 that we prepared ourselves for by clearly
10:41 and nicely structuring our code.
10:43 Well that's it.
10:44 Those are the major concepts
10:45 of what we covered in this course.
10:47 I hope you feel like you learned a lot.
10:49 And I hope this quick review helps them stick.