Building data-driven web apps with Pyramid and SQLAlchemy Transcripts
Chapter: Testing web apps
Lecture: Sitemaps testing in action
0:00 Let me show you the sitemap I've added
0:02 to our website here.
0:04 Again, I don't want to write it from scratch
0:05 cause the details aren't that valuable.
0:07 It's just repetition of what we've been doing.
0:09 But let me show you what's happening here.
0:11 So we actually added this UtilsController
0:13 and we have two things, robot.txt
0:15 which tells search engines how to search the site.
0:19 And notice we're changing the content type
0:20 to plain text as it should be.
0:22 And then this one, we're saying the content type is xml
0:26 and we're going to generate it from the sitemap.xml.
0:29 We saw that before but let's have a quick look.
0:31 Robots, by the way, super easy.
0:34 You explicitly disallow stuff
0:36 with not disallowed is allowed in so, search everything.
0:39 And then here, we're going to say generate the static URL's
0:43 that we know about.
0:45 And then we're going to dynamically go through this.
0:47 Your site might have categories
0:50 and books, and comments, and whatever.
0:53 Put em all in here.
0:54 Whatever you want search engines to find
0:56 you put them in here.
0:57 So let's go see that real quick.
0:58 So we can go to robots.txt, and there's that.
1:01 And we can go to sitemap.xml.
1:05 And here you can see these are the static URL's
1:08 and then these are all of our packages:
1:10 Awscli, Logs, Babel, BeautifulSoup, et cetera, et cetera.
1:15 All right, so this is in place.
1:18 Once this is in place then we can use it for testing, right?
1:22 So let's go down to our tests
1:23 and here's the site map test.
1:26 We're going to create one of these functional tests
1:28 that goes and grabs an app that's already built.
1:32 It's a little bit confused there isn't it?
1:34 There we go. So it's going to use just one app
1:37 for all the tests it does here.
1:39 And just like we saw before it's going to call main
1:41 create a test app, and give it back.
1:44 Want to do that on setup and that's great
1:46 but then what we do is get the site map
1:49 by issuing a get there.
1:51 And then I'm going to drop the namespace
1:53 so we don't have to do it in our queries
1:54 and just return the text.
1:56 And then our tests become, get the text
1:58 load it up as xml, do this x path query on it
2:02 and then we'll just print out a little info.
2:04 So let's run this.
2:06 I've also added it to the alltests so it'll appear.
2:10 And here you go you can see we're testing
2:12 potentially 99 URL's.
2:14 Got the local host, account login, account register
2:17 project /, you know, here's the first one that we hit.
2:21 We took away that part of the break out
2:23 for testing the packages we'd hit em all.
2:26 So it's up to you how you want to do that
2:27 but this can be really helpful.
2:29 It's like I said, often when these pages fail
2:31 or these methods fail, they fail hard
2:33 and you get some 500 server error
2:36 and this would definitely catch it
2:37 cause we're checking there all 200 a-okay.
2:41 One thing to be aware of
2:42 when you go and request basically every URL on your system
2:46 we'll go into the database at this point
2:48 and doing the other things they might do
2:50 so you might want to be careful here
2:52 and mock that stuff out as we saw
2:55 but it's going to be quite involved
2:57 because you're basically mocking out everything.
2:59 Can totally do it and it may be worthwhile
3:01 or it may not be.
3:02 You'll have to try and see whether it's worth it.
3:05 So this definitely counts as some kind of integration test
3:08 but it's really easy to do.
3:10 You can see the whole implementation is 57 lines long
3:13 and you have it here in code so you can just copy and adapt.
3:17 Really nice. All right, so that wraps up testing.
3:19 We've seen the different levels, and the different ways
3:22 of wrapping the web framework infrastructure
3:25 around our tests, providing it to our tests
3:29 as well as replacing some
3:31 of the foundational items like data access.