Modern Python Projects Transcripts
Chapter: Managing Python project
0:00 How do we pin our dependencies?
0:02 Do we go and do this by hand?
0:05 And then each time a new version is released,
0:07 we go back to the requirements.txt and update all those versions by hand.
0:12 No, that would be a terrible waste of time.
0:14 There are automatic tools that can do this for us.
0:17 One of them is called pip tools,
0:19 and that's the one that I use.
0:21 Most of the time. pip tools is a combination of to different tools,
0:25 one called pip-compiled and another one called pip-sync.
0:29 pip-compile is the important one, So let's talk about this one first.
0:33 Basically, what pip compile does is that it takes an input file with dependencies,
0:38 and it generates a list of pinned dependencies for you.
0:42 This input file can be either a setup.py.
0:45 or requirements file, notice that the requirements in the input file can be as specific as
0:50 you like. You can pin some dependencies,
0:52 but you don't have to. You can simply provide the name of a package,
0:56 and it will work fine. When you run,
0:58 pip-compile command pip tools will take that input file and generate a new list of requirements
1:03 this time with each dependency pinned to the more specific version possible.
1:09 Let's see this in action. Let's use a simple requirements.in file with just two
1:14 packages. I want to use Django,
1:16 but it has to be version at least 2.2 and less than 3.
1:21 And I want to use the latest version of pytest, if we run.
1:25 pip compile requirements.in, we get a new file called requirements.txt this
1:31 time with all the dependencies of Django and pytest and their sub dependencies.
1:36 So every single package it's pinned to the most specific version that is available.
1:41 As you can see, most packages use semantic versioning with three numbers.
1:46 But for example, pytz is using a different format,
1:49 and pip tools detected that, also as an additional benefit,
1:53 we can see in the comments from where each package comes from.
1:57 We see that pytest comes from the requirements.in file, packaging library comes from
2:02 pytest and finally, pyparsing,
2:05 comes from the packaging library. So pip tools has pinned all the possible dependencies and if
2:11 two different packages required the same dependency like here,
2:15 the importlib-metadata is required by pluggy and pytest, pip tools will try to find
2:21 a version that satisfies both requirements.
2:23 If it can't find such version,
2:25 it will report an error. And that's it.
2:29 Now you take this requirements.txt file,
2:31 and you tell pip to use it.
2:34 That's the requirements file that you will use on your production server and on your test
2:38 server, to make sure that everything works fine.
2:41 If in the future you see that the new version of Django was released,
2:45 you can just rerun, pip-compile,
2:47 and it will create a new set of dependencies.
2:50 It's a good idea to run pip-compile as part of your continuous integration pipeline.
2:55 That way, your dependencies will be up to date with the latest patch fixes.
2:59 But at the same time, your test will be run on those new dependencies, so
3:03 you can see if something breaks.
3:06 And if something breaks because let's say the latest version of Django has a bug,
3:10 you can go back to the requirements.in file, and pin Django version,
3:15 so you won't use the latest version until you notice that the bug was fixed.
3:20 This probably won't happen often, especially for such a popular package like Django.
3:25 But smaller python packages can sometimes have bugs in the latest version,
3:29 so keep that in mind. pip tools comes with another command called pip-sync.
3:35 You can use it to synchronize which dependencies are installed in your virtual environment,
3:40 with the list of dependencies in the requirements.txt file. Running pip-sync is equivalent
3:46 to uninstalling all the pip packages and installing them again from the requirements file.
3:51 So, this tool is just for convenience and can save you some typing,
3:55 but I usually don't use it.
3:58 If you want to see some more examples of how to structure requirements,
4:02 check out the GitHub repository of pip tools.
4:06 It has a lot of examples of how you can use it.