Modern Python Projects Transcripts
Chapter: Managing Python project
0:00 How do we pin our dependencies? Do we go and do this by hand? And then each time a new version is released,
0:08 we go back to the requirements.txt and update all those versions by hand. No, that would be a terrible waste of time.
0:15 There are automatic tools that can do this for us. One of them is called pip tools, and that's the one that I use.
0:22 Most of the time. pip tools is a combination of to different tools, one called pip-compiled and another one called pip-sync.
0:30 pip-compile is the important one, So let's talk about this one first.
0:34 Basically, what pip compile does is that it takes an input file with dependencies, and it generates a list of pinned dependencies for you.
0:43 This input file can be either a setup.py. or requirements file, notice that the requirements in the input file can be as specific as
0:51 you like. You can pin some dependencies, but you don't have to. You can simply provide the name of a package, and it will work fine. When you run,
0:59 pip-compile command pip tools will take that input file and generate a new list of requirements
1:04 this time with each dependency pinned to the more specific version possible.
1:10 Let's see this in action. Let's use a simple requirements.in file with just two packages. I want to use Django,
1:17 but it has to be version at least 2.2 and less than 3. And I want to use the latest version of pytest, if we run.
1:26 pip compile requirements.in, we get a new file called requirements.txt this
1:32 time with all the dependencies of Django and pytest and their sub dependencies.
1:37 So every single package it's pinned to the most specific version that is available.
1:42 As you can see, most packages use semantic versioning with three numbers. But for example, pytz is using a different format,
1:50 and pip tools detected that, also as an additional benefit, we can see in the comments from where each package comes from.
1:58 We see that pytest comes from the requirements.in file, packaging library comes from pytest and finally, pyparsing,
2:06 comes from the packaging library. So pip tools has pinned all the possible dependencies and if
2:12 two different packages required the same dependency like here, the importlib-metadata is required by pluggy and pytest, pip tools will try to find
2:22 a version that satisfies both requirements. If it can't find such version, it will report an error. And that's it.
2:30 Now you take this requirements.txt file, and you tell pip to use it.
2:35 That's the requirements file that you will use on your production server and on your test server, to make sure that everything works fine.
2:42 If in the future you see that the new version of Django was released, you can just rerun, pip-compile, and it will create a new set of dependencies.
2:51 It's a good idea to run pip-compile as part of your continuous integration pipeline.
2:56 That way, your dependencies will be up to date with the latest patch fixes. But at the same time, your test will be run on those new dependencies, so
3:04 you can see if something breaks. And if something breaks because let's say the latest version of Django has a bug,
3:11 you can go back to the requirements.in file, and pin Django version, so you won't use the latest version until you notice that the bug was fixed.
3:21 This probably won't happen often, especially for such a popular package like Django.
3:26 But smaller Python packages can sometimes have bugs in the latest version, so keep that in mind. pip tools comes with another command called pip-sync.
3:36 You can use it to synchronize which dependencies are installed in your virtual environment,
3:41 with the list of dependencies in the requirements.txt file. Running pip-sync is equivalent
3:47 to uninstalling all the pip packages and installing them again from the requirements file.
3:52 So, this tool is just for convenience and can save you some typing, but I usually don't use it.
3:59 If you want to see some more examples of how to structure requirements, check out the GitHub repository of pip tools.
4:07 It has a lot of examples of how you can use it.