Modern Python Projects Transcripts
Chapter: Managing Python project
Lecture: pip-tools
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
How do we pin our dependencies? Do we go and do this by hand? And then each time a new version is released,
0:08
we go back to the requirements.txt and update all those versions by hand. No, that would be a terrible waste of time.
0:15
There are automatic tools that can do this for us. One of them is called pip tools, and that's the one that I use.
0:22
Most of the time. pip tools is a combination of to different tools, one called pip-compiled and another one called pip-sync.
0:30
pip-compile is the important one, So let's talk about this one first.
0:34
Basically, what pip compile does is that it takes an input file with dependencies, and it generates a list of pinned dependencies for you.
0:43
This input file can be either a setup.py. or requirements file, notice that the requirements in the input file can be as specific as
0:51
you like. You can pin some dependencies, but you don't have to. You can simply provide the name of a package, and it will work fine. When you run,
0:59
pip-compile command pip tools will take that input file and generate a new list of requirements
1:04
this time with each dependency pinned to the more specific version possible.
1:10
Let's see this in action. Let's use a simple requirements.in file with just two packages. I want to use Django,
1:17
but it has to be version at least 2.2 and less than 3. And I want to use the latest version of pytest, if we run.
1:26
pip compile requirements.in, we get a new file called requirements.txt this
1:32
time with all the dependencies of Django and pytest and their sub dependencies.
1:37
So every single package it's pinned to the most specific version that is available.
1:42
As you can see, most packages use semantic versioning with three numbers. But for example, pytz is using a different format,
1:50
and pip tools detected that, also as an additional benefit, we can see in the comments from where each package comes from.
1:58
We see that pytest comes from the requirements.in file, packaging library comes from pytest and finally, pyparsing,
2:06
comes from the packaging library. So pip tools has pinned all the possible dependencies and if
2:12
two different packages required the same dependency like here, the importlib-metadata is required by pluggy and pytest, pip tools will try to find
2:22
a version that satisfies both requirements. If it can't find such version, it will report an error. And that's it.
2:30
Now you take this requirements.txt file, and you tell pip to use it.
2:35
That's the requirements file that you will use on your production server and on your test server, to make sure that everything works fine.
2:42
If in the future you see that the new version of Django was released, you can just rerun, pip-compile, and it will create a new set of dependencies.
2:51
It's a good idea to run pip-compile as part of your continuous integration pipeline.
2:56
That way, your dependencies will be up to date with the latest patch fixes. But at the same time, your test will be run on those new dependencies, so
3:04
you can see if something breaks. And if something breaks because let's say the latest version of Django has a bug,
3:11
you can go back to the requirements.in file, and pin Django version, so you won't use the latest version until you notice that the bug was fixed.
3:21
This probably won't happen often, especially for such a popular package like Django.
3:26
But smaller Python packages can sometimes have bugs in the latest version, so keep that in mind. pip tools comes with another command called pip-sync.
3:36
You can use it to synchronize which dependencies are installed in your virtual environment,
3:41
with the list of dependencies in the requirements.txt file. Running pip-sync is equivalent
3:47
to uninstalling all the pip packages and installing them again from the requirements file.
3:52
So, this tool is just for convenience and can save you some typing, but I usually don't use it.
3:59
If you want to see some more examples of how to structure requirements, check out the GitHub repository of pip tools.
4:07
It has a lot of examples of how you can use it.