Modern Python Projects Transcripts
Chapter: CI
Lecture: GitLab CI in action
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
This time we'll set up a GitLab CI for our calculator project. So again, I have a GitHub repository with just four files and we don't
0:10
have any GitLab CI setup, One way we could set it up is to click this set up. CI/Cd I see the icon and this will generate a GitLab CI yaml file
0:20
which is the configuration file. For GitHub CI, by default, it's empty, but we can search for Python here,
0:30
and this will add a lot of different options. So as you can see, we could use latest Python version.
0:36
Define the cash for pip,so it will speed up our build, then install virtual environment, set it up, source it, install Tox and,
0:47
flake 8, run Tox under Python 3.6. Then we could build a binary well distribution of our application,
0:57
Which makes sense if you're building up pypi, package but doesn't really make sense if we just have a few Python scripts and then in the final step,
1:06
we will be building as sphinx documentation. So all that is cool, but we don't really need most of those steps We
1:12
don't need a binary distribution because we are not building Python package and we don't need these sphinx documentation because, well,
1:20
way don't really want sphinx documentation for now. So, instead of using this, let's go back to our code and let's write a
1:27
simple configuration for gitlab ci from scratch. So I'm going to discard those changes. And let's go to the code editor, here
1:39
We need to create a new file. I think that's the correct name. And now let's copy and paste some code here so I can save you from watching
1:51
me typing all those stuff. Okay, so what's happening here? First, we define which image we want to use. I want to, Python 3.8,
2:00
because that's the same image I used on Git Hub. And then I defined three stages first build, then static analysis and then test,
2:09
and each stage can contain different steps. So first step is install and this belongs to the stage called Build,
2:16
and here will simply up make sure that we have updated pip version and will install for flake 8 black and pytest this first step with updating pip.
2:25
It's not really necessary because we are using Python image, and it should have the latest possible pip version.
2:33
So you might need this step if you are using. Let's say Ubuntu image, but I still have it here to show you how you can execute multiple commands per
2:41
step. Our state called build will just install all the dependencies. And next we have stage called static analysis that has two steps.
2:51
First we want to run Flake 8 and then we want to run back pretty simple And when all that is successful, we want to run our tests.
3:02
So we define pytest, step belonging to stage test, and here we just run pytest. Okay, so let's commit this file and see what happens.
3:23
As you can see, we have this icon here, which means that there is a pipeline running.
3:29
So Gitlab has detected that we have a CI configuration and it's trying to use it. And here we have a nice UI that shows different stages and
3:38
different steps. For each stage. We can go inside the install to see what's happening here.
3:44
The first run is usually the slowest because it's not using any cache, and we have this message saying that the job was successful.
3:51
So let's go back and let's refresh. All right, so this is green now. Those two are running in parallel.
4:02
They should be very fast. And we have a problem because black is not found Actually, both steps should fail. Yeah, and as you can see,
4:15
pytest was not run. So what went wrong here is that I forgot that whatever
4:20
we build in this build stage is not going to be preserved for the static analysis or for the test stages. So we have to modify our configuration file.
4:33
We have to take those two lines, and we can either put them at the beginning of each script.
4:39
Or we can add a config option called before_script and add them there. So let's go for that. So whatever you put inside this before,
4:55
script, will be executed before each of the script. So before flake 8, before black and before pytest.
5:02
And that way we can actually get rid of this install step and the stage build because we are not using it anymore.
5:18
So, one more time, we have another build running,
5:43
and this time we no longer have this build stage. Let's check out black. Great, This time the job was successful.
6:09
Let's check out the other job. Not here, here. So both Black and Flake 8 were successful. And now the pytest is running.
6:27
So again, GitLab CI is pulling the Python 3.8 image. Then it will run Pip, Install, Flake 8 Black and pytest, and then it's going to run our tests.
6:39
This pipeline is quite inefficient because we don't need flake 8 and black in our test stage, but it's much easier to write it like that,
6:52
and the job was successful. If we go back to the main page, we can see that now we have this green mark, saying that the pipeline was successful
7:01
great. So, as always, let's go back to our code and let's try to break something First We're going to mess up with black
7:12
Again I remove some white spaces, and that's actually break the test. While we're here, let's switch to a new branch and let's create a new merge
7:22
request. If we refresh, we get this, create merge request button and we can submit it.
7:48
Now we can see there is a pipeline running. So first the static analysis stage is running If we go here, we can, for example, go to flake 8.
7:57
Actually, the Flake 8 will be successful. So let's go to Black. And go to jobs and here is the black and as you can see, it failed.
8:13
Let's open the pipeline. it's actually flake 8 that failed, not black. That's interesting. So Flake 8 is failing because of the same reasons that
8:31
Black would complain, its expecting to have to blank lines. But we have zero. So let's fix only this part.
8:45
Okay, so our static analysis tools should be happy. But we still have this failed test, so that should fail. Let's add a new commit to our branch.
9:00
Okay, I made a type is going to bother me when we push This will start another pipeline. Let's go back to our merge request up and we
9:18
have the pipeline running. So let's wait for a moment. All right, so the static analysis stage is happy, but our tests are failing. So finally,
9:35
let's go back and fix them. So we have 5 + 15. That's 20, 20 - 15. That should be 5.
10:07
Okay, One last run of our pipeline, but this time everything should be fine. And now everything is green, so we can go back to the merge request and
10:16
actually merge it. This pipeline that we just build,
10:34
is very simple. But if you're looking for some more complex examples, you can go to the GitLab CI examples page and set for a Python.
10:45
So, here we have Python on Heroku, and here we have a standard Python example. So, the first one can be used to automatically.
10:53
Build and deploy your code to Heroku. And the second one is actually the same example that we have when we try to
11:03
create a CI configuration from the Web interface. So, as you can see,
11:08
it will run, test using Tox and then generate documentation with sphinx for every commit that you push on the master branch,
11:16
you can use a setting option called artifacts to preserve some files after the pipeline is
11:21
finished. In this case, I want to keep this sphinx documentation that we just generated inside the public folder.