Modern Python Projects Transcripts
Lecture: GitLab CI in action
0:00 This time we'll set up a GitLab CI for our calculator project.
0:05 So again, I have a GitHub repository with just four files and we don't
0:09 have any GitLab CI setup, One way we could set it up
0:13 is to click this set up.
0:15 CI/Cd I see the icon and this will generate a GitLab CI yaml file
0:19 which is the configuration file.
0:22 For GitHub CI, by default,
0:25 it's empty, but we can search for python here,
0:29 and this will add a lot of different options.
0:31 So as you can see, we could use latest python version.
0:35 Define the cash for pip,so it will speed up our build,
0:39 then install virtual environment, set it up,
0:42 source it, install Tox and,
0:46 flake 8, run Tox under python 3.6.
0:52 Then we could build a binary well distribution of our application,
0:56 Which makes sense if you're building up pypi, package but doesn't really make sense if
1:01 we just have a few python scripts and then in the final step,
1:05 we will be building as sphinx documentation.
1:07 So all that is cool, but we don't really need most of those steps We
1:11 don't need a binary distribution because we are not building python package and we don't need
1:17 these sphinx documentation because, well,
1:19 way don't really want sphinx documentation for now.
1:22 So, instead of using this, let's go back to our code and let's write a
1:26 simple configuration for gitlab ci from scratch.
1:29 So I'm going to discard those changes.
1:35 And let's go to the code editor, here
1:38 We need to create a new file.
1:43 I think that's the correct name.
1:45 And now let's copy and paste some code here so I can save you from watching
1:50 me typing all those stuff. Okay,
1:52 so what's happening here? First,
1:54 we define which image we want to use.
1:57 I want to, python 3.8,
1:59 because that's the same image I used on Git Hub.
2:02 And then I defined three stages first build, then static analysis and then test,
2:08 and each stage can contain different steps.
2:11 So first step is install and this belongs to the stage called Build,
2:15 and here will simply up make sure that we have updated pip version and will install for
2:20 flake 8 black and pytest this first step with updating pip.
2:24 It's not really necessary because we are using python image,
2:28 and it should have the latest possible pip version.
2:32 So you might need this step if you are using.
2:34 Let's say Ubuntu image,
2:36 but I still have it here to show you how you can execute multiple commands per
2:40 step. Our state called build will just install all the dependencies.
2:45 And next we have stage called static analysis that has two steps.
2:50 First we want to run Flake 8 and then we want to run back pretty simple
2:55 And when all that is successful,
2:58 we want to run our tests.
3:01 So we define pytest, step belonging to stage test,
3:05 and here we just run pytest.
3:08 Okay, so let's commit this file and see what happens.
3:22 As you can see, we have this icon here,
3:24 which means that there is a pipeline running.
3:28 So Gitlab has detected that we have a CI configuration and it's trying to
3:32 use it. And here we have a nice UI that shows different stages and
3:37 different steps. For each stage.
3:39 We can go inside the install to see what's happening here.
3:43 The first run is usually the slowest because it's not using any cache,
3:47 and we have this message saying that the job was successful.
3:50 So let's go back and let's refresh.
3:57 All right, so this is green now.
3:59 Those two are running in parallel.
4:01 They should be very fast. And we have a problem because black is not found
4:07 Actually, both steps should fail.
4:12 Yeah, and as you can see,
4:14 pytest was not run. So what went wrong here is that I forgot that whatever
4:19 we build in this build stage is not going to be preserved for the static analysis
4:24 or for the test stages. So we have to modify our configuration file.
4:32 We have to take those two lines,
4:34 and we can either put them at the beginning of each script.
4:38 Or we can add a config option called before_script and add them there.
4:44 So let's go for that. So whatever you put inside this before,
4:54 script, will be executed before each of the script.
4:58 So before flake 8, before black and before pytest.
5:01 And that way we can actually get rid of this install step and the stage build
5:06 because we are not using it anymore.
5:17 So, one more time, we have another build running,
5:42 and this time we no longer have this build stage.
5:45 Let's check out black. Great, This time the job was successful.
6:08 Let's check out the other job.
6:11 Not here, here. So both Black and Flake 8 were successful.
6:23 And now the pytest is running.
6:26 So again, GitLab CI is pulling the Python 3.8 image.
6:31 Then it will run Pip, Install,
6:33 Flake 8 Black and PyTest,
6:35 and then it's going to run our tests.
6:38 This pipeline is quite inefficient because we don't need flake 8 and black in our test
6:43 stage, but it's much easier to write it like that,
6:51 and the job was successful. If we go back to the main page,
6:55 we can see that now we have this green mark, saying that the pipeline was successful
7:00 great. So, as always,
7:03 let's go back to our code and let's try to break something
7:07 First We're going to mess up with black
7:11 Again I remove some white spaces, and that's actually break the test.
7:16 While we're here, let's switch to a new branch and let's create a new merge
7:21 request. If we refresh,
7:35 we get this, create merge request button and we can submit it.
7:47 Now we can see there is a pipeline running. So first the static analysis stage is running
7:52 If we go here, we can,
7:55 for example, go to flake 8.
7:56 Actually, the Flake 8 will be successful.
8:01 So let's go to Black. And go to jobs and here is the black and
8:11 as you can see, it failed.
8:12 Let's open the pipeline. Uh,
8:15 it's actually flake 8 that failed,
8:17 not black. That's interesting. So Flake 8 is failing because of the same reasons that
8:30 Black would complain, its expecting to have to blank lines.
8:33 But we have zero. So let's fix only this part.
8:44 Okay, so our static analysis tools should be happy.
8:47 But we still have this failed test,
8:49 so that should fail. Let's add a new commit to our branch.
8:59 Okay, I made a type is going to bother me when we push
9:08 This will start another pipeline. Let's go back to our merge request up and we
9:17 have the pipeline running. So let's wait for a moment.
9:28 All right, so the static analysis stage is happy,
9:31 but our tests are failing. So finally,
9:34 let's go back and fix them.
9:38 So we have 5 + 15.
9:40 That's 20, 20 - 15. That should be 5.
10:06 Okay, One last run of our pipeline,
10:08 but this time everything should be fine.
10:11 And now everything is green, so we can go back to the merge request and
10:15 actually merge it. This pipeline that we just build,
10:33 is very simple. But if you're looking for some more complex examples,
10:37 you can go to the GitLab CI examples page and set for a python.
10:44 So, here we have python on Heroku,
10:46 and here we have a standard python example.
10:49 So, the first one can be used to automatically.
10:52 Build and deploy your code to Heroku.
10:57 And the second one is actually the same example that we have when we try to
11:02 create a CI configuration from the Web interface.
11:06 So, as you can see,
11:07 it will run, test using Tox and then generate documentation with sphinx for every commit
11:13 that you push on the master branch,
11:15 you can use a setting option called artifacts to preserve some files after the pipeline is
11:20 finished. In this case, I want to keep this sphinx documentation that we just generated inside the public folder.