Getting started with pytest Transcripts
Chapter: pytest Plugins
Lecture: Randomizing test order with pytest-random
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
The last plug in. I want to show you is pytest randomly. I'm gonna demonstrate the pytest randomly because by using just four tests that run in
0:11
order 1234. If we run it, we're in the right directory pytest -V test order and we get these in
0:22
order 1234. And in these there's not much going on here and running them in
0:28
a different order wouldn't make much difference because they're not doing anything. But what if we have an order dependency that we're not aware of.
0:34
Here's an obvious order dependency. We've got test one that assigns a global value to X of X to one and then still one, make sure that it's one, three,
0:47
changes it to three and then still three makes it still three. Now this is intentionally obviously order dependent and it passes like this.
0:57
But if we run them in a different order or maybe in parallel there might be problems. So let's run them in parallel for instance,
1:10
this is a problem to test for order dependencies. One of the best plug ins I like is pytest randomly. So let's pip install pytest randomly.
1:24
And now let's run our test again. So I'm not gonna do it in parallel. Just run it once and now they are in a different order.
1:34
Said one, still three, still one in the three different order. What happens if we run it again? Let's turn off, trace back equals No,
1:43
so it's it's reordering them each time, which is kind of cool. We see the order dependency causes these tests to mess up. And this is, you know,
1:52
clear that this is gonna be a problem within this test, but a lot of times it's not it's not so obvious.
1:57
You've got, you're not going to have the state here, it's going to be state of the system changes with a different order.
2:03
So I really love pytest randomly to run your tests in random order just to make sure that things are okay. One of things I haven't noted yet,
2:13
which is a cool thing is pytest will list the plugins that you have installed in the header, which is really pretty cool.
2:21
And pytest randomly will use the random seed and it shows this. So it picks a different seed each time we got 13, still three and one and if we want,
2:33
I really, if this is the order that causes the problem, you can copy that and then some paste it that work right, then we'll still see the same order.
2:45
It doesn't really randomize that again, it uses the same same seed. So really handy to repeat if you're using randomly and you get a failure in
2:55
a certain order and you really want to run that order again and use the randomly seed which is very handy.