Write Pythonic Code Like a Seasoned Developer Transcripts
Chapter: Methods and Functions
Lecture: Beware: The danger of mutable default arguments
0:01 We've seen the power of default values and methods. But they can have a dark side that you need to watch out for.
0:06 It's not hard to avoid, but you need to be aware of it. Let's see this first in code then we'll come back and look at the graphic.
0:12 Over here I have a method. It does a couple of things and it kind of gives us a hint on what's going on by naming the methods "bad" and "good",
0:19 if we have an add_items_bad and add_items_good, now the good part is not yet good, so don't expect it to behave well.
0:26 Now let's just look at "bad" real quick. What we'd like to be able to do is come over here and pass in a name,
0:32 some number of times and add that name to the list however many times we set. Again, this is a very cheesy example but it's pretty simple
0:40 and it will make it totally understandable what's happening. So we want the behavior to be like this, either you provide us a list that already exists
0:49 and then we'll add to that list or if you don't give us a list, we'll create a brand new list for you using a default value here
0:56 and we'll add to that and we'll pass it back. Sounds like it's going to be great, right? So, let's come over here and comment some of these out,
1:05 so we are going to start out and we are going to add the item "a" 3 times. So our list here should be "a, a, a,"
1:13 and then we are going to call this again and we are going to pass "a" in first we are going to use the default value
1:17 then we are going to pass it and it should onto that list add two more "b"s, so in the end, our list here should be "a, a, a, b, b."
1:27 Let's see that that's the case. First time "a, a, a" was created as part of the default value, and then "a, a, a, b, b."
1:36 OK, so this is what you would expect, there is nothing weird going on here, let's continue though, and here you'll see something go completely wrong.
1:44 So this first time our expectation was: we don't supply list, great, the method will create it for us and give it to us, populated.
1:51 Here, we are going to do the same thing, we worked with "a", we are now done with it, we want to create a new one called "d", "d" for danger,
1:57 we are going to create a new one called "d" and we are going to add "d" to it 4 times, so what you would expect is basically that, these characters.
2:07 But you are going to see that that is not what we get, let's try it. We get "a, a, a", "a, a, a, b, b", right?
2:15 Now this is what we expected but what the heck is that? Well, if I had to guess,
2:22 I would say it reused the list a that it gave us on line 2 and in fact, that's exactly what happened,
2:28 we can even come down here we can say let's compare, if I say I'd like to print out the id of "a", basically the pointer address, and the id(d),
2:38 if those are the same value, we have a problem, we can even print "id(a) == id(b)" because it's going to be kind of a big number,
2:46 who wants to compare those. So our expectation was every time we did a call like this we got a new list, empty, that was "d".
2:55 We got a new list, empty, and it would be filled up, otherwise we specify an existing one but that's not what happened.
3:02 Oh dear, those two lists are the same and of course that's why they look like we added the "d" to the first list,
3:10 so let's go look at the code down here and see why that happens. Well, it all has to do with when is this default value created,
3:18 is the default value created every time the function is called? Or is it created when the function is built - when the function is defined?
3:28 And you know, see PyCharm here it knows something is up it's coloring this in a way that says "no, no" this is probably not what you want,
3:34 so if we hover this it says the default argument is mutable and yet there is only one of them that ever exists,
3:41 so if it's only one of them and it can be changed every time the method is called, this is not a good situation.
3:49 So it's fine to have these singleton immutable things, but this one is mutable and this could be a custom type,
3:55 this could be all sorts of things, in this case it's just a list. So how do we deal with it, how do we get a add_items_good
4:02 so we don't run into this problem? Let's go and hit and just uncomment all these down here,
4:06 run it and you see we have the same problem, even put my test down here. Right, so you can see we have the same problem,
4:16 obviously because we have the same code, but let's fix it, so the way you fix it when you have a mutable type as a default value
4:22 is you create it every time so you need some kind of indicator that this is not set, so we are going to say None, we'll say "if list is None",
4:31 remember we check for None with "is", not "equals", although "equals" would work, this is better, we'll say "lst = new list",
4:40 so this achieves what we were hoping this would achieve for us up here but actually didn't.
4:44 Now, it's worth noting that we could say "if", you might be tempted to say "if not lst",
4:51 then we’re going to do this thing, but that might not be exactly what you want, if they were to pass the list and it was empty,
4:58 like this but they pass it in, this would still be False, remember the truthiness of sequences, so then we would recreate it
5:04 and we would add, possibly to a pointer, that they never grab back, because they thought they passed it in,
5:09 so we are not going to do "if not list", we'll do "if list is None", we'll recreate it.
5:12 Now, let's run the code, the bottom wants you to work the way we expected it, look, PyCharm is not angry with us any more,
5:19 because this is an immutable singleton, let's go. All right, so obviously bad news, perfect,
5:27 this is what we were looking for - "a" - this was a default one created, then we had some more values to it, to the existing one,
5:34 then we are going to call the function again with default, and we get brand new different list they are not the same.
5:40 So be aware of this possibility where you have mutable default arguments. Here I have colored list red because we are using a mutable default argument,
5:49 now what we have learned is that this default argument is created once per process,
5:54 basically, so this is a shared list for everyone who calls this function without passing a value for list but just rather uses the default,
6:02 this is not going to be the behavior you are looking for. So you saw the first two lines below this method here,
6:07 list one, this worked perfectly, then we tried to create a second list calling add_items_bad with the default value,
6:15 and we got some very bizzare behavior, in fact what we saw was list_2 and list_1 were in fact the same list.
6:23 So how do we fix it, we put some kind of indicator here that we need to create the list, in this case we said if they pass None for the list,
6:30 or they don't pass anything, we'll create the list for them, so we check if list is None, we create it,
6:36 and then that way every time we call this function we get a new individual dedicated default list.
6:44 Oh wait, there is one more thing, I almost forgot and this is really cool;
6:48 remember I said notice PyCharm sees the error and it highlights it and says "oh no, this is not going to work out for you probably,
6:56 this is not what you think it is", right? But I ignored this little part they said Alt+Enter fixes it,
7:01 so Alt+Enter in PyCharm will do all sorts of fix-ups and automatic transformations, if it knows there is something, a way you can do something better
7:10 you can hit Alt+Enter and often it will do it better. So I can come over here and hit Alt+Enter,
7:16 and it will say "we would like to replace the mutable default argument", boom, just like that. Alt+Enter, fixed.
7:23 And what did it do- it did exactly the thing we did, below. All right, that was the thing I forgot to show you.