Write Pythonic Code Like a Seasoned Developer Transcripts
Chapter: Methods and Functions
Lecture: Beware: The danger of mutable default arguments
0:01 We've seen the power of default values and methods.
0:03 But they can have a dark side that you need to watch out for.
0:05 It's not hard to avoid, but you need to be aware of it.
0:08 Let's see this first in code then we'll come back and look at the graphic.
0:11 Over here I have a method.
0:13 It does a couple of things and it kind of gives us a hint
0:15 on what's going on by naming the methods "bad" and "good",
0:18 if we have an add_items_bad and add_items_good,
0:22 now the good part is not yet good, so don't expect it to behave well.
0:25 Now let's just look at "bad" real quick.
0:28 What we'd like to be able to do is come over here and pass in a name,
0:31 some number of times and add that name to the list however many times we set.
0:36 Again, this is a very cheesy example but it's pretty simple
0:39 and it will make it totally understandable what's happening.
0:42 So we want the behavior to be like this,
0:44 either you provide us a list that already exists
0:48 and then we'll add to that list or if you don't give us a list,
0:51 we'll create a brand new list for you using a default value here
0:55 and we'll add to that and we'll pass it back.
0:57 Sounds like it's going to be great, right?
0:59 So, let's come over here and comment some of these out,
1:04 so we are going to start out and we are going to add the item "a" 3 times.
1:08 So our list here should be "a, a, a,"
1:12 and then we are going to call this again and we are going to pass "a" in
1:15 first we are going to use the default value
1:16 then we are going to pass it and it should onto that list add two more "b"s,
1:22 so in the end, our list here should be "a, a, a, b, b."
1:26 Let's see that that's the case.
1:29 First time "a, a, a" was created as part of the default value, and then "a, a, a, b, b."
1:35 OK, so this is what you would expect, there is nothing weird going on here,
1:39 let's continue though, and here you'll see something go completely wrong.
1:43 So this first time our expectation was: we don't supply list,
1:46 great, the method will create it for us and give it to us, populated.
1:50 Here, we are going to do the same thing, we worked with "a",
1:53 we are now done with it, we want to create a new one called "d", "d" for danger,
1:56 we are going to create a new one called "d" and we are going to add "d" to it 4 times,
2:00 so what you would expect is basically that, these characters.
2:06 But you are going to see that that is not what we get, let's try it.
2:11 We get "a, a, a", "a, a, a, b, b", right?
2:14 Now this is what we expected but what the heck is that?
2:18 Well, if I had to guess,
2:21 I would say it reused the list a that it gave us on line 2 and in fact,
2:25 that's exactly what happened,
2:27 we can even come down here we can say let's compare,
2:29 if I say I'd like to print out the id of "a", basically the pointer address, and the id(d),
2:37 if those are the same value, we have a problem,
2:40 we can even print "id(a) == id(b)" because it's going to be kind of a big number,
2:45 who wants to compare those.
2:47 So our expectation was every time we did a call like this we got a new list,
2:51 empty, that was "d".
2:54 We got a new list, empty, and it would be filled up,
2:57 otherwise we specify an existing one but that's not what happened.
3:01 Oh dear, those two lists are the same and of course
3:04 that's why they look like we added the "d" to the first list,
3:09 so let's go look at the code down here and see why that happens.
3:13 Well, it all has to do with when is this default value created,
3:17 is the default value created every time the function is called?
3:21 Or is it created when the function is built - when the function is defined?
3:27 And you know, see PyCharm here it knows something is up
3:30 it's coloring this in a way that says "no, no" this is probably not what you want,
3:33 so if we hover this it says the default argument is mutable
3:37 and yet there is only one of them that ever exists,
3:40 so if it's only one of them and it can be changed every time the method is called,
3:45 this is not a good situation.
3:48 So it's fine to have these singleton immutable things,
3:51 but this one is mutable and this could be a custom type,
3:54 this could be all sorts of things, in this case it's just a list.
3:57 So how do we deal with it, how do we get a add_items_good
4:01 so we don't run into this problem?
4:02 Let's go and hit and just uncomment all these down here,
4:05 run it and you see we have the same problem, even put my test down here.
4:12 Right, so you can see we have the same problem,
4:15 obviously because we have the same code, but let's fix it,
4:17 so the way you fix it when you have a mutable type as a default value
4:21 is you create it every time so you need some kind of indicator
4:24 that this is not set, so we are going to say None, we'll say "if list is None",
4:30 remember we check for None with "is",
4:32 not "equals", although "equals" would work, this is better,
4:35 we'll say "lst = new list",
4:39 so this achieves what we were hoping this would achieve
4:41 for us up here but actually didn't.
4:43 Now, it's worth noting that we could say "if", you might be tempted to say "if not lst",
4:50 then we’re going to do this thing, but that might not be exactly what you want,
4:54 if they were to pass the list and it was empty,
4:57 like this but they pass it in, this would still be False,
4:59 remember the truthiness of sequences, so then we would recreate it
5:03 and we would add, possibly to a pointer, that they never grab back,
5:06 because they thought they passed it in,
5:08 so we are not going to do "if not list", we'll do "if list is None", we'll recreate it.
5:11 Now, let's run the code, the bottom wants you to work the way we expected it,
5:15 look, PyCharm is not angry with us any more,
5:18 because this is an immutable singleton, let's go.
5:22 All right, so obviously bad news, perfect,
5:26 this is what we were looking for - "a" - this was a default one created,
5:30 then we had some more values to it, to the existing one,
5:33 then we are going to call the function again with default,
5:35 and we get brand new different list they are not the same.
5:39 So be aware of this possibility where you have mutable default arguments.
5:44 Here I have colored list red because we are using a mutable default argument,
5:48 now what we have learned is that this default argument is created once per process,
5:53 basically, so this is a shared list for everyone who calls this function
5:57 without passing a value for list but just rather uses the default,
6:01 this is not going to be the behavior you are looking for.
6:04 So you saw the first two lines below this method here,
6:06 list one, this worked perfectly, then we tried to create a second list
6:10 calling add_items_bad with the default value,
6:14 and we got some very bizzare behavior,
6:16 in fact what we saw was list_2 and list_1 were in fact the same list.
6:22 So how do we fix it, we put some kind of indicator here
6:25 that we need to create the list, in this case we said if they pass None for the list,
6:29 or they don't pass anything, we'll create the list for them,
6:32 so we check if list is None, we create it,
6:35 and then that way every time we call this function
6:37 we get a new individual dedicated default list.
6:43 Oh wait, there is one more thing, I almost forgot and this is really cool;
6:47 remember I said notice PyCharm sees the error and it highlights it and says "oh no,
6:52 this is not going to work out for you probably,
6:55 this is not what you think it is", right?
6:57 But I ignored this little part they said Alt+Enter fixes it,
7:00 so Alt+Enter in PyCharm will do all sorts of fix-ups and automatic transformations,
7:06 if it knows there is something, a way you can do something better
7:09 you can hit Alt+Enter and often it will do it better.
7:12 So I can come over here and hit Alt+Enter,
7:15 and it will say "we would like to replace the mutable default argument",
7:18 boom, just like that. Alt+Enter, fixed.
7:22 And what did it do- it did exactly the thing we did, below.
7:26 All right, that was the thing I forgot to show you.