MongoDB for Developers with Python Transcripts
Chapter: Mapping classes to MongoDB with the ODM MongoEngine
Lecture: A parting glitch
0:01 Let's look at one final thing, I think it is not beautiful
0:03 but knowing about it and expecting it is really, really important,
0:07 not in the beginning, but as you evolve your application,
0:10 you'll end up with some funky complications.
0:13 I actually chose the cars that I wanted to update very, very carefully,
0:17 let me run this again, if I list the cars, notice here in particular state
0:22 far away from this Ferrari F40 from 2005, there's only one of those, right,
0:27 notice the id it's d15 and ends in 7e, if I try to list the cars again
0:31 there's that one but oh, it ends in ae, list the cars again—
0:36 now it's ending in a1, what is up with that?
0:43 Just to be clear, the other ones are not changing,
0:45 like 0f that's always the value, for the first one 0f,
0:51 there's not a problem with Mongo or anything like that,
0:54 what is going on here is this car was inserted into the database
0:59 when we just had a little bit of our class to find here,
1:02 remember in the beginning, we didn't have this default concept
1:06 when I first introduced it, and somewhere along the way
1:09 after we had inserted a few cars, then we added this,
1:12 let's look at Robomongo.
1:17 If you flip through here, you'll see almost all the cars have a vi number,
1:20 vehicle id number, vehicle id number
1:24 except for that F40 from 2005, right at the top— none.
1:29 Because when it was inserted, there was no definition for a vi number
1:34 what the heck was that thing anyway,
1:36 how was it supposed to know that was not here yet but would eventually exist;
1:39 so we can do a couple of things,
1:41 the reason you keep seeing this changing numbers is that there's a default value
1:45 and basically it gets created every time
1:50 it comes back from the serialization layer,
1:53 but it doesn't get set from the database
1:55 because there is nothing to set it to,
1:57 so every time it goes back, it reruns that lambda
1:59 and gets a new value and we're not saving it.
2:02 So basically what we need to do is we need to upgrade our documents,
2:04 now sometimes like I said, this doesn't matter,
2:07 but this one where we kind of counted on a default value to be there,
2:10 and then it wasn't, well that's unfortunate.
2:13 So here's something, there's a couple of things we could do,
2:20 and apply that, that's one option,
2:22 another option is let's go here and let's write make sure
2:25 we just below configure Mongo I'll say update_doc_versions, or something like this
2:32 define that function here, and what we can do is something like this
2:37 it's not exactly in a work, but it'll give you the idea what we're after;
2:41 so we'll say for car in car.objects so basically
2:44 let's look through everything in this collection and let's save it back,
2:48 I'm going to run this, and let's list the cars and see if we solved it,
2:53 I really wish it wasn't at the top but there is, 19, 02
3:02 what's going on, well, it turns out that unless we somehow forcibly change this object
3:09 it's like hey, this object is not dirty and we don't need to push it in there.
3:16 And say mark as changed, vi number and let's try it again—
3:26 here we go, so we told it that it changed
3:30 sadly even though it generates a default value
3:32 it doesn't look back into the database which I guess would be super expensive,
3:37 it just says hey someone changed this, right
3:39 it didn't really trigger that, it came from the databases none and then we set it,
3:43 so it doesn't know to push that through, so you have to do this little bit of a trick here
3:46 to say mark has changed, and PyCharm says you're not supposed to do that,
3:52 let's just tell it hey, don't make me look bad,
3:55 we have to do this you understand it, right?
3:57 Very well, okay, so now we've got this save back to the database,
4:01 we only want to run this the one time, right, we don't want to run at all the time,
4:05 this is just like a one time upgrade of our documents
4:08 and if you have a 100 thousand records, probably fine,
4:11 if you have a billion records this is not how you want to do it,
4:14 you want to do some kind of in place updater or something better, write the script,
4:18 so let's run this again, and now we should see our car here,
4:23 this is the 2005 F40, you know what,
4:26 it is time for new tires, let's service this puppy.
4:30 Now we come down here and say I don't know how much new tires are in Ferrari,
4:33 but let's say they're 2000, new tires,
4:35 the customer is pretty happy they had the low profile ones they were looking for
4:39 but they could have been like a certain kind, who knows, whatever.
4:43 Perfect, that worked, now if we list our cars again,
4:46 you can see that this one that was basically, we couldn't get to
4:50 because its vin number kept changing is now fixed
4:54 and that's because when we reran that we said hey,
4:58 force of the default of the vin number in there,
5:01 notice that none of these other ones changed,
5:03 it did write them back to the database, but it wrote them back
5:06 in exactly the way they were before, so nothing changed there,
5:09 I'll just run it one more time, this second one is,
5:12 I'll just copy this and we can go pull it back in a second,
5:14 so if we put this back one more time, and then we try to service this car
5:23 one dollar test service they were pretty thrilled with that
5:30 here you can see the test service, okay.
5:37 So the ids are not changing when we do this,
5:39 it's just if they're not there they are created.
5:42 Alright, so if that seemed kind of annoying, I'm sure it was annoying,
5:46 but let's think how this would be in a relational database,
5:50 what would have happened if this was a say SQLAlchemy or something to that effect,
5:55 or if this was some other thing, we would have a lot more trouble
5:58 evolving from one to the next, right,
6:01 so we wouldn't have the problem of
6:03 hey here's a car that doesn't have a vin number,
6:05 because if we didn't actually go and manually changed the database
6:10 every time we added something here, when we added this
6:13 we would have had to go back to the database
6:17 and do like a migration or data transformation SQL query
6:20 to actually change and add this column, same thing for this,
6:23 but none of that was required, it was just this one case
6:26 where we went back in time that we had to do a little bit of work here.
6:29 So, sometimes you still have these scripts you've got to run,
6:32 sometimes you still have these changes, you got to do
6:35 and consider the version history,
6:37 but it's much much less often than with relational databases
6:41 where every little change requires a script
6:43 or it's just oops things are out of sync, bam we can no longer work,
6:46 but I did want to point that out to you that look,
6:49 you're going to have to be really careful, some of the time.
6:52 as these things evolve, how are you going to deal with the fact
6:54 that in the database there is this thing that has no vehicle id number.
6:58 If we're using PyMongo, it would have just come up as none
7:02 or key error or something like that, it would have been a little more obvious
7:06 but that's just one of the trade-offs you get with these ODMs.