Fundamentals of Dask Transcripts
Chapter: Dask Delayed
Lecture: Recap: Delayed
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
All right, everyone. Now it's time to jump into our Jupyter Notebook in Jupyter lab to check out Dask Delayed.
0:06
So I just want to recap a few things about Dask Delayed as I've said and as you know, from the first course, it can be used to parallelize,
0:13
regular Python code important to recognize that it's evaluated lazily. What that means is that you need to call '.compute ( )' in order to get the
0:21
computation to be evaluated. And and that's important cause you don't want all like these big computations to run unless you explicitly stated.
0:29
And we can also generate a 'task graph', which will see, So these are functions that we created in course one as well
0:35
Once an increment function which takes X and adds one to it, the other takes two arguments X and Y.
0:42
And it adds them together. And what we did is initially we put a little
0:46
'sleep( )' into these functions of one second for pedagogical purpose is to show you that when you parallelize the code, essentially,
0:54
that the time is reduced to doing it serially. Executed that code. Now, what we do is we import delayed,
1:02
which allows us to pass the functions that we want to delay as the first argument
1:07
to delayed and then pass to that the argument that we want to pass into the function. Okay, so we execute that Now.
1:16
It returns a Delayed object, it hasn't been computed yet. We're going to compute it now,
1:20
what do we expect? We increment 10 twice and add them together. So that's 11 plus 11. Is it 22? Is what we expect? Okay, that's a good sanity check.
1:30
Now let's visualize the task graph that was created here to really make explicit what did
1:36
Dask Delayed has done. So 'z.visualize( )' what it has done is it does these increments in parallel and then adds them together.
1:47
Okay, so we'll be back in a second to parallelize further Python code with Dask Delay.