Fundamentals of Dask Transcripts
Chapter: Dask Schedulers
Lecture: Distributed scheduler
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
As we mentioned earlier, we always recommend using the "Distributed scheduler" and we've been using it throughout this course already,
0:08
so it's the only scheduler that supports all the diagnostic dashboards and improved memory management
0:14
capabilities. It's also a separate sub project with a separate team of maintainers. So just a few points, the distributed scheduler,
0:23
as we have here in the notebook, will also work well for workloads on a single machine on top of that.
0:29
It is recommended for workloads that do not hold the GIL, such as (dask.bag) and custom code wrapped in (dask.delayed),
0:35
even recommended on a single machine on top of this, it's kind of more intelligent and provides better diagnostics than the processes scheduler,
0:44
It's absolutely required for scaling out work across a Cluster. Now let's not forget our Dask and distributed hygiene as we always close the 'Cluster'.