Getting started with Dask Transcripts
Chapter: Scaling Dask in the cloud
Lecture: Introducing Coiled cloud
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
So far, we have worked with a medium dataset that is too big to fit in RAM, but it fits on your local disk.
0:07
We have used Dask to parallelize computation locally and leverage all the cores of your local machine. But what if you want to scale further?
0:15
Maybe you don't have enough cores on your local machine, or not enough storage, or you just want to get your work done faster.
0:20
This is where you can start considering using the cloud. Like AWS, Azure, GCP and other providers. To show how to do it
0:28
with Dask, we will be using Coiled Cloud. This is not the only way to do cloud deployments with Dask, and this section is not mandatory to follow.
0:36
We're using Coiled Cloud here because honestly, it's the most convenient way to do this today.
0:41
It also offers a generous free tier so you can get started right away and don't feel bothered by paying money for cloud services.