Configure compute for a job run - Azure Tutorial

- [Instructor] One of the more important things that happens when you're dealing with complex packaging in a machine learning operation is that you need to bootstrap your compute, right? If you're running PyTorch, you're running some kind of machine learning library and you don't have it on your compute node, it'll fail. So if you're running a job or an endpoint or notebook you're going to need to get that new library or new configuration onto your compute node. And so one of the ways you can do this for let's say a worker node or a notebook, et cetera, is that you can define a script inside of the compute configuration and attach that script so that it runs either every time the new machine launches or initially on creation. Another thing you can do is you can also customize an environment. So you can take a base Docker image and you can tweak it a little bit and then also do that for one of your worker nodes. So let's go ahead and take a look at how this would work inside of Azure ML Studio. So to start with, I've got a file here which is a config file that is going to allow us to install a package, NumPy package. It's going to activate an environment and then run this inside of a node. So I'm going to go ahead and download this zip. And then what I'll do next is that I will uncompress this thing and I'm going to grab it and I'm going to put this on my desktop and I can actually upload something right inside of this environment here. So all I would need to do in order to do that would be to add files and we'll go ahead and say upload. And let's go ahead and upload this file. So now that this is available, this config file, what I can do is go back to compute which we can get to right here and now if I want to launch a new compute node, whatever and we're going to go to advanced settings here and notice how we can say Provision with script, right. And so I can either select Creation Script or Startup Script. And this is really neat, is that because I uploaded it into the Notebook file share, right, I can just go here and now we've got that script enabled for the startup, and then I could proceed to go down the road and get this configured. So really the provision with setup script is a nice feature to configure the notebooks using scripts that you've written yourself. And you can either use the existing notebook file share or you can do the local file share as well.