Path4.Mod1.a - Training Models with Scripts - Run a Training Script as a Command Job

Question 1

Rem Ref T

Three actions for creating a production-ready script

Answer 1

• Remove nonessential code: ex. print statements for stuff you don’t care about in production logs
• Refactor your code to functions: testability, readability
• Test your script in a terminal: Either in the Notebooks page in the studio, choose save and run in the terminal, OR in the Compute Instance select Terminal from the apps then run the script manually from there.

Question 2

cod com env com d_n e_n

Parameters to Configure a Command Job, then code to create it

Answer 2

• code: folder path containing the script
• command: specify the script to run as a command line
• environment: necessary packages to install on the Compute Instance before running the Command. Can be an environment image or a file
• compute: Compute Instance to use for running the script/job by name
• display_name: the job name
• experiment_name

You need to create the Command instance, then Create the job via mlclient.create_or_update():

from azure.ai.ml import command
job = command(
    code="./src",
    command="python my_python_script.py --var1 'helloworld'",
    environment="AzureML-sklearn-0.24-somesystem-cpu@latest",
    compute="aml-cluster",
    display_name="train_my_model_job",
    experiment_name="error_code_model_training")
returned_job = ml_client.create_or_update(job)

// or alternatively returned_job = ml_client.jobs.create_or_update(job)

Question 3

Using parameters in your script when running a Command Job

Answer 3

Since your command is a script, just make sure your script can take parameters, then define those parameters in your Command Job’s command parameter (DUR!!! Commandline bro!):

command="python my_python_script.py --my-training-data.csv --skip-unknown --output-to-log",

Question 4

How to process parameters into your script in the first place

Answer 4

To actually process params in your script, using a library like argparse:

import argparse
import pandas as pd

// Read data as a Pandas Dataframe
// from the argument "training_data"
def get_dataframe(args):
    df = pd.read_csv(args.training_data)

// Create an instance of an ArgumentParser
// Add arguments then parse them
def parse_args():
    parser = argparse.ArgumentParser()
    parser.add_argument("--training_data", dest='training_data', type=str)
    args = parser.parse_args()

    return args

if \_\_name\_\_ == "\_\_main\_\_":
    args = parse_args()
    get_dataframe(args)

Question 5

Converting a Jupyter Notebook into a script in ML Studio

Answer 5

Authoring > Notebooks > Open your Notebook > Hamburger menu at the top left > Export > select Python (.py)

Question 6

Jupyter Notebooks are ideal for experimentation, scripts are better for production workloads (T/F)

Answer 6

TRUE

Question 7

When converting Jupyter Notebook to a script, ML Studio gives you an interface to provide params after it parses your script for inputs (T/F)

Answer 7

FALSE. When converting to a file, your code is just dumped into the file. It’s up to you to refactor it

Question 8

All jobs with the same experiment name will be grouped under the same experiment name (T/F)

Answer 8

TRUE. See image…

Question 10

Only outputs of a Command Job are tracked, and can be reviewed after the job has completed (T/F)

Answer 10

FALSE. Azure ML will track both inputs AND outputs