Path4.Mod2.a - Training Models with Scripts - Track Model Training with Jobs using MLFlow

Question 1

Two options to track ML Jobs with MLFlow

Answer 1

Option 1: mlflow.autolog() - Enable autologging:
Option 2: mlflow.log_param/metric/artifact/etc...* - Use individual logging functions for specific libraries to track custom metrics per library

Question 2

mlf amlf

Two libraries you’ll need to (pip) install on your Compute for tracking with MLFlow

Answer 2

mlflow and azureml-mlflow:

name: mlflow-env
channels:
  - conda-forge
dependencies:
  - python=3.10
  - pip
  - pip:
    - numpy
    - pandas
    - scikit-learn
    - matplotlib
    - mlflow
    - azureml-mlflow

Question 3

When and where to enable MLFlow Autologgging,

Answer 3

At the beginning of your training script, before training code:

Question 4

Sav Sup

Two advantages of using MLFlow Autologging

Answer 4

• Saves you time and effort wrt logging important Model information, so you can track your model’s performance over time
• Supported by the majority of common libraries: scikit-learn, Tensorflow, LightGBM, Spark, XGBoost, PyTorch, etc.

Question 5

P => l _ p, M => l _ m, MA => l _ a

The three things MLFlow allows you to log, along with their corresponding Custom logging functions that fulfill the majority of logging use cases:
* Inputs vs Outputs
* Which keeps track of value history

Answer 5

• Parameters get logged with mlflow.log_param(): kv-pair for a single param, for inputs
• Metrics get logged with mlflow.log_metric(): kv-pair for a single numeric metric, for output. MLFlow will remember the value history for each metric for tracking purposes
• Model Artifacts get logged with mlflow.log_artifact(): Logs a file. Can use to save plots of the log as a image.

Question 6

Ov Me Im Ou+L

Where to view metrics again…

Answer 6

In a completed Job/Experiement details:
- Details tab - overall view with all logged params under Params
- Metrics tab - numeric metrics specific to the ML type you selected
- Images tab - plots and other metric charting
- Outputs+Logs tab - additional artifacts like model files are stored in the Model folder