Path4.Mod2.b - Training Models with Scripts - Code to support Model Tracking with Jobs using MLFlow

Question 1

le

Code that Lists all Experiments with MLFlow and what parameter to use if you want to limit those results.

Answer 1

Retrive N experiements or all by excluding the max_results parameter

Use mlflow.list_experiments. To limit, use the max_results parameter:

experiments = mlflow.list_experiments(max_results=10)
for exp in experiements:
    print(exp.name);

Question 2

sr

Code that retrieves runs within a single Experiment

Answer 2

Retrive N experiements or all by excluding the max_results parameter

Use mlflow.search_runs, pass it the specific Experiment ID:

mlflow.search_runs(exp.experment_id)

Question 3

se

Code that searches for all Experiments by a certain ViewType

The three ViewTypes

Answer 3

Retrive N experiements or all by excluding the max_results parameter

Use mlflow.search_experiments, pass it the specific ViewType enum value via view_type parameter:

from mlflow.entities import ViewType

experiments = mlflow.search_experiments(view_type=ViewType.ALL)
for exp in experiments:
    print(exp.name)

ViewType: ACTIVE_ONLY DELETED_ONLY ALL

Question 4

gebn

Code that gets an Experiment by name

Answer 4

Retrive N experiements or all by excluding the max_results parameter

Use mlflow.get_experiment_by_name (DUR!), pass it the desired Experiment’s name as a string:

exp = mlflow.get_experiment_by_name(experiment_name)
print(exp)

Question 5

The default field Experiments are ordered by on retrieval

Answer 5

start_time

Question 6

Code that customizes the order_by parameter for the mlflow.search_runs function

Answer 6

Give the order_by a query modifier:

mlflow.search_runs(exp.experiment_id, order_by=["attributes.start_time DESC"], max_results=2)

Question 7

order_by doesn’t support ordering expressions containing these three subexpressions, and the alternative if you want to sort those expressions

Answer 7

Expressions that contain wildcards, specfically metrics.*, params.* and tags.*

The alternative is to use the sort_values method from Pandas:

runs = mlflow.search_runs(experiment_ids=[ "1234-5678-90AB-CDEFG" ])
runs.sort_values("metrics.accuracy", ascending=False)

Question 8

Code that searches for runs based on Hyperparameter values using the mlflow.search_runs function

The only three operators supported when filtering params

Answer 8

Give the filter_string parameter the name and value of the target Hyperparameter in syntax similar to FluxQL:

mlflow.search_runs(
    exp.experiment_id,
    filter_string="params.num_boost_round='100'", max_results=2)

Operator support: = != like

Question 9

Code that queries for runs using the mlflow.search_runs function

Answer 9

Give the filter_string parameter a straight-up full query that looks similar to FluxQL:

query = "metrics.AUC > 0.8 and tags.model_type = 'LogisticRegression'"
mlflow.search_runs(exp.experiment_id, filter_string=query)