MLOps basics | FSDL | Priority

Question 1

Write out a diagram of ML product engineering with a data flywheel.

mlops data-flywheel

Answer 1

(See source material.)

Source: Lecture 1: Course Vision and When to Use ML > But ML-powered products require an outer loop

Question 2

What are some of the items on a checklist to assess the feasibility of an ML project?

mlops ml-powered-products

Answer 2

(See source material.)

mlops ml-powered-products

Source: Lecture 1: Course Vision and When to Use ML > ML Feasibility Assessment

Question 3

Draw a diagram of the model-as-service pattern.

mlops ml-powered-products deployment

Answer 3

(See source material.)

mlops ml-powered-products deployment

Source: Lecture 5: Deployment

Question 4

What are a few of the pros and cons of the model-as-service pattern?

mlops ml-powered-products deployment

Answer 4

(See source material.)

mlops ml-powered-products deployment

Source: Lecture 5: Deployment

Question 5

Explain the various forms of parallelism in distributed training.

mlops training

Answer 5

1. Trivial parallelism: model and data fit on single gpus.
2. Data parallelism: model fits on a single gpu but data is spread across gpus; average gradients are computed by the model across gpus
3. Model parallelism
   a. Sharded data parallelism
   i. DeepSpeed, FairScale, fully-sharded data-parallel (pytorch)
   ii. shards the optimizer states, the gradients, and the model parameters.
   b. Pipelined model parallelism
   i. put each layer of your model on each GPU.
   c. Tensor parallelism
   i. distribute the matrix over multiple GPUs

Source

Question 6

What takes up gpu memory?

mlops training

Answer 6

Model parameters
Gradients
Optimizer states (statistics about gradients)
Batch of data

Source

Question 7

Using a cloud provider, how can you minimize costs?

mlops training

Answer 7

Use the most expensive per-hour GPU in the least expensive cloud.
Startups (e.g., Paperspace) tend to be cheaper than major cloud providers.

Source