Baselines

Question 1

Why are baseline important

Answer 1

You need to decide on a good baseline, so when you do the trading you will be able to decide if you need to focus on getting the results closer to the baseline or improving something else (ex overfitting…) because you are close enough.

Question 2

Where to look for baselines

Answer 2

1. External baselines:
   Business/engineering requirements
   published results
2. Internal baselines:
   Scripted baselines (openCV, tule based - so something that works without ml)
   Simple ml baseline (linear regression…)
   Human performance

Question 3

How to create a Human baseline

Answer 3

From low quality and easy to collect to the oposite:
Random people (Amazon turk)
Ensamble of random people 
Domain experts (eg doctors)
Deep domain experts (specialists)
Mixture of experts 

Spend more on labeling cases that the model is having hard time with