! Hypothesis Testing

Question 1

Hypothesis Space

Answer 1

• set of all hypotheses that can be produced by learning algorithm
• set of all possible finite discrete functions: representable by decision tree(s)

Question 2

Hypothesis

Answer 2

= statement about a population parameter

Question 3

Null Hypothesis H0

Answer 3

= statement about population parameter assumed to be true unless there is convincing evidence to contrary

Question 3

Alternative Hypothesis Ha

Answer 3

= statement about population parameter contradictory to H0 & accepted only if there is convincing evidence

Question 4

Hypothesis Testing

Answer 4

= statistical procedure choose between H0 & Ha based on info in sample

Question 5

Result Options of Hypothesis Testing

Answer 5

• Reject H0 (= accept Ha)
• Fail to reject H0 (= fail to accept Ha) -> H0 cannot be proven to be true

Question 6

Statistically significant

Answer 6

= probability not due to chance

Question 7

p-value

Answer 7

• probability level
• defines when sample results = strong enough to reject H0
• Low p-value = H0 unlikely to be true

Question 8

Error Types

Answer 8

• Type 1: Ho is true but rejected (a = probability of this error)
• Type 2: H0 is false but not rejected

Question 9

Hypothesis Testing Procedure (p-value approach)

Answer 9

1. Identify H0 & H1
2. Identify test statistic & its distribution
3. Compute value of test statistic from data
4. Compute p-value
5. Compare p & ts: reject H0 if p <= a
6. Formulate decision

Question 10

Extrapolation

Answer 10

= drawing a conclusion about something beyond data range e.g. conclusion from a biased sample

Question 11

Sampling distribution

Answer 11

• distribution of sample statistics
• with mean approx. equal to mean in original distribution & sd known as standard error

Question 12

Major Error Sources

Answer 12

1. Sampling Error
2. Sampling Bias

Question 13

Sampling Error

Answer 13

• proportion of overall error attributable to sampling procedure
• how much sample estimates of varibales differ btw samples

Question 14

Sampling Bias

Answer 14

• Systematic favoring of certain outcomes due to the methods employed to obtain the sample (e.g. self-selection bias)