What is the Sum rule?
P(A v B) = P(A) + P(B) - P(A ∧ B)
Prob. of A or B is equal to prob. of A plus prob. of B, minus prob. of A and B
What is the chain rule?
P(A ∧ B) = P(A|B) * P(B)
Probability of A and B is equal to the probability of A given B multiplied by prob. of B
What is the formula for P(A|B)?
P(A|B) = P(A ∧ B) / P(B)
How do we make predictions with multiple pieces of data?
- De couple the multiple pieces of data
- Treat each of them as ‘independent’
What is the formula for naive bayes?
P(h | multiple data) = P(data(1) | h) *P(data(2) | h) … * P(data(n)) * P(h)
What do naive Bayes classifiers assume?
They assume that the value of any particular feature is independent of any other feature
Conditional Independence
What is the issue if one of the features has probability of 0, and what do we do about it?
Then the entire function evaluates to 0.
We apply some addition of a small ( < 0.000001) value to the ones with probability of 0
What are some other ways to use naive bayes?
- Part of a KNN classifier
- Use the priors in membership calculations
What are some applications of naive bayes?
- Used as a standard to compare other algorithms to
- Spam filtering
- Text classification
What happens with values in real-valued domains?
Assume that P(X|Y) lies on a normal gaussian distribution
How can the gaussian distribution be described?
It can be described with a probability density function
What is the gaussian PDF?
p(x) = (1 / ( sq.rt. ( 2pi * std.dev^2) ) )^e^( -0.5 ( ( x - mean ) / variance) ) ^2
When and how is naive Bayes flawed?
It is flawed when two datasets overlap to a large extent.
Because this means that the mean and std. dev. would be similar, giving us a similar gaussian distribution.
What does a bayesian belief network represent?
The state of some model.
They describe how states are related by their probabilities
How can any system be modelled by a BBN?
All the possible states of the model are the possible worlds that can exist, or how all the possible ways that the parts of the system can be configured
What are some examples of BBNs?
- Stock market
- House diagnostics
- Car diagnostics
- Ecosystem simulation
What does a BBN contain?
A set of interconnected nodes, where each node represents a variable in the dependency model
A set of edges, where each edge represents relationships between variables
What are the advantages of Bayes learning?
- No explicit model - easy to modify ‘on the fly’
- Works well on small amounts of data
- Naive Bayes classifiers tend to be robust
- Generally underfit
- Naive Bayes models are efficient
- Easy to implement
What are the disadvantages of Bayes learning?
- Handling of continuous data is bad
- Not everything can be represented by a
Gaussian
- Not everything can be represented by a
- Conditional independence assumption in
Naive Bayes can be restrictive - Sparse data can be problematic
