HW 1 - Question 1. Three funds. Total of $12,000. Muni = 7%, CD = 8%, High Risk = 12%. Constraints: No more than 2K in High Risk Three times more in Muni than CD What are optimal investments for each fund?
Objective Function: Min - (r x+r y+r z)
x = muni, y = CD, z = High Risk
Constraints: z
Understanding the Least Squares (data fitting)
The best fit in the least squares sense minimizes the sum of squared residuals. Residuals are the difference between an observed value and the fitted value provided by the model. Hence - least squares solution is obtained by minimizing the sum of squared distances between the line and each point.
Normalized Vector
The normalized vector of X is a vector in the same direction but with norm (length) 1. It is denoted X^ and given by
X^^=(X)/(|X|),
where |X| is the norm of X. It is also called a unit vector.
What is Norm?
Norm of a mathematical object is a quantitly that in some (possibly abstract) sense describes the length, size and extent of the object.
Most commonly, the unqualified term “norm” refers to the flavor of vector norm technically known as the L2-norm. This norm is variously denoted ||x||2, ||x||, or |x|, and gives the length of an n-vector x=(x1,x2,…,xn). It can be computed as
|x| = sqrt(x12 + x22 +…+ xn2)
While Loop - Matlab
While loop runs until certain condition is met. While loops can continue indefinitely. Very useful if number of iterations are not known.
Sample Code: While – {Logical Expression}
{code}
end.
If the Logical expression is true, it will run the code. If the logical expression is not met, then the code will exit. In the code, something inside the “while” loop must change otherwise the loop will never exit or execute.
The symbols &, |, and ~ are the logical array operators AND, OR, and NOT.
Cardinality of Matrix
The Cardinality of a set is is a measure of the “number of elements of the set.” For example, the set A = {2, 4, 6} contains 3 elements, and therefore A has a cardinality of 3.
First Order Method for Sparse Covariance selection
maximize log det X − 〈Σ, X〉 − ρ Card(X)
**subject to ** αIn ≼ X ≼ βIn
Zeros in the inverse covariance matrix corresponds to conditionally independent variables in the model. This approach can be used to simultaneously determine a robust estimate of a covariance matrix.
In this paper, they are trying to strike a trade-off between maximality of the likelihood and the number of non-zero elements in the inverse covariance matrix. Their approch is particular useful at discovering structure - precisely conditional independence properties in data.
Trace of Matrix
Trace of a matrix is the sum of its diagonal elements
Tridiagonal matrix
a matrix that has nonzero elements only on the main diagonal, the first diagonal below this, and the first diagonal above the main diagonal.
Rank of a Matrix
The column rank of a matrix A is the maximum number of linearly independent column vectors of A. The row rank of A is the maximum number of linearly independent row vectors of A.
Kernel of a Matrix
In linear algebra, the kernel or null space (also nullspace) of a matrix A is the set of all vectors x for which Ax = 0. The kernel of a matrix with n columns is a linear subspace of n-dimensional Euclidean space. The dimension of the null space of A is called the nullity of A.
Orthogonal Matrix and Vector
Two vectors are orthogonal iff their scalar product is zero.
An Orthogonal matrix is if: UTU = I
Cauchy-Schwartz inequality:
xTy < = ||x||2 · ||y||2
Test Set and Training Set
- A test set is a set of data used in various areas of information science to assess the strength and utility of a predictive relationship.
- A test set is a set of data that is independent of the training data, but that follows the same probability distribution as the training data. If a model fit to the training set also fits the test set well, minimal overfitting has taken place. If the model fits the training set much better than it fits the test set, overfitting is likely the cause.
Regularization
- This refers to a process of introducing additional informatin in order to solve an ill-posed problem or to avoid overfitting. This information is usually of the form of penalty for complexity, such as restrictions for smoothness or bounds on the vector space norm.
- In stat and machine learning, regularization methods are used for model selection - in particular to prevent overfitting by penalizing models with extreme parameter values.
- L1 and L2 regularization - which can be added to learning algorithms that minimizes a loss function E(X,Y) by instead minimizing E(X,Y) + Alpha||w|| (where w is model’s weight vector)
