The set of matrices as a vector space
The idea of a vector space can be extended to include objects that you would not initially consider to be ordinary vectors. Matrix spaces. Consider the set of 2 by 3 matrices with real entries. This set is closed under addition, since the sum of a pair of 2 by 3 matrices is again a 2 by 3 matrix, and when such a matrix is multiplied by a real scalar, the resulting matrix is in the set also. Since , with the usual algebraic operations, is closed under addition and scalar multiplication, it is a real Euclidean vector space. The objects in the vector space, the vectors, are now matrices. This proof is trivial and it is left as an exercise for the reader.
Since is a vector space, what is its dimension? First, note that any 2 by 3 matrix is a unique linear combination of the following six matrices:
Therefore, they span .
Furthermore, these elements of the vector space are linearly independent: none of these matrices is a linear combination of the others. (ie, the only way will give the 2 by 3 zero matrix is if each scalar coefficient, , in this combination is zero) These six elemments therefore form a basis for ,
If the entries in a given 2 by 3 matrix are written out in a single row (or column), the result is a vector in . For example,
The rule here is simple: Given a 2 by 3 matrix, form a 6-vector by writing the entries in the first row of the matrix followed by the entries in the second row. Then, to every matrix in there corresponds a unique vector in , and vice versa. This one-to-one correspondence:
is compatible with the vector space operations of addition and scalar multiplication. This means that
The conclusion is that the spaces and are structurally identical, that is, isomorphic, a fact which is denoted .
One consequence of this structural identity is that each basis of the vector space given above for corresponds to the standard basis vector
for .
The only real difference between the spaces and is in the notation: The six entries denoting an element in are written as a single row (or
column), while the six entries denoting an element in are written in two rows of three entries each.
This example can be generalized further.
Recommended for you
High Dimension Data Analysis - A tutorial and review for Dimensionality Reduction Techniques
High Dimension Data Analysis - A tutorial and review for Dimensionality Reduction Techniques
This article explains and provides a comparative study of a few techniques for dimensionality reduction. It dives into the mathematical explanation of several feature selection and feature transformation techniques, while also providing the algorithmic representation and implementation of some other techniques. Lastly, it also provides a very brief review of various other works done in this space.