Matrices represent linear transformations of spaces as described in the article. Not all matrices are well-behaved however. We'll talk first about diagonalizable matrices.
A diagonalizable matrix is a matrix which functions much like a diagonal one—one which is non-zero only on its diagonal. To consider a small example, multiplying an arbitrary 3-dimensional vector by a 3x3 diagonal matrix
What we observe is that a diagonal matrix affects each element of the vector independently. Since a vector represented as a list of numbers merely reads out how "far along" the vector is with each basis vector of the space, this means that a diagonal matrix merely "stretches" vectors along the space's bases.
It's important to remember that vectors do not have a canonical representation. The vector [1 2 3] might be equivalent to [3 2 1]---we could simply be changing the bases, in this case by rearrangement. With this intuition in hand, (most) diagonalizable matrices are merely diagonal matrices in some other basis.
A diagonal matrix M is thus one such that there exists two matrices A and P such that M = P A inv(P) where A is diagonal and P corresponds to a "rotation". In other words, to act upon a vector x by M, to compute Mx, we first rotate with inv(P), apply a basis scaling, and then rotate back with P---we've effectively "stretched" the vector along a different basis.
If you think carefully about it, vectors which already lie along that "other" basis will be purely scaled, affected by only one component of the diagonal matrix. This satisfies the definition of eigenvector and the corresponding diagonal entry of A is the eigenvalue.
---
So far this has been a description of "nice" matrices---nice linear transforms which can be fully described by scaling (or reflecting, consider a negative eigenvalue) along some choice of basis which may or may not be the one we're currently writing our vectors in. Matrices which are not "nice" are called "defective". We might call non-square matrices defective (since a diagonal along a rectangle doesn't "go all the way").
Defective matrices still have eigenvalues and eigenvectors, but they are non-unique or incomplete in some way that makes the transformation "confusable". In particular, there may be multiple decompositions M = P A inv(P) which all work. Or none at all. As a very simple example, consider
technically (0, 0) isn't an eigenvector---for this matrix there is only one eigenvector and so it can't "scale" in two dimensions as it would need to in order to fit the intuition laid out above.
---
Defective matrices are rare. If you pick a random matrix it is highly unlikely (almost impossible) to be defective. Instead, you can think of these are ones where things line up "just right" to eliminate some information.
Geometrically, what occurs is that a defective matrix "shears" the space. Since we cannot describe a shearing motion in terms of a rotation+scaling+reflection then we no longer get the simple eigenvalue picture above.
---
It's worth noting that you can go quite far without leaving the world of matrices which have nice or mostly nice diagonalizations. As stated, random matrices are nearly always diagonalizable. You're more likely to see them in graph theory where structures of certain graphs induce "shearing".
That said, defective matrices can be nice still. For instance, diagonalizability is not necessary for invertibility---shearing transformations can be invertible. That's still a very "nice" matrix.
To consider this further, think about what happens to a diagonalizable matrix if you take one of the eigenvalues to 0. Suddenly, one "axis" of the stretch merely compresses all information away. We're now non-invertible.
I really like your explanation of diagonalizable matrices! I've always seen explanations of how to determine if a matrix is diagonalizable, but hadn't run across the idea that they are just scaled diagonal matrices for a different basis.
A diagonalizable matrix is a matrix which functions much like a diagonal one—one which is non-zero only on its diagonal. To consider a small example, multiplying an arbitrary 3-dimensional vector by a 3x3 diagonal matrix
What we observe is that a diagonal matrix affects each element of the vector independently. Since a vector represented as a list of numbers merely reads out how "far along" the vector is with each basis vector of the space, this means that a diagonal matrix merely "stretches" vectors along the space's bases.It's important to remember that vectors do not have a canonical representation. The vector [1 2 3] might be equivalent to [3 2 1]---we could simply be changing the bases, in this case by rearrangement. With this intuition in hand, (most) diagonalizable matrices are merely diagonal matrices in some other basis.
A diagonal matrix M is thus one such that there exists two matrices A and P such that M = P A inv(P) where A is diagonal and P corresponds to a "rotation". In other words, to act upon a vector x by M, to compute Mx, we first rotate with inv(P), apply a basis scaling, and then rotate back with P---we've effectively "stretched" the vector along a different basis.
If you think carefully about it, vectors which already lie along that "other" basis will be purely scaled, affected by only one component of the diagonal matrix. This satisfies the definition of eigenvector and the corresponding diagonal entry of A is the eigenvalue.
---
So far this has been a description of "nice" matrices---nice linear transforms which can be fully described by scaling (or reflecting, consider a negative eigenvalue) along some choice of basis which may or may not be the one we're currently writing our vectors in. Matrices which are not "nice" are called "defective". We might call non-square matrices defective (since a diagonal along a rectangle doesn't "go all the way").
Defective matrices still have eigenvalues and eigenvectors, but they are non-unique or incomplete in some way that makes the transformation "confusable". In particular, there may be multiple decompositions M = P A inv(P) which all work. Or none at all. As a very simple example, consider
http://www.wolframalpha.com/input/?i=eigensystem+of+%7B%7B1%...
technically (0, 0) isn't an eigenvector---for this matrix there is only one eigenvector and so it can't "scale" in two dimensions as it would need to in order to fit the intuition laid out above.
---
Defective matrices are rare. If you pick a random matrix it is highly unlikely (almost impossible) to be defective. Instead, you can think of these are ones where things line up "just right" to eliminate some information.
Geometrically, what occurs is that a defective matrix "shears" the space. Since we cannot describe a shearing motion in terms of a rotation+scaling+reflection then we no longer get the simple eigenvalue picture above.
---
It's worth noting that you can go quite far without leaving the world of matrices which have nice or mostly nice diagonalizations. As stated, random matrices are nearly always diagonalizable. You're more likely to see them in graph theory where structures of certain graphs induce "shearing".
That said, defective matrices can be nice still. For instance, diagonalizability is not necessary for invertibility---shearing transformations can be invertible. That's still a very "nice" matrix.
To consider this further, think about what happens to a diagonalizable matrix if you take one of the eigenvalues to 0. Suddenly, one "axis" of the stretch merely compresses all information away. We're now non-invertible.