Here's the class description for MAT 202, "Linear Algebra with Applications" at Princeton.[1] Here's a problem set with answers.[2]
It's painful to read. Problem 2: "Find the matrix A describing the following linear transformation from R2 to R'2 : First rotate clockwise by π/6, then scale by a factor of 2, then reflect about the x2 axis." Anyone who programs video games recognizes that as a stack of transformations. You set up the matrix for the rotate, the scale, and the reflection (which is a scale of -1 on one axis). You multiply the matrices to get the result. This is a pain to do with pencil and paper. GPUs have hardware for it.
Books on graphics programming cover this problem. But they usually do it much better, with graphical examples.
The presentation here is done very abstractly, without any motivation.
Most of linear algebra has graphical representations. It takes a useful and relatively easy area of math and makes it harder.
This is apparently on purpose. The class description says:
The calculations are relatively simple once you understand what you need to compute, but it can take time to master the abstract concepts well enough to understand what that might be. The challenge in 201 (a calculus class) is usually how to finish a problem as the technical complications mount, whereas in 202 the challenge is often in seeing how to start the problem.
Right. In calculus, problems usually involve trying to integrate something, where you get stuck and need to transform the problem in some non-obvious way to make forward progress. It's puzzle-solving. In linear algebra, the actual operations are mostly matrix adds and multiplies. It's setting up the problem that's hard.
It's annoying to see this for a useful area of applied math. If you're headed for abstract algebra, where intuition breaks down, this approach might be useful. But if you want to use ordinary linear systems to get work done, which is common in engineering, it's not.
As a high-schooler I did a summer course at Columbia about "graphics programming". In the morning, the instructor would cover the very basics of the theory of what we were trying to model, from the physics of it to the math required to represent the physics. The point of it was that, starting from scratch, we would build our own little ray-tracer in C++.
The course lasted 4 weeks (and was intensive, we were in class 5hrs a day if memory serves me right). It may have been the most unique and interesting educational experience of my life. There was so much motivation to LEARN the math and physics, and I vividly remember going home and looking at objects, and thinking about the angles that the light would bounce off of objects, etc. Plus, there was something truly empowering with starting from a completely blank sheet of code and building something like this, fully understanding what each part of the code was doing.
It was this course that introduced me to vectors and vector operations, as well as matrices, before I fully dove into them in school. It was absolutely perfect. Ever since I've had a soft spot for linear algebra, and am often disappointed at how frequently I've encountered it completely in the abstract, when it may be one of the topics in math that you can most readily connect to interesting, non-trivial real world problems.
This is the only way I've ever learned anything mathematical, at least beyond school-level maths. I wish it were more common to find courses - even online courses - that 'motivate'[0] maths with engaging projects, which for most of us here would mean coding projects. (The same goes for mathsy areas of programming: information theory, the prereqs of compression or computer graphics, modular arithmetic, linear algebra, set theory, topology, etc.)
And it's not just us, either. So many people I speak to have had the same exact experience. I have mathematician friends who agree with this, based on their experience of teaching maths. I think mathematical pedagogy is just broken, with teachers getting high on the perceived difficulty of their special wisdom.
[0] ...to use the semi-slang my university lecturers used. In other words, give you a reason to want to learn it.
I share a similar experience. I have a course on Numerical Optimization, and I feel that I have learned more linear algebra, and matrix calculus in 2 weeks than I did my whole undergraduate course. I also delved a bit into differential geometry to argue the correctness of the implementation of my gradient functions using the finite difference method.
Honestly, this has been the most fun I have had in a course in a long time. It's difficult at times, and there are many things one needs to learn, but this is how learning should be done imho.
That's really cool. If you have any pointers to any resources along those lines, I'd be really interested.
And I absolutely agree. It's a tragedy that maths is taught in such a dry, unengaging way. I think far more people would enjoy it and excel at it if only it were taught in a way that motivated them. Your experience is one of a million demonstrations of that.
For what it's worth, there are a couple of Wikipedia pages which I found to be great jumping-off points - effectively 'sitemaps of mathematics' - to explore mathematics and what parts you may not know yet:
They also present 'ways of looking' at mathematics which I find can be eye-opening with any subject. I don't know if you're a lumper or a splitter (https://en.wikipedia.org/wiki/Lumpers_and_splitters), but I'm a lumper, so it helps to see connections and unities between things deep down. (Anyone who's ever read the annoyingly separate study areas of automata vs state machines, and the replicated abstractions and principles – despite them being literally the fucking same thing – will probably see what I mean.)
Out of a problem set of 12 problems, you picked the only one that is similar to games programming.
Problem 1 teaches about the rank of a linear transform giving different number of solutions.
Problem 2 from this single problem set is the only one you base a critique of an entire course upon. It is a simple 2D problem to teach that 2D problems often have nice matrix formulations.
Problem 3 is a 3D projection, also nice for geometric intuition.
Problem 4 is about rank, the image of a linear transform, and the kernel of a linear transform, extremely important concepts to go further in math, nearly zero use for games programming, and nearly as unused for engineering.
Problem 5 is about basis changes, also of great use in theory.
Problem 6 is about a least squares solution to a higher dimensional transform. Not used in games.
Problem 7 is about finding the determinant of a special arbitrary sized matrix. The determinant map is crazy important in higher math, and this is an entry to starting to think about that.
Problem 8 is about eigenvalues and diagonalization.
Problem 9 is about understanding the relationship between characteristic polynomials of linear transforms and how that relates to rank.
Problem 10 is about looking at quadratic forms as matrices, and being able to compute interesting things from this relation.
Problem 11 is about singular value decomposition, very useful in higher math, very useful in many engineering places, and I've never once seen it used for games.
Problem 12 is a matrix formulation of a dynamical system, and solving the resulting ODE.
The point of all 12 problems is to teach general linear algebra. The students are not being taught game programming. The book types you list teach pretty much none of this.
I took linear algebra the same semester as computer graphics - the first half of CG was all about 2D stuff - Bresenham's line-drawing algorithm, things like that. First half of LA was pretty much everything I needed to know to understand matrix operations in OpenGL, which was the second half of CG. Worked great! I still remember most of the relevant stuff 20 years later.
The second half of linear algebra was a bunch of stuff about eigenvectors and eigenvalues. We never were given an explanation why you'd care about them, and I still have no idea why you'd care about them.
Eigenvectors and eigenvalues are (among other things) connected to something called diagonalization. In math, this is incredibly useful because it lets you describe a matrix as being put together from simpler, easy-to-understand matrixes.
Eigenvectors and eigenvalues are also a major focus of quantum mechanics. For example, when you measure the energy level of an electron in a quantum mechanical system, the measurement itself is a linear operator on the underlying wavefunction (linear operator = think "like a matrix"). The eigenvalues are the different energy levels of the electron, and the eigenvectors are the wavefunctions which are states of the electron with the corresponding energy level. You can experimentally verify that the eigenvalues correspond to spectral lines (the rainbow you see when you look at the substance through diffraction grating).
There are a ton of other things going on with eigenvalues and eigenvectors, they're used all over the place. If you want to understand a Markov process, for example, it can be described in terms of linear equations, and if it has a steady-state, it's an eigenvector.
Decomposing things into eigenvectors turns what used to be a complicated problem of coupled variables (the matrix), into a list of simple single-variable problems. It turns a matrix equation into a list of scalar equations. I hope this helps :)
Eigenvectors tell you which vectors are taken to a scalar multiple of themselves via a linear map. Eigenvalues tell you what those scalars are. That means some subspace is fixed by your linear map.
I'll pile on another common application of eigenvectors and values that hasn't been mentioned: principal component analysis in statistics/machine learning.
> It's painful to read. Problem 2: "Find the matrix A describing the following linear transformation from R2 to R'2 : First rotate clockwise by π/6, then scale by a factor of 2, then reflect about the x2 axis." Anyone who programs video games recognizes that as a stack of transformations. You set up the matrix for the rotate, the scale, and the reflection (which is a scale of -1 on one axis). You multiply the matrices to get the result. This is a pain to do with pencil and paper. GPUs have hardware for it.
> Books on graphics programming cover this problem. But they usually do it much better, with graphical examples.
With all due respect to your impressive graphics programming experience, I think even you would have something to learn from this class in general and this problem specifically.
The columns of a matrix are where the basis vectors of the domain are mapped. As such, this problem can be done by keeping track of what these operations due to the vectors e_1 and e_2. In particular, it is easy to this problem in your head, never multiplying any matrices along the way.
It's worth pointing out that Mathematics is a "purist" scientific pursuit - it focuses on the theory for people for whom the theory is the point. The practical applications are for the engineers.
Arguably even computer science suffers from the same problem. Keeping your domain as an example, it will go one level higher - and teach you why this math is useful for i don't know rotating a 3d object in space - but not actually teach you any of the libraries (OpenGL, Direct3D, Unity, Unreal) to actually get it done.
In either case you come out with a degree with a lot of theoretical understanding, but you are useless to practical applications without further training.
This is by design, but it's debatable if it's a good design. Humanities does a better job of welcoming laypersons and giving them SOME basic awareness of history/politics/philosophy/psychology to survive in the world. Math and Computer Science do no such thing, and so we have people who have no idea how to do their taxes, or automate a basic repeatable process on their computer. Maybe we should fix this.
"Mathematics is a "purist" scientific pursuit - it focuses on the theory for people for whom the theory is the point. The practical applications are for the engineers"
Yes, and it tends to be taught that way. But this is "Linear Algebra with Applications", being offered to non-mathematicians. That should be taught less abstractly.
> If you're headed for abstract algebra, where intuition breaks down, this approach might be useful.
I don’t even buy that. Instead of trying to take away the examples and spatial content of a very concrete subject like basic linear algebra, much better would be to give people a lot of concrete (geometrical and otherwise) examples so that their algebra course is better grounded, before just dumping them into abstract definitions and expecting them to symbol-twiddle their way to solutions of unmotivated problems.
I don't understand what you don't like about that problem. It's not very computationally intensive. Basically it tests if you know what these matrices look like, if you know how matrix multiplication works, and if you know that matrix multiplication is composition. These are all important ideas to reinforce. There aren't graphical examples because it's a single problem on a list, not an explanation in a textbook chapter.
> Problem 2: "Find the matrix A describing the following linear transformation from R2 to R'2 : First rotate clockwise by π/6, then scale by a factor of 2, then reflect about the x2 axis."
> Anyone who programs video games recognizes that as a stack of transformations. You set up the matrix for the rotate, the scale, and the reflection (which is a scale of -1 on one axis). You multiply the matrices to get the result. This is a pain to do with pencil and paper. GPUs have hardware for it.
Did you... look at the actual transformations? They are incredibly simple; nothing about this problem is even minorly challenging to do with pencil and paper. The matrix for "scale by a factor of two" is [[2 0] [0 2]]. The matrix for "reflect about the x_2 axis" is [[1 0] [0 -1]]. Watch me compose them in my head: [[2 0] [0 -2]]
I agree that that appears to be bullshit. When I think back to my courses, I had one in ‘vectors and matrices’ which is roughly equivalent to early courses in what American universities call ‘Linear Algebra’. I think it included:
- a refresher on matrix multiplication and eigenvectors/eigenvalues for those who hadn’t learned them in high school
- inverting arbitrary matricies (done with the horrific formalism and rigour that first year undergraduates sometimes like to see)
- basis transformations
- special matrices (orthogonal, unitary, hermitian, etc)
- suffix notation (by far the best bit)
- bilinear/quadratic/sesquilinear forms
- tensor products/contractions/basis changes (but without using tensor notation—it was all done with suffix notation thankfully)
And an exam question might be like:
- invert or diagonalise a 3x3 matrix of nice numbers. Or do Gaussian elimination to a 4x4. (These would be warm-up/small questions). If you get eigenvalues, maybe make a geometric interpretation.
- do some algebra with some variables known to be some kind of special matrix to prove something. (I.e. apply a property of the special kind of matrix to do the algebra)
- do some suffix notation algebra (in particular using some kronecker deltas or antisymmetric tensors and associated identities)
- something about bilinear/quadratic/sesquilinear forms/algebra
They mostly tried to avoid bullshit exam questions that were just computation. There were some other opportunities to do a bit of matrix computation like solving/linearising/sketching certain ODE systems, or computing Jacobians.
In second year we had a course that was actually called ‘Linear Algebra’ which touched on (numeric) vectors and matrices but mostly talked about linear maps, had lines of juxtaposed symbols with juxtapositions having different meanings between different symbols, and mostly involved statements like ‘let V be a vector space over F and let e_1,e_2,…,e_n be a basis for V’. I think the exams did not involve any computation and maybe involved producing some of the proofs from the course or applying them to something.
> In linear algebra, the actual operations are mostly matrix adds and multiplies
I’d say this misses the point of the subject by a mile. Linear algebra is the study of finite dimensional linear maps. Saying it’s about “matrix adds and multiplies” is like saying the study of algorithms is about “filling arrays and comparing numbers”
I am sorry for the snark, but do you always judge the presentation of a course based on ten sample problems listed on its website? Because if you do then you probably would not take any of my courses: the publicly accessible old exams also do not generally explain the course material very well...
Sorry for the snark but the sample problems listed on the website are a huge part of how the prof is presenting the course to prospective students. Why wouldn't you use that to form judgements?
If I understand you correctly to be saying the way you are currently doing that will put people of your courses consider maybe whether that's one small part of your presentation of your work you could improve to really good effect!
It's painful to read. Problem 2: "Find the matrix A describing the following linear transformation from R2 to R'2 : First rotate clockwise by π/6, then scale by a factor of 2, then reflect about the x2 axis." Anyone who programs video games recognizes that as a stack of transformations. You set up the matrix for the rotate, the scale, and the reflection (which is a scale of -1 on one axis). You multiply the matrices to get the result. This is a pain to do with pencil and paper. GPUs have hardware for it.
Books on graphics programming cover this problem. But they usually do it much better, with graphical examples. The presentation here is done very abstractly, without any motivation. Most of linear algebra has graphical representations. It takes a useful and relatively easy area of math and makes it harder.
This is apparently on purpose. The class description says:
The calculations are relatively simple once you understand what you need to compute, but it can take time to master the abstract concepts well enough to understand what that might be. The challenge in 201 (a calculus class) is usually how to finish a problem as the technical complications mount, whereas in 202 the challenge is often in seeing how to start the problem.
Right. In calculus, problems usually involve trying to integrate something, where you get stuck and need to transform the problem in some non-obvious way to make forward progress. It's puzzle-solving. In linear algebra, the actual operations are mostly matrix adds and multiplies. It's setting up the problem that's hard.
It's annoying to see this for a useful area of applied math. If you're headed for abstract algebra, where intuition breaks down, this approach might be useful. But if you want to use ordinary linear systems to get work done, which is common in engineering, it's not.
[1] https://www.math.princeton.edu/undergraduate/placement/MAT20...
[2] https://www.math.princeton.edu/sites/default/files/2018-04/M...