In terms of math, you will want to have at least a decent understanding of derivatives and integrals, as well as "basics" like logarithms/exponentiation, trigonometry and sums.

While it goes through vectors and such from the bottom up, it does so quite quickly so will probably be a bit steep if you haven't had any linear algebra from before.

Partial derivatives show up a lot, probably good if one has some experience from before.

Overall though the book doesn't assume a lot as I recall, and goes through the constructions from a fairly basic level and up.

Computer Graphics: Principles and Practice is a fairly reasonable substitute for an introductory graphics course, and having a good understanding of vectors and matrices. Other than that all you really need is the willingness to go and find resources when you don’t feel you fully grasp something.

You might also want to try this sort of thing in a study group. You definitely get more out of exercises if you have people to discuss them with.

I've only read the first edition, but I'm a bit surprised to read that as I don't recall them assuming any knowledge, except maybe some basic geometry and vector math.

I think there has always been some assumption that you know a bit about raster images, and colour and compositing. I remember it being the type of thing that anybody who has been involved in graphics code probably already knows so well that it’s second nature, but I also know these are things not every programmer has internalised.