If you want to generalize it even further you can have tensor calculus. An example is each point in a field is a 3x3 matrix with a special property of positive symmetric. A diffusion tensor MRI image has one at each voxel whereas a regular mri image might have a scalar value (grayscale). The principle eigenvector might represent the direction of a muscle fibre for that Voxel.

Taking the derivative of a matrix is equivalent to writing a system of differential equations.

Not really. A matrix derivative is just a derivative -- it doesn't have to be part of an equation.

The derivative of x² is 2x (where x is a scalar)

The derivative of vᵀv is 2v (where v is a vector)

There's no differential equation.

A matrix is an array of expressions. Its derivative is the matrix of derivatives of its expressions.

This is 0 for any matrix of constants, as you can see in the example.