It's funny that you mention this; I was about to type a similar comment! Thinking of the Fourier transform in terms of group theory seems like it would make it more complicated, but it actually makes the fundamental underlying concept simpler to understand.
One can perform the Fourier transform over an arbitrary, compact, non-commutative group G via f(g) = ∑ dᵏ⋅tr(f̂ᵏ⋅ρᵏ(g)), where the sum is from k = 0 to k = ∞, and k indexes the unitary, irreducible representations ρᵏ of G. dᵏ is the dimension of the kth representation. f̂ᵏ is the (matrix) Fourier coefficient of the kth irreducible representation and is computed as f̂ᵏ = ∫ f(g)⋅ρᵏ(g⁻¹) dμ(g), where μ is a Haar measure on G such that ∫ dμ(g) = 1. Note that since the group representations are unitary, ρᵏ(g⁻¹) = ρᵏ(g)ᴴ.
For commutative groups, all of the ρᵏ are one-dimensional, and so the sums and integrals are over scalar values. As you mention, for the circle group the above expression reduces to the "conventional" equation for the Fourier transform.
One can think of the group Fourier transform as decomposing a nonlinear function over a group into a linear combination of orthonormal functions such that cutting off the sum at the kth term provides the best MSE approximation to the function, i.e., f̂ᵏ = argmin ∫ [tr(ĉᵏ⋅ρᵏ(g)) - f(g)]² dμ(g), where the minimization is over ĉᵏ.
One can perform the Fourier transform over an arbitrary, compact, non-commutative group G via f(g) = ∑ dᵏ⋅tr(f̂ᵏ⋅ρᵏ(g)), where the sum is from k = 0 to k = ∞, and k indexes the unitary, irreducible representations ρᵏ of G. dᵏ is the dimension of the kth representation. f̂ᵏ is the (matrix) Fourier coefficient of the kth irreducible representation and is computed as f̂ᵏ = ∫ f(g)⋅ρᵏ(g⁻¹) dμ(g), where μ is a Haar measure on G such that ∫ dμ(g) = 1. Note that since the group representations are unitary, ρᵏ(g⁻¹) = ρᵏ(g)ᴴ.
For commutative groups, all of the ρᵏ are one-dimensional, and so the sums and integrals are over scalar values. As you mention, for the circle group the above expression reduces to the "conventional" equation for the Fourier transform.
One can think of the group Fourier transform as decomposing a nonlinear function over a group into a linear combination of orthonormal functions such that cutting off the sum at the kth term provides the best MSE approximation to the function, i.e., f̂ᵏ = argmin ∫ [tr(ĉᵏ⋅ρᵏ(g)) - f(g)]² dμ(g), where the minimization is over ĉᵏ.