It's not that functions are themselves state machines, but functional languages are sometimes implemented as state machines. E.g. the lambda calculus can be implemented on an abstract machine known as the Krivine machine [1].
Right, what would be the states in it? What would be the state-transitions? How many are there? How does it model a physical system with multiple states and state-transitions?
Can a functional program return to its previous state ever? If not I don't think it should be called "state-machine".
It can calculate what should be the next state of some state-machine, but that does not mean it is that "state-machine".
[1] https://en.wikipedia.org/wiki/Abstract_machine#Functional_pr...