It's a metaphor to help humans grasp it better. Space and time does exists physically.

Does a shortest path between you and your workplace exists that you can traverse? Is it a straight line? No, it's not, but there's one path that you as a being can walk along physically. The buildings, roads, obstacles force you to diverge from the ideal straight line. That's another analogy of what space & time is. Fabric is just shorter.