Maybe the X-factor is simply that having more stimulating experiences and environments makes us more enriched as individuals? I.e, instead of life experiences consisting of three parts - toddler, school , work for 50 years, one gets to add a fourth component and another identity building phaze between school and work. I could believe that.

Although, with this argumentation, one could simply change university with some other novel experience, like military service.