The iPhone X fits perfect in my hand. I upgrade from iPhone 6 and didn't notice any difference in size. Actually use it first.
When I have to use a button to open a phone now it feels like using a fax machine. It's so incredibly sluggish, slow and clunky.
And, when I look at an iPhone with huge borders on the edge (iPhone X goes right to the edge) it gives me claustrophobic anxiety because it's so small.
People are so afraid of change. Aside from the price, there's really no reason not to upgrade.
Seriously I love the SE too. And I was coming from large screen Android world to it as my first iOS device. Maybe they'll revive it by the time mine is ready to die...
Design? Nah. Some of us like a phone that fits comfortably in our hand and pocket, and doesn't look like every other phone out there.