Which is weird. Whenever I'm downtown at night (Belltown not included) it seems like the only people there are older upper-middle class white people who are going out to dinner at fancy restaurants.
Sure, there are a handful of homeless people and people waiting for buses, but there's nothing too scary.
Posting as someone who's lived in the heart of Belltown for the last 3 years... even Belltown isn't that bad. Avoid 3rd, especially around and Bell Street and the worst you'll have to deal with is some rich yuppie trust fund kids from the east side. (which, arguably, is worse than what you'll find on 3rd.)
The difference between Seattle and other cities that are considered dangerous is that generally in dangerous cities it isn't a single demographic that feels uncomfortable in the allegedly dangerous parts of town. In actually dangerous cities, everybody does.
This said, Seattle's downtown does have a rather dead nightlife; you have to go to other neighborhoods for that, and it is something that Seattle should work on.
Suburban middle-aged middle-class white people. My condo building in Seattle has many middle-aged middle-class white people who are apparently quite comfortable living in the city.