The biggest obstacle for me has been the weather. If it rains, you get soaked and all of the shit from the road flies onto your clothes. And during winter, the roads are so icy and uneven that I'm worried I go tumbling downhill and slide right under a car.
Helps that the bike path situation here is decent, where you're usually on a shared-use path with a curb, although you really need to pay attention when the bike path crosses a road. Some people just don't see you and others don't know that they actually need to yield to cyclists in quite a lot of intersections.
I've been commuting by bike daily for the last 20 years, and the only weather that I think is too unpleasant or dangerous to bike in is the same weather that cars shouldn't drive in, too (with one exception: wind. Lord, how I hate the wind).
But, as with all other outdoor activities, I dress for it. I wear raingear when it rains, etc. I've been through plenty of downpours and have never arrived at my destination soaked, once I take the raingear off.
Helps that the bike path situation here is decent, where you're usually on a shared-use path with a curb, although you really need to pay attention when the bike path crosses a road. Some people just don't see you and others don't know that they actually need to yield to cyclists in quite a lot of intersections.