Source availability doesn't really feel relevant to a discussion about why most people don't use it. Most people don't know anything about open source.
It's extremely relevant, as the group of people who are most likely to hear about an alternative browser in the first place are going to be disproportionately favourable and interested in open source specifically.
So by not attracting folks who are interested in open source, the company now has an uphill struggle to market to people who are already bombarded with ads and marketing. And when they do reach someone, if that someone asks a tech-knowledgable friend about it they are more likely to get a response of apathy or negativity towards it because of the closed-source nature.
This is not to say it's impossible to overcome this hurdle, but it makes their job more difficult. And trying to compete with well-established, free products is already pretty hard to do in the first place.
I think it actually was a factor in adoption of Firefox. Enthusiasts switched in part because they understand what open-source means. Ordinary people started using it because their tech-savvy friends insisted it was better than IE, even if they didn’t understand exactly why.
