I found my current job through a recruiter, I think I will find all subsequent jobs through one as well for this reason. Applying for a position is fundamentally an act of sales, so it behooves you to either be a salesman yourself or to hire someone to do it for you. Otherwise you're throwing yourself at the mercy of people's deep-seated fears and prejudices.
I think that recruiters do make sense (clearly). I hope that we can use our position as recruiters to be better than most companies at figuring out who's a good programmer, and better than most candidates at figuring out which companies are good places to work (we're candidate constrained, so we have no incentive to send people to bad companies)
True, but also consider that you're the product in this case, and the salesman (in many cases) doesn't care who he makes the sale to. It may be better to first research and shortlist companies that you want to target, and then find a recruiter who has them as a client? Never done that myself, so don't know how practical it might be.
Ultimately it comes down to how many expectations you're placing on the company you work for. If you're one of those who vets the company harder than the company vets prospects, then by all means cut out the middleman and do the legwork yourself. A hired gun is just going to get in the way.
If your job is just a vehicle to enable your lifestyle, like mine, then you'll be able to establish a set of criteria and only work with a recruiter that respects those criteria.
That's why it's helpful to find and maintain relationships with the good recruiters. Really good recruiters are a very rare extremely valuable thing. They know how to find companies that are a good match and understand the time investment on both sides on the hiring spectrum. They are also good at identifying qualifications and requirements to avoid wasting time when it shouldn't be.
Good recruiters very much care about their reputation, because a sale isn't always a sale when either side ends up unhappy.
Recruiters, in the States at least, typically work for large recruiting shops that have many relationships with many employers. They'll contact you to try to fit a particular job, but are more than willing to match you to other jobs if you don't like that one or if it doesn't fit.
I cycled through like three recruiters before finding one with a job I was interested in. If you really like working with a particular recruiter, what you might do is look for jobs yourself and then forward those listings on to the recruiter, so he can do his thing. It's not how most recruiters are used to operating, and it's not something I've done, but I think it could work based on my previous experiences.
In my experience, recruiters are far more 'loyal' to job hunters than they are to employers, at least in tech fields. Hunters have nothing but options in who they go through and how they conduct a search. Whereas the requirements of companies are usually fixed and often are completely irrational. It's often easier to work with and influence a job seeker than it is to talk sense into an employer.