A cool idea, way overpriced in my opinion, though. $24-$149 a month for "you moderate" packages? I could roll my own in almost any programming language in 1/10th the time it would take me to integrate with their service.
The MTurk services are a little more reasonably priced, 2.9 cents a comment is about what I'd offer the users directly.
I have my doubts about whether the Mturk payments are going to attract moderators capable of handling more than the 'LOL I TROLLED U' comments, though. It seems like more of an anti-spam/swearing tool than anything likely to elevate the quality of discourse; that's useful enough in and of itself for quite a few sites though.
I do think it's fairly pricey. 15k messages/mo ain't much on a busy site. Some political blogs rack that up in a couple of days, but I don't see them putting down several thousand a month for crowd moderation.
Spam is automated, trolling isn't. So it makes sense that spam detection is easier to automate than troll detection.
A different way to say just about the same thing: Trolling can be customized to a specific context, it can rely on sarcasm or social references, etc. I don't think there exists any NLP system that can pick up e.g. written sarcasm.
Spam is not automated anymore. Automated spam doesn't get through modern countermeasures such as human-detection and bayesian filtering, so it is not worth considering. The spam that you actually see today on most sites is hand-pasted by real human beings (at least the stuff that makes it onto my site is. I'm sure you can still find plenty of phpbb sites that are overrun with the old sort).
It's hard to do anything about because they'll simply strip the content of an article that would fit well on your site, and stick a link at the bottom. Or maybe they'll skip the link for the first dozen blog entries until they think they've gained your trust.
Real spam (versus the wikispam v1agra stuff that gets automatically swatted down) is tricky. And these people are a lot more motivated than trolls, since they get paid when they get a working link on your site.
I've actually seen my share of this type of spam as well. Lucky for us all of it came through Indian IPs, while our users were all from the US.
They are really getting creative. They WILL post several messages to "gain your trust" before they throw in a link. I can't believe all this effort is really worth it for whomever is sponsoring this "human spam".
I was thinking more along the lines of an aide to existing human moderators. If you start with a clean slate for each particular website and then use the moderation queue information to slowly train the system. Maybe, just maybe that would be enough.
If not, then why not take the context into account as well? Context based troll filter. Whenever a troll posts something there is usually a resulting discussing with a healthy dose of profanity. We can all just glance at such messages and say with complete certainty that it brings no value, certainly that means there are existing patterns.
The biggest risk in this is the stick situation you come to when somebody who feels that a legitimate comment was moderated and feels the need to make a fuss about it. It's a little hard to come up with a thoughtful response to that. "Well, my moderators made the decision." "Who are they?" "I don't know them."
Kudos.