I guess you'll have to choose wisely your annotations sources... If you see too much spam on 1 source, just unsubscribe. As said earlier, it's not so different of reddit/HN discuussions abouut content. The publisher may not even be aware of the discussion.
Again, it's about discoverability. If browsers support this natively, visiting a website would put up a virtual "link" of some sort to see the discussions for every url, as you visit them. There's a difference between discussion exists somewhere, and discussion is tied to your website/url.
