I create recommender systems for a living. They are powerful and also potentially dangerous. But many people fall into the trap of thinking that just because a computer recommends something it’s objectively good.
It is math but it’s not “just math”. Pharmaceuticals is chemistry but it’s not “just chemistry”. And that is the framework I think we should be thinking about these with. Instagram doesn’t have a God-given right to flood teen girls’ feeds with anorexia-inducing media. The right is granted by people, and can be revoked.
> Because you can’t demand that anyone recommending anything know with certainty whether or not the content they are recommending is good or bad. That puts way too much of a burden on the recommender, and makes the mere process of recommending anything a legal minefield.
Let’s flag for a moment that this is a value judgement. The author is using “can’t” when they really mean “should not”. I also think it is a strawman to suggest anyone is requiring absolute certainty.
When dealing with baby food manufacturers, if their manufacturing process creates poisoned food, we hold the manufacturer liable. Someone might say it’s unfair to require that a food manufacturer guarantee none of their food will be poisoned, and yet we still have a functioning food industry.
> The whole point of a search recommendation is “the algorithm thinks these are the most relevant bits of content for you.”
Sure. But “relevant” is fuzzy and not quantifiable. Computers like things to be quantifiable. So instead we might use a proxy like click. Click will lead to boosting clickbait content. So maybe you include text match. Now you boost websites that are keyword stuffing.
If you continue down the path of maximal engagement somewhere down the line you end up with some kind of cesspool of clickbait and ragebait. But choosing to maximize engagement was itself a choice, it’s not objectively more relevant.
It is math but it’s not “just math”. Pharmaceuticals is chemistry but it’s not “just chemistry”. And that is the framework I think we should be thinking about these with. Instagram doesn’t have a God-given right to flood teen girls’ feeds with anorexia-inducing media. The right is granted by people, and can be revoked.
> Because you can’t demand that anyone recommending anything know with certainty whether or not the content they are recommending is good or bad. That puts way too much of a burden on the recommender, and makes the mere process of recommending anything a legal minefield.
Let’s flag for a moment that this is a value judgement. The author is using “can’t” when they really mean “should not”. I also think it is a strawman to suggest anyone is requiring absolute certainty.
When dealing with baby food manufacturers, if their manufacturing process creates poisoned food, we hold the manufacturer liable. Someone might say it’s unfair to require that a food manufacturer guarantee none of their food will be poisoned, and yet we still have a functioning food industry.
> The whole point of a search recommendation is “the algorithm thinks these are the most relevant bits of content for you.”
Sure. But “relevant” is fuzzy and not quantifiable. Computers like things to be quantifiable. So instead we might use a proxy like click. Click will lead to boosting clickbait content. So maybe you include text match. Now you boost websites that are keyword stuffing.
If you continue down the path of maximal engagement somewhere down the line you end up with some kind of cesspool of clickbait and ragebait. But choosing to maximize engagement was itself a choice, it’s not objectively more relevant.