I just did a quick search and couldn't find any court cases directly on point. Three possible arguments come to mind:
Alternative 1: By making your file available via the Web, you implicitly consented to having anyone download it who wants to do so, even via hot-linking, therefore there's no infringement.
Alternative 2: It's established custom that the implicit consent applies only to people who are downloading the whole page, not individual images --- therefore you didn't implicitly consent to hot-linking, therefore whoever downloads your content that way is an infringer, therefore the hot-linker is liable for contributory infringement or perhaps inducement of infringement.
Alternative 3: The person who created a page including hot-linked images, etc., has thereby created an unauthorized, and therefore infringing, derivative work [1]. (This assumes the hot-linked content is copyrightable.)
Alternative 1: By making your file available via the Web, you implicitly consented to having anyone download it who wants to do so, even via hot-linking, therefore there's no infringement.
Alternative 2: It's established custom that the implicit consent applies only to people who are downloading the whole page, not individual images --- therefore you didn't implicitly consent to hot-linking, therefore whoever downloads your content that way is an infringer, therefore the hot-linker is liable for contributory infringement or perhaps inducement of infringement.
Alternative 3: The person who created a page including hot-linked images, etc., has thereby created an unauthorized, and therefore infringing, derivative work [1]. (This assumes the hot-linked content is copyrightable.)
[1] http://en.wikipedia.org/wiki/Derivative_work#Definition