> Agree with their position 100%. Including the part where they blamed anyone bu... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		pseudalopex on Aug 10, 2023 \| parent \| context \| favorite \| on: CNET is deleting old articles to try to improve it... > Agree with their position 100%. Including the part where they blamed anyone but themselves for hiding old snapshots when robots.txt changed?

npunt on Aug 10, 2023 [–]

no I didn't see that in there, I just agree that robots.txt makes sense for web crawlers but not archival purposes

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact