I'm pretty sure Google is smart enough to recognize the main content of a page, ... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		crazygringo on Aug 10, 2023 \| parent \| context \| favorite \| on: CNET is deleting old articles to try to improve it... I'm pretty sure Google is smart enough to recognize the main content of a page, and ignore things like widgets and navigation. That's Search Engine 101.

snowwrestler on Aug 10, 2023 [–]

Yes, of course, but that analysis happens after the content has been visited by the bot. It’s still a visit, and still hits the “crawl budget.”

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact