Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Server load goes really high on that day, and if you get more popular, you'll need more servers and hence more money.


Maybe it wasn't clear, but in this case the load is still distributed evenly across all days. I always download on Monday, you always download on Tuesday. The author suggests that this is not desirable.

FTA,

The “Total” row looks very nice, it’s exactly what we would like to have. However, our download interval matches the number of days now, with the effect that people always download on the same weekday (and those currently downloading on a Monday will continue downloading on a Monday).


Use a CDN? It's not like the list is tailored to anybody.


Isn't that something that nginx/varnish should easily be able to handle? It is just a static file download after all...


CPU and bandwidth are entirely different issues. Sure, nginx can handle the processing. But do you have the piping to match?

A run of the mill dedicated server has a 100mbit uplink. Do the math. (Hint: it's easy to saturate in no time).



This is just downloading a single static text file so there's nothing to optimize.


Even serving a static file is a burden on the server when you have millions of requests.


What are the exact numbers?

A quote from: http://wiki.nginx.org/Main :

> I currently have Nginx doing reverse proxy of over tens of millions of HTTP requests per day (thats a few hundred per second) on a single server. At peak load it uses about 15MB RAM and 10% CPU on my particular configuration (FreeBSD 6).


https://easylist.adblockplus.org/blog/2011/09/01/easylist-st... is the first thing i found, talking about 11.5 million total users and 80% of them using easylist, 9.2 million. According to the blogpost, the still existing (i just noticed that this was after the update-behaviour change) monday-peak was 118.5% of the expected (week total / 7) - 73 million download in august, so (73 / 4 / 7) * 1.185 = 3 million for that specific list and 3.75 for all of them. Hope i didn't miscalculate ^^ Add to that the growth till than. The easylist.txt seems to have a size of 528kb.


+ SSL




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: