> Even better: Don't use sessions. If there's data you're caching in a session, pre-compute it and store it in Redis (or other suitable NoSQL database).
What do you mean by data you are caching in a session? Session is for maintaining state viz. logged in or not, language preference etc.
How is storing in Redis(or any other data store) any different from storing in Memcache?
> Going statless wherever possible almost always wins.
Using sessions with a data store which can be accessed from multiple machines(no files or in-memory stores) is stateless. The simplest implementation will have a signed cookie with a session id. Your application will have a hook to load the user session from the backing data store before processing the request.
I know about memcache key eviction. For most of the applications, that is an acceptable compromise. For others, back up the sessions in the db every n seconds as mentioned in the article. Session is loaded every request and hitting the db every request is not something I want.
Once you move from storing session data in-memory on the webserver, and add a network call, why not just store the data alongside the users other data in a fast datastore? That could be Redis, Cassandra, whatever.
This isn't babble -- it's honestly a pretty common technique. If your site sees millions of users, storing a session for visitors that aren't logged in is prohibitive and unnecessary. You can store UI customizations and basic memoization in a client-side cookie if you need to.
If you're building something small or basic, then you probably won't have multiple webservers and you can use fast in-memory sessions without concern. This only applies once you need to worry about scale.
A cookie will eat up latency on every request. If the amount of data is non-trivial (and UI customizations can be, there're some fairly demanding users, and oftentimes it's helpful to store a history of what they've done recently so you don't prompt them for things they have no interest in doing), that can be a big usability hit, and even run afoul with HTTP request-size limits.
Using local storage or manipulating the path to avoid the cookie being sent to the server are a solid option if you need to store UI state for non-logged-in visitors of a highly trafficked site.
Obviously solutions are not zero-sum but having to deal with a session store that brings its own latency, state, garbage collection issues, etc, is something I've found to be sub-optimal. But to be clear: We're talking about specialized techniques that don't apply to most websites.
Though most of my experience is at a social network (not the social network) and building out advertising platforms. I've certainly never had to worry about storing so much UI state data that I'd be concerned about its size. Your experience seems a bit different and obviously at this level everything is subjective.
> Once you move from storing session data in-memory on the webserver, and add a network call,
I don't move from storing session data in-memory; I never store session data in-memory. Well, I do indirectly but that's always some distributed memory system. And there isn't necessarily a network call. A single webserver accessing memcache running on the same machine is as fast as accessing in-memory cache. When there is more than one memcache server, there has got to be a network call, but I am yet to see an application where that is an overhead. Compared to disk access, that is a floating point error.
> why not just store the data alongside the users other data in a fast datastore? That could be Redis, Cassandra, whatever.
What advantage would I have from storing the session alongside the user data? My datastore isn't Redis, Cassandra. It's postgresql. Redis is my cache/data structure server. The only nosql solution which I would consider for user data is mongo.
> This isn't babble -- it's honestly a pretty common technique.
Other than your insistence on using Redis, I don't see how your technique is any different from mine.
> If your site sees millions of users, storing a session for visitors that aren't logged in is prohibitive and unnecessary.
If the user aren't logged in and don't have any user data, there is no session to be loaded. When the user logs in, set the secure cookie with the session id. When a new request comes in, if the secure cookie has the session id, load it. For a not-logged in user, there is not session id and nothing is loaded.
> You can store UI customizations and basic memoization in a client-side cookie if you need to.
The only thing I will store in the cookie is session id. Cookie length constraints and network payload for every request is more overhead than loading the session on the server side.
> If you're building something small or basic, then you probably won't have multiple webservers and you can use fast in-memory sessions without concern. This only applies once you need to worry about scale.
For sessions, session id in cookie and memcache/redis as session store works for all scale.
If you've got a single webserver then we're speaking a different language. That's not an insult or anything, it's just that I'm talking about techniques we use to support millions of pageviews a day.
There is no one way to build a system of course. I'd love to hear more about your lessons-learned, but this really isn't a good forum for that. Though even at your scale you should consider redis as a drop-in replacement for memcache. It benchmarks faster in many cases, and has support for data structures (lists, sorted sets, etc) that make your life easier.
And see my comment below clarifying what you said about cookies.
> For sessions, session id in cookie and memcache/redis as session store works for all scale.
> If you've got a single webserver then we're speaking a different language. That's not an insult or anything, it's just that I'm talking about techniques we use to support millions of pageviews a day.
And where did I talk about single webserver? You said "Once you move from storing session data in-memory on the webserver, and add a network call,", to which I said I never do in-memory session, even for a single webserver. Get over yourself - you aren't a special snowflake who deals with more than one more webserver.
> Though even at your scale you should consider redis as a drop-in replacement for memcache. It benchmarks faster in many cases, and has support for data structures (lists, sorted sets, etc) that make your life easier.
"Though even at your scale "
As I said, get over yourself. All you have offered is somehow storing session in Redis makes you scale.
How on earth would you know what scale I
am talking about? I don't remember mentioning it.
And I know what redis does. Cargo cult mentality viz. "redis is really better than memcached" are the main reason behind fucked up systems.
> And see my comment below clarifying what you said about cookies.
Yes, I saw your comment. Local storage is not the solution for coming year or so. I am paid to design systems that work, not systems that might work.
> For sessions, session id in cookie and memcache/redis as session store works for all scale.
>> Kind of a bold statement? GL with that.
I don't know where you are getting your numbers from, but million pageviews/day as you mention again and again is a very nominal number for a generic webapp(unless you are very cache unfriendly viz. reddit). That isn't something you even have to think about. A standard rails app sitting behind nginx with 4-5 webservers will do it just fine.
And for the last time, "session id in cookies and then load the session before request" fucking works for every one including facebook and google. The only thing that differs is choice of session store, and no, redis isn't the catchall solution. For most of the cases, memcache is faster.
You have taken this all very personally. I apologize for offering a different take on system design than what you apparently believe in very strongly? I've said a few times there's "no one way."
The systems we've built have served over-billion-page-view months. That's not common or easy. HN is a site that values a back-and-forth about that kind of experience and I'd have loved to hear your tips thrown-out there too. But this has become some strange ego thing for you so it's time for me to bow out.
> You have taken this all very personally. I apologize for offering a different take on system design than what you apparently believe in very strongly?
No, you haven't offered anything other than "you should use Redis". You started with "if you are caching in sessions". I don't know where you get the idea the sessions are caches.
> The systems we've built have served over-billion-page-view months. That's not common or easy.
Good for you. But have you actually used local storage for storing user specific data? And have you compared using redis and memcached for session storage? I don't have a problem with difference in opinion - it's just that your opinions aren't valid.
What do you mean by data you are caching in a session? Session is for maintaining state viz. logged in or not, language preference etc.
How is storing in Redis(or any other data store) any different from storing in Memcache?
> Going statless wherever possible almost always wins.
Using sessions with a data store which can be accessed from multiple machines(no files or in-memory stores) is stateless. The simplest implementation will have a signed cookie with a session id. Your application will have a hook to load the user session from the backing data store before processing the request.