"We didn’t have any page caching in place at all."
Cache me if you can! If you're a high-traffic site and you are not caching static content (I recommend Varnish highly), then your are throwing money or good user experience out the window.
The key thing I got a bit wrong was high-traffic mostly static "content", vs high-traffic "very custom" which is a) what I was used to at envato and b) what I thought we had on our hands at goodfilms.
I think in a growing site with a lot of custom per-user content (like a social network) the extra complexity of a cache layer and managing expiry is more pain than it's worth while iterating the product quickly. If you're mostly a content site, it's definitely the #1 thing you should be doing.
Realising that we're both at the same time, depending on the user or page, means that sometimes the cache stuff is the right thing to do, and sometimes not. I was leaning too far towards not, and happy now with the balance we've picked.
Absolutely. At theconversation.edu.au we run a content site—we publish the news, which is the same for everyone. This means we can cache the front page and all the articles as static HTML, and then annotate the page with user info for signed-in commenters, editorial controls for signed-in editors, and so on.
(We have a separate cookie that is present for signed-in users, so the fronted knows whether it should fire the annotation request.)
The result is that we can serve a sudden influx of unauthenticated users (e.g. from Google News or StumbleUpon) from nginx alone, which gives us massive scale from very little hardware. It's likely that the network is actually the bottleneck in this case, and not nginx.
The cached page contains content suitable for everyone, so it looks the the user is logged out.
An extra AJAX request grabs the users logged in status, CSRF token and similar data as JSON and then modifies the page so the user sees what they expect (a logout button, a comment form, etc).
If you're running Ruby on Rails record cache will do it all for you (I've yet to run into any issues, the one I thought was an issue was actually a pgpool-II bug).
If you're on Django I think cache-machine does the same thing. There are some things you won't get cached this way that you could manually (functions and procedures), but I think they're both conservative enough that you won't return stale resources.
One of the great things I love about Rails 3 is how easy it is to start caching. It can, and does, cause issues so make sure you're including caching in your test suite. I'm definitely looking forward to seeing how Rails 4 pushes this forward.
When you say static content, do you mean actual static content, or dynamic-but-still-kind-of-static? Actual static content like CSS files and background images doesn't get any benefit from varnish. Something like the HN front page, where it is dynamic but consistent for all users at a specific point in time, is what you need to put behind varnish.
Both. Varnish defaults to caching jpg,gif,png,css,etc. but we also having it caching things that are pulled from the db and processed. So it leaves our cpu-intensive backend untouched for the vast majority of requests.
In high traffic caching css and images in varnish as well makes sense as it is much faster at evaluating and responding to the HTTP request than say Apache which will be busy building dynamic requests.
And I agree completely. Varnish is goddamn magical and a fascinating exercise in a principled approach to kernel-oriented memory management (a mindset I don't normally undertake).
Cache me if you can! If you're a high-traffic site and you are not caching static content (I recommend Varnish highly), then your are throwing money or good user experience out the window.