The Cache Money Theory

I've just spent 72 hours trapped in the seventh circle of hell. For the non-religious, I refer to the dark place in which a dedicated developer attempts to use a memory cache to solve his web performance problems. The shiny gates of promise were opened to me and I walked right in, only to find myself in a twisting, spiraling funhouse of doom. My heart is aflutter and my nerves are still raw; I'll need an appointment with a cardiologist by week's end.

Rather than belabor the problem itself, or dive into a deep technical analysis of the app to look for inefficiencies (something we've already done off-the-record), I'd rather take a higher-level look at the idea of caching within an app. One thing I've learned over the years, and that's been made manifest to me yet again these last few days, is that the devil is in the details.

To start, let me clarify what I mean by caching in the scope of a web application, just to ensure we're all on the same page. Any reasonable web framework offers a number of vehicles for caching data, content, and/or actions:

  • Request caching: cached at the lowest level using the requested URL. This can be done either within the application or on an external proxy
  • Action caching: cached before the view is rendered
  • Query caching: storing a database query according to the parameters passed into it
  • Page caching: cached after the view is rendered
  • Content caching, commonly known as "frag caching": storing small chunks of content within a page
  • Client-side caching: allow the browser to keep a local copy of a resource

Each of these has its valid use case. For example, static assets should always be cached at the lowest level possible, which in most cases would be the browser or an external proxy. There's no reason for your application to continue to serve up an image file that hasn't changed in two years.

In the case of my three-day nightmare, we were dealing with a combination of all six of these caching techniques. Which brings me to my first major point: DON'T DO THAT. Don't cache requests at the middleware layer while also doing action caching on pages that are telling the browser to hold on to them for an extended period of time, because when you need to manipulate cached content you won't know where to find it. You'll end up making the technical decision to invalidate more than needs to be invalidated just to ensure you've got it all. I'd like to propose we call this the Cache Money Theory:

Every layer of caching you add to an app exponentially increases the difficulty you'll have in controlling the state of your content.

Since we all know that time is a currency our clients often can't afford, caching problems can become an expensive issue very quickly. For each component of your app, pick one or MAYBE two caching techniques and apply them accordingly, otherwise you'll find that maintaining a 6-tier layer cake of caches may cost you more in effort and time (not to mention lost hair) than you or your client bargained for.

Consider minimizing costs by reducing the amount of cache layers you build in to your app, opting to raise performance instead by adding additional processing power, memory, or throughput availability. Consider the monthly cost of an additional server in your load balancer (less than $100/month unless you're getting ripped off) versus the cost of even one of your development hours per month. Think about it: you can improve profitability by reducing expenses or by raising income, and the same applies to app development.

If, however, and I mean IF, you simply can't spend more money on resources and have to find a way to squeeze another drop of performance out of your app, even at the cost of complexity, you can still choose to be smart about how you cache. If you're doing action caching, for example, there's no need to do query caching within the same action. If you're invalidating your action cache periodically, don't allow the browser to cache it as if it was a static asset. Remember what we learned in a classic Seinfeld episode about good and bad:

Jerry: "Well, I hit the wall yesterday with Lady Godiva. She did a full body
flex on a pickle jar."

George: "Did you explain to her about the good naked and the bad naked?"

Jerry: "Where am I gonna get a fat guy and a cannonball?"

There's good caching and bad caching. Since I'm an optimistic guy, here are some qualifications that make up the idea of "good caching" in my view:

  1. All cached items are objects or content than can be produced consistently outside of the cache.
  2. All objects within the cache can be invalidated on an individual basis or as part of a larger group of invalidations.
  3. Timeouts or content expiration times should be consistent for the same piece of content across multiple cache types.

I'm sure there are all sorts of additional bullet points I could add to that list, but I'll leave it at that for now. Each scenario is different, though, so it's going to come down to your judgment as an application developer when contemplating the problem of content performance. Please be smart about how you apply multiple tiers of caching, though, and always remember that when you have performance issues, adding another layer of caching isn't the only way to solve the problem.