TwentySevenEighteen

memcached is a great tool for building and running large sites. The SME Toolkit, a partnership between IBM and the International Finance Corporation (and a Ruby on Rails application) uses memcached extensively to reduce the load on the database and make the application really fly. In addition to caching HTML fragments in memory, we also use the Robot Coop’s simple CachedModel plugin to ease some of the repetitive queries for some key models.

One of the limitations of the MemCacheStore mechanism is that it doesn’t support regular-expression expiration of cache fragments. With the setup of the Toolkit site, various partners publish updates to their sites (for example: new content, updated categories, granting permission to view premium content). One instance of the application powers many brands, and the whole instance shares one memcached process. When a partner publishes an update to their site, we need to expire the cache fragments for their site, and their site only. With a simple disk-based cache, this would be simple; pass a regular expression to the expire_fragment method and be done with it. With memcached, it’s not so simple.

To work around this limitation, we exploit the ability of memcached to hold any piece of data. To keep track of each brand’s cache data, we maintain a hash of all fragment names for a brand, and save that in memcached as well. The code for this is remarkably simple:

 module ActionController

  module Caching

    module Fragments

      def write_fragment_with_memcache(name, content, options = nil)

        return unless ret = write_fragment_without_memcache(name, content, options)

        # Append the item's key to the array of items for this site

        if fragment_cache_store.is_a? ActionController::Caching::Fragments::MemCacheStore

          site = @site_brand.id  #@site_brand is set by the application controller

          doc = CACHE.get "fragments_for_site_#{site}"

          doc = [] unless Array === doc

          doc << fragment_cache_key(name)

          CACHE.set("fragments_for_site_#{site}", doc)

        end

        ret

      end

      alias_method_chain :write_fragment, :memcache

    end

  end

end

Then in our sweeper for the site model, we simply iterate over the entire hash of fragments for the site and delete each one.

  def expire_fragments(record)

    doc = CACHE.get "fragments_for_site_#{record.id.to_s}"

    if Array === doc

      doc.each do |page|

        CACHE.delete page

      end

      CACHE.delete "fragments_for_site_#{record.id.to_s}"

    end

  end