OutputCacheProvider for Redis

Following this series of posts about caching and OutputCache in particular, I will now show you how to use Redis to store the output cache items in your ASP.NET MVC application. You can use the same concepts to store in any other data store, such as AppFabric, MongoDb, File System, etc.

First, we will need a Redis client. In this implementation i’m using StackExchange.Redis, so you will need to run this in the package manager console (or via nuget package manager UI):

Install-Package StackExchange.Redis

Next, let’s implement the provider. The first thing you should do is to create a class that inherits from System.Web.Caching.OutputCacheProvider, and then provide implementations for Add, Set, Get and Remove methods. The stubs look like this:

public class RedisOutputCacheProvider : OutputCacheProvider
{
    public override object Add(string key, object entry, DateTime utcExpiry)
    {
        throw new NotImplementedException();
    }

    public override object Get(string key)
    {
        throw new NotImplementedException();
    }

    public override void Remove(string key)
    {
        throw new NotImplementedException();
    }

    public override void Set(string key, object entry, DateTime utcExpiry)
    {
        throw new NotImplementedException();
    }
}

Next, we will connect to the Redis cluster. Since i’m only using my local development machine for testing, i’ll create a ConnectionMultiplexer like so:

private static ConnectionMultiplexer Redis = ConnectionMultiplexer.Connect("localhost:6379");

Also, since this Redis client needs to receive a byte[] when storing the entries, we will need to handle the serialization as well. I created an abstraction for this, also because I wanted to test with different serializers:

public interface IItemSerializer
{
    byte[] Serialize(object item);
    object Deserialize(byte[] itemBytes);
}

Now, for the provider implementation:

public class RedisOutputCacheProvider : OutputCacheProvider
{
    private static ConnectionMultiplexer Redis = ConnectionMultiplexer.Connect("localhost:6379");
    private static IItemSerializer Serializer = new BinaryFormatterItemSerializer();

    public override object Add(string key, object entry, DateTime utcExpiry)
    {
        // return the existing value if any
        var existingValue = Get(key);
        if (existingValue != null)
        {
            return existingValue;
        }

        var expiration = utcExpiry - DateTime.UtcNow;
        var entryBytes = Serializer.Serialize(entry);

        var db = Redis.GetDatabase();

        // set the cache item with the defined expiration, only if an item does not exist
        db.StringSet(key, entryBytes, expiration, When.NotExists);

        return entry;
    }

    public override void Set(string key, object entry, DateTime utcExpiry)
    {
        var expiration = utcExpiry - DateTime.UtcNow;
        var entryBytes = Serializer.Serialize(entry);

        var db = Redis.GetDatabase();

        // set the cache item with the defined expiration, overriding existing items
        db.StringSet(key, entryBytes, expiration, When.Always);
    }

    public override object Get(string key)
    {
        var db = Redis.GetDatabase();

        var valueBytes = db.StringGet(key);

        if (!valueBytes.HasValue)
        {
            return null;
        }

        var value = Serializer.Deserialize(valueBytes);

        return value;
    }

    public override void Remove(string key)
    {
        var db = Redis.GetDatabase();

        // signal key removal but don't wait for the result
        db.KeyDelete(key);
    }
}

The provider implementation must follow the documentation guidelines in the MSDN page.

In particular, these are the main design rules for the Add method:

If there is already a value in the cache for the specified key, the provider must return that value.
The Add method stores the data if it is not already in the cache.
If the data is in the cache, the Add method returns it.

In the example above, i’m using a implementation of the IItemSerializer interface that uses the BinaryFormatter for the serialization:

public class BinaryFormatterItemSerializer : IItemSerializer
{
    public byte[] Serialize(object item)
    {
        var formatter = new BinaryFormatter();

        byte[] itemBytes;

        using (var ms = new MemoryStream())
        {
            formatter.Serialize(ms, item);
            itemBytes = ms.ToArray();
        }

        return itemBytes;
    }

    public object Deserialize(byte[] bytes)
    {
        var formatter = new BinaryFormatter();

        object item;
        using (var ms = new MemoryStream(bytes))
        {
            item = formatter.Deserialize(ms);
        }

        return item;
    }
}

Unit testing

I wrote some unit tests so that we can check that everything is working correctly before using it in our application. The first test checks all the basic functionality, and the second one checks the ability of the cache provider to remove expired items:

[TestClass]
public class RedisOutputCacheProviderTests
{
    [TestMethod]
    public void CanAddAndGetAndSetAndRemoveItem()
    {
        var key = "mykey" + Guid.NewGuid();
        var provider = new RedisOutputCacheProvider();

        // test add and get

        provider.Add(key, "myvalue", DateTime.UtcNow.Add(TimeSpan.FromMinutes(1)));

        var value = (string)provider.Get(key);

        Assert.AreEqual("myvalue", value);

        // test set and get

        provider.Set(key, "anothervalue", DateTime.UtcNow.Add(TimeSpan.FromMinutes(1)));

        var valueAfterSet = (string)provider.Get(key);

        Assert.AreEqual("anothervalue", valueAfterSet);

        // test remove and get

        provider.Remove(key);

        var valueAfterRemove = (string)provider.Get(key);
        Assert.Is Null(valueAfterRemove);
    }

    [TestMethod]
    public async Task EntryExpiresAfterAbsoluteTime()
    {
        var key = "mykey" + Guid.NewGuid();
        var provider = new RedisOutputCacheProvider();

        // add the value and check that it is in cache

        provider.Add(key, "myvalue", DateTime.UtcNow.Add(TimeSpan.FromSeconds(3)));
        var value = (string)provider.Get(key);
        Assert.AreEqual("myvalue", value);

        await Task.Delay(3000);

        var valueAfterExpiration = (string)provider.Get(key);
        Assert.Is Null(valueAfterExpiration);
    }
}

Configure your web application

After checking that the basic behavior is fine, we can integrate the provider into our web application. Assuming that you are already using the OutputCacheAttribute, you can do this by adding the following xml to your web.config:

<system.web>
  (...)
  <caching>
    <outputcache defaultprovider="RedisOutputCache">
      <providers>
        <add name="RedisOutputCache" type="YourNamespace.RedisOutputCacheProvider, YourAssemblyName">
      </add></providers>
    </outputcache>
  </caching>
  (...)
</system.web>

Then, if you go back to your web application, your OutputCache attributes should all be using this new provider!

Further work

I was also experimenting with a faster binary serializer, protobuf-net. It works for unit tests, but since the type that the OutputCache is trying to store is internal, we cannot use this implementation at the moment. I will try to figure out a way to use protobuf-net for this, and if you have any suggestions please drop a comment below:

...
using ProtoBuf;
...
public class ProtoBufItemSerializer : IItemSerializer
{
    public byte[] Serialize(object entry)
    {
        var item = new CacheItem { Value = entry };

        byte[] itemBytes;

        using (var ms = new MemoryStream())
        {
            Serializer.Serialize<cacheitem>(ms, item);
            itemBytes = ms.ToArray();
        }

        return itemBytes;
    }

    public object Deserialize(byte[] bytes)
    {
        CacheItem item;
        using (var ms = new MemoryStream(bytes))
        {
            item = Serializer.Deserialize<cacheitem>(ms);
        }

        return item.Value;
    }

    [ProtoContract]
    public class CacheItem
    {
        [ProtoMember(1, DynamicType = true)]
        public object Value { get; set; }
    }
}

Finally, since I didn’t want my web application to fail when temporary connection issues to the Redis cluster arise, I wrapped all the implementations in a try-catch, so that if the cache fails, MVC will ignore the cache and execute the action.

Example for the Get method:

public override object Get(string key)
{
    try
    {
        // implementation goes here
    }
    catch (RedisConnectionException e)
    {
        Trace.TraceError(e.ToString());
        return null;
    }
}

You can test this by loading up your web application, making a request to a OutputCached action, and then shutting down redis and reloading the page. The page should still show up, but the content is being regenerated in every request.

Invalidating cache when data changes

You can invalidate a cache item over the response context. As an example, I created one more action in my controller to invalidate the Index item cache:

public string InvalidateCache()
{
    Response.RemoveOutputCacheItem(Url.Action("Index", "Home"));
    return "Done";
}

Code on GitHub

I created a repository on github to host these and other OutputCache-related extensions. You can find the full code at https://github.com/danielbcorreia/OutputCacheExtensions.

If you have any comments or suggestions about this article, go ahead and write in the comments.