Redis backed response caching in ASP.NET Core

ASP.NET Core comes with out-of-the-box support for server side response caching. It's easy to use and (when configured properly) can give you nice performance boost. But it also has some shortcomings.

Under the hood it utilizes in-memory caching which means that the cache has low latency at price of increased memory usage. In high load scenarios this can lead to memory pressure and memory pressure can lead to entries being evicted prior to its expiration. This also means that cache is not durable. If the process goes down for any reason the cache needs to be repopulated. Last but not least, it provides no support for load balancing scenarios - every node has to keep its own full cache.

Load balancing with in-memory response cache

None of those limitations may be a problem for you, but if it is, you might want to trade some cache latency to solve them. One of approaches can be using a distributed cache like Redis. This way the nodes are no longer responsible for holding the cache, the memory usage is lower and when an instance recycles it doesn't have to warm up again.

Load balancing with Redis backed response cache

Implementing Redis backed IResponseCache

The heart of server side response caching in ASP.NET Core is ResponseCachingMiddleware. It orchestrates the entire process and makes other components like IResponseCachingPolicyProvider, IResponseCachingKeyProvider and IResponseCache talk to each other. The component, which needs to be implemented in order to switch caching from in-memory to Redis, is IResponseCache as it represent the storage for entries. It needs to be able to set or get IResponseCacheEntry by a string key. The IResponseCacheEntry doesn't make any assumptions about the shape of an entry (it's an empty interface) so the only thing that can be done with an instance of it is to blindly attempt binary serialization. That might not be a good idea, so it might be better to focus on its implementations: CachedResponse and CachedVaryByRules. They can be stored in Redis by using Hashes. I'm going to focus only on CachedResponse as CachedVaryByRules is simpler and can be done by replicating same approach.

An instance of CachedResponse can't be represented by single hash because it contains headers collection. What can be done is represent it by two separated hashes, which share a key pattern. First, some helper methods which will take care of conversion (I will be using StackExchange.Redis).

internal class RedisResponseCache : IResponseCache
{
    ...

    private HashEntry[] CachedResponseToHashEntryArray(CachedResponse cachedResponse)
    {
        MemoryStream bodyStream = new MemoryStream();
        cachedResponse.Body.CopyTo(bodyStream);

        return new HashEntry[]
        {
            new HashEntry("Type", nameof(CachedResponse)),
            new HashEntry(nameof(cachedResponse.Created), cachedResponse.Created.ToUnixTimeMilliseconds()),
            new HashEntry(nameof(cachedResponse.StatusCode), cachedResponse.StatusCode),
            new HashEntry(nameof(cachedResponse.Body), bodyStream.ToArray())
        };
    }

    private HashEntry[] HeaderDictionaryToHashEntryArray(IHeaderDictionary headerDictionary)
    {
        HashEntry[] headersHashEntries = new HashEntry[headerDictionary.Count];

        int headersHashEntriesIndex = 0;
        foreach (KeyValuePair<string, StringValues> header in headerDictionary)
        {
            headersHashEntries[headersHashEntriesIndex++] = new HashEntry(header.Key, (string)header.Value);
        }

        return headersHashEntries;
    }
}

With the conversion in place the entry in cache can be set (I will show only async version). It is important to set expiration for both hashes.

internal class RedisResponseCache : IResponseCache
{
    private ConnectionMultiplexer _redis;

    public RedisResponseCache(string redisConnectionMultiplexerConfiguration)
    {
        if (String.IsNullOrWhiteSpace(redisConnectionMultiplexerConfiguration))
        {
            throw new ArgumentNullException(nameof(redisConnectionMultiplexerConfiguration));
        }

        _redis = ConnectionMultiplexer.Connect(redisConnectionMultiplexerConfiguration);
    }

    ...

    public async Task SetAsync(string key, IResponseCacheEntry entry, TimeSpan validFor)
    {
        if (entry is CachedResponse cachedResponse)
        {
            string headersKey = key + "_Headers";

            IDatabase redisDatabase = _redis.GetDatabase();

            await redisDatabase.HashSetAsync(key, CachedResponseToHashEntryArray(cachedResponse));
            await redisDatabase.HashSetAsync(headersKey, HeaderDictionaryToHashEntryArray(cachedResponse.Headers));

            await redisDatabase.KeyExpireAsync(headersKey, validFor);
            await redisDatabase.KeyExpireAsync(key, validFor);
        }
        else if (entry is CachedVaryByRules cachedVaryByRules)
        {
            ...
        }
    }

    ...
}

Getting entry from cache is similar. An opposite conversion methods are needed.

internal class RedisResponseCache : IResponseCache
{
    ...

    private CachedResponse CachedResponseFromHashEntryArray(HashEntry[] hashEntries)
    {
        CachedResponse cachedResponse = new CachedResponse();

        foreach (HashEntry hashEntry in hashEntries)
        {
            switch (hashEntry.Name)
            {
                case nameof(cachedResponse.Created):
                    cachedResponse.Created = DateTimeOffset.FromUnixTimeMilliseconds((long)hashEntry.Value);
                    break;
                case nameof(cachedResponse.StatusCode):
                    cachedResponse.StatusCode = (int)hashEntry.Value;
                    break;
                case nameof(cachedResponse.Body):
                    cachedResponse.Body = new MemoryStream(hashEntry.Value);
                    break;
            }
        }

        return cachedResponse;
    }

    private IHeaderDictionary HeaderDictionaryFromHashEntryArray(HashEntry[] headersHashEntries)
    {
        IHeaderDictionary headerDictionary = new HeaderDictionary();

        foreach (HashEntry headersHashEntry in headersHashEntries)
        {
            headerDictionary.Add(headersHashEntry.Name, (string)headersHashEntry.Value);
        }

        return headerDictionary;
    }
}

So the hashes can be retrieved and entry recreated (only async version again).

internal class RedisResponseCache : IResponseCache
{
    ...

    public async Task<IResponseCacheEntry> GetAsync(string key)
    {
        IResponseCacheEntry responseCacheEntry = null;

        IDatabase redisDatabase = _redis.GetDatabase();

        HashEntry[] hashEntries = await redisDatabase.HashGetAllAsync(key);

        string type = hashEntries.First(e => e.Name == "Type").Value;
        if (type == nameof(CachedResponse))
        {
            HashEntry[] headersHashEntries = await redisDatabase.HashGetAllAsync(key + "_Headers");

            if ((headersHashEntries != null) && (headersHashEntries.Length > 0)
                && (hashEntries != null) && (hashEntries.Length > 0))
            {
                CachedResponse cachedResponse = CachedResponseFromHashEntryArray(hashEntries);
                cachedResponse.Headers = HeaderDictionaryFromHashEntryArray(headersHashEntries);

                responseCacheEntry = cachedResponse;
            }
        }
        else if (type == nameof(CachedVaryByRules))
        {
            ...
        }

        return responseCacheEntry;
    }

    ...
}

At this point adding sync versions and code for CachedVaryByRules shouldn't be hard.

Having the implementation is step one, step two is using it.

Using custom IResponseCache

Back in ASP.NET Core 1.1 ResponseCachingMiddleware had a constructor which allowed for providing your own implementations of IResponseCache. This constructor is gone in 2.0 in order to guarantee a limit on memory usage by making ResponseCachingMiddleware use its own private instance of cache. The IResponseCache implementation can still be replaced by (please don't throw rocks at me) using reflection. Yes, reflection is not a perfect solution. It results in less readable and harder to maintain code. But here a very little of it is needed, just enough to gain access to a single field _cache. A custom middleware can be delivered from ResponseCachingMiddleware and expose this field through property.

internal class RedisResponseCachingMiddleware : ResponseCachingMiddleware
{
    private RedisResponseCache Cache
    {
        set
        {
            FieldInfo cacheFieldInfo = typeof(ResponseCachingMiddleware)
                .GetField("_cache", BindingFlags.NonPublic | BindingFlags.Instance);

            cacheFieldInfo.SetValue(this, value);
        }
    }

    public RedisResponseCachingMiddleware(RequestDelegate next, IOptions<RedisResponseCachingOptions> options,
        ILoggerFactory loggerFactory, IResponseCachingPolicyProvider policyProvider,
        IResponseCachingKeyProvider keyProvider)
        : base(next, options, loggerFactory, policyProvider, keyProvider)
    {
        Cache = new RedisResponseCache(options.Value.RedisConnectionMultiplexerConfiguration);
    }
}

The RedisResponseCachingOptions extends ResponseCachingOptions by adding option needed to establish connection to Redis. Now this all can be put together by providing the options and registering custom middleware instead of calling UseResponseCaching method.

public class Startup
{
    public void ConfigureServices(IServiceCollection services)
    {
        services.AddResponseCaching();
        services.Configure((options) =>
        {
            ...
            options.RedisConnectionMultiplexerConfiguration = "localhost";
        });
    }

    public void Configure(IApplicationBuilder app)
    {
        ...

        app.UseMiddleware<RedisResponseCachingMiddleware>();

        ...
    }
}

This is a fully working solution. Redis is used just as an example, you can use a distribute cache of your choosing or even a database if you wish (although I wouldn't recommend that). It's all about implementing IResponseCache and following the pattern for replacing it.