In general WebHooks don't use any kind of authentication or authorization. The delivery is based on unique (preferably unguessable) URLs. But, in unlikely circumstances of such a URL leaking it would expose the receiver to unwanted content. This is why it has become a standard practice for WebHooks to send a signature (hash) of content in dedicated header so receiver can validate it as consistency check. In most cases the signature is a HMAC hash of payload, where a shared secret value is used as key. Validating such signature will be subject of my fourth post in Under the hood of ASP.NET Core WebHooks series:

The signature generation and verification is not officially standardized, which results in implementation differences between various WebHooks. Because of that, instead of providing 'one size fits all' solution, ASP.NET Core WebHooks approach is for every receiver to have its own implementation. This approach is supported by WebHookVerifySignatureFilter base class which gives developers a set of shared tools. With its help the signature can be verified in few, not too complicated steps.

Step 1 - Ensuring secure connection

This step is not directly connected to signature validation. It is also not required. But from privacy perspective it's strongly encouraged. In order to prevent exposing the payload to third parties, WebHooks requests should be using HTTPS. Ensuring that connection is secure is as simple as calling EnsureSecureConnection method. If check fails the method will return an IActionResult which should be used to short-circuit the execution.

public class WebSubWebHookSecurityFilter : WebHookVerifySignatureFilter, IAsyncResourceFilter
{
    ...

    public override string ReceiverName => "websub";

    ...

    public async Task OnResourceExecutionAsync(ResourceExecutingContext context,
        ResourceExecutionDelegate next)
    {
        ...

        IActionResult secureConnectionCheckResult = EnsureSecureConnection(ReceiverName,
            context.HttpContext.Request);
        if (secureConnectionCheckResult != null)
        {
            context.Result = secureConnectionCheckResult;
            return;
        }

        ...
    }
}

The behaviour of EnsureSecureConnection is configurable through WebHooks:DisableHttpsCheck configuration key (also the check is disabled when ASPNETCORE_ENVIRONMENT is set to development).

Step 2 - Retrieving signature generated by sender

Retrieving desired header value is as simple as calling GetRequestHeader method. It will even prepare erroneous response if the header is not present.

public class WebSubWebHookSecurityFilter : WebHookVerifySignatureFilter, IAsyncResourceFilter
{
    ...

    public async Task OnResourceExecutionAsync(ResourceExecutingContext context,
        ResourceExecutionDelegate next)
    {
        ...

        if (HttpMethods.IsPost(context.HttpContext.Request.Method))
        {
            string signatureHeader = GetRequestHeader(request, "X-Hub-Signature",
                out IActionResult erroneousResult);
            if (erroneousResult != null)
            {
                context.Result = erroneousResult;
                return;
            }

            ...
        }

        ...
    }
}

But the header value rarely contains only the signature. Usually it carries additional information, like for example identifier of used algorithm. The task of parsing the header value is left to the developers. If the value can be split into tokens based on separators and you are looking for inspiration, the TrimmingTokenizer is a good place to start. It's used by several receivers but not publicly exposed.

After extracting signature from header value there is one more thing which needs to be done. The signature must be encoded to Base64 or hexadecimal representation so it can be put into header. Decoding is covered by FromBase64 and FromHex methods (there are also CreateBadBase64EncodingResult and CreateBadHexEncodingResult methods for creating responses when decoding fails).

public class WebSubWebHookSecurityFilter : WebHookVerifySignatureFilter, IAsyncResourceFilter
{
    ...

    public async Task OnResourceExecutionAsync(ResourceExecutingContext context,
        ResourceExecutionDelegate next)
    {
        ...

        if (HttpMethods.IsPost(context.HttpContext.Request.Method))
        {
            ...

            byte[] expectedSignature = FromHex(parsedSignature, "X-Hub-Signature");
            if (expectedSignature == null)
            {
                context.Result = CreateBadHexEncodingResult("X-Hub-Signature");
                return;
            }

            ...
        }

        ...
    }
}

Step 3 - Computing your own signature

The WebHookVerifySignatureFilter provides ready to use implementation of signature calculation for SHA-1 and SHA-256 algorithms. If one of those algorithms is sufficient for given receiver needs it is enough to call ComputeRequestBodySha1HashAsync or ComputeRequestBodySha1HashAsync and pass request and the secret value as parameters. If different algorithms are needed (for example SHA-384 or SHA-512) it requires a little bit more effort (which makes one wish for more generic implementation, maybe with option for providing HMAC instance through parameter). The additional effort involves managing the request body stream. The current stream position must be pointing to its beginning before and after the hashing. If the stream doesn't support seeking, it requires enabling request buffering.

public class WebSubWebHookSecurityFilter : WebHookVerifySignatureFilter, IAsyncResourceFilter
{
    ...

    public async Task OnResourceExecutionAsync(ResourceExecutingContext context,
        ResourceExecutionDelegate next)
    {
        ...

        if (HttpMethods.IsPost(context.HttpContext.Request.Method))
        {
            ...

            byte[] actualSignature = await ComputeRequestBodySha512HashAsync(request,
                Encoding.UTF8.GetBytes(secret));
            if (actualSignature == null)
            {
                context.Result = new BadRequestResult();
                return;
            }

            ...
        }

        ...
    }

    private static async Task<byte[]> ComputeRequestBodySha512HashAsync(HttpRequest request,
        byte[] secret)
    {
        await PrepareRequestBody(request);

        using (HMACSHA512 hasher = new HMACSHA512(secret))
        {
            try
            {
                Stream inputStream = request.Body;

                int bytesRead;
                byte[] buffer = new byte[4096];

                while ((bytesRead = await inputStream.ReadAsync(buffer, 0, buffer.Length)) > 0)
                {
                    hasher.TransformBlock(buffer, inputOffset: 0, inputCount: bytesRead,
                        outputBuffer: null, outputOffset: 0);
                }

                hasher.TransformFinalBlock(Array.Empty(), inputOffset: 0, inputCount: 0);

                return hasher.Hash;
            }
            finally
            {
                request.Body.Seek(0L, SeekOrigin.Begin);
            }
        }
    }

    private static async Task PrepareRequestBody(HttpRequest request)
    {
        if (!request.Body.CanSeek)
        {
            request.EnableBuffering();

            await request.Body.DrainAsync(CancellationToken.None);
        }

        request.Body.Seek(0L, SeekOrigin.Begin);
    }
}

Step 4 - Comparing signatures

We have actual and expected signatures, all that is left is to compare them. It is important not to forget about the security while doing this. Naive implementation could left the receiver open for timing attacks. Here we should use SecretEqual method which provides a time consistent comparison.

public class WebSubWebHookSecurityFilter : WebHookVerifySignatureFilter, IAsyncResourceFilter
{
    ...

    public async Task OnResourceExecutionAsync(ResourceExecutingContext context,
        ResourceExecutionDelegate next)
    {
        ...

        if (HttpMethods.IsPost(context.HttpContext.Request.Method))
        {
            ...

            if (!SecretEqual(expectedSignature, actualSignature))
            {
                context.Result = CreateBadSignatureResult("X-Hub-Signature");
                return;
            }
        }

        ...
    }
}

Of course CreateBadSignatureResult is provided for us as well.

This is third part of my Under the hood of ASP.NET Core WebHooks series:

I was expecting this post subject (verification requests) to be plain and simple, yet it managed to surprise me.

What's verification request about

Some webhooks perform a verification request as part of creating a subscription. This request is typically a GET (while notifications are POSTs) with a dedicated query parameter which serves as challenge. The goal is to verify that the provided URL in fact supports the webhook. To confirm that, the application is expected to echo the challenge parameter value in response body. Dropbox webhook is textbook example of this flow.

What's in the box for verification request

ASP.NET Core WebHooks provides support for verification requests through WebHookGetHeadRequestFilter. The filter is "triggered" when IWebHookGetHeadRequestMetadata interface is implemented by metadata. I wanted to see if it provides what I needed for my WebSub compliant receiver so I've added it to metadata, which enforced implementation of three properties.

public class WebSubMetadata : WebHookMetadata, IWebHookGetHeadRequestMetadata, ...
{
    ...

    public bool AllowHeadRequests => false;

    public string ChallengeQueryParameterName => "hub.challenge";

    public int SecretKeyMinLength => 0;
}

The AllowHeadRequests and ChallengeQueryParameterName are self-explanatory. As I didn't care about any keys at this stage I've decided to set SecretKeyMinLength to zero. I've run the demo application, fired the prepared request from Postman and (to my surprise) received a 500 response. I've quickly looked through the logs and found this:

System.InvalidOperationException: Could not find a valid configuration for the 'websub' WebHook receiver. Configure secret keys for this receiver.

But I didn't want any secret keys...

Further examination of WebHookGetHeadRequestFilter revealed that responding to verification requests isn't its only responsibility. It also confirms that secret key is properly configured for given receiver and not having one is not an option. This is surprising as secret keys are not related to verification requests but signature verification (hopefully the subject of next post in this series). Regardless, I had to configure one.

The WebHookGetHeadRequestFilter is looking for secret key under WebHooks:{webHookReceiver}:SecretKey:{id} configuration key. If the webhook URL doesn't contain id, the value default will be used in its place. This means that scenarios where all webhooks URLs are not known upfront is becoming challenging and requires smart usage of in-memory configuration provider or implementation of custom configuration provider.

For testing purposes I've put a dummy value into configuration file and received the response I wanted.

What if that's not what you need

The secret key requirement is a pain, but for most cases it's bearable. What if you have special requirements (which ended up being the case for my WebSub receiver)? The WebHookGetHeadRequestFilter doesn't provide way for customization, so the only solution seems to be using IWebHookFilterMetadata (which was briefly described in the introduction post) and creating your own filter. The ASP.NET Core WebHooks helps a little bit here by exposing static Order property on every filter, which allows placing the equivalent in the right spot. The filter should implement IResourceFilter or IAsyncResourceFilter.

public class WebSubWebHookIntentVerificationFilter : IAsyncResourceFilter, IOrderedFilter
{
    ...

    public int Order => WebHookGetHeadRequestFilter.Order;

    ...
}

This gives full freedom in handling verification requests.

This is a second post in my series about ASP.NET Core WebHooks:

As promised it will be focused on machinery which makes it possible for a WebHook request to find matching action.

Basics of WebHooks routing

You may know from the previous post, that key component responsible for configuring routing is WebHookSelectorModelProvider. Its job is to iterate all discovered actions and for those decorated with WebHookAttribute check for conflicts, set attribute routing template and inject action constraints. The most important thing is the template. It has following value (currently there is no built-in customization mechanism available): /api/webhooks/incoming/{webHookReceiver}/{id?}. Let's split it up:

  • /api/webhooks/incoming - static part which is the same for all WebHooks receivers. It might be best to consider all paths starting with it as reserved.
  • {webHookReceiver} - route parameter which must provide the intended receiver name.
  • {id?} - route parameter which may provide unique identifier.

The fact, that all actions decorated with WebHookAttribute share the route template means that when WebHooks request comes in all registered WebHooks actions are possible candidates. How specific action is being chosen? As stated above the WebHookSelectorModelProvider injects actions constraints. The one constraint which is always injected is WebHookReceiverNameConstraint. It validates two things. First is check for receiver "completeness" (does it metadata implement IWebHookBodyTypeMetadataService) and second is comparison between webHookRecevier route parameter and receiver name from metadata. Only if both conditions are met the action will be selected.

More about the {id?}

The purpose of second parameter defined by route is to provide a way for WebHooks URLs to be unique and give them possibility of being capability URLs. Imagine a situation where your application gives users option to receive WebHooks from GitHub. In order to distinguish requests for different users it is best to have unique URLs, so the application can generate them like this: https://{host}/api/webhooks/incoming/github/user1, https://{host}/api/webhooks/incoming/github/user2. In result the action will be able to access the unique identifier through id parameter.

public class GitHubController : ControllerBase
{
    // /api/webhooks/incoming/github/{id}
    [GitHubWebHook]
    public IActionResult GitHubHandler(string id, ...)
    {
        ...
    }
}

That's not all. The unique identifier can also participate in action selection. The WebHookAttribute provides an Id property. If that property is set for specific action the WebHookSelectorModelProvider will inject WebHookIdConstraint. This constraint makes sure that value of id route parameter matches the value provided by property. Going back to the GitHub example, if the application knows that it will need to handle a specific WebHook (for example its own repository), it can create a dedicated action.

public class GitHubController : ControllerBase
{
    // /api/webhooks/incoming/github/tpeczek
    [GitHubWebHook(Id = "tpeczek")]
    public IActionResult GitHubHandlerForTPeczek(...)
    {
        ...
    }

    // /api/webhooks/incoming/github/{id}
    [GitHubWebHook]
    public IActionResult GitHubHandler(string id, ...)
    {
        ...
    }
}

Adding events to the mix

There is one more WebHooks concept which relates to routing - events. Some of the WebHooks providers use events to share information about action which has triggered the WebHook and ASP.NET Core WebHooks provides a set of building blocks for utilizing them. Adding support for event to WebHooks receiver starts by implementing IWebHookEventMetadata as part of metadata. The interface provides properties for defining source of events information. Currently one can provide a query parameter or HTTP header name. The presence of IWebHookEventMetadata will cause WebHookSelectorModelProvider to inject yet another constraint. The WebHookEventNameMapperConstraint constraint will retrieve value from request based on IWebHookEventMetadata and treat it as comma separated list of events. The list must contain at least one entry (unless default event have been specified through IWebHookEventMetadata.ConstantValue property) for constraint to accept the request. Additionally the events are being added to route values, which makes them accessible as action parameter (if multiple events are expected an array should be used as parameter type).

public class GitHubController : ControllerBase
{
    // /api/webhooks/incoming/github/tpeczek
    [GitHubWebHook(Id = "tpeczek")]
    public IActionResult GitHubHandlerForTPeczek(string @event, ...)
    {
        ...
    }

    // /api/webhooks/incoming/github/{id}
    [GitHubWebHook]
    public IActionResult GitHubHandler(string id, string @event, ...)
    {
        ...
    }
}

Events, similarly to unique identifier, can participate in action selection. To enable this the receivers WebHookAttribute must implement IWebHookEventSelectorMetadata, which will provide EventName property. As you probably already expect the WebHookSelectorModelProvider is just waiting to inject related constraint. In this case it's WebHookEventNameConstraint which validates EventName property value against previously parsed list of events, if event is present the action is selected.

public class GitHubController : ControllerBase
{
    // /api/webhooks/incoming/github/tpeczek [X-GitHub-Event --> push]
    [GitHubWebHook(Id = "tpeczek", EventName = "push")]
    public IActionResult GitHubHandlerForTPeczekPushEvent(...)
    {
        ...
    }

    // /api/webhooks/incoming/github/tpeczek
    [GitHubWebHook(Id = "tpeczek")]
    public IActionResult GitHubHandlerForTPeczek(string @event, ...)
    {
        ...
    }

    // /api/webhooks/incoming/github/{id} [X-GitHub-Event --> push]
    [GitHubWebHook(EventName = "push")]
    public IActionResult GitHubHandlerForPushEvent(string id, ...)
    {
        ...
    }

    // /api/webhooks/incoming/github/{id}
    [GitHubWebHook]
    public IActionResult GitHubHandler(string id, string @event, ...)
    {
        ...
    }
}

This covers the most important information regarding ASP.NET Core WebHooks routing. Of course I haven't described everything (for example special support for Ping event). In next post I'm planning to take a look at verification requests.

I was thinking about digging into ASP.NET Core WebHooks for quite some time, but I had other things to do and I was telling myself it will be best to do it when it reaches RTM. Those other things are done now, but ASP.NET Core WebHooks has been moved out of ASP.NET Core 2.1. I could either keep waiting for RTM and postpone further or admit that it was only excuse and go for it. After briefly looking at code one more time I've decided I'm curious about the internals and I want to do it, especially that I had a project idea which will be a good context for it - implementing a WebSub compliant subscriber. WebSub is a specification of common mechanism for communication between publishers and subscribers of Web content. The content distribution requires subscriber to expose a web hook. This web hook is very specific so I'm not sure if I will be able to deliver it, but it certainly will be interesting.

With this post I'm aiming at starting a series which for me will be a log of what I have learned and for you (hopefully) a source of knowledge. I don't have full roadmap for this series. I'm planning on touching following subjects:

I'll keep this list updated so it could serve as TOC. Please also remember that I'm starting writing this based on 1.0.0-preview3-final version so things might change.

Let's start with general overview.

What happens when you "Add" WebHooks

An adventure with every WebHook starts with AddXXXWebHooks() extension method. In fact there are always two methods. One is extending IMvcCoreBuilder while the other is extending IMvcBuilder. The reason is that WebHooks doesn't require "full" ASP.NET Core MVC. If we are not setting them together with MVC based frontend or Web API following will be enough for working application.

public class Startup
{
    public void ConfigureServices(IServiceCollection services)
    {
        services.AddMvcCore()
            .AddWebSubWebHooks();
    }

    public void Configure(IApplicationBuilder app, IHostingEnvironment env)
    {
        app.UseMvc();
    }
}

Are both method the same? Usually not. The method which extends IMvcCoreBuilder must register required formatters even if it's JSON. Skipping this difference, the flow is the same.

public static IMvcCoreBuilder AddWebSubWebHooks(this IMvcCoreBuilder builder)
{
    if (builder == null)
    {
        throw new ArgumentNullException(nameof(builder));
    }

    WebHookMetadata.Register<WebSubMetadata>(builder.Services);

    // Registration of required formatters

    return builder.AddWebHooks();
}

Two things are happening here: metadata describing the receiver are being registered and the core part of WebHooks is being added. What's happening there?

The heart of ASP.NET Core WebHooks - Application Models

The components of an ASP.NET Core MVC application are represented by application model. Most of the time developers don't have to be aware of it. The default implementation registers filters, discovers actions, configures routing and provides conventions which are used to interact with it. But for special cases there is an option for modifying application model by providing implementations of IApplicationModelProvider. This is how WebHooks integrate themselves with routing and create dedicated filters pipeline. As part of AddWebHooks() four implementations are being registered (listed in order of execution):

  1. WebHookActionModelPropertyProvider - responsible for adding receiver metadata to ActionModel.Properties of related actions (later implementations depend on it).
  2. WebHookSelectorModelProvider - responsible for adding routing template and constraints.
  3. WebHookActionModelFilterProvider - responsible for adding common and receiver specific filters to related actions.
  4. WebHookBindingInfoProvider - responsible for adding model binding information.

The future posts in this series will look closer at some of them.

Describing a receiver with metadata

The subject of receiver metadata will be probably coming back in every post in this series. Most of the aspects of receiver behaviour are described by one of interfaces which can be used to compose metadata (those interfaces can be found in Microsoft.AspNetCore.WebHooks.Metadata namespace). The bare minimum which every metadata needs to implement is IWebHookBodyTypeMetadataService. This is enforced by WebHookMetadata base class which also takes care of most of the implementation.

public class WebSubMetadata : WebHookMetadata
{
    public WebSubMetadata()
        : base(WebSubConstants.ReceiverName)
    { }

    public override WebHookBodyType BodyType => WebHookBodyType.Xml;
}

This makes two information obligatory. First is receiver name and second its request body type. The body type is potentially problematic one. It can have one of three values: Form, Json or Xml. The WebHookActionModelPropertyProvider validates if one of those values is set before adding metadata to action and later WebHookVerifyBodyTypeFilter (added by WebHookActionModelFilterProvider) validates if Content-Type of incoming request is in line with the provided value. For WebSub the web hook should support multiple body types so this will be one of challenging parts.

There is one more requirement for the metadata, enforced by WebHookReceiverExistsFilter. The metadata must implement IWebHookVerifyCodeMetadata or provide filter which implements IWebHookReceiver (the only out-of-the-box implementations is WebHookVerifySignatureFilter). The goal here seems to be security - enforcing some kind of request validation. For cases where such validation is not (yet) needed one can implement IWebHookFilterMetadata for metadata to inject some simple implementation of IWebHookReceiver.

public class WebSubMetadata : WebHookMetadata, IWebHookFilterMetadata
{
    private class WebSubWebHookReceiverFilter : IFilterMetadata, IWebHookReceiver
    {
        public string ReceiverName => WebSubConstants.ReceiverName;

        public bool IsApplicable(string receiverName)
        {
            if (receiverName == null)
            {
                throw new ArgumentNullException(nameof(receiverName));
            }

            return String.Equals(ReceiverName, receiverName, StringComparison.OrdinalIgnoreCase);
        }
    }

    private WebSubWebHookReceiverFilter _webHookReceiverFilter = new WebSubWebHookReceiverFilter();

    ...

    public void AddFilters(WebHookFilterMetadataContext context)
    {
        context.Results.Add(_webHookReceiverFilter);
    }
}

Associating action with WebHooks endpoint

There is one more piece missing - a way to associate an action with WebHooks endpoint. This is achieved by decorating action with attribute derived from WebHookAttribute.

public class WebSubWebHookAttribute : WebHookAttribute
{
    public WebSubWebHookAttribute()
        : base(WebSubConstants.ReceiverName)
    { }
}

The receiver name must match the one provided in metadata.

This is enough to create a "working" receiver. Its capabilities are strongly limited, but it is capable of routing matching requests to an action. What that exactly means will be the focus of next post in this series.

Recently I've been looking into using some "special tasks" databases with ASP.NET Core. One of them was RethinkDB. In general RethinkDB is a NoSQL database, which stores schemaless JSON documents. What's special about it, is its real-time capabilities - changefeeds. Changefeeds provide an option of continuously pushing updated query results to applications in real-time. The data can come to RethinkDB from various sources and be perceived as stream of events. I wanted to expose that stream through the application to the clients.

Setting up RethinkDB connection

There is a .NET Standard 2.0 driver for RethinkDB available as open source project (free to use if SSL/TLS encryption is not required). For interaction with the driver a globally provided singleton RethinkDb.Driver.RethinkDB.R should be used. This singleton together with open connection allows for using the database (the driver supports connections pooling with couple possible strategies, but I'm not going to use it here). Establishing connection requires hostname or IP and optionally port and timeout.

internal class RethinkDbOptions
{
    public string HostnameOrIp { get; set; }

    public int? DriverPort { get; set; }

    public int? Timeout { get; set; }
}

The connection object is expensive to create and it is advised to you use only one in the application. To achieve that the connection object can be encapsulated (together with the global singleton) in dedicated service.

internal class RethinkDbSingletonProvider : IRethinkDbSingletonProvider
{
    public RethinkDb.Driver.RethinkDB RethinkDbSingleton { get; }

    public RethinkDb.Driver.Net.Connection RethinkDbConnection { get; }

    public RethinkDbSingletonProvider(IOptions<RethinkDbOptions> options)
    {
        if (options == null)
        {
            throw new ArgumentNullException(nameof(options));
        }

        if (String.IsNullOrWhiteSpace(options.Value.HostnameOrIp))
        {
            throw new ArgumentNullException(nameof(RethinkDbOptions.HostnameOrIp));
        }

        var rethinkDbSingleton = RethinkDb.Driver.RethinkDB.R;

        var rethinkDbConnection = rethinkDbSingleton.Connection()
            .Hostname(options.Value.HostnameOrIp);

        if (options.Value.DriverPort.HasValue)
        {
            rethinkDbConnection.Port(options.Value.DriverPort.Value);
        }

        if (options.Value.Timeout.HasValue)
        {
            rethinkDbConnection.Timeout(options.Value.Timeout.Value);
        }

        RethinkDbConnection = rethinkDbConnection.Connect();

        RethinkDbSingleton = rethinkDbSingleton;
    }
}

To ensure a single instance the service should be registered as singleton.

internal static class ServiceCollectionExtensions
{
    public static IServiceCollection AddRethinkDb(this IServiceCollection services,
        Action<RethinkDbOptions> configureOptions)
    {
        if (services == null)
        {
            throw new ArgumentNullException(nameof(services));
        }
        if (configureOptions == null)
        {
            throw new ArgumentNullException(nameof(configureOptions));
        }

        services.Configure(configureOptions);
        services.TryAddSingleton<IRethinkDbSingletonProvider, RethinkDbSingletonProvider>();
        services.TryAddTransient<IRethinkDbService, RethinkDbService>();

        return services;
    }
}

The second service (RethinkDbService) will handle the RethinkDB specific logic.

Creating a data source

The changefeeds capability is great for solving specific use cases. But for demo purposes it's better to have something simple for brevity. Good enough example can be an IHostedService for gathering ThreadPool statistics.

internal class ThreadStats
{
    public int WorkerThreads { get; set; }

    public int MinThreads { get; set; }

    public int MaxThreads { get; set; }

    public override string ToString()
    {
        return $"Available: {WorkerThreads}, Active: {MaxThreads - WorkerThreads}, Min: {MinThreads}, Max: {MaxThreads}";
    }
}

internal class ThreadStatsService : IHostedService
{
    private readonly IRethinkDbService _rethinkDbService;

    ...

    private async Task GatherThreadStatsAsync(CancellationToken cancellationToken)
    {
        while (!cancellationToken.IsCancellationRequested)
        {
            ThreadPool.GetAvailableThreads(out var workerThreads, out var _);
            ThreadPool.GetMinThreads(out var minThreads, out var _);
            ThreadPool.GetMaxThreads(out var maxThreads, out var _);

            _rethinkDbService.InsertThreadStats(new ThreadStats
            {
                WorkerThreads = workerThreads,
                MinThreads = minThreads,
                MaxThreads = maxThreads
            });

            await Task.Delay(TimeSpan.FromSeconds(1), cancellationToken);
        }
    }
}

If this code looks familiar it is because its based on one of David Fowler demos from NDC London 2018. Adding controller from that demo to the application provides an easy way to use up some threads.

The programming model of RethinkDB driver is a little bit specific. In general one builds the query and at the end runs it providing the connection. This makes inserting the stats look like below.

internal class RethinkDbService: IRethinkDbService
{
    private const string DATABASE_NAME = "Demo_AspNetCore_RethinkDB";
    private const string THREAD_STATS_TABLE_NAME = "ThreadStats";

    private readonly RethinkDb.Driver.RethinkDB _rethinkDbSingleton;
    private readonly Connection _rethinkDbConnection;

    public RethinkDbService(IRethinkDbSingletonProvider rethinkDbSingletonProvider)
    {
        if (rethinkDbSingletonProvider == null)
        {
            throw new ArgumentNullException(nameof(rethinkDbSingletonProvider));
        }

        _rethinkDbSingleton = rethinkDbSingletonProvider.RethinkDbSingleton;
        _rethinkDbConnection = rethinkDbSingletonProvider.RethinkDbConnection;
    }

    public void InsertThreadStats(ThreadStats threadStats)
    {
        _rethinkDbSingleton.Db(DATABASE_NAME).Table(THREAD_STATS_TABLE_NAME)
            .Insert(threadStats).Run(_rethinkDbConnection);
    }
}

With the data source in place, the changefeed can be exposed.

Exposing changefeed with Server-Sent Events

Server-Sent Events feels like natural choice here. This is exactly the scenario this technology has been designed for. Also with ready to use library the implementation should be straightforward. First the required service and middleware needs to be registered.

public class Startup
{
    public void ConfigureServices(IServiceCollection services)
    {
        services.AddRethinkDb(options =>
        {
            options.HostnameOrIp = "127.0.0.1";
        });

        services.AddServerSentEvents();

        services.AddSingleton<IHostedService, ThreadStatsService>();

        services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_1);
    }

    public void Configure(IApplicationBuilder app)
    {
        app.MapServerSentEvents("/thread-stats-changefeed");

        app.UseStaticFiles();

        app.UseMvc();

        app.Run(async (context) =>
        {
            await context.Response.WriteAsync("-- Demo.AspNetCore.RethinkDB --");
        });
    }
}

In order to acquire the changefeed from RethinkDB one needs to build the query, call Changes() on it and then run it by calling RunChanges()/RunChangesAsync(). This returns a cursor which can be iterated later.

internal class RethinkDbService: IRethinkDbService
{
    ...

    public Task<Cursor<Change<ThreadStats>>> GetThreadStatsChangefeedAsync(CancellationToken cancellationToken)
    {
        return _rethinkDbSingleton.Db(DATABASE_NAME).Table(THREAD_STATS_TABLE_NAME)
            .Changes().RunChangesAsync<ThreadStats>(_rethinkDbConnection, cancellationToken);
    }
}

With all the pieces in place the application can start sending events. Again IHostedService comes with help here by providing a nice way of utilizing the changefeed in background.

internal class ThreadStatsChangefeedService : IHostedService
{
    private readonly IRethinkDbService _rethinkDbService;
    private readonly IServerSentEventsService _serverSentEventsService;

    ...

    private async Task ExposeThreadStatsChangefeedAsync(CancellationToken cancellationToken)
    {
        var threadStatsChangefeed = await _rethinkDbService
            .GetThreadStatsChangefeedAsync(cancellationToken);

        while (!cancellationToken.IsCancellationRequested
               && (await threadStatsChangefeed.MoveNextAsync(cancellationToken)))
        {
            string newThreadStats = threadStatsChangefeed.Current.NewValue.ToString();
            await _serverSentEventsService.SendEventAsync(newThreadStats);
        }
    }
}

This is it. All that is left is registering the service and adding some UI. The complete demo can be found here.

WebSockets fallback

The sad truth is that not all major browsers support Server-Sent Events. It might not always be needed (in the context mentioned at the beginning the target clients were known to use browsers with support), but it might be nice to add a fallback for that 11% (at the moment of writing this). Such fallback can be implemented based on WebSockets. I've been writing about using WebSockets in ASP.NET Core several times before and I have a demo project up on GitHub, so there is no need to repeat anything of that here. For completeness the demo for this post contains the fallback, which is a simple WebSockets implementation capable only of broadcasting text messages.

Why I didn't use SignalR

This question will most likely come up. The honest answer is that I didn't need it. The Server-Sent Events fit the use case perfectly and the implementation was even shorter than it would be with SignalR (it's no longer true with WebSockets fallback, but even in that case the code is still pretty simple). Also the usage of SignalR wouldn't be typical here. It would either have to handle the changefeed in Hub or attempt to access clients in IHostedService. In my opinion this approach is cleaner.

Older Posts