Rate limiting (sometimes also referred to as throttling) is a key mechanism when it comes to ensuring API responsiveness and availability. By enforcing usage quotas, it can protect an API from issues like:

  • Denial Of Service (DOS) attacks
  • Degraded performance due to traffic spikes
  • Monopolization by a single consumer

Despite its importance, the typical approach to rate limiting is far from perfect when it comes to communicating usage quotas by services and (as a result) respecting those quotas by clients. It shouldn't be a surprise that various services were experimenting with different approaches to solve this problem. The common pattern in the web world is that some of such experiments start to gain traction which results in standardization efforts. This is exactly what is currently happening around communicating services usage quotas with RateLimit Fields for HTTP Internet-Draft. As rate limiting will have built-in support with .NET 7, I thought it might be a good time to take a look at what this potential standard is bringing. But before that, let's recall how HTTP currently supports rate limiting (excluding custom extensions).

Current HTTP Support for Rate Limiting

When it comes to current support for rate limiting in HTTP, it's not much. If a service detects that a client has reached the quota, instead of a regular response it may respond with 429 (Too Many Request) or 503 (Service Unavailable). Additionally, the service may include a Retry-After header in the response to indicate how long the client should wait before making another request. That's it. It means that client can be only reactive. There is no way for a client to get the information about the quota to avoid hitting it.

In general, this works. That said, handling requests which are above the quota still consumes some resources on the service side. Clients would also prefer to be able to understand the quota and adjust their usage patterns instead of handling it as an exceptional situation. So as I said, it's not much.

Proposed Rate Limit Headers

The RateLimit Fields for HTTP Internet-Draft proposes four new headers which aim at enabling a service to communicate usage quotas and policies:

  • RateLimit-Limit to communicate the total quota within a time window.
  • RateLimit-Remaining to communicate the remaining quota within the current time window.
  • RateLimit-Reset to communicate the time (in seconds) remaining in the current time window.
  • RateLimit-Policy to communicate the overall quota policy.

The most interesting one is RateLimit-Policy. It is a list of quota policy items. A quota policy item consists of a quota limit and a single required parameter w which provides a time window in seconds. Custom parameters are allowed and should be treated as comments. Below you can see an example of RateLimit-Policy which informs that client is allowed to make 10 requests per second, 50 requests per minute, 1000 requests per hour, and 5000 per 24 hours.

RateLimit-Policy: 10;w=1, 50;w=60, 1000;w=3600, 5000;w=86400

The only headers which intend to be required are RateLimit-Limit and RateLimit-Reset (RateLimit-Remaining is strongly recommended). So, how an ASP.NET Core based service can server those headers.

Communicating Quotas When Using ASP.NET Core Middleware

As I've already mentioned, built-in support for rate limiting comes to .NET with .NET 7. It brings generic purpose primitives for writing rate limiters as well as a few ready-to-use implementations. It also brings a middleware for ASP.NET Core. The below example shows the definition of a fixed time window policy which allows 5 requests per 10 seconds. The OnRejected callback is also provided to return 429 (Too Many Request) status code and set the Retry-After header value based on provided metadata.

using System.Globalization;
using System.Threading.RateLimiting;
using Microsoft.AspNetCore.RateLimiting;

var builder = WebApplication.CreateBuilder(args);

var app = builder.Build();

app.UseHttpsRedirection();

app.UseRateLimiter(new RateLimiterOptions
{
    OnRejected = (context, cancellationToken) =>
    {
        if (context.Lease.TryGetMetadata(MetadataName.RetryAfter, out var retryAfter))
        {
            context.HttpContext.Response.Headers.RetryAfter = ((int)retryAfter.TotalSeconds).ToString(NumberFormatInfo.InvariantInfo);
        }

        context.HttpContext.Response.StatusCode = StatusCodes.Status429TooManyRequests;

        return new ValueTask();
    }
}.AddFixedWindowLimiter("fixed-window", new FixedWindowRateLimiterOptions(
    permitLimit: 5,
    queueProcessingOrder: QueueProcessingOrder.OldestFirst,
    queueLimit: 0,
    window: TimeSpan.FromSeconds(10),
    autoReplenishment: true
)));

app.MapGet("/", context => context.Response.WriteAsync("-- Demo.RateLimitHeaders.AspNetCore.RateLimitingMiddleware --"))
    .RequireRateLimiting("fixed-window");

app.Run();

The question is, if and how this can be extended to return the rate limit headers? The answer is, sadly, that there seems to be no way to provide the required ones right now. All the information about rate limit policies is well hidden from public access. It would be possible to provide RateLimit-Limit and RateLimit-Policy as they are a direct result of provided options. It is also possible to provide RateLimit-Remaining, but it requires rewriting a lot of the middleware ecosystem to get the required value. What seems completely impossible to get right now is RateLimit-Reset as timers are managed centrally deep in System.Threading.RateLimiting core without any access to their state. There is an option to provide your own timers, but it would mean rewriting the entire middleware stack and taking a lot of responsibility from System.Threading.RateLimiting. Let's hope that things will improve.

Communicating Quotas When Using AspNetCoreRateLimit Package

That built-in support for rate limiting is something that is just coming to .NET. So far the ASP.NET Core developers were using their own implementations or non-Microsoft packages for this purpose. Arguably, the most popular rate limiting solution for ASP.NET Core is AspNetCoreRateLimit. The example below provides similar functionality to the one from the built-in rate limiting example.

using AspNetCoreRateLimit;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddMemoryCache();

builder.Services.Configure<IpRateLimitOptions>(options =>
{
    options.EnableEndpointRateLimiting = true;
    options.StackBlockedRequests = false;
    options.HttpStatusCode = 429;
    options.GeneralRules = new List<RateLimitRule>
    {
        new RateLimitRule { Endpoint = "*", Period = "10s", Limit = 5 }
    };
});

builder.Services.AddInMemoryRateLimiting();

builder.Services.AddSingleton<IRateLimitConfiguration, RateLimitConfiguration>();

var app = builder.Build();

app.UseHttpsRedirection();

app.UseIpRateLimiting();

app.MapGet("/", context => context.Response.WriteAsync("-- Demo.RateLimitHeaders.AspNetCore.RateLimitPackage --"));

app.Run();

The AspNetCoreRateLimit has its own custom way of communicating quotas with HTTP headers. In the case of the above example, you might receive the following values in response.

X-Rate-Limit-Limit: 10s
X-Rate-Limit-Remaining: 4
X-Rate-Limit-Reset: 2022-07-24T11:30:47.2291052Z

As you can see, they provide potentially useful information but not in a way that RateLimit Fields for HTTP is going for. Luckily, AspNetCoreRateLimit is not as protective about its internal state and algorithms, the information needed can be accessed and served in a different way.

The information about the current state is kept in IRateLimitCounterStore. This is a dependency that could be accessed directly, but the method for generating needed identifiers lives in ProcessingStrategy so it will be better to create an implementation of it dedicated for purpose of just getting the counters state.

internal interface IRateLimitHeadersOnlyProcessingStrategy : IProcessingStrategy
{ }

internal class RateLimitHeadersOnlyProcessingStrategy : ProcessingStrategy, IRateLimitHeadersOnlyProcessingStrategy
{
    private readonly IRateLimitCounterStore _counterStore;
    private readonly IRateLimitConfiguration _config;

    public RateLimitHeadersOnlyProcessingStrategy(IRateLimitCounterStore counterStore, IRateLimitConfiguration config) : base(config)
    {
        _counterStore = counterStore;
        _config = config;
    }

    public override async Task ProcessRequestAsync(ClientRequestIdentity requestIdentity, RateLimitRule rule,
        ICounterKeyBuilder counterKeyBuilder, RateLimitOptions rateLimitOptions, CancellationToken cancellationToken = default)
    {
        string rateLimitCounterId = BuildCounterKey(requestIdentity, rule, counterKeyBuilder, rateLimitOptions);

        RateLimitCounter? rateLimitCounter = await _counterStore.GetAsync(rateLimitCounterId, cancellationToken);
        if (rateLimitCounter.HasValue)
        {
            return new RateLimitCounter
            {
                Timestamp = rateLimitCounter.Value.Timestamp,
                Count = rateLimitCounter.Value.Count
            };
        }
        else
        {
            return new RateLimitCounter
            {
                Timestamp = DateTime.UtcNow,
                Count = _config.RateIncrementer?.Invoke() ?? 1
            };
        }
    }
}

The second thing that is needed are rules which apply to specific endpoint and identity. Those can be retrieved from specific (either based on IP or client identifier) IRateLimitProcessor. The IRateLimitProcessor is also a tunnel to IProcessingStrategy, so it's nice we have a dedicated one. But what about the identity I've just mentioned? The algorithm to retrieve it lies in RateLimitMiddleware, so access to it will be needed. There are two options here. One is to inherit from RateLimitMiddleware and the other is to create an instance of one of its implementation and use it as a dependency. The first case would require hiding the base implementation of Invoke as it can't be overridden. I didn't like that, so I went with keeping an instance as a dependency approach. This led to the following code.

internal class IpRateLimitHeadersMiddleware
{
    private readonly RequestDelegate _next;
    private readonly RateLimitOptions _rateLimitOptions;
    private readonly IpRateLimitProcessor _ipRateLimitProcessor;
    private readonly IpRateLimitMiddleware _ipRateLimitMiddleware;

    public IpRateLimitHeadersMiddleware(RequestDelegate next,
        IRateLimitHeadersOnlyProcessingStrategy processingStrategy, IOptions<IpRateLimitOptions> options, IIpPolicyStore policyStore,
        IRateLimitConfiguration config, ILogger<IpRateLimitMiddleware> logger)
    {
        _next = next;
        _rateLimitOptions = options?.Value;
        _ipRateLimitProcessor = new IpRateLimitProcessor(options?.Value, policyStore, processingStrategy);
        _ipRateLimitMiddleware = new IpRateLimitMiddleware(next, processingStrategy, options, policyStore, config, logger);
    }

    public async Task Invoke(HttpContext context)
    {
        ClientRequestIdentity identity = await _ipRateLimitMiddleware.ResolveIdentityAsync(context);

        if (!_ipRateLimitProcessor.IsWhitelisted(identity))
        {
            var rateLimitRulesWithCounters = new Dictionary<RateLimitRule, RateLimitCounter>();

            foreach (var rateLimitRule in await _ipRateLimitProcessor.GetMatchingRulesAsync(identity, context.RequestAborted))
            {
                rateLimitRulesWithCounters.Add(
                    rateLimitRule,
                    await _ipRateLimitProcessor.ProcessRequestAsync(identity, rateLimitRule, context.RequestAborted)
                 );
            }
        }

        await _next.Invoke(context);

        return;
    }
}

The rateLimitRulesWithCounters contains all the rules applying to the endpoint in the context of the current request. This can be used to calculate the rate limit headers values.

internal class IpRateLimitHeadersMiddleware
{
    private class RateLimitHeadersState
    {
        public HttpContext Context { get; set; }

        public int Limit { get; set; }

        public int Remaining { get; set; }

        public int Reset { get; set; }

        public string Policy { get; set; } = String.Empty;

        public RateLimitHeadersState(HttpContext context)
        {
            Context = context;
        }
    }

    ...

    public async Task Invoke(HttpContext context)
    {
        ...
    }

    private RateLimitHeadersState PrepareRateLimitHeaders(HttpContext context, Dictionary<RateLimitRule, RateLimitCounter> rateLimitRulesWithCounters)
    {
        RateLimitHeadersState rateLimitHeadersState = new RateLimitHeadersState(context);

        var rateLimitHeadersRuleWithCounter = rateLimitRulesWithCounters.OrderByDescending(x => x.Key.PeriodTimespan).FirstOrDefault();
        var rateLimitHeadersRule = rateLimitHeadersRuleWithCounter.Key;
        var rateLimitHeadersCounter = rateLimitHeadersRuleWithCounter.Value;

        rateLimitHeadersState.Limit = (int)rateLimitHeadersRule.Limit;

        rateLimitHeadersState.Remaining = rateLimitHeadersState.Limit - (int)rateLimitHeadersCounter.Count;

        rateLimitHeadersState.Reset = (int)(
            (rateLimitHeadersCounter.Timestamp+ (rateLimitHeadersRule.PeriodTimespan ?? rateLimitHeadersRule.Period.ToTimeSpan())) - DateTime.UtcNow
            ).TotalSeconds;

        rateLimitHeadersState.Policy = String.Join(
            ", ",
            rateLimitRulesWithCounters.Keys.Select(rateLimitRule =>
                $"{(int)rateLimitRule.Limit};w={(int)(rateLimitRule.PeriodTimespan ?? rateLimitRule.Period.ToTimeSpan()
            ).TotalSeconds}")
        );

        return rateLimitHeadersState;
    }
}

The only thing that remains is setting the headers on the response.

internal class IpRateLimitHeadersMiddleware
{
    ...

    public async Task Invoke(HttpContext context)
    {
        ...

        if (!_ipRateLimitProcessor.IsWhitelisted(identity))
        {
            ...

            if (rateLimitRulesWithCounters.Any() && !_rateLimitOptions.DisableRateLimitHeaders)
            {
                context.Response.OnStarting(
                    SetRateLimitHeaders,
                    state: PrepareRateLimitHeaders(context, rateLimitRulesWithCounters)
                );
            }
        }

        ...
    }

    ...

    private Task SetRateLimitHeaders(object state)
    {
        var rateLimitHeadersState = (RateLimitHeadersState)state;

        rateLimitHeadersState.Context.Response.Headers["RateLimit-Limit"] = rateLimitHeadersState.Limit.ToString(CultureInfo.InvariantCulture);
        rateLimitHeadersState.Context.Response.Headers["RateLimit-Remaining"] = rateLimitHeadersState.Remaining.ToString(CultureInfo.InvariantCulture);
        rateLimitHeadersState.Context.Response.Headers["RateLimit-Reset"] = rateLimitHeadersState.Reset.ToString(CultureInfo.InvariantCulture);
        rateLimitHeadersState.Context.Response.Headers["RateLimit-Policy"] = rateLimitHeadersState.Policy;

        return Task.CompletedTask;
    }
}

After registering the RateLimitHeadersOnlyProcessingStrategy and IpRateLimitHeadersMiddleware (I've registered it after the IpRateLimitMiddleware) response will contain values similar to the following ones.

RateLimit-Limit: 5
RateLimit-Remaining: 4
RateLimit-Reset: 9
RateLimit-Policy: 5;w=10
X-Rate-Limit-Limit: 10s
X-Rate-Limit-Remaining: 4
X-Rate-Limit-Reset: 2022-07-25T20:57:32.0746592Z

The code works but certainly isn't perfect, so I've created an issue in hope that AspNetCoreRateLimit will get those headers built in.

Limiting the Number of Outbound Requests in HttpClient

The general rule around rate limit headers is that they should be treated as informative, so the client doesn't have to do anything specific with them. They are also described as generated at response time without any guarantee of consistency between requests. This makes perfect sense. In the simple examples above, multiple clients would be competing for the same quota so received headers values don't exactly tell how many requests a specific client can make within a given window. But, real-life scenarios are usually more specific and complex. It's very common for quotas to be per client or per IP address (this is why AspNetCoreRateLimit has concepts like request identity as a first-class citizen, the ASP.NET Core built-in middleware also enables sophisticated scenarios by using PartitionedRateLimiter at its core). In a such scenario, the client might want to use rate limit headers to avoid making requests which have a high likelihood of being throttled. Let's explore that, below is a simple code that can handle 429 (Too Many Request) responses and utilize the Retry-After header.

HttpClient client = new();
client.BaseAddress = new("http://localhost:5262");

while (true)
{
    Console.Write("{0:hh:mm:ss}: ", DateTime.UtcNow);

    int nextRequestDelay = 1;

    try
    {
        HttpResponseMessage response = await client.GetAsync("/");
        if (response.IsSuccessStatusCode)
        {
            Console.WriteLine(await response.Content.ReadAsStringAsync());
        }
        else
        {
            Console.Write($"{(int)response.StatusCode}: {await response.Content.ReadAsStringAsync()}");

            string? retryAfter = response.Headers.GetValues("Retry-After").FirstOrDefault();
            if (Int32.TryParse(retryAfter, out nextRequestDelay))
            {
                Console.Write($" | Retry-After: {nextRequestDelay}");
            }

            Console.WriteLine();
        }
    }
    catch (Exception ex)
    {
        Console.WriteLine(ex.Message);
    }

    await Task.Delay(TimeSpan.FromSeconds(nextRequestDelay));
}

Let's assume that the service is sending all rate limit headers and that they are dedicated to the client. We can rate limit the HttpClient by creating a DelegatingHandler which will read the RateLimit-Policy header value and instantiate a FixedWindowRateLimiter based on it. The FixedWindowRateLimiter will be used to rate limit the outbound requests - if a lease can't be acquired a locally created HttpResponseMessage will be returned.

internal class RateLimitPolicyHandler : DelegatingHandler
{
    private string? _rateLimitPolicy;
    private RateLimiter? _rateLimiter;

    private static readonly Regex RATE_LIMIT_POLICY_REGEX = new Regex(@"(\d+);w=(\d+)", RegexOptions.Compiled);

    public RateLimitPolicyHandler() : base(new HttpClientHandler())
    { }

    protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    {
        if (_rateLimiter is not null)
        {
            using var rateLimitLease = await _rateLimiter.WaitAsync(1, cancellationToken);
            if (rateLimitLease.IsAcquired)
            {
                return await base.SendAsync(request, cancellationToken);
            }

            var rateLimitResponse = new HttpResponseMessage(HttpStatusCode.TooManyRequests);
            rateLimitResponse.Content = new StringContent($"Service rate limit policy ({_rateLimitPolicy}) exceeded!");

            if (rateLimitLease.TryGetMetadata(MetadataName.RetryAfter, out var retryAfter))
            {
                rateLimitResponse.Headers.Add("Retry-After", ((int)retryAfter.TotalSeconds).ToString(NumberFormatInfo.InvariantInfo));
            }

            return rateLimitResponse;
        }

        var response = await base.SendAsync(request, cancellationToken);

        if (response.Headers.Contains("RateLimit-Policy"))
        {
            _rateLimitPolicy = response.Headers.GetValues("RateLimit-Policy").FirstOrDefault();

            if (_rateLimitPolicy is not null)
            {
                Match rateLimitPolicyMatch = RATE_LIMIT_POLICY_REGEX.Match(_rateLimitPolicy);

                if (rateLimitPolicyMatch.Success)
                {
                    int limit = Int32.Parse(rateLimitPolicyMatch.Groups[1].Value);
                    int window = Int32.Parse(rateLimitPolicyMatch.Groups[2].Value);

                    _rateLimiter = new FixedWindowRateLimiter(new FixedWindowRateLimiterOptions(
                        limit,
                        QueueProcessingOrder.NewestFirst,
                        0,
                        TimeSpan.FromSeconds(window),
                        true
                    ));

                    string? rateLimitRemaining = response.Headers.GetValues("RateLimit-Remaining").FirstOrDefault();
                    if (Int32.TryParse(rateLimitRemaining, out int remaining))
                    {
                        using var rateLimitLease = await _rateLimiter.WaitAsync(limit - remaining, cancellationToken);
                    }
                }
            }
        }

        return response;
    }
}

The above code also uses the RateLimit-Remaining header value to acquire leases for requests which are no longer available in the initial window.

Now depending if the sample code is run with the RateLimitPolicyHandler in the HttpClient pipeline or not, the console output will be different as those 429 will be coming from a different place.

Opinions

The rate limit headers seem like an interesting addition for communicating services usage quotas. Properly used in the right situations might be a useful tool, it is just important not to treat them as guarantees.

Serving rate limit headers from ASP.NET Core right now has its challenges. If they will become a standard and gain popularity, I think this will change.

If you want to play with the samples, they are available on GitHub.

I'm continuing my series on implementing the samples from Micro Frontends in Action in ASP.NET Core:

In the previous post, I described how I've deployed the two services which provide fragments of the frontend to the single Container Apps environment and then hidden them behind a YARP-based proxy. This technique (called server-side routing) solves some problems related to bad user experience (multiple domains), browser security (CORS), or search engines. That said, there are some other problems on the table.

The first common problem is performance. Server-side routing does improve performance over solution where fragments are loaded directly from different domains by removing multiple DNS lookups, SSL handshakes, etc. Still, separated AJAX calls can have a very negative impact on overall page load time, especially on slower connections.

The second common problem is layout shifts. When fragments are being loaded, it can often cause already visible page content to "jump". This is frustrating for the end-users.

A solution to both of these problems can be providing a complete page to the browser in a response to the first request.

Server-Side Composition

Server-side composition is a technique, where the page is being fully assembled (which means requesting all the required fragments) on the server. The composition can be done either by a central service (a proxy) or can be done in a decentralized manner, where every service requests fragments it requires to build its own UI. As the current solution already has a proxy in place, I've decided to start with a centralized approach. There are two possible mechanisms discussed in the book for this purpose: Server-Side Includes (SSI) and Edge-Side Includes (ESI).

Server-Side Includes (SSI)

Server-Side Includes is a quite old mechanism, it dates back to the NCSA HTTPd web server. It defines a set of directives (called commands or elements in some implementations) that can be placed in HTML and evaluated by the server while the page is being served. Currently, SSI is supported by Apache, Nginx, and IIS. The subset of supported directives varies between implementations, but the most useful and always available one is include, which the server replaces with a file or result of a request. All that a service needs to do, is put that directive as part of the returned markup.

<html>
  ...
  <body class="decide_layout">
    ...
    <aside class="decide_recos">
      <!--#include virtual="/inspire/fragment/recommendations/porsche" -->
    </aside>
  </body>
</html>

All the magic needs to happen at the proxy level, the only question is how?

Supporting SSI in YARP With Response Body Transform

Every well-respected reverse proxy provides more capabilities than just routing and YARP is no different. One of the capabilities provided by YARP, which goes beyond routing, is transforms. Transforms allow for modifying parts of the request or response as part of the flow. Currently, there are three categories of transforms:

  • Request
  • Response
  • Response Trailers

In every one of those categories, YARP provides a number of built-in transforms which allow for modifying path, query string, client certificates, and headers. There are no built-in transforms for request and response body, which probably makes sense as transforms including request and response body are slightly tricky and potentially dangerous. The first tricky part is that direct forwarding ignores any modifications to the response body which transforms could make, so the proxy service needs to be switched to a "full" reverse proxy experience.

app.Run();

var builder = WebApplication.CreateBuilder(args);

...

builder.Services.AddReverseProxy();

var app = builder.Build();

app.MapReverseProxy();

app.Run();

Moving away from direct forwarding means that there is no longer a way to define path prefixes directly. The out-of-the-box approach for configuring the reverse proxy requires is through configuration files. This would mean that I would have to change the way in which the deployment workflow provides the proxy with URLs to other services. That's something I really didn't want to do. Luckily, there is a possibility of implementing configuration providers to load the configuration programmatically from any source. The documentation even contains an example of an in-memory configuration provider which I've basically copy-pasted (looking at the YARP backlog it seems that the team has noticed its usefulness and it will be available out-of-the-box as well). This allowed me to keep the routes and clusters (this is how destinations are represented in YARP) configuration in the code.

...

var routes = new[]
{
    ...
    new RouteConfig
    {
        RouteId = Constants.DECIDE_ROUTE_ID,
        ClusterId = Constants.DECIDE_CLUSTER_ID,
        Match = new RouteMatch { Path = "/decide/{**catch-all}" }
    },
    ...
};

var clusters = new[]
{
    new ClusterConfig()
    {
        ClusterId = Constants.DECIDE_CLUSTER_ID,
        Destinations = new Dictionary<string, DestinationConfig>(StringComparer.OrdinalIgnoreCase)
        {
            { Constants.DECIDE_SERVICE_URL, new DestinationConfig() { Address = builder.Configuration[Constants.DECIDE_SERVICE_URL] } }
        }
    },
    ...
};

builder.Services.AddReverseProxy()
    .LoadFromMemory(routes, clusters);

...

With the configuration in place, it's time for the transform. There are two main ways for adding transforms. One is a callback and the other is ITransformProvider implementation. The ITransformProvider implementation gives more flexibility and isolation, so I've decided to go with it. As the implementation will be a registered dependency, it gives full access to dependency injection. It also gives validation capabilities for routes and clusters.

The simplest ("no-op") implementation of ITransformProvider can look like below.

internal class SsiTransformProvider : ITransformProvider
{
    public void ValidateRoute(TransformRouteValidationContext context)
    { }

    public void ValidateCluster(TransformClusterValidationContext context)
    { }

    public void Apply(TransformBuilderContext transformBuildContext)
    {
        transformBuildContext.AddResponseTransform(TransformResponse);
    }

    private ValueTask TransformResponse(ResponseTransformContext responseContext)
    {
        return default;
    }
}

In order to register that ITransformProvider implementation (and make TranformResponse part of the flow) it is enough to call AddTransforms.

...

builder.Services.AddReverseProxy()
    .LoadFromMemory(routes, clusters)
    .AddTransforms<SsiTransformProvider>();

...

This is where it is important to understand further specifics of transforms that are working with the request or response body. As a result of the Apply method from the above implementation, the TranformResponse will be executed for every flow going through YARP. This is too broad because if that transform deals with request or response body, it will come with a performance penalty. It will have to read (essentially buffer) the response from the destination. The moment the body has been read, it will also have to be written to the HttpContext of the YARP response, otherwise the response will be empty. This is happening because YARP doesn't buffer the response as part of the flow (due to performance reasons), instead it attempts to read the stream which in this case is at the end.

The performance penalty means that there is a need to limit the scope of impact by adding the transform only to certain routes. The routes which should be transformed need to be somehow marked. For that purpose I've decided to include additional information in the routes through metadata. Metadata is a dictionary available on RouteConfig. I've defined a specific key and value for which the Apply method will check before adding the transform. I've also added a statically available dictionary which can be used to set the metadata.

internal class SsiTransformProvider : ITransformProvider
{
    private const string SSI_METADATA_FLAG = "SSI";
    private const string SSI_METADATA_FLAG_ON = "ON";

    public static IReadOnlyDictionary<string, string> SsiEnabledMetadata { get; } = new Dictionary<string, string>()
    {
        { SSI_METADATA_FLAG, SSI_METADATA_FLAG_ON }
    };

    ...

    public void Apply(TransformBuilderContext transformBuildContext)
    {
        if (transformBuildContext.Route.Metadata is not null
            && transformBuildContext.Route.Metadata.ContainsKey(SSI_METADATA_FLAG)
            && transformBuildContext.Route.Metadata[SSI_METADATA_FLAG] == SSI_METADATA_FLAG_ON)
        {
            transformBuildContext.AddResponseTransform(TransformResponse);
        }
    }

    ...
}

Thanks to the in-memory configuration provider, including those metadata mean just setting one more property. While doing this I've also increased the granularity of routes to further limit the affected scope.

...

var routes = new[]
{
    ...
    new RouteConfig
    {
        RouteId = Constants.DECIDE_ROUTE_ID + "-static",
        ClusterId = Constants.DECIDE_CLUSTER_ID,
        Match = new RouteMatch { Path = "/decide/static/{**catch-all}" }
    },
    new RouteConfig
    {
        RouteId = Constants.DECIDE_ROUTE_ID,
        ClusterId = Constants.DECIDE_CLUSTER_ID,
        Match = new RouteMatch { Path = "/decide/{**catch-all}" },
        Metadata = SsiTransformProvider.SsiEnabledMetadata
    },
    ...
};

...

Now it's time for the actual transformation. To focus on the logic needed to support SSI include directive, let's get the response body reading and writing out of the way.

internal class SsiTransformProvider : ITransformProvider
{
    ...

    private ValueTask TransformResponse(ResponseTransformContext responseContext)
    {

        string proxyResponseContent = await responseContext.ProxyResponse.Content.ReadAsStringAsync();

        responseContext.SuppressResponseBody = true;

        ...

        byte[] proxyResponseContentBytes = Encoding.UTF8.GetBytes(proxyResponseContent);
        responseContext.HttpContext.Response.ContentLength = proxyResponseContentBytes.Length;
        await responseContext.HttpContext.Response.Body.WriteAsync(proxyResponseContentBytes);
    }
}

In order to get the include directive from the response body, I've decided to use (I don't believe I'm saying this) regex. The snippet below will grab all the directives. The group with index one of the resulting match will contain the directive name (all others than include are to be ignored) while the group with index two will contain parameters for further parsing (I'm doing this with another regex, but ultimately I care only for virtual parameter).

internal class SsiTransformProvider : ITransformProvider
{
    private static readonly Regex SSI_DIRECTIVE_REGEX = new Regex(
        @"<!--\#([a-z]+)\b([^>]+[^\/>])?-->",
        RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.IgnorePatternWhitespace
    );

    ...

    private ValueTask TransformResponse(ResponseTransformContext responseContext)
    {

        ...

        var directives = SSI_DIRECTIVE_REGEX.Matches(proxyResponseContent)

        ...
    }
}

To get the content to which the include directive is pointing, I will need to make an HTTP request to a specific service. The configuration provided for YARP already has an URL for that service. YARP also maintains a HttpClient instance dedicated to the cluster to which the service belongs. It would be nice to reuse it. In order to do that, first I needed to identify the endpoint for the path to which the virtual parameter is pointing.

Endpoint? virtualEndpoint = null;

var endpointDataSource = context.RequestServices.GetService<EndpointDataSource>();
if (endpointDataSource is not null)
{
    var virtualPath = new PathString(directive.Parameters[VIRTUAL_PARAMETER]);
    foreach (Endpoint possibleVirtualEndpoint in endpointDataSource.Endpoints)
    {
        var routeEndpoint = possibleVirtualEndpoint as RouteEndpoint;
        if (routeEndpoint is not null)
        {
            var routeTemplateMatcher = new TemplateMatcher(new RouteTemplate(routeEndpoint.RoutePattern), _emptyRouteValueDictionary);
            if (routeTemplateMatcher.TryMatch(virtualPath, _emptyRouteValueDictionary))
            {
                virtualEndpoint = possibleVirtualEndpoint;
                break;
            }
        }
    }
}

The endpoint also has a metadata collection. In this collection, YARP keeps the route model, which includes the cluster model.

ClusterModel? cluster = null;

foreach (var endpointMetadata in virtualEndpoint.Metadata)
{
    var proxyRoute = endpointMetadata as RouteModel;
    if (proxyRoute is not null)
    {
        cluster = proxyRoute.Cluster?.Model;
        break;
    }
}

The cluster model contains the configured destinations (with URLs) and that mentioned HttpClient instance. All that remains is to build the request URI, make the request, and read the content.

string virtualUri = cluster.Config.Destinations.FirstOrDefault().Value.Address + parameters["virtual"];

HttpResponseMessage response = await cluster.HttpClient.SendAsync(new HttpRequestMessage(HttpMethod.Get, virtualUri), CancellationToken.None);

string directiveOutputContent = await response.Content.ReadAsStringAsync();

Once the content has been read, the directive can be replaced in the body.

proxyResponseContent = proxyResponseContent.Substring(0, directives[directiveIndex].Index)
    + directiveOutputContent
    + proxyResponseContent.Substring(directives[directiveIndex].Index + directives[directiveIndex].Length);

This Is Just a Proof of Concept

Yes, this code (even the little more polished version available in the repository) is just a POC. It's missing constraints, error checking, support for multiple destinations in a cluster, support for other parameters of the include directive, and much more.

There should be also further performance considerations. The approach that the sample code takes (buffer the body, request content for include directives in parallel, and then build the final response body) is typical for SSI, but an approach that streams the body whenever possible could be considered. This is for example how ESI (which is a more modern mechanism) is sometimes implemented.

The only goal of this post (and related sample) is to show some YARP capabilities which can be used for achieving server-side composition at its level.

Recently, I've been reading Micro Frontends in Action and while doing it I've decided that I want to implement the samples in ASP.NET Core and host them on Azure. Then I had a thought, that maybe I can use this as an opportunity to play with some technologies I didn't have a chance to play with yet, like YARP or Azure Container Apps. Once I did that, the next thought was that maybe I should write down some of this and maybe someone will find it useful. So here we are, possibly starting a new series around micro frontends techniques:

Micro Frontends in Action

One of the praises for Micro Frontends in Action says that it's an excellent starting point to understanding how to introduce micro frontends in your projects. I'm about one-third through it and I'm willing to agree with that statement. The book starts with very simple techniques like page transition via links, composition via iframe, and composition via Ajax. Then it moves to server-side composition, advanced techniques for client-side composition, communication patterns, and more (I don't know, I haven't gotten there yet). Up to this point, it has been interesting and engaging (this is my opinion and just to be clear - nobody is paying me to read the book and provide an opinion about it).

The book contains samples for every discussed technique and it was just too tempting not to implement them with technologies I like. That doesn't mean I will implement every single one of those samples. I will also certainly not describe every single one of those implementations. I'm doing it for fun.

I also recommend you read the book if you are interested in the subject - a blog post describing some specific implementation will not provide you with information about benefits, drawbacks, and when to use a certain technique.

The Landscape So Far

The technique I'm going to describe in this post is server-side routing. Prior to introducing that technique, the project consists of two services: Decide and Inspire. The Decide service is loading frontend fragments provided by Inspire service via Ajax request.

Composition via Ajax Frontend Layout

Under the hood, both services are simple ASP.NET Core MVC applications that serve views based on static HTML from original samples, where the URLs are generated based on configuration.

@using Microsoft.Extensions.Configuration
@inject IConfiguration Configuration
@{string decideServiceUrl = Configuration["DECIDE_SERVICE_URL"];}
<link href="@(Context.Request.Scheme + "://" + Context.Request.Host + "/static/fragment.css")" rel="stylesheet" />
<div class="inspire_fragment">
  <h2 class="inspire_headline">Recommendations</h2>
  <div class="inspire_recommendations">
    <a href="@(decideServiceUrl + "/product/fendt")"><img src="https://mi-fr.org/img/fendt.svg" /></a>
    <a href="@(decideServiceUrl + "/product/eicher")"><img src="https://mi-fr.org/img/eicher.svg" /></a>
  </div>
</div>

The Inspire service additionally defines a CORS policy to enable requesting the fragments from different domain via Ajax.

var builder = WebApplication.CreateBuilder(args);

string decideServiceCorsPolicyName = "decide-service-cors-policy";

builder.Services.AddCors(options =>
{
    options.AddPolicy(name: decideServiceCorsPolicyName, policy =>
    {
        policy.WithOrigins(builder.Configuration["DECIDE_SERVICE_URL"]);
        policy.AllowAnyHeader();
        policy.WithMethods("GET");
    });
});

...

app.UseRouting();

app.UseCors(decideServiceCorsPolicyName);

...

Both services are containerized and deployed as two Container Apps within a single Container Apps environment.

az containerapp create \
  -n ${DECIDE_CONTAINERAPP} \
  -i ${CONTAINER_REGISTRY}.azurecr.io/decide:latest \
  -g ${RESOURCE_GROUP} \
  --environment ${CONTAINERAPPS_ENVIRONMENT} \
  --ingress external \
  --target-port 3001 \
  --min-replicas 1 \
  --registry-server ${CONTAINER_REGISTRY}.azurecr.io

az containerapp create \
  -n ${INSPIRE_CONTAINERAPP} \
  -i ${CONTAINER_REGISTRY}.azurecr.io/inspire:latest \
  -g ${RESOURCE_GROUP} \
  --environment ${CONTAINERAPPS_ENVIRONMENT} \
  --ingress external \
  --target-port 3002 \
  --min-replicas 1 \
  --registry-server ${CONTAINER_REGISTRY}.azurecr.io

This results in the following Container Apps solution.

Composition via Ajax Container Apps Solution

I've created a GitHub Actions workflow that performs all the necessary steps (creating Azure resources, building and pushing containers images, and deploying Container Apps) to set up the entire solution from scratch. There is one tricky step in that workflow. Both services must know each other URLs but those are available only after Container Apps are created. To solve this I'm first getting the ingress information for every app (with az containerapp ingress show) and then write them to environment variables (using the multiline strings approach).

jobs:
  ..
  deploy-to-container-apps:
  ..
  steps:
    ..
    - name: Get Services Ingress
      run: |
        echo 'DECIDE_CONTAINERAPP_INGRESS_JSON<<EOF' >> $GITHUB_ENV
        az containerapp ingress show -n ${DECIDE_CONTAINERAPP} -g ${RESOURCE_GROUP} >> $GITHUB_ENV
        echo 'EOF' >> $GITHUB_ENV
        echo 'INSPIRE_CONTAINERAPP_INGRESS_JSON<<EOF' >> $GITHUB_ENV
        az containerapp ingress show -n ${INSPIRE_CONTAINERAPP} -g ${RESOURCE_GROUP} >> $GITHUB_ENV
        echo 'EOF' >> $GITHUB_ENV
    ..

Next, I'm using the fromJson expression to get FQDN from ingress information and update the Container Apps.

jobs:
  ..
  deploy-to-container-apps:
  ..
  steps:
    ..
    - name: Configure Services URLs
      run: |
        az containerapp update -n ${DECIDE_CONTAINERAPP} -g ${RESOURCE_GROUP} --set-env-vars INSPIRE_SERVICE_URL=https://${{ fromJSON(env.INSPIRE_CONTAINERAPP_INGRESS_JSON).fqdn }}
        az containerapp update -n ${INSPIRE_CONTAINERAPP} -g ${RESOURCE_GROUP} --set-env-vars DECIDE_SERVICE_URL=https://${{ fromJSON(env.DECIDE_CONTAINERAPP_INGRESS_JSON).fqdn }}
    ..

The Challenge

There is a problem with this solution. There are two services and each of them is available to public requests under a different domain. This has several drawbacks:

  • Bad user experience (requests flying to different domains).
  • Internal infrastructure of the solution is exposed publicly.
  • Performance (two DNS lookups, two SSL handshakes, etc.).
  • Indexing by search engines.

To solve this a central service (a proxy), where all requests will arrive, is needed.

Introducing YARP as a Solution

For about a year, the ASP.NET Core stack have its own reverse proxy - YARP. It provides a lot of routing features like headers-based routing, session affinity, load balancing, or destination health checks. In this scenario I'm going to use direct forwarding to simply forward requests to specific services based on a path. I've also decided to set up the rules from code instead of using configuration (it seemed simpler as I still could provide service URLs through environment variables). For this purpose, I've created a MapForwarder extension method.

public static void MapForwarder(this IEndpointRouteBuilder endpoints, string pattern, string serviceUrl)
{
    var forwarder = endpoints.ServiceProvider.GetRequiredService<IHttpForwarder>();
    var requestConfig = new ForwarderRequestConfig { ActivityTimeout = TimeSpan.FromMilliseconds(500) };

    endpoints.Map(pattern, async httpContext =>
    {
        var error = await forwarder.SendAsync(httpContext, serviceUrl, HttpClient, requestConfig, HttpTransformer.Default);

        if (error != ForwarderError.None)
        {
            var errorFeature = httpContext.GetForwarderErrorFeature();
            var exception = errorFeature?.Exception;
        }
    });
}

I've followed the approach of defining service-specific path prefixes as well as specific prefixes for important pages.

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddHttpForwarder();

var app = builder.Build();

string decideServiceUrl = app.Configuration["DECIDE_SERVICE_URL"];
string inspireServiceUrl = app.Configuration["INSPIRE_SERVICE_URL"];

app.UseRouting();
app.UseEndpoints(endpoints =>
{
    // Per service prefixes
    endpoints.MapForwarder("/", decideServiceUrl);

    // Per service prefixes
    endpoints.MapForwarder("/decide/{**catch-all}", decideServiceUrl);
    endpoints.MapForwarder("/inspire/{**catch-all}", inspireServiceUrl);

    // Per page prefixes
    endpoints.MapForwarder("/product/{**catch-all}", decideServiceUrl);
    endpoints.MapForwarder("/recommendations/{**catch-all}", inspireServiceUrl);
});

app.Run();

As it will be now the proxy responsibility to know the addresses of both services, they no longer need to be able to point to each other. All the paths can now be relative, as long as they follow the established rules.

<link href="/inspire/static/fragment.css" rel="stylesheet" />
<div class="inspire_fragment">
  <h2 class="inspire_headline">Recommendations</h2>
  <div class="inspire_recommendations">
    <a href="/product/fendt"><img src="https://mi-fr.org/img/fendt.svg" /></a>
    <a href="/product/eicher"><img src="https://mi-fr.org/img/eicher.svg" /></a>
  </div>
</div>

There is also no need for CORS anymore, as from a browser perspective there are no cross-origin requests. So I've removed that code from the Inspire service.

Now it's time to hide the services from the public. For starters let's change their ingress to internal.

az containerapp create \
  -n ${DECIDE_CONTAINERAPP} \
  -i ${CONTAINER_REGISTRY}.azurecr.io/decide:latest \
  -g ${RESOURCE_GROUP} \
  --environment ${CONTAINERAPPS_ENVIRONMENT} \
  --ingress internal \
  --target-port 3001 \
  --min-replicas 1 \
  --registry-server ${CONTAINER_REGISTRY}.azurecr.io

az containerapp create \
  -n ${INSPIRE_CONTAINERAPP} \
  -i ${CONTAINER_REGISTRY}.azurecr.io/inspire:latest \
  -g ${RESOURCE_GROUP} \
  --environment ${CONTAINERAPPS_ENVIRONMENT} \
  --ingress internal \
  --target-port 3002 \
  --min-replicas 1 \
  --registry-server ${CONTAINER_REGISTRY}.azurecr.io

This way the services are now accessible only from within the Container Apps environment. The proxy can be deployed to the same Container Apps environment and expose the desired traffic outside.

az containerapp create \
  -n ${PROXY_CONTAINERAPP} \
  -i ${CONTAINER_REGISTRY}.azurecr.io/proxy:latest \
  -g ${RESOURCE_GROUP} \
  --environment ${CONTAINERAPPS_ENVIRONMENT} \
  --ingress external \
  --target-port 3000 \
  --min-replicas 1 \
  --registry-server ${CONTAINER_REGISTRY}.azurecr.io \
  --env-vars INSPIRE_SERVICE_URL=https://${INSPIRE_CONTAINERAPP_FQDN} DECIDE_SERVICE_URL=https://${DECIDE_CONTAINERAPP_FQDN}

After this change, the Container Apps solution looks like in the below diagram.

Server-Side Routing via YARP Container Apps Solution

That's All Folks

At least for this post. You can find the final solution here and its GitHub Actions workflow here. The services have become a little bit strange in the process (they server static HTML in an unnecessarily complicated way) but they are not the focus here, the server-side routing technique is.

I don't know if or when I'm going to play with another technique, but if I will it's probably going to be something around server-side composition.

Back in July, I've shared my experiments around new JSON async streaming capabilities in ASP.NET Core 6. Last week I've received a question about utilizing these capabilities in the Blazor WebAssembly application. The person asking the question has adopted the DeserializeAsyncEnumerable based client code, but it didn't seem to work properly. All the results were always displayed at once. As I didn't have a Blazor WebAssembly sample as part of my streaming JSON objects demo, I've decided I'll add one and figure out the answer to the question along the way.

Blazor WebAssembly and IAsyncEnumerable

Before I focus on the problem of results not being received in an async stream manner, I think it is worth discussing the way of working with IAsyncEnumerable in Blazor WebAssembly. What's the challenge here? The first one is that await foreach can't be used in the page markup, only in the code block. So the markup must use a synchronous loop.

<table>
    <thead>
        <tr>
            <th>Date</th>
            <th>Temp. (C)</th>
            <th>Temp. (F)</th>
            <th>Summary</th>
        </tr>
    </thead>
    <tbody>
        @foreach (WeatherForecast weatherForecast in weatherForecasts)
        {
            <tr>
                <td>@weatherForecast.DateFormatted</td>
                <td>@weatherForecast.TemperatureC</td>
                <td>@weatherForecast.TemperatureF</td>
                <td>@weatherForecast.Summary</td>
            </tr>
        }
    </tbody>
</table>

That brings us to the second challenge. If the await foreach can be used only in the code block, how the streamed results can be rendered as they come? Here the solution comes in form of the StateHasChanged method. Calling this method will trigger a render. With this knowledge, we can adopt the DeserializeAsyncEnumerable based code from my previous post.

@code {

    private List<WeatherForecast> weatherForecasts = new List<WeatherForecast>();

    private async Task StreamWeatherForecastsJson()
    {
        weatherForecasts = new List<WeatherForecast>();

        StateHasChanged();

        using HttpResponseMessage response = await Http.GetAsync("api/WeatherForecasts/negotiate-stream", HttpCompletionOption.ResponseHeadersRead);

        response.EnsureSuccessStatusCode();

        using Stream responseStream = await response.Content.ReadAsStreamAsync();

        await foreach (WeatherForecast weatherForecast in JsonSerializer.DeserializeAsyncEnumerable<WeatherForecast>(
            responseStream,
            new JsonSerializerOptions
            {
                PropertyNameCaseInsensitive = true,
                DefaultBufferSize = 128
            }))
        {
            weatherForecasts.Add(weatherForecast);

            StateHasChanged();
        }
    }
}

Running that code put me in the exact same spot where the person asking the question was. All the results were rendered at once after the entire wait time. What to do, when you have no idea what might be wrong and where? Dump what you can to the console ;). No, I'm serious. Debugging through console.log is in fact quite useful in many situations and I'm not ashamed of using it here. I've decided that the diagnostic version will perform direct response stream reading.

@code {

    ...

    private async Task StreamWeatherForecastsJson()
    {
        Console.WriteLine($"-- {nameof(StreamWeatherForecastsJson)} --");
        Console.WriteLine($"[{DateTime.UtcNow:hh:mm:ss.fff}] Requesting weather forecasts . . .");

        using HttpResponseMessage response = await Http.GetAsync("api/WeatherForecasts/negotiate-stream", HttpCompletionOption.ResponseHeadersRead);

        response.EnsureSuccessStatusCode();

        Console.WriteLine($"[{DateTime.UtcNow:hh:mm:ss.fff}] Receving weather forecasts . . .");

        using Stream responseStream = await response.Content.ReadAsStreamAsync();

        Console.WriteLine($"[{DateTime.UtcNow:hh:mm:ss.fff}] Weather forecasts stream obtained . . .");

        while (true)
        {
            byte[] buffer = ArrayPool<byte>.Shared.Rent(128);
            int bytesRead = await responseStream.ReadAsync(buffer);

            Console.WriteLine($"[{DateTime.UtcNow:hh:mm:ss.fff}] ({bytesRead}/{buffer.Length}) {Encoding.UTF8.GetString(buffer[0..bytesRead])}");

            ArrayPool<byte>.Shared.Return(buffer);

            if (bytesRead == 0)
            {
                break;
            }
        }

        Console.WriteLine($"[{DateTime.UtcNow:hh:mm:ss.fff}] Weather forecasts has been received.");
        Console.WriteLine();
    }
}

Below you can see the output from browser developer tools.

-- StreamWeatherForecastsJson --
[08:04:01.183] Requesting weather forecasts . . .
[08:04:01.436] Receving weather forecasts . . .
[08:04:02.420] Weather forecasts stream obtained . . .
[08:04:02.426] (128/128) [{"dateFormatted":"06.12.2021","temperatureC":28,"temperatureF":82,"summary":"Hot"},{"dateFormatted":"07.12.2021","temperatureC"
[08:04:02.429] (128/128) :36,"temperatureF":96,"summary":"Scorching"},{"dateFormatted":"08.12.2021","temperatureC":-7,"temperatureF":20,"summary":"Mild"}
[08:04:02.431] (128/128) ,{"dateFormatted":"09.12.2021","temperatureC":-6,"temperatureF":22,"summary":"Hot"},{"dateFormatted":"10.12.2021","temperatureC"
[08:04:02.433] (128/128) :40,"temperatureF":103,"summary":"Cool"},{"dateFormatted":"11.12.2021","temperatureC":44,"temperatureF":111,"summary":"Swelterin
[08:04:02.434] (128/128) g"},{"dateFormatted":"12.12.2021","temperatureC":-3,"temperatureF":27,"summary":"Balmy"},{"dateFormatted":"13.12.2021","temperat
[08:04:02.435] (128/128) ureC":1,"temperatureF":33,"summary":"Sweltering"},{"dateFormatted":"14.12.2021","temperatureC":3,"temperatureF":37,"summary":"Ho
[08:04:02.437] (88/128) t"},{"dateFormatted":"15.12.2021","temperatureC":19,"temperatureF":66,"summary":"Mild"}]tureC":3,"temperatureF":37,"summary":"Ho
[08:04:02.438] (0/128) t"},{"dateFormatted":"15.12.2021","temperatureC":19,"temperatureF":66,"summary":"Mild"}]tureC":3,"temperatureF":37,"summary":"Ho
[08:04:02.439] Weather forecasts has been received.

So the call which is blocking the whole thing seems to be ReadAsStreamAsync, when it returns the entire response is already available. All I knew at this point was that Blazor WebAssembly is using a special HttpMessageHandler. I needed to dig deeper.

Digging Into BrowserHttpHandler

There is a number of things that have dedicated implementations for Blazor WebAssembly. The HttpClient stack is one of those things. Well, there is no access to native sockets in the browser, so the HTTP calls must be performed based on browser-provided APIs. The BrowserHttpHandler is implemented on top of Fetch API. Inspecting its code shows that it can provide response content in one of two forms.

The first one is BrowserHttpContent, which is based on arrayBuffer method. This means, that it will always read the response stream to its completion, before making the content available.

The second one is StreamContent wrapping WasmHttpReadStream, which is based on readable streams. This one allows for reading response as it comes.

How does BrowserHttpHandler decide which one to use? In order for WasmHttpReadStream to be used, two conditions must be met - the browser must support readable streams and the WebAssemblyEnableStreamingResponse option must be enabled on the request. Now we are getting somewhere. Further search for WebAssemblyEnableStreamingResponse will reveal a SetBrowserResponseStreamingEnabled extension method. Let's see what happens if it's used.

@code {

    ...

    private async Task StreamWeatherForecastsJson()
    {
        Console.WriteLine($"-- {nameof(StreamWeatherForecastsJson)} --");
        Console.WriteLine($"[{DateTime.UtcNow:hh:mm:ss.fff}] Requesting weather forecasts . . .");

        HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Get, "api/WeatherForecasts/negotiate-stream");
        request.SetBrowserResponseStreamingEnabled(true);

        using HttpResponseMessage response = await Http.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);

        response.EnsureSuccessStatusCode();

        ...
    }
}

This gives the desired output.

-- StreamWeatherForecastsJson --
[08:53:14.722] Requesting weather forecasts . . .
[08:53:15.002] Receving weather forecasts . . .
[08:53:15.009] Weather forecasts stream obtained . . .
[08:53:15.018] (84/128) [{"dateFormatted":"06.12.2021","temperatureC":31,"temperatureF":87,"summary":"Cool"}
[08:53:15.057] (84/128) ,{"dateFormatted":"07.12.2021","temperatureC":18,"temperatureF":64,"summary":"Cool"}
[08:53:15.166] (86/128) ,{"dateFormatted":"08.12.2021","temperatureC":10,"temperatureF":49,"summary":"Chilly"}
[08:53:15.276] (84/128) ,{"dateFormatted":"09.12.2021","temperatureC":33,"temperatureF":91,"summary":"Mild"}
[08:53:15.386] (88/128) ,{"dateFormatted":"10.12.2021","temperatureC":-14,"temperatureF":7,"summary":"Freezing"}
[08:53:15.492] (84/128) ,{"dateFormatted":"11.12.2021","temperatureC":12,"temperatureF":53,"summary":"Warm"}
[08:53:15.600] (86/128) ,{"dateFormatted":"12.12.2021","temperatureC":6,"temperatureF":42,"summary":"Bracing"}
[08:53:15.710] (85/128) ,{"dateFormatted":"13.12.2021","temperatureC":48,"temperatureF":118,"summary":"Mild"}
[08:53:15.818] (89/128) ,{"dateFormatted":"14.12.2021","temperatureC":13,"temperatureF":55,"summary":"Scorching"}
[08:53:15.931] (88/128) ,{"dateFormatted":"15.12.2021","temperatureC":44,"temperatureF":111,"summary":"Chilly"}]
[08:53:15.943] (0/128) 
[08:53:15.946] Weather forecasts has been received.

Corrected Implementation

This means, that in order to have stream-based access to the response body (regardless if it's JSON or something else), one needs to explicitly enable it on the request. So the code which is able to receive async streamed JSON and properly deserialize it to IAsyncEnumerable should look like this.

@code {

    ...

    private async Task StreamWeatherForecastsJson()
    {
        weatherForecasts = new List<WeatherForecast>();

        StateHasChanged();

        HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Get, "api/WeatherForecasts/negotiate-stream");
        request.SetBrowserResponseStreamingEnabled(true);

        using HttpResponseMessage response = await Http.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);

        response.EnsureSuccessStatusCode();

        using Stream responseStream = await response.Content.ReadAsStreamAsync();

        await foreach (WeatherForecast weatherForecast in JsonSerializer.DeserializeAsyncEnumerable<WeatherForecast>(
            responseStream,
            new JsonSerializerOptions
            {
                PropertyNameCaseInsensitive = true,
                DefaultBufferSize = 128
            }))
        {
            weatherForecasts.Add(weatherForecast);

            StateHasChanged();
        }
    }
}

This works exactly as expected - the results are being rendered as they are being returned from the backend (with precision resulting from the size of the buffer).

Lesson Learned

Despite the fact that there is no dedicated TFM for Blazor WebAssembly, and it uses the "runs everywhere" one (net5.0, net6.0, etc.) there are platform differences. You will notice the APIs which aren't supported very quickly because they will throw PlatformNotSupportedException. But there are also more sneaky ones, ones which behavior is different. Blazor WebAssembly needs to be approached with this thought in the back of your mind. This will allow you to properly handle situations when things are not working the way you've expected.

A couple of weeks ago the Cosmos DB team has announced support for patching documents. This is quite a useful and long-awaited feature, as up to this point the only way to change the stored document was to completely replace it. This feature opens up new scenarios. For example, if you are providing a Web API on top of Cosmos DB, you can now directly implement support for the PATCH request method. I happen to have a small demo of such Web API, so I've decided to refresh it and play with leveraging this new capability.

Adding PATCH Requests Support to an ASP.NET Core Web API

An important part of handling a PATCH request is deciding on the request body format. The standard approach for that, in the case of JSON-based APIs, is JSON Patch. In fact, Cosmos DB is also using JSON Patch, so it should be easier to leverage it this way.

ASP.NET Core provides support for JSON Patch, but I've decided not to go with it. Why? It's designed around applying operations to an instance of an object and as a result, has internal assumptions about the supported list of operations. In the case of Cosmos DB, the supported operations are different, and that "applying" capability is not needed (it was a great way to implement PATCH when Cosmos DB provided on replace option). I've figured out it will be better to start fresh, with a lightweight model.

public class JsonPatchOperation
{
    [Required]
    public string Op { get; set; }

    [Required]
    public string Path { get; set; }

    public object Value { get; set; }
}

public class JsonPatch : List<JsonPatchOperation>
{ }

This class is quite generic and should allow for handling any request which body is compliant with JSON Patch structure. The first concretization I want to introduce is the list of available operations. Cosmos DB currently supports five operations: Add, Set, Remove, Replace, and Increment. In this demo (at least for now) I'm skipping the Increment because it would require a little bit different handling than others.

public enum JsonPatchOperationType
{
    Add,
    Set,
    Remove,
    Replace,,
    Invalid
}

As you can see, in the above enumeration I've also added the Invalid value. This will give me a way to represent the operations I don't intend to support through a specific value instead of e.g. throwing an exception.

public class JsonPatchOperation
{
    private string _op;
    private JsonPatchOperationType _operationType;

    [Required]
    public string Op
    {
        get { return _op; }

        set
        {
            JsonPatchOperationType operationType;
            if (!Enum.TryParse(value, ignoreCase: true, result: out operationType))
            {
                operationType = JsonPatchOperationType.Invalid;
            }

            _operationType = operationType;

            _op = value;
        }
    }

    public JsonPatchOperationType OperationType => _operationType;

    ...
}

Having not supported operations represented through a specific value also allows me to implement IValidatableObject which checks for them. This way, if the class is used as an action model, making a request with an unsupported operation will trigger a 400 Bad Request response.

public class JsonPatchOperation : IValidatableObject
{
    ...

    public IEnumerable<ValidationResult> Validate(ValidationContext validationContext)
    {
        if (OperationType == JsonPatchOperationType.Invalid)
        {
            yield return new ValidationResult($"Not supported operation: {Op}.", new[] { nameof(Op) });
        }
    }
}

Now all is needed is an action that will support the PATCH method.

[Route("api/[controller]")]
[ApiController]
public class CharactersController : Controller
{
    ...

    [HttpPatch("{id}")]
    public async Task<ActionResult<Character>> Patch(string id, JsonPatch update)
    {
        ...
    }

    ...
}

The next step will be proxying the deserialized JSON Patch to Cosmos DB .NET SDK.

Utilizing Cosmos DB Partial Updates

The Cosmos DB .NET SDK exposes partial document updates through the PatchItemAsync method on a container. This method expects a collection of PatchOperation instances. Instances representing specific operations can be created through static methods on PatchOperation which names correspond to operations names. So the conversion from JsonPatch to PatchOperation collection requires calling the appropriate method for every JsonPatchOperation. This is something a simple extensions method should handle.

public static class JsonPatchExtensions
{
    public static IReadOnlyList<PatchOperation> ToCosmosPatchOperations(this JsonPatch jsonPatchOperations)
    {

        List<PatchOperation> cosmosPatchOperations = new List<PatchOperation>(jsonPatchOperations.Count);
        foreach (JsonPatchOperation jsonPatchOperation in jsonPatchOperations)
        {
            switch (jsonPatchOperation.OperationType)
            {
                case JsonPatchOperationType.Add:
                    cosmosPatchOperations.Add(PatchOperation.Add(jsonPatchOperation.Path, jsonPatchOperation.Value));
                    break;
                case JsonPatchOperationType.Remove:
                    cosmosPatchOperations.Add(PatchOperation.Remove(jsonPatchOperation.Path));
                    break;
                case JsonPatchOperationType.Replace:
                    cosmosPatchOperations.Add(PatchOperation.Replace(jsonPatchOperation.Path, jsonPatchOperation.Value));
                    break;
                case JsonPatchOperationType.Set:
                    System.Int32 test = 25;
                    cosmosPatchOperations.Add(PatchOperation.Set(jsonPatchOperation.Path, jsonPatchOperation.Value));
                    break;
            }
        }

        return cosmosPatchOperations;
    }
}

Making the proper call in our action.

[Route("api/[controller]")]
[ApiController]
public class CharactersController : Controller
{
    ...

    [HttpPatch("{id}")]
    public async Task<ActionResult<Character>> Patch(string id, JsonPatch update)
    {
        ...

        ItemResponse<Character> characterItemResponse = await _starWarsCosmosClient.Characters.PatchItemAsync<Character>(
            id,
            PartitionKey.None,
            update.ToCosmosPatchOperations());

        return characterItemResponse.Resource;
    }

    ...
}

And we have some testable code. In order to test I've decided to attempt two operations: Set on /height and Add on /weight. This is represented by below JSON Patch body.

[
  {
    "op": "set",
    "path": "/height",
    "value": 195
  },
  {
    "op": "add",
    "path": "/weight",
    "value": 90
  }
]

I've hit F5, crated the request in Postman, and clicked Send. What I've received was a 500 Internal Server Error with a weird Newtonsoft.Json deserialization exception. A quick look into Cosmos DB Explorer revealed that the document now looks like this.

{
    ...
    "height": {
        "valueKind": 4
    },
    "weight": {
        "valueKind": 4
    },
    ...
}

This is not what I was expecting. What happened? The fact that PatchItemAsync worked without an exception suggested that this must be how Cosmos DB .NET SDK interpreted the JsonPatchOperation.Value. Through debugging I've quickly discovered that what JsonPatchOperation.Value actually holds is System.Text.Json.JsonElement. The Cosmos DB .NET SDK has no other way of dealing with that than serializing public properties - regardless of how smart the implementation is. This is because Cosmos DB .NET SDK (at least for now) is based on Newtonsoft.Json.

So, this is going to be a little bit harder. Conversion from JsonPatch to PatchOperation collection will require deserializing the value along the way. As this is still part of deserializing the request, I've figured out it will be best to put it into JsonPatchOperation.

public class JsonPatchOperation : IValidatableObject
{
    ...

    public T GetValue<T>()
    {
        return ((JsonElement)Value).Deserialize<T>();
    }
}

This will need to be called with the right type parameter, so a mapping between paths and types needs to be obtained. I've decided to make this information available through JsonPatch by making it generic and spicing with reflection.

public class JsonPatch<T> : List<JsonPatchOperation>
{
    private static readonly IDictionary<string, Type> _pathsTypes;

    static JsonPatch()
    {
        _pathsTypes = typeof(T).GetProperties().ToDictionary(p => $"/{Char.ToLowerInvariant(p.Name[0]) + p.Name[1..]}", p => p.PropertyType);
    }

    public Type GetTypeForPath(string path)
    {
        return _pathsTypes[path];
    }
}

A disclaimer is in order. This simple code will work only in this simple case. I'm looking only at top-level properties because my document only has top-level properties. But, in general, a path can be targeting a nested property e.g. /mather/familyName. This code will have to get more complicated to handle such a case.

To make the conversion from JsonPatch to PatchOperation work correctly, sadly, more reflection is needed as those generic calls have to be made with the right parameters at the runtime. The needed pieces can live together with the conversion extension method.

public static class JsonPatchExtensions
{
    private static MethodInfo _createAddPatchOperationMethodInfo = typeof(JsonPatchExtensions)
        .GetMethod(nameof(JsonPatchExtensions.CreateAddPatchOperation), BindingFlags.NonPublic | BindingFlags.Static);
    private static MethodInfo _createReplacePatchOperationMethodInfo = typeof(JsonPatchExtensions)
        .GetMethod(nameof(JsonPatchExtensions.CreateReplacePatchOperation), BindingFlags.NonPublic | BindingFlags.Static);
    private static MethodInfo _createSetPatchOperationMethodInfo = typeof(JsonPatchExtensions)
        .GetMethod(nameof(JsonPatchExtensions.CreateSetPatchOperation), BindingFlags.NonPublic | BindingFlags.Static);

    ...

    private static PatchOperation CreateAddPatchOperation<T>(JsonPatchOperation jsonPatchOperation)
    {
        return PatchOperation.Add(jsonPatchOperation.Path, jsonPatchOperation.GetValue<T>());
    }

    private static PatchOperation CreateReplacePatchOperation<T>(JsonPatchOperation jsonPatchOperation)
    {
        return PatchOperation.Replace(jsonPatchOperation.Path, jsonPatchOperation.GetValue<T>());
    }

    private static PatchOperation CreateSetPatchOperation<T>(JsonPatchOperation jsonPatchOperation)
    {
        return PatchOperation.Set(jsonPatchOperation.Path, jsonPatchOperation.GetValue<T>());
    }

    private static PatchOperation CreatePatchOperation<T>(
        MethodInfo createSpecificPatchOperationMethodInfo,
        JsonPatch<T> jsonPatchOperations,
        JsonPatchOperation jsonPatchOperation)
    {
        Type jsonPatchOperationValueType = jsonPatchOperations.GetTypeForPath(jsonPatchOperation.Path);

        MethodInfo createSpecificPatchOperationWithValueTypeMethodInfo =
            createSpecificPatchOperationMethodInfo.MakeGenericMethod(jsonPatchOperationValueType);

        return (PatchOperation)createSpecificPatchOperationWithValueTypeMethodInfo.Invoke(null, new object[] { jsonPatchOperation });
    }
}

The code above has been structured to isolate the reflection related part and cache the well-known things. Of course, this is a subject of personal preference, but I hope it's readable.

The last remaining thing is modifying the extension method.

public static class JsonPatchExtensions
{
    ...

    public static IReadOnlyList<PatchOperation> ToCosmosPatchOperations<T>(this JsonPatch jsonPatchOperations)
    {
        List<PatchOperation> cosmosPatchOperations = new List<PatchOperation>(jsonPatchOperations.Count);
        foreach (JsonPatchOperation jsonPatchOperation in jsonPatchOperations)
        {
            switch (jsonPatchOperation.OperationType)
            {
                case JsonPatchOperationType.Add:
                    cosmosPatchOperations.Add(CreatePatchOperation(_createAddPatchOperationMethodInfo, jsonPatchOperations, jsonPatchOperation));
                    break;
                case JsonPatchOperationType.Remove:
                    cosmosPatchOperations.Add(PatchOperation.Remove(jsonPatchOperation.Path));
                    break;
                case JsonPatchOperationType.Replace:
                    cosmosPatchOperations.Add(CreatePatchOperation(_createReplacePatchOperationMethodInfo, jsonPatchOperations, jsonPatchOperation));
                    break;
                case JsonPatchOperationType.Set:
                    cosmosPatchOperations.Add(CreatePatchOperation(_createSetPatchOperationMethodInfo, jsonPatchOperations, jsonPatchOperation));
                    break;
            }
        }

        return cosmosPatchOperations;
    }

    ...
}

Adjusting the action code, building the demo, running, going to Postman, sending a request, and it works!

What’s Missing?

This is demoware, so there is always something missing. I've already mentioned that parts of the code are handling only simple cases. But there are two additional subjects which require more attention.

One is validation. This code is not validating if paths provided for operations represent valid properties. It's also not validating if values can be converted to the right types and if the operations result in an invalid state of the entity. First, two things should be achievable at the JsonPatch level. The last one will require additional code in action.

The other subject is performance around that reflection code. It can be improved by caching the generic methods for specific types, but as the solution grows it might not be enough. It is worth thinking about another option here.

I might tackle those two subjects, but I don't know if and when. You can keep an eye on the demo project because if I do, the code will certainly end there.

Older Posts