Recently I've published code and written posts about working with asynchronous streaming data sources over HTTP using NDJSON. One of the follow-up questions I've received was how does it relate to async streaming coming in ASP.NET Core 6. As the answer isn't obvious, I've decided to describe the subject in more detail.

ASP.NET Core and IAsyncEnumerable

Since ASP.NET Core 3, IAsyncEnumerable can be returned directly from controller actions.

[Route("api/[controller]")]
[ApiController]
public class WeatherForecastsController : Controller
{
    ...

    [HttpGet("weather-forecast")]
    public IAsyncEnumerable<WeatherForecast> GetWeatherForecastStream()
    {
        async IAsyncEnumerable<WeatherForecast> streamWeatherForecastsAsync()
        {
            for (int daysFromToday = 1; daysFromToday <= 10; daysFromToday++)
            {
                WeatherForecast weatherForecast = await _weatherForecaster.GetWeatherForecastAsync(daysFromToday);

                yield return weatherForecast;
            };
        };

        return streamWeatherForecastsAsync();
    }
}

The ASP.NET Core will iterate IAsyncEnumerable in an asynchronous manner, buffer the result, and send it down the wire. The gain here is no blocking of calls and no risk of thread pool starvation, but there is no streaming of data to the client. If one would like to test this by making some requests, a possible first attempt could look like this.

private static async Task ConsumeJsonStreamAsync()
{
    Console.WriteLine($"[{DateTime.UtcNow:hh:mm:ss.fff}] Receving weather forecasts . . .");

    using HttpClient httpClient = new();

    using HttpResponseMessage response = await httpClient.GetAsync(
        "...",
        HttpCompletionOption.ResponseHeadersRead
    ).ConfigureAwait(false);

    response.EnsureSuccessStatusCode();

    IAsyncEnumerable<WeatherForecast> weatherForecasts = await response.Content
        .ReadFromJsonAsync<IAsyncEnumerable<WeatherForecast>>().ConfigureAwait(false);

    await foreach (WeatherForecast weatherForecast in weatherForecasts)
    {
        Console.WriteLine($"[{DateTime.UtcNow:hh:mm:ss.fff}] {weatherForecast.Summary}");
    }

    Console.WriteLine($"[{DateTime.UtcNow:hh:mm:ss.fff}] Weather forecasts has been received.");
}

This will fail. Until .NET 6 System.Text.Json doesn't support IAsyncEnumerable, the only option is to use IEnumerable.

private static async Task ConsumeJsonStreamAsync()
{
    ...

    IEnumerable<WeatherForecast> weatherForecasts = await response.Content
        .ReadFromJsonAsync<IEnumerable<WeatherForecast>>().ConfigureAwait(false);

    foreach (WeatherForecast weatherForecast in weatherForecasts)
    {
        ...
    }

    ...
}

The output will look like below (assuming it takes around 100ms to generate a single forecast).

[08:12:59.184] Receving weather forecasts . . .
[08:13:01.380] Cool
[08:13:01.381] Warm
[08:13:01.381] Sweltering
[08:13:01.381] Hot
[08:13:01.381] Chilly
[08:13:01.382] Scorching
[08:13:01.382] Hot
[08:13:01.382] Freezing
[08:13:01.382] Chilly
[08:13:01.382] Bracing
[08:13:01.382] Weather forecasts has been received.

Here the gain of properly using NDJSON is clear, as in such case the output would look more like this.

[08:13:01.400] Receving weather forecasts . . .
[08:13:01.538] Mild
[08:13:01.633] Freezing
[08:13:01.755] Mild
[08:13:01.862] Warm
[08:13:01.968] Warm
[08:13:02.075] Sweltering
[08:13:02.184] Freezing
[08:13:02.294] Chilly
[08:13:02.401] Freezing
[08:13:02.506] Hot
[08:13:02.513] Weather forecasts has been received.

Async Streaming in ASP.NET Core 6

One of ASP.NET Core improvements in .NET 6 is support for async streaming of IAsyncEnumerable. In .NET 6, System.Text.Json can serialize incoming IAsyncEnumerable in asynchronous manner. Thanks to that, the ASP.NET Core no longer buffers IAsyncEnumerable at ObjectResult level, the decision is made at output formatter level and the buffering occurs only in the case of Newtonsoft.Json based one.

Also, deserialization to IAsyncEnumerable is now supported by System.Text.Json, so the client code which was failing now works.

private static async Task ConsumeJsonStreamAsync()
{
    ...

    IAsyncEnumerable<WeatherForecast> weatherForecasts = await response.Content
        .ReadFromJsonAsync<IAsyncEnumerable<WeatherForecast>>().ConfigureAwait(false);

    await foreach (WeatherForecast weatherForecast in weatherForecasts)
    {
        ...
    }

    ...
}

Unfortunately, the result of running that code is disappointing. There is no difference from deserialization to IEnumerable. That shouldn't be a surprise as DeserializeAsync method (which is being used under the hood) signature doesn't allow streaming. This is why a new API, DeserializeAsyncEnumerable method, has been introduced to handle streaming deserialization.

private static async Task ConsumeJsonStreamAsync()
{
    ...

    Stream responseStream = await response.Content.ReadAsStreamAsync().ConfigureAwait(false);
    IAsyncEnumerable<WeatherForecast> weatherForecasts = JsonSerializer.DeserializeAsyncEnumerable<WeatherForecast>(
        responseStream,
        new JsonSerializerOptions
        {
            PropertyNameCaseInsensitive = true,
            DefaultBufferSize = 128
        });

    await foreach (WeatherForecast weatherForecast in weatherForecasts)
    {
        ...
    }

    ...
}

This time I was surprised because the result also didn't change. As I had no idea if my expectations about the behavior were correct, I've decided to ask. Turns out I've hit a bug. I've quickly downgraded from Preview 6 to Preview 4 and finally achieved the result I was fishing for.

[08:28:51.662] Receving weather forecasts . . .
[08:28:51.967] Cool
[08:28:52.068] Sweltering
[08:28:52.288] Cool
[08:28:52.289] Freezing
[08:28:52.396] Freezing
[08:28:52.614] Cool
[08:28:52.614] Cool
[08:28:52.723] Cool
[08:28:52.851] Cool
[08:28:52.851] Chilly
[08:28:52.854] Weather forecasts has been received.

You may have noticed that I've set DefaultBufferSize while passing JsonSerializerOptions to the DeserializeAsyncEnumerable method. This is very important if one wants to achieve streaming behavior. Internally, DeserializeAsyncEnumerable will read from the stream until the buffer is full or the stream has ended. If the buffer size is large (and the default is 16KB) there will be a significant delay in asynchronous iteration (in fact you can see irregularity in the above output resulting from exactly that).

The Conclusion

Async Streaming in ASP.NET Core 6 will allow achieving similar effects to NDJSON, if you understand your data very well and know how to configure the deserialization. That's the main difference from NDJSON. In the case of NDJSON streaming capability is a consequence of the format. It is possible to implement it on top of JSON serializers/deserializers available in different platforms and languages. In the case of async streaming, it's a consequence of serializer/deserializer internal nature. That internal nature will be different on different platforms. The first example coming to mind are browsers.

The good thing is that in ASP.NET Core 6 one doesn't have to choose between async streaming and NDJSON. Because ObjectResult is no longer buffering IAsyncEnumerable, content negotiation becomes possible. I'm showing exactly that capability in ASP.NET Core 6 branch of my demo project (which will become main after GA).