Leveraging Azure Cosmos DB Partial Document Update With JSON Patch in an ASP.NET Core Web API
A couple of weeks ago the Cosmos DB team has announced support for patching documents. This is quite a useful and long-awaited feature, as up to this point the only way to change the stored document was to completely replace it. This feature opens up new scenarios. For example, if you are providing a Web API on top of Cosmos DB, you can now directly implement support for the PATCH request method. I happen to have a small demo of such Web API, so I've decided to refresh it and play with leveraging this new capability.
Adding PATCH Requests Support to an ASP.NET Core Web API
An important part of handling a PATCH request is deciding on the request body format. The standard approach for that, in the case of JSON-based APIs, is JSON Patch. In fact, Cosmos DB is also using JSON Patch, so it should be easier to leverage it this way.
ASP.NET Core provides support for JSON Patch, but I've decided not to go with it. Why? It's designed around applying operations to an instance of an object and as a result, has internal assumptions about the supported list of operations. In the case of Cosmos DB, the supported operations are different, and that "applying" capability is not needed (it was a great way to implement PATCH when Cosmos DB provided on replace option). I've figured out it will be better to start fresh, with a lightweight model.
public class JsonPatchOperation
{
[Required]
public string Op { get; set; }
[Required]
public string Path { get; set; }
public object Value { get; set; }
}
public class JsonPatch : List<JsonPatchOperation>
{ }
This class is quite generic and should allow for handling any request which body is compliant with JSON Patch structure. The first concretization I want to introduce is the list of available operations. Cosmos DB currently supports five operations: Add, Set, Remove, Replace, and Increment. In this demo (at least for now) I'm skipping the Increment because it would require a little bit different handling than others.
public enum JsonPatchOperationType
{
Add,
Set,
Remove,
Replace,,
Invalid
}
As you can see, in the above enumeration I've also added the Invalid value. This will give me a way to represent the operations I don't intend to support through a specific value instead of e.g. throwing an exception.
public class JsonPatchOperation
{
private string _op;
private JsonPatchOperationType _operationType;
[Required]
public string Op
{
get { return _op; }
set
{
JsonPatchOperationType operationType;
if (!Enum.TryParse(value, ignoreCase: true, result: out operationType))
{
operationType = JsonPatchOperationType.Invalid;
}
_operationType = operationType;
_op = value;
}
}
public JsonPatchOperationType OperationType => _operationType;
...
}
Having not supported operations represented through a specific value also allows me to implement IValidatableObject which checks for them. This way, if the class is used as an action model, making a request with an unsupported operation will trigger a 400 Bad Request response.
public class JsonPatchOperation : IValidatableObject
{
...
public IEnumerable<ValidationResult> Validate(ValidationContext validationContext)
{
if (OperationType == JsonPatchOperationType.Invalid)
{
yield return new ValidationResult($"Not supported operation: {Op}.", new[] { nameof(Op) });
}
}
}
Now all is needed is an action that will support the PATCH method.
[Route("api/[controller]")]
[ApiController]
public class CharactersController : Controller
{
...
[HttpPatch("{id}")]
public async Task<ActionResult<Character>> Patch(string id, JsonPatch update)
{
...
}
...
}
The next step will be proxying the deserialized JSON Patch to Cosmos DB .NET SDK.
Utilizing Cosmos DB Partial Updates
The Cosmos DB .NET SDK exposes partial document updates through the PatchItemAsync method on a container. This method expects a collection of PatchOperation instances. Instances representing specific operations can be created through static methods on PatchOperation which names correspond to operations names. So the conversion from JsonPatch to PatchOperation collection requires calling the appropriate method for every JsonPatchOperation. This is something a simple extensions method should handle.
public static class JsonPatchExtensions
{
public static IReadOnlyList<PatchOperation> ToCosmosPatchOperations(this JsonPatch jsonPatchOperations)
{
List<PatchOperation> cosmosPatchOperations = new List<PatchOperation>(jsonPatchOperations.Count);
foreach (JsonPatchOperation jsonPatchOperation in jsonPatchOperations)
{
switch (jsonPatchOperation.OperationType)
{
case JsonPatchOperationType.Add:
cosmosPatchOperations.Add(PatchOperation.Add(jsonPatchOperation.Path, jsonPatchOperation.Value));
break;
case JsonPatchOperationType.Remove:
cosmosPatchOperations.Add(PatchOperation.Remove(jsonPatchOperation.Path));
break;
case JsonPatchOperationType.Replace:
cosmosPatchOperations.Add(PatchOperation.Replace(jsonPatchOperation.Path, jsonPatchOperation.Value));
break;
case JsonPatchOperationType.Set:
System.Int32 test = 25;
cosmosPatchOperations.Add(PatchOperation.Set(jsonPatchOperation.Path, jsonPatchOperation.Value));
break;
}
}
return cosmosPatchOperations;
}
}
Making the proper call in our action.
[Route("api/[controller]")]
[ApiController]
public class CharactersController : Controller
{
...
[HttpPatch("{id}")]
public async Task<ActionResult<Character>> Patch(string id, JsonPatch update)
{
...
ItemResponse<Character> characterItemResponse = await _starWarsCosmosClient.Characters.PatchItemAsync<Character>(
id,
PartitionKey.None,
update.ToCosmosPatchOperations());
return characterItemResponse.Resource;
}
...
}
And we have some testable code. In order to test I've decided to attempt two operations: Set on /height and Add on /weight. This is represented by below JSON Patch body.
[
{
"op": "set",
"path": "/height",
"value": 195
},
{
"op": "add",
"path": "/weight",
"value": 90
}
]
I've hit F5, crated the request in Postman, and clicked Send. What I've received was a 500 Internal Server Error with a weird Newtonsoft.Json deserialization exception. A quick look into Cosmos DB Explorer revealed that the document now looks like this.
{
...
"height": {
"valueKind": 4
},
"weight": {
"valueKind": 4
},
...
}
This is not what I was expecting. What happened? The fact that PatchItemAsync worked without an exception suggested that this must be how Cosmos DB .NET SDK interpreted the JsonPatchOperation.Value. Through debugging I've quickly discovered that what JsonPatchOperation.Value actually holds is System.Text.Json.JsonElement. The Cosmos DB .NET SDK has no other way of dealing with that than serializing public properties - regardless of how smart the implementation is. This is because Cosmos DB .NET SDK (at least for now) is based on Newtonsoft.Json.
So, this is going to be a little bit harder. Conversion from JsonPatch to PatchOperation collection will require deserializing the value along the way. As this is still part of deserializing the request, I've figured out it will be best to put it into JsonPatchOperation.
public class JsonPatchOperation : IValidatableObject
{
...
public T GetValue<T>()
{
return ((JsonElement)Value).Deserialize<T>();
}
}
This will need to be called with the right type parameter, so a mapping between paths and types needs to be obtained. I've decided to make this information available through JsonPatch by making it generic and spicing with reflection.
public class JsonPatch<T> : List<JsonPatchOperation>
{
private static readonly IDictionary<string, Type> _pathsTypes;
static JsonPatch()
{
_pathsTypes = typeof(T).GetProperties().ToDictionary(p => $"/{Char.ToLowerInvariant(p.Name[0]) + p.Name[1..]}", p => p.PropertyType);
}
public Type GetTypeForPath(string path)
{
return _pathsTypes[path];
}
}
A disclaimer is in order. This simple code will work only in this simple case. I'm looking only at top-level properties because my document only has top-level properties. But, in general, a path can be targeting a nested property e.g. /mather/familyName. This code will have to get more complicated to handle such a case.
To make the conversion from JsonPatch to PatchOperation work correctly, sadly, more reflection is needed as those generic calls have to be made with the right parameters at the runtime. The needed pieces can live together with the conversion extension method.
public static class JsonPatchExtensions
{
private static MethodInfo _createAddPatchOperationMethodInfo = typeof(JsonPatchExtensions)
.GetMethod(nameof(JsonPatchExtensions.CreateAddPatchOperation), BindingFlags.NonPublic | BindingFlags.Static);
private static MethodInfo _createReplacePatchOperationMethodInfo = typeof(JsonPatchExtensions)
.GetMethod(nameof(JsonPatchExtensions.CreateReplacePatchOperation), BindingFlags.NonPublic | BindingFlags.Static);
private static MethodInfo _createSetPatchOperationMethodInfo = typeof(JsonPatchExtensions)
.GetMethod(nameof(JsonPatchExtensions.CreateSetPatchOperation), BindingFlags.NonPublic | BindingFlags.Static);
...
private static PatchOperation CreateAddPatchOperation<T>(JsonPatchOperation jsonPatchOperation)
{
return PatchOperation.Add(jsonPatchOperation.Path, jsonPatchOperation.GetValue<T>());
}
private static PatchOperation CreateReplacePatchOperation<T>(JsonPatchOperation jsonPatchOperation)
{
return PatchOperation.Replace(jsonPatchOperation.Path, jsonPatchOperation.GetValue<T>());
}
private static PatchOperation CreateSetPatchOperation<T>(JsonPatchOperation jsonPatchOperation)
{
return PatchOperation.Set(jsonPatchOperation.Path, jsonPatchOperation.GetValue<T>());
}
private static PatchOperation CreatePatchOperation<T>(
MethodInfo createSpecificPatchOperationMethodInfo,
JsonPatch<T> jsonPatchOperations,
JsonPatchOperation jsonPatchOperation)
{
Type jsonPatchOperationValueType = jsonPatchOperations.GetTypeForPath(jsonPatchOperation.Path);
MethodInfo createSpecificPatchOperationWithValueTypeMethodInfo =
createSpecificPatchOperationMethodInfo.MakeGenericMethod(jsonPatchOperationValueType);
return (PatchOperation)createSpecificPatchOperationWithValueTypeMethodInfo.Invoke(null, new object[] { jsonPatchOperation });
}
}
The code above has been structured to isolate the reflection related part and cache the well-known things. Of course, this is a subject of personal preference, but I hope it's readable.
The last remaining thing is modifying the extension method.
public static class JsonPatchExtensions
{
...
public static IReadOnlyList<PatchOperation> ToCosmosPatchOperations<T>(this JsonPatch jsonPatchOperations)
{
List<PatchOperation> cosmosPatchOperations = new List<PatchOperation>(jsonPatchOperations.Count);
foreach (JsonPatchOperation jsonPatchOperation in jsonPatchOperations)
{
switch (jsonPatchOperation.OperationType)
{
case JsonPatchOperationType.Add:
cosmosPatchOperations.Add(CreatePatchOperation(_createAddPatchOperationMethodInfo, jsonPatchOperations, jsonPatchOperation));
break;
case JsonPatchOperationType.Remove:
cosmosPatchOperations.Add(PatchOperation.Remove(jsonPatchOperation.Path));
break;
case JsonPatchOperationType.Replace:
cosmosPatchOperations.Add(CreatePatchOperation(_createReplacePatchOperationMethodInfo, jsonPatchOperations, jsonPatchOperation));
break;
case JsonPatchOperationType.Set:
cosmosPatchOperations.Add(CreatePatchOperation(_createSetPatchOperationMethodInfo, jsonPatchOperations, jsonPatchOperation));
break;
}
}
return cosmosPatchOperations;
}
...
}
Adjusting the action code, building the demo, running, going to Postman, sending a request, and it works!
What’s Missing?
This is demoware, so there is always something missing. I've already mentioned that parts of the code are handling only simple cases. But there are two additional subjects which require more attention.
One is validation. This code is not validating if paths provided for operations represent valid properties. It's also not validating if values can be converted to the right types and if the operations result in an invalid state of the entity. First, two things should be achievable at the JsonPatch level. The last one will require additional code in action.
The other subject is performance around that reflection code. It can be improved by caching the generic methods for specific types, but as the solution grows it might not be enough. It is worth thinking about another option here.
I might tackle those two subjects, but I don't know if and when. You can keep an eye on the demo project because if I do, the code will certainly end there.