Building a simple ETag in ASP.NET Core

APIs must be developed in such a way that it returns data to the requests in a minimal time and bandwidth. It can also dictate the clients on how to store certain data on its side, So that it can still show the same data for a time specified by the API during the response, instead of making repetitive requests for the same data.

What if the data in the API has not changed even after that time? Its not a good design to still request and receive the same data that was already present. It wastes the bandwidth of the Server.

One way to solve this is by letting the API know what version of the data the client has previously received, and let the API return new data or tell the client that the data the client already has is “not yet modified”.

This exchange technique is called ETag.

What is an ETag?

ETag or an “Entity Tag” is a string which represents the “version” information of the data that the client has. The client passes this ETag to the server along with the request, so that the server knows what data the client currently holds.

If the Server finds that there is no new “version” of the data other than the one the client already has, it simply returns a status code “304 Not Modified”

On the other hand, if there is a new “version” of the data available, the server returns the updated data to the client. It also returns a new ETag which represents this new “version”, which the client then updates on its side. This cycle continues, thereby avoiding unnecessary response bandwidth for the API, which can improve server efficiency.

Does the ETag contain any information? Nope. It is purely “opaque”, meaning that there’s nothing that can be found about the data from this value.

ETag solves two problems:

1. Unchanged resource caching
2. Mid-air edit collision detection

You can find detailed guide on how we can detect and prevent mid-air edit collision using ETag here.

How does Resource caching work with ETag?

The entire flow can be sequenced as below:

1. Client sends a request to the server for some data
2. Server returns an ETag for a specific version of a resource in the response header “ETag”
3. Client stores the ETag value and
4. Client passes ETag for the successive API calls for the same resource under the request header “If-None-Match”
5. Server extracts the ETag from the request and it matches this with the computed value from the resource that is currently available
   6a. If both the tags match, the data is unchanged – server sends an empty response with StatusCode 304 (Not Modified)
   7a. Client can then use the same data for a little longer duration
   6b. If the tags don’t match, the data is changed – server sends the new data with a new ETag computed in the response header.
   7b. Client updates its ETag and uses the new data it received.

If the request doesn’t contain any ETag, the Server treats it as a “new” request and sends a computed ETag along with the data.

How to compute ETag in Server?

The Web Specification advocates what an ETag is supposed to do and how it must be exchanged between the client and the server. It doesn’t explain how it must be computed at the server end. Since the client doesn’t need to process the ETag for any information, the server can use any method to compute an ETag.

A simple approach would be to compute an ETag just before it is sent out of the API to the client. The computation method can be as simple as using the LastModified timestamp, or computing a hash function using the response body content.

For example, an API to fetch a single blog content can hash all the content and compute an ETag. Or it can simply use the LastModified date value for the same.

Since we’re generating an ETag once after the result is prepared, it doesn’t really impact the functionality of the API. We’re still making a database READ, still processing it and preparing a result.

But since we’re not writing the content to the response stream out of the Server, we could be saving the Server’s response bandwidth. Hence it is called a “shallow ETag”.

Implementing ETag for Unchanged Response Caching

As mentioned above, the simplest way to implement an ETag is to compute it and attach it to the response just before it leaves the API. In an ASP.NET Core WebAPI, we can use an ActionFilter for the same purpose.

To demonstrate, let’s assume we have an API that returns a single Item from the store for a requested Id. Typically, it means that the API has to READ into the datastore for the Id and fetch all the details for the record and return it back.

Let’s assume that the record contains a “LastModified” property which is strictly modified for every WRITE request.

The entity Item looks like below:

namespace ContainerNinja.Contracts.Data.Entities
{
    public abstract class BaseEntity
    {
        public virtual int Id { get; set; }
        public virtual DateTime LastModified { get; set; }
    }

    public class Item : BaseEntity
    {
        public string Name { get; set; }
        public string Description { get; set; }
        public string Categories { get; set; }
        public string ColorCode { get; set; }
    }
}

A simple GetById() API retrieves this Item from the database and returns an ItemDTO as below.

namespace ContainerNinja.Contracts.DTO
{
    public class ItemDTO : BaseEntity
    {
        public string Name { get; set; }
        public string Description { get; set; }
        public string Categories { get; set; }
        public string ColorCode { get; set; }
    }
}

namespace ContainerNinja.Controllers
{
    [Route("api/[controller]")]
    [ApiController]
    public class ItemsController : ControllerBase
    {
        private readonly IMediator _mediator;

        public ItemsController(IMediator mediator)
        {
            _mediator = mediator;
        }
        
        [HttpGet]
        [Route("{id}")]
        [ETagFilter]
        [ProducesResponseType(typeof(ItemDTO), (int)HttpStatusCode.OK)]
        [ProducesErrorResponseType(typeof(BaseResponseDTO))]
        public async Task<IActionResult> GetById(int id)
        {
            try
            {
                var query = new GetNinjaByIdQuery(id);
                var response = await _mediator.Send(query);
                return Ok(response);
            }
            catch (EntityNotFoundException ex)
            {
                return NotFound(new BaseResponseDTO
                {
                    IsSuccess = false,
                    Errors = new string[] { ex.Message }
                });
            }
        }
    }
}

The API fetches data from the backend using CQRS pattern, via Mediator library. You can find more information about CQRS and MediatR here. It is decorated with a custom ActionFilter [ETagFilter] that encapsulates the functionality to validate and set the ETag response header.

Within this filter, we capture the response that is produced by the API and extract the response content to compute the ETag. I’m using the entire response content object to compute Hash and use it as my ETag.

How the ETag approach works?

1. We compute ETag string based on the result content only for GET requests which are successful in execution (200 OK response)
2. Here we extract the result object and then compute ETag out of it (using any mechanism, be it hashing or any method of developer’s choice)
3. We check if there’s already an Etag being sent in the request
4. If there’s one available we compare it with the computed etag to check if both the tags are equal. Equality means that the data isn’t modified.
5. If the tags are equal, we send a Not Modified Status response.
6. Else we add the computed etag to the response header and then let the response be passed on to the client.

The functionality looks like below:

private void ValidateETagForResponseCaching(ActionExecutedContext executedContext)
{
    if (executedContext.Result == null)
    {
        return;
    }

    var request = executedContext.HttpContext.Request;
    var response = executedContext.HttpContext.Response;

    var result = (BaseEntity)(executedContext.Result as ObjectResult).Value;

    // generate ETag from LastModified property
    //var etag = GenerateEtagFromLastModified(result.LastModified);

    // generates ETag from the entire response Content
    var etag = GenerateEtagFromResponseBodyWithHash(result);

    if (request.Headers.ContainsKey(HeaderNames.IfNoneMatch))
    {
        // fetch etag from the incoming request header
        var incomingEtag = request.Headers[HeaderNames.IfNoneMatch].ToString();

        // if both the etags are equal
        // raise a 304 Not Modified Response
        if (incomingEtag.Equals(etag))
        {
            executedContext.Result = new StatusCodeResult((int)HttpStatusCode.NotModified);
        }
    }

    // add ETag response header 
    response.Headers.Add(HeaderNames.ETag, new[] { etag });
}

The ActionFilter [ETagFilter] looks like below:

namespace ContainerNinja.API.Filters
{
    // prevents the action filter methods to be invoked twice
    [AttributeUsage(AttributeTargets.Method | AttributeTargets.Class, AllowMultiple = false)]
    public class ETagFilter : ActionFilterAttribute, IAsyncActionFilter
    {
        public override async Task OnActionExecutionAsync(ActionExecutingContext executingContext, ActionExecutionDelegate next)
        {
            var request = executingContext.HttpContext.Request;

            var executedContext = await next();
            var response = executedContext.HttpContext.Response;

            // Computing ETags for Response Caching on GET requests
            if (request.Method == HttpMethod.Get.Method && response.StatusCode == (int)HttpStatusCode.OK)
            {
                ValidateETagForResponseCaching(executedContext);
            }
        }
    }
}

Testing the ETag mechanism

The below are a series of cURL requests and their responses, which demonstrate the series of above mentioned points.

1. Make a GET request “without” any ETag. The server returns data with an ETag response header.

curl --location --request GET 'https://localhost:5001/api/Items/3'

< HTTP/1.1 200 OK
< Content-Type: application/json; charset=utf-8
< Date: Wed, 08 Dec 2021 06:55:46 GMT
< Server: Kestrel
< ETag: Gm9UoL03Pqru3bp6jVR3QAS4myCMw6cd922HyeJaQgM=
< Transfer-Encoding: chunked

{
    "name": "Shuriken",
    "description": "One has to throw it over attackers to kill or distract them even when they try to chase you.",
    "categories": "Weaponry",
    "colorCode": "Black",
    "id": 3,
    "lastModified": "2021-12-08T11:10:27.7694364"
}

2. Make a GET request “with” an ETag received in the response above. Data is not yet modified, so the Server returns 304.

curl --location --request GET 'https://localhost:5001/api/Items/3' 
--header 'If-None-Match: Gm9UoL03Pqru3bp6jVR3QAS4myCMw6cd922HyeJaQgM='

< HTTP/1.1 304 Not Modified
< Date: Wed, 08 Dec 2021 06:59:12 GMT
< Server: Kestrel
< ETag: Gm9UoL03Pqru3bp6jVR3QAS4myCMw6cd922HyeJaQgM=

3. Update the Entity in the Server with some new values.

curl -X 'PUT' 
  'https://localhost:5001/api/Items/3' 
  -H 'accept: text/plain' 
  -H 'Content-Type: application/json' 
  -d '{
  "name": "Shuriken",
  "description": "Throwing weapons are unique and famous in the world.",
  "categories": "Weaponry,Ninjitsu",
  "colorCode": "Kuro"
}'

< content-type: application/json; charset=utf-8 
< date: Wed,08 Dec 2021 07:01:34 GMT 
< server: Kestrel 
{
    "name": "Shuriken",
    "description": "Throwing weapons are unique and famous in the world.",
    "categories": "Weaponry,Ninjitsu",
    "colorCode": "Kuro",
    "id": 3,
    "lastModified": "2021-12-08T12:36:42.1015231"
}

4. Client makes a GET request with the “same” ETag, but the data has been modified. The server sends the updated data with a “new” ETag.

curl --location --verbose --request GET "https://localhost:5001/api/Items/3" 
    --header "If-None-Match: Gm9UoL03Pqru3bp6jVR3QAS4myCMw6cd922HyeJaQgM="


< HTTP/1.1 200 OK
< Content-Type: application/json; charset=utf-8
< Date: Wed, 08 Dec 2021 07:07:20 GMT
< Server: Kestrel
< ETag: RkKoHlIIDsVMR0hwSMChWZ5N2ZxNAJk8lDfRLv0UjOo=
< Transfer-Encoding: chunked
{
    "name": "Shuriken",
    "description": "Throwing weapons are unique and famous in the world.",
    "categories": "Weaponry,Ninjitsu",
    "colorCode": "Kuro",
    "id": 3,
    "lastModified": "2021-12-08T12:36:42.1015231"
}

Found this article helpful? Please consider supporting!

Conclusion and… boilerplate 🥳

ETag is a simple yet effective implementation for caching in any RESTful API. This helps in reducing workload on the backend, and also avoiding unnecessary calls from the client; benefiting both.

Implementing ETag is quite simple, with the builtin action filters present in the ASP.NET Core framework. You can check out the code snippets used in this article, in the Clean Architecture boilerplate I have built keeping best practices in mind.

Please do check it out and leave a star ⭐ if you find it useful – https://github.com/referbruv/ContainerNinja.CleanArchitecture