How to build a simple ETag in ASP.NET Core

Introduction

We must develop APIs in such a way that they return data to requests in minimal time and bandwidth. We can also instruct clients on how to store certain data on their side so that it can still display the same data for a time specified by the API during the response, instead of making repetitive requests for the same data.

But what if the data in the API hasn’t changed even after that time?

It’s not good design to still request and receive the same data that was already present. It wastes the server’s bandwidth.

One way we can solve this is by informing the API about the version of the data the client has previously received and having the API return new data or inform the client that the data it already possesses is ‘not yet modified’

This exchange technique is known as ETag.

What is an ETag?

ETag, or ‘Entity Tag,’ is a string that represents the ‘version’ information of the data we have. We pass this ETag to the server along with our request so that the server knows which data we currently possess.

If the server finds that there is no new ‘version’ of the data other than the one we already have, it simply returns a status code of ‘304 Not Modified.’

On the other hand, if there is a new ‘version’ of the data available, the server returns the updated data to us. It also provides a new ETag that represents this new ‘version,’ which we then update on our side. This cycle continues, thereby avoiding unnecessary response bandwidth for the API and improving server efficiency.

Does the ETag contain any information?

No, it is purely ‘opaque,’ meaning that it doesn’t reveal any details about the data.

ETag solves two problems:

1. Caching unchanged resources.
2. Detecting mid-air edit collisions

How does Resource caching work with ETag?

The entire flow can be sequenced as follows:

1. The client sends a request to the server for some data.
   
2. The server returns an ETag for a specific version of a resource in the response header labeled as “ETag.”
   
3. The client stores the ETag value.
   
4. The client passes the ETag for successive API calls for the same resource under the request header labeled as “If-None-Match.”
   
5. The server extracts the ETag from the request and matches it with the computed value from the currently available resource.
   
6. If both tags match, the data remains unchanged, and the server responds with an empty response having StatusCode 304 (Not Modified).
   • If the tags don’t match, it indicates that the data has changed. The server sends the new data with a newly computed ETag in the response header.
     
7. The client can then use the same data for a little longer duration.
   • The client updates its ETag and uses the new data it received.

If the request doesn’t contain any ETag, the server treats it as a “new” request and sends a computed ETag along with the data.

How to generate an ETag?

The Web Specification outlines the purpose and exchange of an ETag between the client and the server. However, it doesn’t specify how the server should compute the ETag. Since the client doesn’t need to process the ETag for any information, we can use any method to compute it on the server.

One straightforward approach is to compute the ETag just before sending it to the client via the API. The computation method can be as simple as using the LastModified timestamp or generating a hash function based on the response body content.

For instance, when our API retrieves a single blog content, we can hash all the content to compute an ETag. Alternatively, we can use the LastModified date value for this purpose.

Since we generate an ETag once the result is prepared, it doesn’t significantly impact the functionality of the API. We still perform a database READ, process it, and prepare a result. However, by not immediately writing the content to the response stream from the Server, we can potentially save the Server’s response bandwidth. This practice is referred to as a “shallow ETag.”

How to use ETag for Unchanged Responses (304 Not Modified)

As previously mentioned, the simplest way to implement an ETag is to compute it and attach it to the response just before it leaves our API. In an ASP.NET Core WebAPI, we can achieve this by using an ActionFilter for the same purpose.

To illustrate, let’s consider that we have an API responsible for returning a single item from the store based on a requested ID. Typically, this means our API has to read from the data store for the specified ID, fetch all the details for the record, and then return it.

Assuming that the record includes a “LastModified” property that gets strictly modified with every WRITE request, we can proceed with implementing ETag functionality.

The entity Item looks like below:

namespace ContainerNinja.Contracts.Data.Entities
{
    public abstract class BaseEntity
    {
        public virtual int Id { get; set; }
        public virtual DateTime LastModified { get; set; }
    }

    public class Item : BaseEntity
    {
        public string Name { get; set; }
        public string Description { get; set; }
        public string Categories { get; set; }
        public string ColorCode { get; set; }
    }
}

A simple GetById() API retrieves this Item from the database and returns an ItemDTO as below.

namespace ContainerNinja.Contracts.DTO
{
    public class ItemDTO : BaseEntity
    {
        public string Name { get; set; }
        public string Description { get; set; }
        public string Categories { get; set; }
        public string ColorCode { get; set; }
    }
}

namespace ContainerNinja.Controllers
{
    [Route("api/[controller]")]
    [ApiController]
    public class ItemsController : ControllerBase
    {
        private readonly IMediator _mediator;

        public ItemsController(IMediator mediator)
        {
            _mediator = mediator;
        }

        [HttpGet]
        [Route("{id}")]
        [ETagFilter]
        [ProducesResponseType(typeof(ItemDTO), (int)HttpStatusCode.OK)]
        [ProducesErrorResponseType(typeof(BaseResponseDTO))]
        public async Task<IActionResult> GetById(int id)
        {
            try
            {
                var query = new GetNinjaByIdQuery(id);
                var response = await _mediator.Send(query);
                return Ok(response);
            }
            catch (EntityNotFoundException ex)
            {
                return NotFound(
                    new BaseResponseDTO { IsSuccess = false, Errors = new string[] { ex.Message } }
                );
            }
        }
    }
}

The API retrieves data from the backend using the CQRS pattern, facilitated by the Mediator library. You can find more information about CQRS and MediatR here.

We have decorated it with a custom ActionFilter called [ETagFilter], which encapsulates the functionality for validating and setting the ETag response header.

Within this filter, we capture the response produced by the API and extract the response content to compute the ETag. We use the entire response content object to compute the hash and employ it as our ETag.

How the ETag approach works?

1. We calculate the ETag string based on the content of the result, but only for successful GET requests (those with a 200 OK response).
   
2. In this step, we take the result object and generate the ETag from it using a chosen method, such as hashing or another developer’s choice.
   
3. We then examine if there is an ETag included in the request.
   
4. If an ETag is found, we compare it to the computed ETag to determine if they match. If they match, it means the data hasn’t been modified.
   
5. When the tags match, we respond with a “Not Modified” status.
   
6. If the tags don’t match, we include the computed ETag in the response header and allow the response to be sent to the client.

Here’s how the functionality appears. The ActionFilter, [ETagFilter], is shown below:

namespace ContainerNinja.API.Filters
{
    // prevents the action filter methods to be invoked twice
    [AttributeUsage(AttributeTargets.Method | AttributeTargets.Class, AllowMultiple = false)]
    public class ETagFilter : ActionFilterAttribute, IAsyncActionFilter
    {
        public override async Task OnActionExecutionAsync(
            ActionExecutingContext executingContext,
            ActionExecutionDelegate next
        )
        {
            var request = executingContext.HttpContext.Request;

            var executedContext = await next();
            var response = executedContext.HttpContext.Response;

            // Computing ETags for Response Caching on GET requests
            if (
                request.Method == HttpMethod.Get.Method
                && response.StatusCode == (int)HttpStatusCode.OK
            )
            {
                ValidateETagForResponseCaching(executedContext);
            }
        }

        private void ValidateETagForResponseCaching(ActionExecutedContext executedContext)
        {
            if (executedContext.Result == null)
            {
                return;
            }

            var request = executedContext.HttpContext.Request;
            var response = executedContext.HttpContext.Response;

            var result = (BaseEntity)(executedContext.Result as ObjectResult).Value;

            // generate ETag from LastModified property
            //var etag = GenerateEtagFromLastModified(result.LastModified);

            // generates ETag from the entire response Content
            var etag = GenerateEtagFromResponseBodyWithHash(result);

            if (request.Headers.ContainsKey(HeaderNames.IfNoneMatch))
            {
                // fetch etag from the incoming request header
                var incomingEtag = request.Headers[HeaderNames.IfNoneMatch].ToString();

                // if both the etags are equal
                // raise a 304 Not Modified Response
                if (incomingEtag.Equals(etag))
                {
                    executedContext.Result = new StatusCodeResult((int)HttpStatusCode.NotModified);
                }
            }

            // add ETag response header
            response.Headers.Add(HeaderNames.ETag, new[] { etag });
        }
    }
}

Testing the ETag Implementation

Below, we have a set of cURL requests and their corresponding responses that illustrate the points mentioned earlier. We will perform the following operations in sequence –

1. We send a GET request without an ETag. The server responds with data containing an ETag in the header.
   
2. We send a GET request with the ETag received in the previous response. Since the data hasn’t changed, the server replies with a 304 status code.
   
3. We update the entity on the server with new values.
   
4. The client sends a GET request with the same ETag, but the data has been altered. The server returns the updated data with a new ETag.
   

1. GET request without ETag – Server returns with ETag

curl --location --request GET 'https://localhost:5001/api/Items/3'

    < HTTP/1.1 200 OK
    < Content-Type: application/json; charset=utf-8
    < Date: Wed, 08 Dec 2021 06:55:46 GMT
    < Server: Kestrel
    < ETag: Gm9UoL03Pqru3bp6jVR3QAS4myCMw6cd922HyeJaQgM=
    < Transfer-Encoding: chunked

    {
        "name": "Shuriken",
        "description": "One has to throw it over attackers to kill or distract them even when they try to chase you.",
        "categories": "Weaponry",
        "colorCode": "Black",
        "id": 3,
        "lastModified": "2021-12-08T11:10:27.7694364"
    }

2. GET request with ETag – server returns 304 Not Modified

curl --location --request GET 'https://localhost:5001/api/Items/3' 
    --header 'If-None-Match: Gm9UoL03Pqru3bp6jVR3QAS4myCMw6cd922HyeJaQgM='

    < HTTP/1.1 304 Not Modified
    < Date: Wed, 08 Dec 2021 06:59:12 GMT
    < Server: Kestrel
    < ETag: Gm9UoL03Pqru3bp6jVR3QAS4myCMw6cd922HyeJaQgM=

3. Update the Entity in the Server

curl -X 'PUT' 
      'https://localhost:5001/api/Items/3' 
      -H 'accept: text/plain' 
      -H 'Content-Type: application/json' 
      -d '{
      "name": "Shuriken",
      "description": "Throwing weapons are unique and famous in the world.",
      "categories": "Weaponry,Ninjitsu",
      "colorCode": "Kuro"
    }'

    < content-type: application/json; charset=utf-8 
    < date: Wed,08 Dec 2021 07:01:34 GMT 
    < server: Kestrel 
    {
        "name": "Shuriken",
        "description": "Throwing weapons are unique and famous in the world.",
        "categories": "Weaponry,Ninjitsu",
        "colorCode": "Kuro",
        "id": 3,
        "lastModified": "2021-12-08T12:36:42.1015231"
    }

4. GET request with the old ETag – Server sends updated data with a new ETag

curl --location --verbose --request GET "https://localhost:5001/api/Items/3" 
        --header "If-None-Match: Gm9UoL03Pqru3bp6jVR3QAS4myCMw6cd922HyeJaQgM="


    < HTTP/1.1 200 OK
    < Content-Type: application/json; charset=utf-8
    < Date: Wed, 08 Dec 2021 07:07:20 GMT
    < Server: Kestrel
    < ETag: RkKoHlIIDsVMR0hwSMChWZ5N2ZxNAJk8lDfRLv0UjOo=
    < Transfer-Encoding: chunked
    {
        "name": "Shuriken",
        "description": "Throwing weapons are unique and famous in the world.",
        "categories": "Weaponry,Ninjitsu",
        "colorCode": "Kuro",
        "id": 3,
        "lastModified": "2021-12-08T12:36:42.1015231"
    }

Conclusion

ETag offers a straightforward and effective way to implement caching in any RESTful API, reducing the workload on the backend and preventing unnecessary client calls. It’s a win-win for both sides.

Implementing ETag is a breeze, thanks to the built-in action filters provided by the ASP.NET Core framework. You can explore the code snippets used in this article within the Clean Architecture boilerplate I’ve created, following best practices.

Feel free to check it out and consider giving it a star ⭐ if you find it useful: https://github.com/referbruv/ContainerNinja.CleanArchitecture