In a previous article we’ve looked in detail about what is Memcached and how it helps in developing a simple and scalable in-memory data store for caching frequently used data objects in an API, thereby improving the overall performance and throughput.
We have also looked in detail about how we can setup a Memcached instance running and configure our ASP.NET Core API to use the same.
Recap: What is Memcached?
Memcached is an open-source, high-performance, distributed memory object caching system which helps in reducing database load. It maintains data as an in-memory key-value store for small chunks of arbitrary data (strings, objects) which can be result of API calls, database reads and so on.
While this setup works in a development environment, creating and maintaining our own Memcached instances for production-grade application loads is generally not preferred by clients.
Instead, we can let Memcached be created and hosted in a provider environment and connect our application to the same for all the functionalities. AWS provides Memcached as a scalable and fully-managed solution as part of AWS ElastiCache service.
In this article, let’s talk about how we can create an ElastiCache for Memcached instance and connect our ASP.NET Core API to use this Memcached instance for caching.
What is AWS ElastiCache?
Fully managed in-memory data store, compatible with Redis or Memcached. Power real-time applications with sub-millisecond latency. – AWS Documentation.
AWS ElastiCache helps developers to easily create cloud-based in-memory datastores which the applications deployed in AWS cloud environment can easily connect to and integrate. Since these datastores are created as cloud instances, these come with typical performance, scalability and security aspects of AWS cloud services such as VPC, subnets and firewalls.
Amazon ElastiCache for Memcached is a Memcached-compatible in-memory key-value store service that can be used as a cache or a data store. – AWS Documentation.
Since Memcached instance created inside ElastiCache is the same as a typical Memcached datastore, we can configure and access the ElastiCache instance in the same way as we access our local Memcached instance. Which means that we can use the same EnyimMemcachedCore package we used for connecting to our local Memcached instance.
Creating a Memcached instance in AWS ElastiCache
Creating an Memcached instance in AWS ElastiCache is a pretty simple affair and you can boot up a new instance in a few seconds. Keep in mind that to access any of the AWS cloud resources, one must posses an AWS Cloud subscription which one can sign-up for free with a one year access to resources for development or learning purposes.
To create a Memcached instance, sign-in to AWS Console https://console.aws.amazon.com/console/home and in the “Find Services” box, search for AWS ElastiCache.
On the side panel, click on “Memcached” which would open up the Memcached dashboard showing all the Memcached instances previously created. Click on the “Create” button on the top left.
It’d redirect you to the “Create Cluster” page with options as below:
Select “Memcached” for the Cluster Engine and then provider a unique name for the cluster. Node Type defaults to “large”, which you can change according to your API needs.
Expand the Advanced Memcached Settings and take a look at which subnet VPC group the Cluster shall be created at and keep a note of it. This is Important!
Finally, click on “Create” at the bottom which creates a new Memcached Cluster with a single Memcached node for caching.
A Memcached cluster is a load-balanced solution consisting of a number of individual Memcached nodes interfaced together by a load-balancer engine.
Back to Memcached cluster home, click on the cluster which you’ve now created and click on the “description” tab.
Take note of the “Configuration Endpoint” shown in the description. We shall use this configuration endpoint in our API in order to access this Memcached cluster for our caching.
Configuring ElastiCache for Memcached in ASP.NET Core
We can reuse our previous setup of EnyimMemcachedCore library, using which we were successfully able to access our Local Memcached instance.
For accessing Memcached inside ElastiCache, we need to configure the EnyimMemcachedCore library to connect to the Configuration Endpoint we copied just now from the AWS Console.
The configuration looks like below:
services.AddEnyimMemcached(memcachedClientOptions =>
{
memcachedClientOptions.Servers.Add(new Server
{
Address = "my-memchache-for-aspnetcoreapi.cmvmnw.cfg.usw2.cache.amazonaws.com",
Port = 11211
});
});
Note: You can also externalize this setting into your appsettings file as described in my previous post about configuring Memcached in ASP.NET Core.
Now we are all set for accessing the Memcached instance from AWS ElastiCache inside our application. To test it, let’s run the application and see how it goes.
What we get is this:
Enyim.Caching.Configuration.MemcachedClientConfiguration[0]
Failed to CreateSocketAsync to x.x.x.x:11211
System.TimeoutException: Timeout to connect to x.x.x.x:11211.
at Enyim.Caching.Memcached.PooledSocket.ConnectAsync()
at Enyim.Caching.Memcached.MemcachedNode.CreateSocketAsync()
fail: Enyim.Caching.Configuration.MemcachedClientConfiguration[0]
Failed to put PooledSocket 0 in Pool
System.TimeoutException: Timeout to connect to x.x.x.x:11211.
at Enyim.Caching.Memcached.PooledSocket.ConnectAsync()
at Enyim.Caching.Memcached.MemcachedNode.CreateSocketAsync()
at Enyim.Caching.Memcached.Protocol.Binary.BinaryNode.CreateSocketAsync()
at Enyim.Caching.Memcached.MemcachedNode.InternalPoolImpl.CreateSocketAsync()
at Enyim.Caching.Memcached.MemcachedNode.InternalPoolImpl.InitPoolAsync()
fail: Enyim.Caching.Configuration.MemcachedClientConfiguration[0]
Basically what’s happening is, the EnyimMemcachedCore library tries to establish a Socket connection to the Memcached instance it discovered using the Configuration Endpoint we’ve provided. But it fails to connect within the threshold – so it times out. Why?
One of the limitations of using AWS ElastiCache clusters is that applications configured to use them can only work inside an AWS environment.
which means that we must “run” our application “inside” any AWS deployment such as an Elastic Beanstalk or AWS Lambda which are Inside the same VPC group as the ElastiCache cluster it needs to use for this setup to work.
This is the reason I asked to keep a note of the VPC group in which the Memcached cluster is created.
Once our application is deployed in an Elastic Beanstalk or Lambda, we can check whether our application is able to connect to Memcached by checking in the Logs our application writes into AWS CloudWatch.
The logs from Memcached are a bulk of things like below:
Enyim.Caching.Configuration.MemcachedClientConfiguration[0]
Memcached server address - x.x.x.x
Enyim.Caching.Configuration.MemcachedClientConfiguration: Memcached server address - x.x.x.x
Enyim.Caching.Configuration.MemcachedClientConfiguration: [0]: Memcached server address - x.x.x.x
Enyim.Caching.Configuration.MemcachedClientConfiguration[0]
Use KeyTransformer Type : 'Enyim.Caching.Memcached.DefaultKeyTransformer'
Use Transcoder Type : 'Enyim.Caching.Memcached.DefaultTranscoder'
Enyim.Caching.Configuration.MemcachedClientConfiguration: ExecuteOperationAsync(Enyim.Caching.Memcached.Protocol.Binary.GetOperation)
ExecuteOperationAsync(Enyim.Caching.Memcached.Protocol.Binary.GetOperation)
Pool has been inited for x.x.x.x:11211 with 5 sockets
Enyim.Caching.Configuration.MemcachedClientConfiguration: Pool has been inited for x.x.x.x:11211 with 5 sockets
Enyim.Caching.Configuration.MemcachedClientConfiguration: [0]: Pool has been inited for x.x.x.x:11211 with 5 sockets
MemcachedInitPool-cost: 27.8175ms
Acquiring stream from pool. x.x.x.x:11211
Socket abcd-abcd-abcd-abcd was reset
Socket was reset. abcd-abcd-abcd-abcd
pooledSocket.WriteAsync...
Enyim.Caching.Memcached.Protocol.Binary.GetOperation.ReadResponseAsync...
Enyim.Caching.Configuration.MemcachedClientConfiguration: Enyim.Caching.Memcached.Protocol.Binary.GetOperation.ReadResponseAsync result: Not found
Releasing socket abcd-abcd-abcd-abcd
[Debug] Enyim.Caching.Configuration.MemcachedClientConfiguration: Are we alive? True
[Debug] Enyim.Caching.Configuration.MemcachedClientConfiguration: [0]: Are we alive? True
Application started. Press Ctrl+C to shut down.
The last two lines:
[Debug] Enyim.Caching.Configuration.MemcachedClientConfiguration: Are we alive? True
Indicate that the connection was successfully established and Memcached is ready to be written onto.
In this way, we can connect to and work with Memcached for production-grade deployments using AWS ElastiCache and EnyimMemcachedCore library for ASP.NET Core.
Found this article helpful? Please consider supporting!
