Introduction – Why Caching?
Caching is the idea of persisting a frequently read item in a nearby place to pick it up faster than to fetch it only when required. While designing an application or a system, caching is an important technique to be applied, because of the added advantage of speed and skipping unwanted wait times.
When you are trying to fetch when it is actually required, you are waiting the system 🙃
It is also a way of cost optimizing, because you will reduce your storage hits and in some cases the less the storage hits the less you are charged (e.g Cloud). A cache is generally a high-speed memory which is present closest to the system that uses it.
When an item is present in the cache, we call it a “Hit” and the retrieval is faster – you’ll just pull it out of the memory and use it. If it is not present, we call it a “Miss” and you’ll need to fetch data from your storage (or from a service) and place it in the cache. This potentially reduces turn-around times for requests.
Generally applications use a Caching layer b/w their Presentation (Endpoint Layer) and Data tiers and data flows via the Caching Layer into the data. Based on how the cache is populated and maintained, we have Caching Strategies.
What are Caching Strategies?
Like mentioned above, Caching Strategies are ways in which we populate and maintain data in the Caching layer. These approaches depend on what kind of data we are storing in the cache and the frequency in which the data is fetched or invalidated (expired or forced to update).
There are two Caching Strategies which are generally used.
Lazy loading… and its Issues
In this approach, when a client requests for a data, your API looks up for the requested data in the cache. If the cache has data, API fetches the data from the cache and returns it to the client.
If the cache doesn’t contain the requested data, the API queries for the same from its data store and writes this data to the cache. Then it returns the data to the client.
- The downside is that a cache miss takes more time – since you need to both query and write to cache. This is called a “Miss Penalty”.
- There is a risk of returning stale (old) data if that record which is cached is updated.
get_book(book_id)
book = cache.get(book_id)
if (book == null)
book = db_query("SELECT * FROM Books WHERE id = {0}", book_id)
cache.set(book_id, book)
return bookWhat is write-through Caching?
In this strategy, the cache is first filled with the data from the data store. Then for all requests, the cache is queried and the data is returned. In case when the data is updated, first the cache is updated and then the record is synced with the data store.
update_book_price(book_id, price)
book = db_query("UPDATE Books SET price = {0} WHERE id = {1}", price, book_id)
cache.set(book_id, book)
return successAdvantages of a write-through Cache
A write-through caching strategy has the following notable advantages:
- Data present in the cache is always the latest and never stale, because you are also updating the cache when updating the data store.
- Although there is no read-penalty (turnaround when there is a cache miss), there is a write-penalty: data is written twice – at the cache and at the database. But a write-penalty is always acceptable, since end users generally have an opinion that updates take time than reads.
These two advantages put a write-through cache, bit more efficient than a lazy loading cache. Having said that, it too does have its own disadvantages.
Downsides of a write-through Cache
- Since a write-through cache has all the data that is available in the database (because of its write-first), most of the data is never actually read which can potentially impact on cache performance.
- In a distributed caching environment, when a new node is added – there is data miss. This situation continues until all the data is written at least once onto the database, which generally never happens.
As mentioned earlier, in a write-through cache, both the cache and the database are updated simultaneously. This happens synchronously, meaning the system waits until both the cache and the database are updated.
What is write-behind Caching?
A write-behind or a write-back caching strategy works similar to a write-through cache, but in this case only the cache is updated by the API. The cache is responsible for writing the changes to the database, and this happens generally when the data is about to be replaced.
In this approach, the cache generally maintains a dirty flag to indicate if the record has been modified and updates the data to the database later via Event handlers.
This operation takes place asynchronously.
So the API layer doesn’t need to wait for the write to complete and instead query over the cache for the latest data which is already present because of its write-to-cache-first approach.
The advantage of a write-behind cache
- The API layer doesn’t need to perform synchronous writes to database, cache does it.
- There is no write-penalty involved, since API only interacts with the cache and cache updates the records later.
Developers should also keep these things in mind –
- Irrespective of the approach, If there is a node failure or some issue, the data in the cache could be lost forever and so do the updates – although in a partitioned replica cache environment this is already handled.
- Since cache updates records asynchronously, if there is any error while updating, the cache throws an error and logs it. Developers can have the option to catch and decide on where to persist data in such scenarios.
Developers should first verify if their cache supports this approach
Some examples of caching providers who provide a write-behind cache are Redis, NCache etc.
In this article, we’ll see how we can implement write-through and write-behind caching strategies using NCache.
Implementing a write-through Cache in NCache
In NCache, both the write-through and write-behind strategies are implemented in a similar way. It is implemented in 3 steps:
- Provide an implementation of IWriteThruProvider
- Register the implementation in NCache cluster (via GUI or Powershell) and deploy the binary into cache
- Insert or update a cache item with WriteThru options
Implementing a WriteThru Provider
As mentioned above, we first need to implement Alachisoft.NCache.Runtime.DatasourceProviders.IWriteThruProvider. It is highly recommended to implement this in a separate class library project, as we need to upload the binary onto the cluster.
> dotnet new classlib --name NCachedBookStore.CachingProvidersIn the project, we have to install the NCache SDK to access the interface and APIs.
> dotnet add package Next, we have to implement IWriteThruProvider interface. It has different overloads of WriteToDataSource() method. NCache calls this method accordingly, where we have to write our logic to write to the database as required.
The interface looks like below:
namespace Alachisoft.NCache.Runtime.DatasourceProviders
{
public interface IWriteThruProvider
{
void Init(IDictionary parameters, string cacheId);
OperationResult WriteToDataSource(WriteOperation operation);
ICollection<OperationResult> WriteToDataSource(ICollection<WriteOperation> operations);
ICollection<OperationResult> WriteToDataSource(ICollection<DataTypeWriteOperation> dataTypeWriteOperations);
void Dispose();
}
}We add a new implementation that works on the Book entity and adds/updates to the database accordingly. Our implementation looks like below.
namespace NCachedBookStore.CachingProviders.Providers
{
public class NCachedWriteThruProvider : IWriteThruProvider
{
private SqlConnection _connection;
public NCachedWriteThruProvider()
{
}
public void Dispose()
{
// final cleanup
if (_connection != null)
{
_connection.Close();
}
}
public void Init(IDictionary parameters, string cacheId)
{
// initialization functionality
try
{
string connString = GetConnectionString(parameters);
if (!string.IsNullOrEmpty(connString))
{
_connection = new SqlConnection(connString);
_connection.Open();
}
}
catch (Exception)
{
// Handle exception
}
}
public OperationResult WriteToDataSource(WriteOperation operation)
{
ProviderCacheItem cacheItem = operation.ProviderItem;
Book book = cacheItem.GetValue<Book>();
var dtBook = new DataTable();
// transform book object into a database
switch (operation.OperationType)
{
case WriteOperationType.Add:
{
// Insert logic for any Add operation
var commandText = string.Format(
"INSERT INTO dbo.Books (Name, Description, ISBN, Price, AuthorName) VALUES ({0}, {1}, {2}, {3}, {4})",
book.Name, book.Description, book.ISBN, book.Price, book.AuthorName);
new SqlCommand(commandText, _connection).ExecuteNonQuery();
}
break;
case WriteOperationType.Delete:
{
// Insert logic for any Delete operation
var commandText = string.Format(
"DELETE FROM dbo.Books WHERE Id = {0}", book.Id);
new SqlCommand(commandText, _connection).ExecuteNonQuery();
}
break;
case WriteOperationType.Update:
{
// Insert logic for any Update operation
// Insert logic for any Add operation
var commandText = string.Format(
"UPDATE dbo.Books SET Name = {0}, Description = {1}, ISBN = {2}, Price = {3}, AuthorName = {4} WHERE Id = {5}",
book.Name, book.Description, book.ISBN, book.Price, book.AuthorName, book.Id);
new SqlCommand(commandText, _connection).ExecuteNonQuery();
}
break;
}
// Write Thru operation status can be set according to the result.
return new OperationResult(operation, OperationResult.Status.Success);
}
public ICollection<OperationResult> WriteToDataSource(ICollection<WriteOperation> operations)
{
var operationResult = new List<OperationResult>();
foreach (WriteOperation operation in operations)
{
// Write Thru operation status can be set according to the result
operationResult.Add(WriteToDataSource(operation));
}
return operationResult;
}
public ICollection<OperationResult> WriteToDataSource(ICollection<DataTypeWriteOperation> dataTypeWriteOperations)
{
var operationResult = new List<OperationResult>();
foreach (DataTypeWriteOperation operation in dataTypeWriteOperations)
{
var list = new List<Book>();
ProviderDataTypeItem<object> cacheItem = operation.ProviderItem;
Book book = (Book)cacheItem.Data;
switch (operation.OperationType)
{
case DatastructureOperationType.CreateDataType:
// Insert logic for creating a new List
IList myList = new List<Book>();
myList.Add(book.Id);
break;
case DatastructureOperationType.AddToDataType:
// Insert logic for any Add operation
list.Add(book);
break;
case DatastructureOperationType.DeleteFromDataType:
// Insert logic for any Remove operation
list.Remove(book);
break;
case DatastructureOperationType.UpdateDataType:
// Insert logic for any Update operation
list.Insert(0, book);
break;
}
// Write Thru operation status can be set according to the result.
operationResult.Add(new OperationResult(operation, OperationResult.Status.Success));
}
return operationResult;
}
// Parameters specified in Manager are passed to this method
// These parameters make the connection string
private string GetConnectionString(IDictionary parameters)
{
string connectionString = string.Empty;
string server = parameters["server"] as string, database = parameters["database"] as string;
string userName = parameters["username"] as string, password = parameters["password"] as string;
try
{
connectionString = string.IsNullOrEmpty(server) ? "" : "Server=" + server + ";";
connectionString = string.IsNullOrEmpty(database) ? "" : "Database=" + database + ";";
connectionString += "User ID=";
connectionString += string.IsNullOrEmpty(userName) ? "" : userName;
connectionString += ";";
connectionString += "Password=";
connectionString += string.IsNullOrEmpty(password) ? "" : password;
connectionString += ";";
}
catch (Exception)
{
// Handle exception
}
return connectionString;
}
// Deploy this class on cache
}
}The parameter IDictionary is a key-value dictionary which we provide in the GUI and is passed onto the implementation by NCache when it is called. We need to build this class library so that it can be exported to NCache cluster.
> dotnet clean && dotnet publish -c:ReleaseDeploying the Provider
We need to add the provider implementation created in the previous step to the cluster.
In the Cluster GUI, under the Cluster where we need to add provider, click on View Details.
It will open up a suite of configurations
1. Navigate to “Advanced Settings (Clustered Cache)”.
2. On the left, click on Backing Source tab.
3. Scroll down to Write-Through section
4. Tick on “Enable Write Through”, it will now enable “Add Provider”
5. Click on “Add Provider”, it will open up a form.
6. Give your provider a unique name, upload the binary by clicking browse. It will auto detect the implementation and populate the fields
7. Under the parameters section, provide your connection string parameters as NCache will read them and use in the GetConnectionString() method from the IDictionary
8. Once all done, click Ok. You can now see your provider details under the tab
9. Finally, click on Deploy Provider to replicate the binary across the cluster and Click save changes. Your provider is now deployed across the cluster.
Usage
To use the provider now created and deployed, you need to add your items to cache in the following way:
var cache = CacheManager.GetCache(configuration.GetValue<string>("CacheId"));
var book = new Book
{
AuthorName = entity.AuthorName,
Description = entity.Description,
ISBN = entity.ISBN,
Name = entity.Name,
Price = entity.Price
};
// add book - now we don't do it here
//await _db.Books.AddAsync(book);
try
{
// Pre-Condition: Cache is already connected
string key = $"book.Id";
// Create a new cacheItem with the product
var cacheItem = new CacheItem(key);
// Enable write through for the cacheItem created
var writeThruOptions = new WriteThruOptions(
WriteMode.WriteThru, "MyNCachedWriteThruProvider");
// Add the item in the cache with WriteThru enabled
CacheItemVersion itemVersion = cache.Insert(key, cacheItem, writeThruOptions);
}
catch (OperationFailedException ex)
{
if (ex.ErrorCode == NCacheErrorCodes.BACKING_SOURCE_NOT_AVAILABLE)
{
// Backing source is not available
}
else if (ex.ErrorCode == NCacheErrorCodes.SYNCHRONIZATION_WITH_DATASOURCE_FAILED)
{
// Synchronization of data with backing source is failed due to any error
}
else
{
// Exception can occur due to:
// Connection Failures
// Operation Timeout
// Operation performed during state transfer
}
}
catch (Exception)
{
// Any generic exception like ArgumentNullException or ArgumentException
}Implementing a write-behind Cache in NCache
A write-behind cache uses the same Provider that write-through strategy uses. The basic difference is when the provider is called. In a write-through cache the call is synchronous, once cache is updated the database is called and written to immediately.
But in a write-behind cache, the call is asynchronously done sometime later. This is taken care by the NCache cluster and doesn’t require any intervention. Hence it too uses the WriteToDataSource() method when the cache is updated.
We provide WriteOperation as WriteMode.WriteBehind for a write-behind operation.
var cache = CacheManager.GetCache(configuration.GetValue<string>("CacheId"));
// Pre-Condition: Cache is already connected
string key = $"book.Id";
// Create a new cacheItem with the product
var cacheItem = new CacheItem(key);
// Enable write through for the cacheItem created
var writeThruOptions = new WriteThruOptions(WriteMode.WriteBehind, "MyNCachedWriteThruProvider");
// Add the item in the cache with WriteThru enabled
CacheItemVersion itemVersion = cache.Insert(key, cacheItem, writeThruOptions);Conclusion
Caching is a very important design choice which can have a positive impact on the application performance, if implemented correctly. The various caching strategies offer distinctive features and come with their own advantages and drawbacks. Developers can also combine one or more caching strategies (like a Lazy loading read and write-through write) as required.
Write-behind follows the same pattern as write-through, but it overcomes the write-penalty that write-through faces. Using an efficient caching provider such as NCache, which has great support and feature set for write-behind caching, you can consider this design choice if needed.
Reference – NCache Docs
Found this article helpful? Please consider supporting!
