Setting up the Context:
In the world of RESTful API services where client and server communicate in a request and response fashion, it is important for the client and server to interact on a data language which is known and understandable by both the parties. In other words, when a client sends a request which is not in a known format to the server or vice versa, communication cannot be fluent. In order to maintain this balance, both when a request is built in the client for the server or when a response is generated for the client at the server, both specify the language of the content of the request or response so that the parties on the other side aren’t informed off it. This is where the Media types or the MIME types come into play. These types, which we specify under a "ContentType" request or response headers are responsible for this communication between the client and the server API endpoints.
A media type can be used as both an input formatter or an output formatter. Input formatters received from a request "ContentType" are used by the server for model binding onto the input payload (such as json or xml) whereas the Output formatters specified from an "Accept" header from the client or sent in the "ContentType" of the response from the server can be used for formatting the response content as per the prescribed data format expected by the client or given by the server.
Some of the examples of these Media types are:
- application/json
- application/xml
- application/javascript
- text/html
- text/css
And then we have custom media formatters, which are used for providing data exchange between client and server in a type which is not a standard media type in the likes of above. Let’s see how we can implement a media formatter which enables us to have data exchange between client and server using a media called Protocol Buffers aka ProtoBuf.
What is ProtoBuf?
Protocol Buffers aka ProtoBuf is a method of encoding and sending out structured data (such as a json or xml) in an efficient manner. These were developed by Google and touted to be language-neutral, platform-neutral and highly performant. This way of formatting media is light-weight and highly optimised when compared to the standard media types (such as json) and helps optimizing performance when working over low bandwidth networks. Although this might not be used on applications which are directly consumed on a web browser or can’t be a complete replacement for JSON in all ways, ProtoBuf is still considered ideal for internal API communications within a large platform. ProtoBuf converts the actual content data into an encrypted and encoded format which is light-weight when compared to the actual data size but is barely human readable, which can also be seen as a help to data security. The ProtoBuf data content is indicated by the MIME type "application/x-protobuf"
While ProtoBuf has several implementations for various platforms such as C++, Java, Python and so on, we have an open source dotnet core implementation of ProtoBuf, which can be used directly via Nuget. The library we are using is provided by WebApiContrib.Core git project which provides several useful utilities and extensions for making development easier.
Adding ProtoBuf to our existing API involves a few couple of changes, first add the ProtoBuf library to the project via CLI as below:
> dotnet add package WebApiContrib.Core.Formatter.Protobuf --version 2.1.3
This adds the ProtoBuf media formatter functionality to the project. Next, we register the ProtoBuf media formatters onto the Mvc which adds this as an input and output formatter. This is done in the Startup class as below:
// Startup.ConfigureServices method
services.AddControllers()
.AddProtobufFormatters();
Next, we would need to decorate our model classes with either [DataContract] or [ProtoContract] to mark them serializable for output. But we shall prefer [DataContract] over [ProtoContract] because we can have these models reusable with other media formatters such as json as well for serialization.
namespace ReadersApi.Models
{
[DataContract]
public class Reader
{
[DataMember]
public int Id { get; set; }
[DataMember]
public string UserName { get; set; }
[DataMember]
public string EmailAddress { get; set; }
}
}
And when we invoke our Get Readers API via any client such as Postman, all we see is response in an unreadable format for a sample request such as below:
GET /api/reader/all HTTP/1.1
Host: localhost:5000
Accept: application/x-protobuf
Cache-Control: no-cache
Host: localhost:5000
Accept-Encoding: gzip, deflate
Connection: keep-alive
cache-control: no-cache
This is because the response content is in media format application/x-protobuf as requested by the client which is generally not readable for a standard web client. For this would write a simple client application which creates a request and deserializes the response stream.
Parsing ProtoBuf Response at Client:
We shall use the same ProtoBuf library in an aspnetcore console client to make a GET request and deserialize the response, which is as below:
namespace ProtoBufClient
{
public class Program
{
public static void Main(string[] args)
{
var res = CallServerAsync().Result;
if (res.Count > 0)
{
foreach (var r in res)
{
Console.WriteLine($"Reader#{r.Id}");
}
}
}
static async Task<List<Reader>> CallProtoBufApiAsync()
{
var client = new HttpClient();
var request = new HttpRequestMessage(HttpMethod.Get, "http://localhost:5000/api/reader/all");
request.Headers.Accept.Add(new MediaTypeWithQualityHeaderValue("application/x-protobuf"));
var result = await client.SendAsync(request);
var readers = ProtoBuf.Serializer.Deserialize<List<Reader>>(await result.Content.ReadAsStreamAsync());
return readers;
}
}
}
In the code above, we read the response as a stream and deserialize it to the serializable model Reader we have seen before. Now when we run this program, we can see the output printed in the console.
Reader#0
Reader#1
POST ProtoBuf Content from Client to API:
Similarly we can POST data to an API accepting protobuf media type as below, Wherein we serialize the content to be posted to the API into a memory stream and write the stream onto the request body.
static async Task<Reader> PostProtoBufApiAsync()
{
var client = new HttpClient();
var request = new HttpRequestMessage(HttpMethod.Post, "http://localhost:5000/api/reader/new");
request.Headers.Accept.Add(new MediaTypeWithQualityHeaderValue("application/x-protobuf"));
var newReader = new Reader
{
EmailAddress = "reader3@abc.com",
UserName = "Reader#3"
};
using (MemoryStream stream = new MemoryStream())
{
ProtoBuf.Serializer.Serialize(stream, newReader);
request.Content = new ByteArrayContent(stream.ToArray());
var response = await client.SendAsync(request);
if (response.IsSuccessStatusCode)
{
var reader = ProtoBuf.Serializer.Deserialize<Reader>(await response.Content.ReadAsStreamAsync());
return reader;
}
}
return null;
}
Can I Still GET or POST data onto the API using JSON?
Obviously, because so far we have added ProtoBuf as another supported Media Formatter along with the already supported Media types provided implicitly with the MVC framework. All we need to do is to differentiate in our Accept request header, which specifies what type of response content the client expects from the server. And when we provide application/json in place of application/x-protobuf all things work same as before.
Conclusion:
Since the ProtoBuf can’t be recognised and parsed directly by any client, it’s not much preferred for standard client server operations, but it is highly used in internal communications involving high data sizes to be sent over limited bandwidth. In this way, we can create an API which generates content in ProtoBuf media format and digest it in a client using the same library.
Found this article helpful? Please consider supporting!
