Introduction
AWS offers libraries for a range of popular programming platforms, enabling interaction and collaboration with DynamoDB. In this article, we will understand the process of establishing a connection and engaging with DynamoDB within an Express.js application. We will first see how to create a basic Express.js API application using the Express framework. We will integrate our API with the capabilities of DynamoDB.
What is DynamoDB?
DynamoDB is a high-performance NoSQL database service offered by AWS as a part of its AWS Cloud Stack. It is quite popular for its scalability and performance and is used in usecases which require real-time data intensive applications such as Gaming, Leaderboards and so on.
Getting started with Integrating AWS DynamoDB in Express.js
To link up with DynamoDB from our computer, we require either the AWS Command Line or Developer Tools like Visual Studio set up. Whenever the application on our local machine communicates with AWS resources, the SDK searches for a credentials file. This file holds an access token and secret generated from our AWS subscription credentials. To learn how to install AWS CLI and set it up, you can refer the AWS documentation. Once development is finished, we can place this API on an AWS Compute service such as AWS Lambda. We just need to ensure the service has the necessary permissions to access DynamoDB resources.
This article assumes that we have a table ‘MailSources’ already created in DynamoDB. If not created, you can simply create a new Table MailSources with a HashKey ‘Id’ of type String using the AWS Console.
Let us create a new API application using Node and Express. Please make sure that the system already has NodeJS installed, if not install NodeJS to continue.
In a new folder open a Visual Studio Code editor and give the below command which creates a new package.json for the node application.
> npm initThe CLI requests a few inputs, which you can adjust or keep as they are. After the package.json is generated, create a new JavaScript file named “server.js.” Inside this file, we will place the application logic to operate a node API. To simplify things, we will use Express along with NodeJS, a lightweight web framework for developing APIs and applications.
To install Express, run the below command:
> npm install express --saveImplementing GET Endpoint for Reading Data from a DynamoDB table
After installation, open the previously created server.js file. We will begin by adding a GET endpoint to our application. This endpoint will eventually retrieve data from a DynamoDB table and return it as JSON.
const express = require('express');
const app = express();
// port on which the server listens
const port = 3000;
app.get("/rows/all", (req, res) => {
// magic happens here
});
app.listen(port, () => {
console.log(`Listening on port ${port}`);
});Installing the AWS SDK for Node.js
The process for accessing or adding data is as follows:
- Set up a client
- Construct an Input object containing the Table name, and optionally a Query or Projection.
- Invoke the DynamoDB client with the relevant operation, alongside the Input object.
First we require the AWS SDK to connect with AWS components and services. We are focusing on accessing the AWS DynamoDB service. Our initial step is to install the aws-sdk module in our application.
npm install aws-sdkAfter installation, we include the library in our application. Add the following lines of code at the top of server.js, just below our Express modules.
// import the aws sdk to use the DynamoDB
// libraries in the app
const AWS = require('aws-sdk');We will generate a DynamoDB client instance. This instance assists in executing operations on the DynamoDB setup, as demonstrated below:
// create a new DynamoDB client
// which provides connectivity b/w the app
// and the db instance
const client = new AWS.DynamoDB.DocumentClient();
const tableName = 'MailSources';We’ve also added a new field, “tableName,” which points to the specific table in DynamoDB where actions occur. DynamoDB structures data in Tables, with each table being a group of JSON documents. It’s important to set the Region where DynamoDB is hosted; this configuration is applied within our application.
// update the region to
// where DynamoDB is hosted
AWS.config.update({ region: 'us-west-2' });We will use client.scan() method from the library which returns all the rows from the table specified.
// Input object with Table specified
// Since we're using scan() method, no query
// is required for us
var params = {
TableName: tableName
};
// client.scan() returns all the documents
// in the table. you can also use client.query()
// in case of adding a condition for selection
client.scan(params, (err, data) => {
if (err) {
console.log(err);
} else {
var items = [];
// the rows are present in the Items property
// of the data object returned in the callback
// extract the Name property from the rows and
// push them into a new array
for (var i in data.Items)
items.push(data.Items[i]['Name']);
// send the obtained rows onto the response
res.contentType = 'application/json';
res.send(items);
}
});The GET endpoint is complete. It is as follows –
app.get("/rows/all", (req, res) => {
var params = {
TableName: tableName
};
client.scan(params, (err, data) => {
if (err) {
console.log(err);
} else {
var items = [];
for (var i in data.Items)
items.push(data.Items[i]['Name']);
res.contentType = 'application/json';
res.send(items);
}
});
});To launch the application, enter the command below in the directory. This will start a local node server on port 3000.
npm startIf you send a request like “http://localhost:3000/rows/all,” the corresponding response will be as follows.
GET http://localhost:3000/rows/all
[
"abc.com",
"xyz.com"
]Implementing POST Endpoint for Writing Data
To add data to DynamoDB, follow the same steps as mentioned earlier, but replace the “scan()” method with the “put()” method. The “put()” method is used to insert a new record into DynamoDB. For initiating the insertion process, we’ll create a POST endpoint. This endpoint will accept data from the request body and then store that content in the database.
app.post("/rows/add", (req, res) => {
// magic happens here
});For easy reading of the request body, we employ the Express module “body-parser.” This module converts the incoming request body into a designated content format, like “application/json.” To utilize this module, we include the required statement at the top alongside other “require()” statements. Additionally, we insert a middleware into the app that triggers bodyParser for every incoming request.
npm install body-parserconst bodyParser = require('body-parser');
// transforms every input request body
// into json objects for reading
app.use(bodyParser.json());For reading the request body, we just use req.body, which provides the JSON-formatted request payload, processed by the bodyParser middleware. Moving forward, we construct an Input object that now holds the document to be added to the table, along with the table’s name.
DynamoDB doesn’t generate incremental PrimaryKeys automatically and each document needs a unique PrimaryKey (also known as HashKey). Hence we employ an additional module that generates a unique identifier (UUID) for every operation.
npm install uuidAfter installation, we add the module into the script by including a “require()” statement at the top. The uuid module offers various versions for usage, and we’ll utilize version 4 for our purposes.
const { v4: uuidv4 } = require('uuid');Lastly, we construct our Input object to be used in the put() function. We then use this params object as depicted below. If the insertion is successful, the callback will display a success log in the console. Conversely, if an error occurs, the raised error will be printed.
app.post("/rows/add", (req, res) => {
var body = req.body;
var params = {
TableName: tableName,
Item: {
// creates a new uuid
"Id": uuidv4(),
// name property passed from body
"Name": body["name"]
}
};
client.put(params, (err, data) => {
if (err) {
console.error("Unable to add item.");
console.error("Error JSON:", JSON.stringify(err, null, 2));
} else {
console.log("Added item:", JSON.stringify(data, null, 2));
}
});
});After modifying the node script, when we execute it and trigger the POST API /rows/add with the provided payload, we should receive a successful response.
POST /rows/add HTTP/1.1
Host: localhost:3000
Content-Type: application/json
{
"name":"abc2.com"
}
Response:
{
"success": true
}We can also verify by calling our GET API again, which should now include the previously added records as well.
[
"abc2.com",
"abc.com",
"xyz.com"
]Conclusion
This approach allows us to make the most of DynamoDB’s capabilities within our Express.js API. We achieve this by utilizing the AWS SDK and DynamoDBClient.
In this article we have looked into how to connect with DynamoDB from an Express.js API and perform simple GET and PUT methods. As a developer we must need to keep in mind the following 5 rules while developing applications with DynamoDB –
- Plan Your Queries: Know what data you need and plan your queries ahead of time.
- Choose the Right Keys: Use suitable primary and sort keys to match your queries.
- Use Indexes Wisely: Secondary indexes can help, but choose them carefully.
- Be Efficient: Retrieve only the necessary data to save time and cost.
- Adjust Throughput: Set provisioned capacity right for your query workload.
You can find the functional source code in the repository https://github.com/referbruv/dynamodb-nodejs-sample. This integration enhances the performance of our Express.js applications by leveraging the power of DynamoDB.
