Introduction
DynamoDB is a high-performance NoSQL database service offered by AWS as a part of its AWS Cloud Stack. It is quite popular for its scalability and performance and is used in usecases which require real-time data intensive applications such as Gaming, Leaderboards and so on.
AWS provides libraries for various popular programming stacks to connect and work with DynamoDB.
In this article, let’s look at how we can connect and work with DynamoDB in a Express.js application. We shall first look at how we can setup a simple Express.js API application using Express framework and then add the DynamoDB capabilities to our API.
To integrate with DynamoDB from our development machine, we need to have AWS Command Line or Developer Tools for IDEs such as Visual Studio installed and configured on our machine.
Once we’re done with the development, we can deploy this API into any of the AWS Compute service (most preferably AWS Lambda) and ensure that it has a Role to access Dynamo DB resources.
You can check out this article, which explains in detail on how to deploy an Express.js API into AWS Lambda – Deploying Express.js API to AWS Lambda and API Gateway
For all the calls the application makes to AWS resources from our local machine, the SDK looks for a credentials file which contains access token and secret created using our AWS subscription credentials. For details on how to install AWS CLI and configure please refer to the AWS documentation.
This article assumes that we have a table ‘MailSources’ already created in DynamoDB. If not created, you can simply create a new Table MailSources with a HashKey ‘Id’ of type String using the AWS Console.
Getting started with Integrating AWS DynamoDB in Express.js
To get started, lets create a new API application using Node and Express. Please make sure that the system already has NodeJS installed, if not install NodeJS to continue.
In a new folder open a Visual Studio Code editor and give the below command which creates a new package.json for the node application.
> npm initThe CLI asks for a few inputs, which can either be modified or left unchanged. Once the package.json is created, add a new js file and name it “server.js”. This file contains the application logic to run a node API.
To make things simple, we shall use Express on top of NodeJS which is a minimalistic web framework to build APIs and applications on NodeJS.
To install Express, run the below command:
> npm install express --saveImplementing GET Endpoint for Reading Data
Once installed, open the server.js file we created previously and start adding a GET endpoint for our application, which later reads data from a table in DynamoDB and returns as a JSON.
const express = require('express');
const app = express();
// port on which the server listens
const port = 3000;
app.get("/rows/all", (req, res) => {
// magic happens here
});
app.listen(port, () => {
console.log(`Listening on port ${port}`);
});Installing the AWS SDK for Node.js
Next, we need AWS SDK to access components and services from AWS. In our case, we’re interested in accessing the DynamoDB service from AWS. We begin by installing aws-sdk module in our application.
> npm install aws-sdkOnce installed, we would include the library in our application. We add the below lines of code at the top of server.js below our Express modules.
// import the aws sdk to use the DynamoDB
// libraries in the app
const AWS = require('aws-sdk');Following this, we create an instance of the DynamoDb client which helps in performing operations on the DynamoDB instance as below:
// create a new DynamoDB client
// which provides connectivity b/w the app
// and the db instance
const client = new AWS.DynamoDB.DocumentClient();
const tableName = 'MailSources';Observe that we also added another field tableName, which represents the name of the Table inside the DynamoDB on which the operations are performed.
DynamoDB organizes data in the form of Tables, although each table is a collection of JSON documents. We are required to configure the Region in which the DynamoDB is hosted and is used in our application.
// update the region to
// where DynamoDB is hosted
AWS.config.update({ region: 'us-west-2' });
We’re now almost done with our prerequisites, let’s begin by adding logic to fetch rows from the specified table – “MailDomainsList”.
The steps for accessing data or pushing data is simple:
- Create a client
- Create an Input object with Table name and an optional Query or Projection
- Call the DynamoDB client with the respective operation along with the Input object
We’ve already done with our step 1 and have a Client object with us.
For the next two steps, we use client.scan() method from the library which returns all the rows from the table specified.
// Input object with Table specified
// Since we're using scan() method, no query
// is required for us
var params = {
TableName: tableName
};
// client.scan() returns all the documents
// in the table. you can also use client.query()
// in case of adding a condition for selection
client.scan(params, (err, data) => {
if (err) {
console.log(err);
} else {
var items = [];
// the rows are present in the Items property
// of the data object returned in the callback
// extract the Name property from the rows and
// push them into a new array
for (var i in data.Items)
items.push(data.Items[i]['Name']);
// send the obtained rows onto the response
res.contentType = 'application/json';
res.send(items);
}
});The endpoint finally looks like below:
app.get("/rows/all", (req, res) => {
var params = {
TableName: tableName
};
client.scan(params, (err, data) => {
if (err) {
console.log(err);
} else {
var items = [];
for (var i in data.Items)
items.push(data.Items[i]['Name']);
res.contentType = 'application/json';
res.send(items);
}
});
});
To run this setup, type the below command within the directory which runs a localhost node server listening on port 3000.
> npm start
For an input request “http://localhost:3000/rows/all” the response would be:
[
"abc.com",
"xyz.com"
]
Implementing POST Endpoint for Writing Data
To insert data into DynamoDB, we use the same steps as above but make use of another method put() in place of scan() which inserts a new record into the DynamoDB.
To get started with insertion, we shall add a POST endpoint that receives a payload from the request body and then pushes the content into the database.
app.post("/rows/add", (req, res) => {
// magic happens here
});
To make reading the request body simple, we make use of another express module “body-parser” which transforms input request body into specified content format (say application/json). To add the module,
> npm install body-parser
And to use the module, we add the include statement on the top along with other require() statements. We also add a middleware to the app, which invokes bodyParser for every request made.
const bodyParser = require('body-parser');
// transforms every input request body
// into json objects for reading
app.use(bodyParser.json());
To read the request body, we simply call req.body which returns a json formatted request payload transformed thanks to the bodyParser middleware. In the next step, we create an Input object which now contains the document to be inserted into the table along with the table name.
Since DynamoDB doesn’t have incremental PrimaryKeys auto generated and every document requires a unique PrimaryKey (also known as HashKey), we make use of another module which generates a unique identifier (uuid) for each invocation.
> npm install uuid
Once installed, we include the module inside the script by adding a require() statement on the top. There are several versions of uuid available for use by means of the uuid module, and we’re gonna use the v4 of the libraries.
const { v4: uuidv4 } = require('uuid');
Finally we create our Input object to be passed into our put() function as below:
app.post("/rows/add", (req, res) => {
var body = req.body;
var params = {
TableName: tableName,
Item: {
// creates a new uuid
"Id": uuidv4(),
// name property passed from body
"Name": body["name"]
}
};
});
And we push this params object as below:
client.put(params, (err, data) => {
if (err) {
console.error("Unable to add item.");
console.error("Error JSON:", JSON.stringify(err, null, 2));
} else {
console.log("Added item:", JSON.stringify(data, null, 2));
}
});
If the insertion is successful, the callback would print the success log in the console, else the raised error would be printed. The complete endpoint looks like below:
app.post("/rows/add", (req, res) => {
var body = req.body;
var params = {
TableName: tableName,
Item: {
"Id": uuidv4(),
"Name": body["name"]
}
};
client.put(params, (err, data) => {
if (err) {
console.error("Unable to add item.");
console.error("Error JSON:", JSON.stringify(err, null, 2));
} else {
console.log("Added item:", JSON.stringify(data, null, 2));
}
});
});
Found this article helpful? Please consider supporting!
When we run the now modified node script and invoke the POST API /rows/add with the below payload, we should get a success response.
POST /rows/add HTTP/1.1
Host: localhost:3000
Content-Type: application/json
{
"name":"abc2.com"
}
Response:
{
"success": true
}
We can also verify by calling our GET API again, which should now include the previously added records as well.
[
"abc2.com",
"abc.com",
"xyz.com"
]
In this way, we can levarage the capabilities of DynamoDB inside our Express.js API using the AWS SDK and DynamoDBClient. The working source code is available under the repo https://github.com/referbruv/dynamodb-nodejs-sample
