We use cookies to provide you with a great user experience, analyze traffic and serve targeted promotions.   Learn More   Accept
Card image cap

Reading and Writing to AWS DynamoDB using NodeJS with Example

AWS NodeJS Express DynamoDB  • Posted 27 days ago

DynamoDB is a high-performance NoSQL database service offered by AWS as a part of its AWS Cloud Stack. It is quite popular for its scalability and performance and is used in usecases which require real-time data intensive applications such as Gaming, Leaderboards and so on. AWS also provides libraries for various popular programming stacks to connect and work with DynamoDB. In this article, let's look at how we can connect and work with DynamoDB in a NodeJS application. We shall first look at how we can setup a simple NodeJS API application using Express framework and then add the DynamoDB capabilities to our API.

Please note that to connect to and work with DynamoDB from local environment, we would need to have AWS CLI or Developer Tools for IDEs such as Visual Studio installed and configured in our system. For all the calls the application makes to AWS resources from our local machine, the SDK looks for a credentials file which contains access token and secret created using our AWS subscription credentials. For details on how to install AWS CLI and configure please refer to the AWS documentation.

This article assumes that we have a table 'MailSources' already created in DynamoDB. If not created, you can simply create a new Table MailSources with a HashKey 'Id' of type String using the AWS Console.

To get started, lets create a new API application using Node and Express. Please make sure that the system already has NodeJS installed, if not install NodeJS to continue. In a new folder open a Visual Studio Code editor and give the below command which creates a new package.json for the node application.

> npm init

The CLI asks for a few inputs, which can either be modified or left unchanged. Once the package.json is created, add a new js file and name it "server.js". This file contains the application logic to run a node API. To make things simple, we shall use Express on top of NodeJS which is a minimalistic web framework to build APIs and applications on NodeJS.

To install Express, run the below command:

> npm install express --save

Once installed, open the server.js file we created previously and start adding a GET endpoint for our application, which later reads data from a table in DynamoDB and returns as a JSON.

const express = require('express');
const app = express();

// port on which the server listens
const port = 3000;

app.get("/rows/all", (req, res) => {
    // magic happens here
});

app.listen(port, () => {
    console.log(`Listening on port ${port}`);
});

Next, we would need AWS SDK to access components and services from AWS. In our case, we're interested in accessing the DynamoDB service from AWS. We begin by installing aws-sdk module in our application.

> npm install aws-sdk

Once installed, we would include the library in our application. We add the below lines of code at the top of server.js below our Express modules.

// import the aws sdk to use the DynamoDB
// libraries in the app
const AWS = require('aws-sdk');

Following this, we create an instance of the DynamoDb client which helps in performing operations on the DynamoDB instance as below:

// create a new DynamoDB client
// which provides connectivity b/w the app
// and the db instance
const client = new AWS.DynamoDB.DocumentClient();
const tableName = 'MailSources';

Observe that we also added another field tableName, which represents the name of the Table inside the DynamoDB on which the operations are performed. DynamoDB organizes data in the form of Tables, although each table is a collection of JSON documents. We're also required to configure the Region in which the DynamoDB is hosted and is used in our application.

// update the region to 
// where DynamoDB is hosted
AWS.config.update({ region: 'us-west-2' });

We're now almost done with our prerequisites, let's begin by adding logic to fetch rows from the specified table - "MailDomainsList". The steps for accessing data or pushing data is simple:

  1. Create a client
  2. Create an Input object with Table name and an optional Query or Projection
  3. Call the DynamoDB client with the respective operation along with the Input object

We've already done with our step 1 and have a Client object with us. For the next two steps, we use client.scan() method from the library which returns all the rows from the table specified.

// Input object with Table specified
// Since we're using scan() method, no query
// is required for us
var params = {
    TableName: tableName
};

// client.scan() returns all the documents
// in the table. you can also use client.query()
// in case of adding a condition for selection
client.scan(params, (err, data) => {
    if (err) {
        console.log(err);
    } else {
        var items = [];

        // the rows are present in the Items property
        // of the data object returned in the callback
        // extract the Name property from the rows and
        // push them into a new array
        for (var i in data.Items)
            items.push(data.Items[i]['Name']);

        // send the obtained rows onto the response
        res.contentType = 'application/json';
        res.send(items);
    }
});

The endpoint finally looks like below:

app.get("/rows/all", (req, res) => {
    var params = {
        TableName: tableName
    };

    client.scan(params, (err, data) => {
        if (err) {
            console.log(err);
        } else {
            var items = [];
            for (var i in data.Items)
                items.push(data.Items[i]['Name']);

            res.contentType = 'application/json';
            res.send(items);
        }
    });
});

To run this setup, type the below command within the directory which runs a localhost node server listening on port 3000.

> npm start

For an input request "http://localhost:3000/rows/all" the response would be:

[
    "abc.com",
    "xyz.com"
]

To insert data into DynamoDB, we use the same steps as above but make use of another method put() in place of scan() which inserts a new record into the DynamoDB.

To get started with insertion, we shall add a POST endpoint that receives a payload from the request body and then pushes the content into the database.

app.post("/rows/add", (req, res) => {
   // magic happens here
});

To make reading the request body simple, we make use of another express module "body-parser" which transforms input request body into specified content format (say application/json). To add the module,

> npm install body-parser

And to use the module, we add the include statement on the top along with other require() statements. We also add a middleware to the app, which invokes bodyParser for every request made.

const bodyParser = require('body-parser');

// transforms every input request body 
// into json objects for reading
app.use(bodyParser.json());

To read the request body, we simply call req.body which returns a json formatted request payload transformed thanks to the bodyParser middleware. In the next step, we create an Input object which now contains the document to be inserted into the table along with the table name. Since DynamoDB doesn't have incremental PrimaryKeys auto generated and every document requires a unique PrimaryKey (also known as HashKey), we make use of another module which generates a unique identifier (uuid) for each invocation.

> npm install uuid

Once installed, we include the module inside the script by adding a require() statement on the top. There are several versions of uuid available for use by means of the uuid module, and we're gonna use the v4 of the libraries.

const { v4: uuidv4 } = require('uuid');

Finally we create our Input object to be passed into our put() function as below:

app.post("/rows/add", (req, res) => {
    var body = req.body;
    var params = {
        TableName: tableName,
        Item: {
            // creates a new uuid
            "Id": uuidv4(),
            // name property passed from body
            "Name": body["name"]
        }
    };
});

And we push this params object as below:

client.put(params, (err, data) => {
    if (err) {
        console.error("Unable to add item.");
        console.error("Error JSON:", JSON.stringify(err, null, 2));
    } else {
        console.log("Added item:", JSON.stringify(data, null, 2));
    }
});

If the insertion is successful, the callback would print the success log in the console, else the raised error would be printed. The complete endpoint looks like below:

app.post("/rows/add", (req, res) => {
    var body = req.body;
    var params = {
        TableName: tableName,
        Item: {
            "Id": uuidv4(),
            "Name": body["name"]
        }
    };

    client.put(params, (err, data) => {
        if (err) {
            console.error("Unable to add item.");
            console.error("Error JSON:", JSON.stringify(err, null, 2));
        } else {
            console.log("Added item:", JSON.stringify(data, null, 2));
        }
    });
});

When we run the now modified node script and invoke the POST API /rows/add with the below payload, we should get a success response.

POST /rows/add HTTP/1.1
Host: localhost:3000
Content-Type: application/json

{
    "name":"abc2.com"
}

Response: 

{
    "success": true
}

We can also verify by calling our GET API again, which should now include the previously added records as well.

[
    "abc2.com",
    "abc.com",
    "xyz.com"
]

In this way, we can levarage the capabilities of DynamoDB inside our NodeJS API using the AWS SDK and DynamoDBClient. The working source code is available under the repo https://github.com/referbruv/dynamodb-nodejs-sample

node js dynamodb tutorial node js write to dynamodb dynamodb nodejs nodejs to dynamodb node js read dynamodb dynamodb node js put example dynamodb json node js dynamodb node.js insert node js import dynamodb dynamodb query example nodejs dynamodb using node js dynamodb node js dynamodb node js example dynamodb node js tutorial nodejs read data from dynamodb nodejs read from dynamodb dynamodb nodejs example github nodejs express dynamodb dynamodb example nodejs node js dynamodb nodejs aws dynamodb