Amazon DynamoDB with OutSystems

Introduction

“We would like to see the birth date of the employees in the personal details, plus the age”. “We want to display the price of the product, next to the quantity ordered by the user”.

These examples show one typical requirement asked by customers: adding new information to their applications. From a technical point of view, this could mean that some kind of change will happen to our data model:

• A new field needs to be added to an existing table.
• A new table needs to be created.
• New tables/fields might need to be deleted and/or the relations between them rearranged.
• Others.

In most cases, these seem to be very intuitive answers, particularly if we assume that our data is being represented by a relational data model. However, there are also cases where non-relational data models might be more suitable for representing our data. The latter is going to be the main topic of this article.

The Problem

One of the main requirements – that quickly turned on into a big challenge – was a request by the customer to build an application with few screens but with a lot of data in them. Because all of the data comes from external APIs – that don’t perform the best – performance issues begin to rise, particularly in the screens that required calls to more than one API.

In the first approach, a relational data model was built in to provide a way to save data that would be further sent to the APIs. As the screens grew up with more and more data, the relational data model also increased, as well as the number of queries needed.

After a global analysis, it was clear that this solution was not scalable and not performable. The number of calls to the database needed to be reduced but the data still needed to be accessed. This is where the idea of applying a non-relational data model appeared.

The Solution: Non-Relational Data Models – The Amazon DynamoDB in OutSystems

As the name suggests, non-relational data models don’t follow the traditional pattern of relations between the different data. This means that in these kinds of models there are no explicit connections that link data across various tables.

The example I used here in this article is about Amazon DynamoDB, a non-relational database service that allows companies to store their data in a fast and intuitive way. The main advantages of this service are:

• Ability to store and retrieve large amounts of data.
• Serve high levels of access traffic.
• Improve the performance, particularly of large-scale apps.
• Free capacity management.

To use this service in OutSystems, we can start by going to the OutSystems Forge, download the Amazon DynamoDB Connector and install it on the platform. After the installation is complete, there are only 3 steps that we need to perform to implement and run the DynamoDB code in our applications.

The first step is to create our own OutSystems actions that will call the DynamoDB actions. Currently, the DynamoDB actions that are available allow you to:

• Create a table.
• Delete a table.
• Get data from a table.
• Save data to a table.
• Delete data from a table.

As an example, if we want to load certain data from one table when we enter a web screen, this means that we should create an OutSystems action that will call the DynamoDB get method, and then use it in the screen’s preparation (see screenshot 1).

After having our OutSystems actions created we need to understand how we are going to send the data to DynamoDB. DynamoDB’s actions have a parameter called Item, which is nothing more and nothing less than a JSON. This JSON can be:

• A single field, such as an identifier, if we are using the get method.
• A complex and long combination of fields, records, and lists, if we are using the put method.

The JSON, however, has a very unique format. DynamoDB doesn’t work with the basic field types, but instead with objects. This means that:

• If we have a boolean field on our application and we want to save it in DynamoDB, we need to have a Dynamo Boolean Object. To do this, we create a structure and we add a boolean field to it (see screenshot 2);
• If we have a text field on our application and we want to save it in DynamoDB, we need to have a Dynamo String Object. To do this, we create a structure and we add a text field to it;
• For a record of basic type fields, we need to create a structure in which each field is an object of a basic type.

For more information on how to map OutSystems structures with DynamoDB’s structures, you can check the documentation on the service website: https://aws.amazon.com/dynamodb/.

Finally, the third step is to do all configurations needed to use DynamoDB’s services. These include:

• Generating the service credentials (Region, AccessKeyId, and SecretAccessKey).
• Create a table (can also be done in runtime, using DynamoDB’s create table action).
• Define the primary key.
• Define access capacity.

These configurations are done in the site of Amazon DynamoDB, which requires an account to start using the services (see screenshot 3).

As stated before, DynamoDB can be used to improve the performance of large-scale apps, particularly in screens that contain a lot of data from various tables. Instead of doing multiple and repeated queries in different locations of the screen to get the needed data, we can store our tables in a single object in Dynamo.

When entering the screen, we do a single call to DynamoDB’s get method and then we can work with our data in memory (local variables). Each time we need to update our data, we can simply change our DynamoDB’s object stored in memory (using the OutSystems platform actions) and then call DynamoDB’s put method.

This has the advantage to not only centralize all of the data into a single object but it also allows for minimizing the calls to the database to get the information we need. Adding new fields to our object could be easier than adding fields to our relational data model’s tables, because the latter might also involve considerations into how the relations might be affected.

Conclusions 

DynamoDB was a very helpful tool that allowed a big improvement in performance and, as a consequence, a better experience for the users. 

The improvements were particularly relevant in screens that were loaded with a lot of data and that required multiple calls to APIs. DynamoDB also helped in building a more uniform and centralized data model that prevented the use of multiple queries to different tables to get the required data.