API Builder

Arrow Builder API Server Sizing using Load Testing

So, you’ve designed and tested your Arrow Builder Project (APIs) and are ready to publish it. What size container should you choose? Small, Medium, Large, XLarge? How many servers should you deploy? The answer depends on the complexity of the project as well as the load that will be placed on the APIs.

For example, if your application will have many concurrent API requests then you may need to increase the number of servers. The number of concurrent API requests will depend on the number of users and an estimate of concurrent users. If your project is complex with a lot of processing (which requires more CPU/RAM) then you may need to increase the size of each server/container.

Load testing is one way to help determine server sizing and Loader is an API load test tool that can be used to help. This blog post will walk you through setting up Loader with an Arrow Builder project and provide an example of how Loader is used to determine suitable server sizing.

The basic steps for enabling your app for use with Loader are:

  1. Publish your Arrow App and get your URL base address
  2. Create a new Loader Target Host and enter the Arrow App URL base address
  3. Download a verification token from Loader and place this file in your Arrow project
  4. Republish your Arrow Builder Project
  5. Verify that Loader can reach the Arrow Builder App
  6. Create a new Loader test that will create a load on the desired API
  7. Run the Loader test

Publish Your Arrow Builder App

In order to use Loader, you will first need a Loader account. This is free to create and you can test one host (e.g. one Arrow Builder app) with the free tier. In order to setup a Loader test, you will need the Arrow Builder application’s URL. If you haven’t published your application then you will need to do that using the following CLI command:

appc publish

The base URL will be displayed in response, as shown below, for example:

blog-arrowbuilder-image1

You can find your URL using the following CL command:

appc cloud list

Create a new Loader Target Host

In the Loader web site (https://loader.io) create a new Host and enter your Arrow Builder URL.

blog-arrowbuilder-loader1

And click next.

blog-arrowbuilder-loader2

On the verify screen above, you can download a file with the verification token that loader needs in order to run a load test against the URL.

Place this token file in the Arrow Builder project in the web/public folder.

blog-arrowbuilder-image2

Now you should re-publish the project using the following CLI commands:

appc publish --force

On the Loader verify page, click verify. If all is working well, you will see a screen similar to below with a Congrats message in a light green box.

blog-arrowbuilder-loader3

Create a Loader Test and Run It

On the Loader verification screen or in the Loader main navigation, you can create a new Test.

blog-arrowbuilder-loader4

Give your test a name and select the type and parameters (e.g. API path, methid, protocol, client, duration, …). If your API uses basic authentication, you can enter the information by clicking the Advanced Settings button. My API uses API Key authentication, so I created the APIKey header entry.

At this point you can now run your test and see the results.

The following results are for 250 calls made over a 1 minute duration:

blog-arrowbuilder-loader5
blog-arrowbuilder-loader6

The following results are for 5,000 calls made over a 15 second duration:

blog-arrowbuilder-loader7
blog-arrowbuilder-loader8

With the load tests above, I can see that my container size is not suitable for a load of 5000 requests made in a 15 second period due to long response times and timeouts, which will result in a poor user experience in the mobile app or web app (that is using these APIs). If I have enough container points in my account, I should consider changing the size of my container to Medium, Large or XLarge and/or change the number of servers I am allocating to this application.

Use the appc cloud list CLI command to check the current container size and your container point quota and availability and use the appc cloud server –set CLI command to change the container size for your application. Use the appc cloud config –maxsize and appc cloud config –minsize to set the number of servers allocated to your application.

Once you determine the correct sizing and number of servers then these parameters can be set in your Arrow application appc.json file.

Another Example

I created a simple Arrow Builder application that exposed data stored in ArrowDB. The model is simple:

var Arrow = require('arrow');
var Model = Arrow.createModel('employee', {
    fields: {
        fname: {
            type: String
        },
        lname: {
            type: String
        },
        title: {
            type: String
        }
    },
    connector: 'appc.arrowdb',
    actions: [
        'create',
        'read',
        'update',
        'delete',
        'deleteAll'
    ]
});
module.exports = Model;

I published the app and set to small size using:

appc cloud server --set Small

I estimated that the peak load I wanted to support was 2,000 requested in a 15 second period. This is based on the number of users of my app and an estimate on the number of concurrent users and potential concurrent API requests.

I set up a Loader test and had the following results which were consistent on all load tests.

blog-arrowbuilder-loader9
blog-arrowbuilder-loader10

You can see that many requests time out above (see orange area of the 2nd chart). This will result in a poor user experience since the user will need to request the data again.

As can be seen below, setting the container to Large using the following command does not help:

appc cloud server --set Small

blog-arrowbuilder-loader11
blog-arrowbuilder-loader12

The reason that increasing the container size to Large does not help is because this application does not do any processing but simply passes data from ArrowDB to the client. So it does not need more CPU and/or RAM. What we need is more servers.

I increased the number of servers to 2 using the following two commands:

appc cloud config --minsize 2
appc cloud config --maxsize 2

I also reduced the server size to medium using the following command:

appc cloud server --set Medium

Repeated load tests show that the 2 servers can consistently handle this load:

blog-arrowbuilder-loader13
blog-arrowbuilder-loader14

Now that I am happy with the application performance I can set my application’s appc.json from it’s default setting:

{
  "type": "api",
  "group": "arrow",
  "dependencies": {
    "connector/appc.arrowdb": "*",
    "connector/appc.composite": "*"
  },
  "cloud": {
    "guid": "57a3e647e8e127fb1071a7e1",
    "container": "Dev",
    "minimum": 1,
    "maximum": 1,
    "maxqueuesize": 50,
    "environment": {},
    "cname": null,
    "certificate": null
  }
}

with this configuration:

{
  "type": "api",
  "group": "arrow",
  "dependencies": {
    "connector/appc.arrowdb": "*",
    "connector/appc.composite": "*"
  },
  "cloud": {
    "guid": "57a3e647e8e127fb1071a7e1",
    "container": "Medium",
    "minimum": 2,
    "maximum": 2,
    "maxqueuesize": 50,
    "environment": {},
    "cname": null,
    "certificate": null
  }
}

and publish my application.