API Development

Use Axway Integration Builder in a Sentiment Analysis Application: Part 1

| Leor Brenman |

Introduction

Sentiment analysis (also known as opinion mining or emotion AI) refers to the use of natural language processing, text analysis, computational linguistics, and biometrics to identify systematically, extract, quantify and study affective states and subjective information. Sentiment analysis is widely applied to voice of the customer materials such as reviews and survey responses, online and social media, and healthcare materials for applications that range from marketing to customer service to clinical medicine.

We’ll use Integration Builder to extract the text from our data source, make a call to Amazon Comprehend natural language processing (NLP) service, through API Builder, to determine the sentiment and send a notification to the appropriate stakeholder.

We’ll look at two examples:

In both cases, we would like to notify a stakeholder if a negative sentiment is discovered. For our example, we’ll send a notification to Microsoft Teams but we could also or alternatively, use Integration Builder’s Twilio connector to send an SMS to a mobile phone or the Slack connector to send a notification to Slack.

This will give the stakeholder a means to be proactively aware of negative sentiment and take immediate corrective action instead of waiting for the data to be analyzed manually.

In this blog post, we’ll focus on the Net Promoter Score survey example. In the next blog post, we’ll look at the Salesforce example.

Setup

The diagram below illustrates the flow of data:

Dataflow Diagram
  • We’ll use Typeform for our NPS Survey since it is (a) easy to use, (b) provides a free tier and (c) provides a means to call a webhook when the user submits the completed form.
  • In this example, our NPS Survey will contain two questions:
    • “How likely are you to recommend us to a friend or colleague?”
    • “What is the primary reason for your score?”
  • We’ll look for the sentiment in the second question (What is the primary reason for your score?)
  • We’ll configure our Integration Builder flow for manual trigger since that will give us an API in AMPLIFY Central in order to trigger the flow as described here. This is the webhook that we’ll use in Typeform
  • We’ll create an API Builder project to expose AWS Comprehend’s sentiment service as a sentiment API as described here. We’ll publish this API Builder project to Axway AMPLIFY Runtime Services (ARS) as described here
  • We’ll use the swagger docs for our API Builder microservice to create a connector in Integration Builder as described here to make it easy to call the sentiment API in Integration Builder
  • If the sentiment is determined to be negative, we’ll send a notification to MS Teams as shown below:
MS Teams Notification

Note that information in the notification contains the sentiment, the answer to the question, the form name and id and the token corresponding to the particular response

Typeform NPS Survey

In this section, we’ll create a Typeform NPS Survey.

You can review the Typeform Help Center for help on how to create a survey.

Here is what we’ll do:

  • Create a simple survey
  • Assign a reference to the question that you are concerned about
  • Configure a webhook so that when the survey is submitted, our flow will be triggered

I created a simple NPS Survey made up of two questions as shown below:

NPS Survey Question 1
NPS Survey Question 2

As you are creating your form, note the form id in the URL as shown below:

Form ID in URL

Note that my form has a form id of QsM3CQ. We will use this in our flow to make sure we are processing the correct form.

For our NPS Survey, we are mainly concerned about the second question, “What is the primary reason for your score?” since that is where the user may express their sentiment. When our Integration Builder flow parses the survey data we will look for the second question. Typeform enables you to assign a unique name to the question to make the coding easier as shown below:

Question Reference

Note that I named the question ref sentimentQuestion. We’ll see this later when we look our Integration Builder flow.

Finally, I configured the survey to send the survey data to a webhook as shown below:

Form Webhook

The webhook is the AMPLIFY Central API corresponding to my Integration Builder manually triggered flow.

Note that at this point we don’t know the webhook URL yet because we haven’t created our flow but we’ll get to that soon

We can test our survey and webhook by using a service like pipedream’s requestbin, a modern request bin to inspect HTTP events and see what is posted, as shown below:

{
    "event_id": "01E82GJ98P8YCTF6N8TRDV8PA1",
    "event_type": "form_response",
    "form_response": {
        "form_id": "QsM3CQ",
        "token": "5ndeib6dyv37m70eo06ae5ndeib6gezh",
        "landed_at": "2020-05-11T18:39:37Z",
        "submitted_at": "2020-05-11T18:39:44Z",
        "definition": {
            "id": "QsM3CQ",
            "title": "Axway NPS Poll May 2020",
            "fields": [
                {
                    "id": "kYJjqO9kwidB",
                    "title": "How likely are you to recommend us to a friend or colleague?",
                    "type": "opinion_scale",
                    "ref": "c525dc67-d2d0-438f-81a8-6efa4c0443fc",
                    "properties": {}
                },
                {
                    "id": "tVprjNrOIfM2",
                    "title": "What is the primary reason for your score?",
                    "type": "long_text",
                    "ref": "sentimentQuestion",
                    "properties": {}
                }
            ]
        },
        "answers": [
            {
                "type": "number",
                "number": 8,
                "field": {
                    "id": "kYJjqO9kwidB",
                    "type": "opinion_scale",
                    "ref": "c525dc67-d2d0-438f-81a8-6efa4c0443fc"
                }
            },
            {
                "type": "text",
                "text": "Terrible Support",
                "field": {
                    "id": "tVprjNrOIfM2",
                    "type": "long_text",
                    "ref": "sentimentQuestion"
                }
            }
        ]
    }
}

Note the question ref sentimentQuestion at the bottom of the response. We will parse the response for that answer in our Integration Builder flow.

At this point, we’re done with building our Typeform NPS Survey. We’ll have to come back later and set up the webhook URL but for now we can move onto API Builder.

API Builder Sentiment API

In this section, we’ll create an API Builder microservice that will expose the AWS Comprehend sentiment service.

Here is what we’ll do:

  • Create an API Builder project to expose AWS Comprehend’s sentiment service as a sentiment API as described here
  • Publish this API Builder project to Axway AMPLIFY Runtime Services (ARS) as described here
  • Get the swagger docs for this deployed microservice

Follow this blog post to create your API Builder project but instead of using the AWS Comprehend detectDominantLanguage service, use the detectSentiment service as shown below:

var APIBuilder = require('@axway/api-builder-runtime');

var AWS = require('aws-sdk');
AWS.config.update({
  region: 'us-east-1'
});
var comprehend = new AWS.Comprehend();

var detectsentiment = APIBuilder.API.extend({
    group: 'detectsentiment',
    path: '/api/detectsentiment',
    method: 'GET',
    description: 'this is an api that detects the sentiment of the text passed in the text query parameter',
    parameters: {
        text: { description: 'the text to detect the sentiment of' }
    },
    action: function (req, resp, next) {

        console.log(req.params.text);

        var params = {
        LanguageCode: "en",
        Text: req.params.text
      };

        comprehend.detectSentiment(params, function(err, data) {
        if (err) {
                console.log(err, err.stack);
                resp.response.status(400);
                resp.send(err);
            }
        else {
                console.log(data);
                resp.response.status(200);
                resp.send(data);
            }
            next();
      });

    }
});

module.exports = detectsentiment;

Note that we are assuming that the language is english in our API above. Instead we could have used the AWS Comprehend language detection service described here and then called the detectSentiment service with the proper language

Test your API and when you are satisfied that it’s working properly, you can follow the instructions here to create a Docker image and deploy in any docker platform or to publish to the Axway’s platform runtime service, ARS.

The swagger for your microservice is accessible from /apidoc/swagger.json. If you used ARS, then it looks similar to following:

https://<SUB_DOMAIN_TOKEN>.cloudapp-enterprise.appcelerator.com/apidoc/swagger.json

Paste the swagger contents into a swagger.json file on your computer. We’ll use this later when we create an Integration Builder connector to this microservice.

Integration Builder

In this section we’ll create our Integration Builder flow and create a connector for the API Builder microservice. You can download the flow here.

Here is what we will do:

  • Create a connector to our API Builder microservice
  • Create a manually triggered flow so that when the Typeform survey is submitted the flow will be triggered via a webhook
  • Make sure the flow is processing the correct survey
  • Call the sentiment API Builder microservice via the connector we created
  • Check the sentiment to see if it’s negative (or mixed)
  • If so, send a notification to MS Teams
  • Create an instance of the flow, enter the variables and then get the URL that is proxied in Central. This will be the webhook that we will use for the Typeform webhook when the survey is submitted

The resulting flow is shown below:

The Flow

Create a connector to our API Builder microservice

Follow this blog post to create a connector to the API Builder microservice you published above. I called mine Detect Sentiment. Test your connector in the API Docs:

Test Your Sentiment API

Create a manually triggered flow

Follow the instructions here to create a manually triggered flow. I called mine NPS Sentiment Notifier.

I created the following flow variables:

  • *detectSentiment* – connector instance variable for the API Builder connector
  • *formId* – value variable for the survey id
  • *msTeamsWebhookURL* – value variable for the MS Teams webhook
  • *sentimentRef* – value variable for the survey question reference who’s sentiment we are looking for

Make sure the flow is processing the correct survey

Add a JS Filter step, shouldProcess, to the output of the trigger. The contents are as follows:

let form_id = trigger.args.form_response.form_id;
let answers = trigger.args.form_response.answers;
let obj = answers.find(obj => obj.field.ref == config.sentimentRef);
config.myData = {
  answerObject: obj
}
done((form_id === config.formId) && ( obj && obj !== "null" && obj !== "undefined" ));

The javascript above performs the following:

  • Checks the form id to make sure we are processing the correct Typeform survey
  • Checks to see if the user answered the second question (What is the primary reason for your score)
  • Stores the answer to the question for later (in the config object)

Call the sentiment API

We’ll use a Connector API Request step to call the API Builder sentiment API. First we use a JS Script step called prepareGetSentiment to calculate the query parameters as follows:

let query = {
  text: config.myData.answerObject.text
}
done({query: query});

The Connector API Request step, getSentiment, is shown below:

getSentiment Connector API Request Step

Check the sentiment response

Add a JS Filter step, shouldNotify, to check if the sentiment response is negative or mixed as follows:

done(steps.getSentiment.response.body.Sentiment === 'NEGATIVE' || steps.getSentiment.response.body.Sentiment === 'MIXED');

Send a notification to MS Teams

If the sentiment is negative or mixed, we’ll send a notification to MS Teams.

We’ll use an HTTP Request step to send a notification to MS Teams via a webhook. First, we use a JS Script step called prepareNotifyTeams to calculate the body as follows:

var sentiment;
if(steps.getSentiment.response.body.Sentiment === 'NEGATIVE') {
  sentiment = 'Negative';
} else if(steps.getSentiment.response.body.Sentiment === 'MIXED') {
  sentiment = 'Mixed';
} else {
  sentiment = 'Unknown';
}

let resultsUrl = "https://admin.typeform.com/form/"+config.formId+"/results";

let body = {
    "@type": "MessageCard",
    "@context": "http://schema.org/extensions",
    "themeColor": "0076D7",
    "summary": "NPS Score Sentiment Notifier",
    "sections": [{
        "activityTitle": "NPS Score Sentiment Notifier",
        "activitySubtitle": sentiment+" sentiment detected",
        "facts": [{
            "name": "Sentiment",
            "value": sentiment
        },{
            "name": "Form Name",
            "value": trigger.args.form_response.definition.title
        },{
            "name": "Form ID",
            "value": trigger.args.form_response.form_id
        },{
            "name": "Note",
            "value": "'"+config.myData.answerObject.text+"'"
        },{
            "name": "Form Results",
            "value": "[Click to View]("+resultsUrl+")"
        },{
            "name": "Token",
            "value": trigger.args.form_response.token
        }],
        "markdown": true
    }]
}

done({body: body});

The HTTP Request step, notifyTeams, is shown below:

notifyTeams HTTP Request Step

Create an instance of the flow

Create a flow instance and attach the API Builder Detect Sentiment connector and enter the other variables for your Typeform survey and MS Teams webhook:

Flow Instance

Test Your Flow

Follow the instructions here to get your flow’s API in AMPLIFY Central. This is the webhook you’ll enter in your Typeform survey.

Fill out your survey and enter a negative sentiment: “Terrible support”. You should get a notification in MS Teams as follows:

MS Team Notification

Summary

In this blog post we saw how Integration Builder and API Builder can be used together with AWS Comprehend Natural Language Processing service to implement a sentiment analysis application to alert marketing via MS Teams. This enables a company to be more proactive in analyzing Net Promoter Score survey responses.

Discover more about Integration Builder: Graceful Troubleshooting During Development.

Click Here