Site iconAxway Blog

Use Axway Integration Builder in a Sentiment Analysis Application: Part 1

Introduction

Sentiment analysis (also known as opinion mining or emotion AI) refers to the use of natural language processing, text analysis, computational linguistics, and biometrics to identify systematically, extract, quantify and study affective states and subjective information. Sentiment analysis is widely applied to voice of the customer materials such as reviews and survey responses, online and social media, and healthcare materials for applications that range from marketing to customer service to clinical medicine.

We’ll use Integration Builder to extract the text from our data source, make a call to Amazon Comprehend natural language processing (NLP) service, through API Builder, to determine the sentiment and send a notification to the appropriate stakeholder.

We’ll look at two examples:

In both cases, we would like to notify a stakeholder if a negative sentiment is discovered. For our example, we’ll send a notification to Microsoft Teams but we could also or alternatively, use Integration Builder’s Twilio connector to send an SMS to a mobile phone or the Slack connector to send a notification to Slack.

This will give the stakeholder a means to be proactively aware of negative sentiment and take immediate corrective action instead of waiting for the data to be analyzed manually.

In this blog post, we’ll focus on the Net Promoter Score survey example. In the next blog post, we’ll look at the Salesforce example.

Setup

The diagram below illustrates the flow of data:

Dataflow Diagram
MS Teams Notification

Note that information in the notification contains the sentiment, the answer to the question, the form name and id and the token corresponding to the particular response

Typeform NPS Survey

In this section, we’ll create a Typeform NPS Survey.

You can review the Typeform Help Center for help on how to create a survey.

Here is what we’ll do:

I created a simple NPS Survey made up of two questions as shown below:

NPS Survey Question 1
NPS Survey Question 2

As you are creating your form, note the form id in the URL as shown below:

Form ID in URL

Note that my form has a form id of QsM3CQ. We will use this in our flow to make sure we are processing the correct form.

For our NPS Survey, we are mainly concerned about the second question, “What is the primary reason for your score?” since that is where the user may express their sentiment. When our Integration Builder flow parses the survey data we will look for the second question. Typeform enables you to assign a unique name to the question to make the coding easier as shown below:

Question Reference

Note that I named the question ref sentimentQuestion. We’ll see this later when we look our Integration Builder flow.

Finally, I configured the survey to send the survey data to a webhook as shown below:

Form Webhook

The webhook is the AMPLIFY Central API corresponding to my Integration Builder manually triggered flow.

Note that at this point we don’t know the webhook URL yet because we haven’t created our flow but we’ll get to that soon

We can test our survey and webhook by using a service like pipedream’s requestbin, a modern request bin to inspect HTTP events and see what is posted, as shown below:

{
    "event_id": "01E82GJ98P8YCTF6N8TRDV8PA1",
    "event_type": "form_response",
    "form_response": {
        "form_id": "QsM3CQ",
        "token": "5ndeib6dyv37m70eo06ae5ndeib6gezh",
        "landed_at": "2020-05-11T18:39:37Z",
        "submitted_at": "2020-05-11T18:39:44Z",
        "definition": {
            "id": "QsM3CQ",
            "title": "Axway NPS Poll May 2020",
            "fields": [
                {
                    "id": "kYJjqO9kwidB",
                    "title": "How likely are you to recommend us to a friend or colleague?",
                    "type": "opinion_scale",
                    "ref": "c525dc67-d2d0-438f-81a8-6efa4c0443fc",
                    "properties": {}
                },
                {
                    "id": "tVprjNrOIfM2",
                    "title": "What is the primary reason for your score?",
                    "type": "long_text",
                    "ref": "sentimentQuestion",
                    "properties": {}
                }
            ]
        },
        "answers": [
            {
                "type": "number",
                "number": 8,
                "field": {
                    "id": "kYJjqO9kwidB",
                    "type": "opinion_scale",
                    "ref": "c525dc67-d2d0-438f-81a8-6efa4c0443fc"
                }
            },
            {
                "type": "text",
                "text": "Terrible Support",
                "field": {
                    "id": "tVprjNrOIfM2",
                    "type": "long_text",
                    "ref": "sentimentQuestion"
                }
            }
        ]
    }
}

Note the question ref sentimentQuestion at the bottom of the response. We will parse the response for that answer in our Integration Builder flow.

At this point, we’re done with building our Typeform NPS Survey. We’ll have to come back later and set up the webhook URL but for now we can move onto API Builder.

API Builder Sentiment API

In this section, we’ll create an API Builder microservice that will expose the AWS Comprehend sentiment service.

Here is what we’ll do:

Follow this blog post to create your API Builder project but instead of using the AWS Comprehend detectDominantLanguage service, use the detectSentiment service as shown below:

var APIBuilder = require('@axway/api-builder-runtime');

var AWS = require('aws-sdk');
AWS.config.update({
  region: 'us-east-1'
});
var comprehend = new AWS.Comprehend();

var detectsentiment = APIBuilder.API.extend({
    group: 'detectsentiment',
    path: '/api/detectsentiment',
    method: 'GET',
    description: 'this is an api that detects the sentiment of the text passed in the text query parameter',
    parameters: {
        text: { description: 'the text to detect the sentiment of' }
    },
    action: function (req, resp, next) {

        console.log(req.params.text);

        var params = {
        LanguageCode: "en",
        Text: req.params.text
      };

        comprehend.detectSentiment(params, function(err, data) {
        if (err) {
                console.log(err, err.stack);
                resp.response.status(400);
                resp.send(err);
            }
        else {
                console.log(data);
                resp.response.status(200);
                resp.send(data);
            }
            next();
      });

    }
});

module.exports = detectsentiment;

Note that we are assuming that the language is english in our API above. Instead we could have used the AWS Comprehend language detection service described here and then called the detectSentiment service with the proper language

Test your API and when you are satisfied that it’s working properly, you can follow the instructions here to create a Docker image and deploy in any docker platform or to publish to the Axway’s platform runtime service, ARS.

The swagger for your microservice is accessible from /apidoc/swagger.json. If you used ARS, then it looks similar to following:

https://<SUB_DOMAIN_TOKEN>.cloudapp-enterprise.appcelerator.com/apidoc/swagger.json

Paste the swagger contents into a swagger.json file on your computer. We’ll use this later when we create an Integration Builder connector to this microservice.

Integration Builder

In this section we’ll create our Integration Builder flow and create a connector for the API Builder microservice. You can download the flow here.

Here is what we will do:

The resulting flow is shown below:

The Flow

Create a connector to our API Builder microservice

Follow this blog post to create a connector to the API Builder microservice you published above. I called mine Detect Sentiment. Test your connector in the API Docs:

Test Your Sentiment API

Create a manually triggered flow

Follow the instructions here to create a manually triggered flow. I called mine NPS Sentiment Notifier.

I created the following flow variables:

Make sure the flow is processing the correct survey

Add a JS Filter step, shouldProcess, to the output of the trigger. The contents are as follows:

let form_id = trigger.args.form_response.form_id;
let answers = trigger.args.form_response.answers;
let obj = answers.find(obj => obj.field.ref == config.sentimentRef);
config.myData = {
  answerObject: obj
}
done((form_id === config.formId) && ( obj && obj !== "null" && obj !== "undefined" ));

The javascript above performs the following:

Call the sentiment API

We’ll use a Connector API Request step to call the API Builder sentiment API. First we use a JS Script step called prepareGetSentiment to calculate the query parameters as follows:

let query = {
  text: config.myData.answerObject.text
}
done({query: query});

The Connector API Request step, getSentiment, is shown below:

getSentiment Connector API Request Step

Check the sentiment response

Add a JS Filter step, shouldNotify, to check if the sentiment response is negative or mixed as follows:

done(steps.getSentiment.response.body.Sentiment === 'NEGATIVE' || steps.getSentiment.response.body.Sentiment === 'MIXED');

Send a notification to MS Teams

If the sentiment is negative or mixed, we’ll send a notification to MS Teams.

We’ll use an HTTP Request step to send a notification to MS Teams via a webhook. First, we use a JS Script step called prepareNotifyTeams to calculate the body as follows:

var sentiment;
if(steps.getSentiment.response.body.Sentiment === 'NEGATIVE') {
  sentiment = 'Negative';
} else if(steps.getSentiment.response.body.Sentiment === 'MIXED') {
  sentiment = 'Mixed';
} else {
  sentiment = 'Unknown';
}

let resultsUrl = "https://admin.typeform.com/form/"+config.formId+"/results";

let body = {
    "@type": "MessageCard",
    "@context": "http://schema.org/extensions",
    "themeColor": "0076D7",
    "summary": "NPS Score Sentiment Notifier",
    "sections": [{
        "activityTitle": "NPS Score Sentiment Notifier",
        "activitySubtitle": sentiment+" sentiment detected",
        "facts": [{
            "name": "Sentiment",
            "value": sentiment
        },{
            "name": "Form Name",
            "value": trigger.args.form_response.definition.title
        },{
            "name": "Form ID",
            "value": trigger.args.form_response.form_id
        },{
            "name": "Note",
            "value": "'"+config.myData.answerObject.text+"'"
        },{
            "name": "Form Results",
            "value": "[Click to View]("+resultsUrl+")"
        },{
            "name": "Token",
            "value": trigger.args.form_response.token
        }],
        "markdown": true
    }]
}

done({body: body});

The HTTP Request step, notifyTeams, is shown below:

notifyTeams HTTP Request Step

Create an instance of the flow

Create a flow instance and attach the API Builder Detect Sentiment connector and enter the other variables for your Typeform survey and MS Teams webhook:

Flow Instance

Test Your Flow

Follow the instructions here to get your flow’s API in AMPLIFY Central. This is the webhook you’ll enter in your Typeform survey.

Fill out your survey and enter a negative sentiment: “Terrible support”. You should get a notification in MS Teams as follows:

MS Team Notification

Summary

In this blog post we saw how Integration Builder and API Builder can be used together with AWS Comprehend Natural Language Processing service to implement a sentiment analysis application to alert marketing via MS Teams. This enables a company to be more proactive in analyzing Net Promoter Score survey responses.

Discover more about Integration Builder: Graceful Troubleshooting During Development.

 

 

Exit mobile version