Tutorial: use the Watson cognitive service with FRED

with No Comments

This tutorial will show you how to use FRED — a cloud-based Node-RED — with the IBM Watson cognitive services. We will be using the node-red-node-watson module developed by IBM, and you will be able to find the node module on FRED. We will briefly describe the usages of these Watson nodes, and also show you a small application that tracks the general moods of a specific twitter topic in real time using the Watson Tone Analyze service.

Watson is a fast growing cloud-based cognitive service developed by IBM and is combined with artificial intelligence and analytical services. Watson products are available on IBM’s Bluemix platform. With its rich APIs, Watson makes it an accessible tool for cognitive services developers.

Node-RED provides a great way to connect and manage data flows, and it is capable of performing various processing tasks upon receiving requests from users. This makes using FRED with the Watson cognitive service very exciting.

We have provided detailed sample code here. If you do not know how to import flows, you can refer to this article.

Create a FRED Account

If you don’t already have an account on FRED (Cloud Node-RED), set one up now. It’s free for a demo account, which can be used for these tutorials and any other experiments you may have with Node-RED. Register for a free account at http://fred.sensetecnic.com.

After registering, make sure to activate your account via your email. You will not be able to log in until you validate your account.

About IBM Watson

IBM Watson is a fast growing software platform that provides easy access to artificial intelligent (AI) services for commercial and development usages. Currently, the Watson services on Bluemix focus mainly in processing human language by interpreting the intent, content and generating action or analytic results according to the provided training data model. The IBM Watson platform is composed of different services specialized in specific tasks. You can find more information about the IBM Watson from here.


About Sense Tecnic: Sense Tecnic Systems Inc have been building IoT applications and services since 2010. We provide FRED, cloud hosted Node-RED as a service to the community. We also offer a commercial version to our customers, as well as professional services. Learn more.


Prerequisites

To try out the Watson service with FRED, you will need the following:

  1. If you don’t already have a free FRED account, set one up
  2. If you don’t already have an IBM Bluemix account, set one up
  3. You will need to install the node-red-node-watson module.
  4. You will need to set up the Watson services for each section that we will be talking about. We will cover on that.

Outline of the tutorial

  1. Install the node-red-node-watson module in FRED
  2. Setup Watson application and obtain the service credentials.
  3. A brief introduction on some of these Watson services.
  4. A detailed example of using the Watson Tone Analyzer service.

Install the Watson node modules

You will need to install the node-red-node-watson module in order to use the services on Watson with FRED. You can simply search “watson” in the “Add or Remove Nodes” list

Screen Shot 2017-04-12 at 2.59.29 PM

If your instance is already running when you install the node module, you will need to restart your instance in order to pick up the node module. After you finish the installation, you should see the Watson nodes in your node list:

Screen Shot 2017-04-12 at 2.59.42 PM

Note:  some nodes from the original node-red-node-watson module are currently not supported on FRED, and thus are not shown on the list here.

 

Setup Watson application and obtain the credentials.

Once you have setup your IBM Bluemix account, you can go to the Bluemix console, and check out the available Watson services in the catalog.

Each of the Watson nodes you installed on FRED will be matching to one of the Watson services.

Screen Shot 2017-04-12 at 3.10.22 PM

 

The general steps to set up a Watson service on IBM Bluemix are very straight forward. You will need to click on the service you want to use. If you agree with the plan standards/usage restriction, you can go ahead and click the “create” button. For example, if we want to activate a service of “Tone Analyzer”, we will click on this item in the catalog, and the browser will be redirected to the following page:

Screen Shot 2017-04-12 at 3.20.51 PM If you agree with the plan, just go ahead to click the “Create” button at the bottom right corner.

Then you will see a page like this with links to the documentation and demo of your newly-created service

Screen Shot 2017-04-12 at 3.23.14 PM

You will need to get the service credentials by clicking the “Service credentials” tab at the top left corner. Then you will see

Screen Shot 2017-04-12 at 3.23.23 PM

Mark down the service username and password as you will be using these on FRED later on.

 

A brief introduction on some of these Watson services

As you can see, there are quite some nodes in the Watson node modules, and they are all used for different purposes. Each of these nodes is corresponding to a specific Watson service on the IBM Bluemix platform. You are able to view the documentation of each service in the info tab of these nodes.

Conversation: This service was used in our other tutorial. It is a great tool to interpret human language and host a simple conversation. You will need to design the conversation with your training data.

Discovery: This service helps prepare your unstructured data, create a query that will pinpoint the information you need, and then integrate those insights into your new application or existing solution

Language identify: As the name suggested, this node will identify the language of the data sent to this node.

Natural Language Classifier: The Natural Language Classifier service uses machine learning algorithms to return the top matching predefined classes for short text inputs. These predefined classes are configurated in the training data.

Natural Language Understanding: The Natural Language Understanding will analyze a block of text, an article or HTML data to output a JSON object that matches the input data with various index.

Personality insights: The Personality Insights service uses linguistic analytics to infer personality characteristics from a text. The service can infer consumption preferences based on the results of its analysis.

Tone Analyzer: The Tone Analyzer service uses linguistic analysis to detect emotional tones, social propensities, and writing styles in written communication. We will be using this node in the following example.

Tradeoff analytics: The Tradeoff Analytics helps people make better choices when faced with multiple, often conflicting goals and alternatives. Mathematical filtering techniques to identify the top options based on multiple criteria, helps users explore the tradeoffs between options when making complex decisions.

 

A detailed example of using the Watson Tone Analyzer service

We will build an interesting simple application that uses the Watson Tone Analyzer service. Let’s say we want to know about people’s general opinion on a specific topic on Twitter. We can use a twitter in node to obtain tweet feeds, we will then send the content to Watson Tone Analyzer service. When we get the analytic result, we will keep track of the result over time and visualize the average score of each index.

The flow in FRED will look like this

Here we use #NationalPetDay as the Twitter topic that we want to monitor. We first drag the twitter in node onto the editor. We then configure the node to listen to all public tweets about #NationalPetDay.

 Screen Shot 2017-04-12 at 4.26.50 PM 

Since this topic is VERY popular by the time this blog post was written, we have to avoid the flood of the incoming tweets overwhelming the FRED instance. Let’s add a delay node after the twitter in node, and we would like the set the action to “Limit rate to” 1 message per 2 seconds. This should leave enough time for the Watson service to handle each request. However, you might have to adjust this rate depending on your Watson Tone Analyzer plan limit.

We then pull out the Watson Tone Analyzer node onto the FRED editor. You will need to fill in the service credentials that you obtain from the previous section of this blog post. In this example, we want to only look at the emotion tones, so we will select the Emotion option in the Tones field.

Screen Shot 2017-04-12 at 4.36.15 PM

Since the result from the Watson Tone Analyzer will be in msg.response, we can add a change node to move msg.response to msg.payload.

Screen Shot 2017-04-12 at 4.41.04 PM

Then, we have a function node named “Add analyze score to total scores”, we have the following code:

 

var defaultResult = {
  "emotion_tone":{
  "Anger":0,
  "Disgust":0,
  "Joy":0,
  "Sadness":0,
  "Fear":0
  },
  "count":0
}

if(msg.payload) {
 
  var result = context.get('twitterAnalyze')||defaultResult;
 
  msg.payload.document_tone.tone_categories.forEach(function(toneCategory){
    if(toneCategory.tones){
      toneCategory.tones.forEach(function(tone){
        result[toneCategory.category_id][tone.tone_name] += tone.score;
      })
    }
  })
  result.count += 1;

  context.set('twitterAnalyze', result);
 
  return {payload:result};
}

As you can see, we add the scores of each tone and then save into a context variable named “twitterAnalyze”. We also keep track of the count so we can use the number for the average in the next function node. Now we will send the result to the next node.

Now we have a function node named “Calculate the average” with this code

if(msg.payload.count){
 
  msg.type = "newMsg";

  for (var toneCatogory in msg.payload) {
    // skip loop if the property is from prototype
    if (!msg.payload.hasOwnProperty(toneCatogory)) continue;
 
    var obj = msg.payload[toneCatogory];
    for (var prop in obj) {
      // skip loop if the property is from prototype
      if(!obj.hasOwnProperty(prop)) continue;
 
      obj[prop] = obj[prop]/msg.payload.count;
    }
  } 
}

return msg;

As you can see in this function node, we average out each score with the total count we got from the previous function node.

In order to show the scores of each tone, we need to separate the scores into different topics before sending to the dashboard chart node. We use the change node to do this, here is an example of the Anger score

Screen Shot 2017-04-12 at 4.58.44 PM

You will need to do the same changes for the other tones. Once you have everything ready and you have data sending into the chart node, you will see a chart similar to this after a while:

Screen Shot 2017-04-12 at 4.45.54 PM

So from this diagram, we can see that most people feel joy about #NationalPetDay (this should be as expected). More surprising is that the second highest feeling of this topic is Disgust.

Ok, that’s it – Hope you’ve enjoyed exploring data analysis with the Watson services. What’s more, this basic example can now be modified to adapt to your specific needs – have fun!

 

About Sense Tecnic: Sense Tecnic Systems Inc have been building IoT applications and services since 2010. We provide FRED, cloud hosted Node-RED as a service to the community. We also offer a commercial version to our customers, as well as professional services. Learn more.