Tutorial: Using the OSIsoft Pi Web API node with FRED (Cloud Node-RED)

with No Comments

OSIsoft Pi System is an industrial data management system that is widely used to capture, process, analyze, and store any form of real-time data. With the flexibility and powerful features of FRED, users can add more values to their current OSIsoft Pi System. The new node-red-contrib-osisoft-web-api node from Sense Tecnic makes it possible for you to create your own data-flow application with an existing Pi System in a few hours.

The sample flow for this demo is available on our Github repo. If you do not know how to import flows, you can refer to this article.

 

About OSIsoft Pi System

To stay competitive, companies must continually innovate, improve and streamline their operations. But accurately identifying how and where to improve operations can be extraordinarily challenging if the data that holds the answers is scattered among different incompatible systems, formats and processes.

Enter the PI System and its ability to collect, analyze, visualize and share large amounts of high-fidelity, time-series data from multiple sources to people and systems across all operations. By accessing key data and insights, the PI System has helped leading companies deliver greater operational improvements and breakthroughs that lead ultimately to comprehensive business transformation.

That’s the essence – and the power – of PI System operational intelligence.

-From OSIsoft Pi System product page.

About Sense Tecnic

Sense Tecnic Systems Inc have been building IoT applications and services since 2010. We provide FRED, cloud-hosted Node-RED as a service to the community. We also offer a commercial version to our customers, as well as professional services. Learn more.

 

Create a FRED Account

If you don’t already have an account on FRED (Cloud Node-RED), set one up now. It’s free for a demo account, which can be used for these tutorials and any other experiments you may have with Node-RED. Register for a free account at http://fred.sensetecnic.com.

After registering, make sure to activate your account via your email. You will not be able to log in until you validate your account.

 

Getting Started

So, here are the steps in the tutorial:

    1. Prerequisites
    2. Setting up your Pi Server
    3. Setting up your Pi security
    4. Setting up your AF servers on your Pi System
    5. Setting up on FRED
    6. Using FRED to write data to Pi
    7. Using FRED to query data from Pi
    8. Conclusion
    9. Troubleshooting

Prerequisites

To use the node-red-contrib-osisoft-web-api node with OSIsofi Pi system, you’ll need:

  • A running Pi Server (more notes below)
  • An OSIsoft Web API server that is connected to your Pi Server, and can be accessed on the Internet
  • A Pi user account that has the proper privileges to read/write data on your Pi system
  • Pi System Explorer installed
  • Pi System Management Tool is installed

For using the node-red-contrib-osisoft-web-api node on FRED, our cloud-based platform, you will also need to add the node module into your bundle list. You can do that by searching ‘osisoft’ in ‘Add or remove nodes’ from the sidebar.

Screen Shot 2017-10-25 at 1.49.27 PM

Setting up your Pi Server

Setting up a Pi server involves a learning curve and requires a number of steps. The Pi System has multiple components integrated to deliver its functionality. You might find this Pi System installation order for Pi Server 2016 R2 useful in figuring out what you need to install, as well as this video series.

For setting up the demo in this tutorial, you will need to confirm that you have the Pi System Explorer and Pi System Management Tools installed on your Pi System.

You will also need to install the Pi Web API server on your Pi server. This is an application that simplifies your setup procedures to allow REST requests to access the Pi servers and databases. You can find more info in the Pi Web API installation documentation, and you should also find this video helpful in configuring the Web API server.

If you are hosting your Pi Server on a local server, you will want to generate a self-signed certificate for the Web API server.  You will also need to configure your firewall settings to allow access to the server from the FRED cloud service.

If you are hosting the Pi Server in a public cloud service such as AWS you can use this trick to avoid re-configuring the Web API server every time you get a new IP address:

  1. Add your hostname in your hosts file. On Windows Server 2016, it is located at C:\Windows\System32\drivers\etc\hosts. Just simply add the hostname near the end:
    1
    2
    # 127.0.0.1 localhost
    EC2AMAZ-xxxxxxx localhost        #EC2AMAZ-xxxxxxx is your hostname
  2. Then, when you configure your Web API server, set the server URL to
    1
    https://<your-host-name>/piwebapi

    for example,

    1
    https://EC2AMAZ-xxxxxxx/piwebapi
  3. Once you are set, you can just access the Web API server by accessing ‘https://<your-instance-ip>/piwebapi’

 

Setting up Pi system security

In order to access the Pi System successfully from your FRED instance, you need to also configure a Pi user account with the proper READ/WRITE privileges on your Pi System. Keep in mind that, the security configuration on your Pi system is configured at different levels. They are:

  • Server level,
  • Databases level,
  • Web API server level,
  • Pi Points level

Currently, the node-red-contrib-osisoft-web-api node module only works with ‘Basic‘(READ/WRITE) or ‘Anonymous‘(READ ONLY) authentication on your Pi server. If you are not familiar with the Pi System identities, mapping, and trusts(on some older version Pi System), you can take a look at this video series.

For this tutorial we created a Windows user account called “piWebApiUser” in the domain ‘EC2AMAZ-xxxxxxx’. Then, on your Pi Server, you need to create a Pi Identities or a Pi user and map it to your new Windows user account by using the Pi System Management Tools. You can find more information on how to map Pi identities from the video series.

Screen Shot 2017-10-25 at 11.40.51 AM

You will also need to update the Database security for this new Pi user account. You can do that by adding your new created Pi user to the appropriate Pi group or manually add the individual rules in the Database Security section.

Screen Shot 2017-10-25 at 11.48.31 AM

Note: You don’t have to follow all the security rules we added in this table. You can modify only the databases that need to be updated, for example, PIPOINT. 

Setting up AF servers on your Pi System

For this tutorial, lets use FRED to gather weather data and upload the data to the Pi System. FRED, although often used to gather and manipulate IIoT data, actually excels at integrating data from a myriad set of sources ranging from physical devices, through third party data sources and websites to social media sources.  We will be using darksky as our weather data source. A potential use case would be ‘mashing up’ this weather data building or factory HVAC data to understand energy costs and their correlation to weather.

First, let’s use the point builder tool in the Pi System Management Tools to create some Pi Points. We first create a “summary” string data point.

Screen Shot 2017-10-25 at 2.35.55 PM

Notice how we put down String as the point type.

Screen Shot 2017-10-25 at 2.36.05 PM

For demonstration purposes, we will also turn off the compression feature.

Screen Shot 2017-10-25 at 2.36.17 PM

Also, make sure that our ‘piWebApiUser’ user account has both READ/WRITE access as it is needed for this demo.

Screen Shot 2017-10-25 at 2.36.49 PM

Once you save this point, you should see the status indicating that this Pi Point is created.

Screen Shot 2017-10-25 at 2.41.55 PM

We also want to create these other Pi Points. Notice how the other point types are Float32 type.

Now, let’s go to the Pi System explorer to create a new Asset Database named ‘weather’.

Screen Shot 2017-10-25 at 3.45.38 PM

We will then head to the Library section in the left sidebar to create a new element template named “weather_template”.

Screen Shot 2017-10-25 at 2.42.41 PM

Let’s go ahead to the “Attribute Templates” tab to add the attributes that are related to the Pi Points that we created earlier.

Screen Shot 2017-10-25 at 2.44.16 PM

We then create a new attribute by clicking the “new Attribute template”, we put down “summary” for the Name field, then String for the value Type. It is also important to select ‘PI Point’ for Data Reference field. We also need to set the correct PI Point reference values by clicking the “Settings” button.

As you can see, We find the correct values to the ‘Data Server’ and the ‘Tag name’ from the drop-down menu of this pop-up PI Point Data Reference window. In our case, since we are also writing to the Pi server, we will need to uncheck the “Read only” flag near the bottom.

We also need to add new attributes for the other Pi Points that we just created. Beware that we select “Double” as the value type for the other Pi Points.

Screen Shot

Don’t forget to check in your changes after you finish adding the Element Template.

Screen Shot 2017-10-25 at 2.47.52 PM

Then, we can create an element to store incoming data. We go to the element tab, right click on the element icon to create a new element from a template.

Screen Shot 2017-10-25 at 2.48.17 PM

Then, we name our new element ‘weather_from_fred’

Screen Shot 2017-10-25 at 4.06.29 PM

And you should see the attributes in the Attributes tab:

Screen Shot 2017-10-25 at 2.48.30 PM

 

Now we are ready to move onto FRED!

 

Setting up on FRED

 

On FRED, we will simply get the weather data using a Darksky node

Screen Shot 2017-10-25 at 4.26.07 PM

And the configuration of the darksky node would be:

Screen Shot 2017-10-25 at 4.19.39 PM

As you can see, we set the darksky node to receive a timestamp and check the weather data with the geo-coordinator of Vancouver, Canada. You will also need to enter your darksky developer API token in the credentials section.

Now if you deploy the flow and check the output of the darsky node, you should be able to see the weather data in the output!

 

Write data to Pi

Now we can start writing data to the Pi server. We start with dragging a web api write node onto the editor. We will also need a change node to:

  • delete the msg.payload object to clear out unnecessary data
  • move the current data from msg.data to msg.payload.Value

Screen Shot 2017-10-25 at 4.37.32 PM

Inside the change node:

Screen Shot 2017-10-25 at 4.37.44 PM

Since we had created the data point for ‘summary’ in the Pi server, we can just post the value by path. This is how we set the web-api-write node:

 

Screen Shot 2017-10-25 at 4.39.44 PM

Inside the Web API server configuration, we need to add the URL and the credentials:

Screen Shot 2017-10-25 at 3.05.49 PM

Notice that we check ‘Enable secure (SSL/TLS) connection‘, but we unchecked the ‘verify certification‘ field inside the TLS configuration. This is because the OSIsoft Web API uses SSL connection by default.

When you deploy, you will be able to inject values into your Pi Server. Normally if the node does not return error messages, then the data should be posted. We will talk about how you can query the values to confirm whether it’s updated or not in the next section, but let’s finish adding the web-api-write nodes for the rest of the Pi Points.

Screen Shot 2017-10-25 at 4.45.00 PM

Query data from Pi

There are two ways to check the data on your Pi System:

  1. Access the Web API server via a browser such as Chrome
  2. Query Web API server using the web-api-query node in FRED.

For the first method, it is pretty straightforward as you can find tutorials from the OSIsoft learning video series. After you log in to your Web API server, you should see the top level of your Web API server.

Screen Shot 2017-10-25 at 4.52.58 PM

Then, you should see the Pi Points you created in “DataServers -> Points”, in which the Web API server will return a list of the Pi Points.

 

Screen Shot 2017-10-25 at 4.54.42 PM

If you click the “Value” links of each Pi Point item, you will be able to see the current values of these points.

 

To use the web-api-query node to query the values from Pi Server is also very straightforward. Simply drag a web-api-query node onto the editor.Screen Shot 2017-10-25 at 5.00.34 PM

And set the configuration:

Screen Shot 2017-10-25 at 5.00.21 PM

Again, at this point, we are querying a specific Pi Point in the Data Servers. After we trigger the query node, we should see something like this in the output.

Screen Shot 2017-10-25 at 5.00.41 PM

In the web-api-query node, we can also choose to query for different types of information we want. For example, we can select “RecordedData” to query for all recorded data for this Pi Point in the past.

Screen Shot 2017-10-25 at 5.07.16 PM

As you can see, the “query by point paths in Data servers” and “query by Web ID for points in Data servers” features only work for Pi Points in the Data servers. If you want to query items in the AF servers, for example, to list out all the Attributes we created in that ‘weather_from_fred’ element (located on the Asset Server), we can first find the element via browser:

Screen Shot 2017-10-25 at 5.13.04 PM

and you can see the link for the “Value” item. We just need to copy down the relative link after

1
https://34.214.255.133/piwebapi

,

and copy it into the web-api-query node in the ‘query by custom URL’ mode:

Screen Shot 2017-10-25 at 5.15.54 PM

Screen Shot 2017-10-25 at 3.11.11 PM

and you should see the output message contains all the attribute values as you can find in the browser:

Screen Shot 2017-10-25 at 5.17.56 PM

The “query by custom URL” mode allows you to query almost all endpoints on your Pi Server. So in theory, you don’t even need to use a browser to find relative links and WebIDs!

You can also start your workflow with the query node in the ‘List‘ mode to obtain web-links, webIDs, databases or tags from the top level of your Pi server to start your task. By doing this, You will not need to use a browser to manually copy down values, and you can design your workflow programmatically. The currently available ‘List’ features include:

  • List all Assert servers: List all the assert servers from your Pi server
  • List all DBs in Assert Servers: List all the databases from all assert servers
  • List all Data Servers: List all the data servers from the Pi Server
  • List all points in Data Servers: List all points from all data servers.

 

Don’t forget that we can use the Node-RED dashboard nodes to generate graphs for our data in no time! You can import the sample flow from our sample code to see how we did that! From the graph below, we can tell that it is a beautiful day in Vancouver!

Screen Shot 2017-10-26 at 11.19.23 AM

 

Conclusion

In this demo, we learned how to rapidly set up a Pi System, and use the node-red-contrib-osisoft-web-api node to query and write data into the Pi server via the OSIsoft Web API server. We also see the different modes in querying and writing to the Pi Server where some common requests are built into the nodes for users already. We also see how we can use the query node and node-red-dashboard node to visualize the current values and historical data. If you have encountered any errors, you can refer to the following troubleshooting section for some common issues.

 

Troubleshooting

  • Unauthorized access denied by OSIsoft Web API:

    There are multiple possible causes for this error. You can check the following settings:

    • Did you set your OSIsoft Web API authentication method to “Basic” or “Anonymous”?
    • Did you enter the correct credentials?
    • Does your Pi User have the proper READ/WRITE privileges?
    • Did you set up the correct mapping plans in Pi System Management tools?
    • Did you set up the correct READ/WRITE privileges at Pi Point level? and other security levels?
    • If you are writing, did you enable writing access for the Pi Point?
    • If you are writing, did you set ‘disableWrite’ to false in Pi Web API configuration in Pi System Explorer?
  • Cannot find resource/path/element/point

    There are multiple possible reasons why you see this error, this means that the path, Web ID or the custom URL you enter is not correct. You can check:

    • Is your Web ID valid?
    • If you are using path method, is the Data server name and the Pi tag name correct? Do they exist in your Pi server? Do you have the right access to this Pi Point?
    • If you are using custom URL, is the URL correct? Beware that the custom URL should be relative to the base URL, which is “https://<your-pi-server>/piwebapi“, so your custom URL should look like “/points/<some-web-ID>/value” with the “/” at the beginning.
    • If you are using path method, beware that the “path” feature only supports Pi Points in the data servers. If you are trying to access an element on the AF server, then you should use the custom URL feature as the AF structure could be more complicated.
    • Did you connect the query node to a debug node?
  • You post, but do not see the updated value

    You can double check on the following fields:

    • Is your Web API server URL correct?
    • Do you have WRITE access to that specific Pi Point and Database? Remember that you need to set your WRITE permission in:
      • Web API server level
      • Database level
      • Pi Point level
    • Are you posting to the correct Web ID/path/custom URL? Are you checking the correct Web ID/path/custom URL?
    • You might want to double check on your Pi Server system configuration, such as your AF server, MySQL database, Web API server etc.
    • You can check the logs in your Windows Event Log Viewer, where you will find the WebAPI logs under Applications
  • Any other questions:

    You can start a technical support issue with OSIsoft, or send an email to us at info@sensetecnic.com.