Monitor your Azure API Management Instance with PowerBI

Summary

This document outlines the steps involved to monitor your Azure API management instance with PowerBI.

Note: I will refer to Azure API management as APIM in this document.

Steps will include:

  • Add a logger, using the APIM REST API, to your APIM instance to send events to an event hub
  • Set up a Stream Analytics job – It consists of one or more input data sources, a query expressing the data transformation, and one or more output targets that results are written to. Together these enable the user to perform data analytics processing for streaming data scenarios
  • Build a PowerBI dashboard to see your APIM data in a format that suits your business requirements.

Adding a logger to APIM

First thing you need to do is add a logger to your APIM instance using the APIM REST API. I will use Postman to do this. (https://www.getpostman.com)

Firstly, in your APIM instance, enable the REST API. You will find this in the security section:

https://4.bp.blogspot.com/-n44nEUMqWtQ/WDusTJ77X-I/AAAAAAAAfXc/n6wmaCDYmoUnVZQXh5Kju8FbLErB0BBxACLcB/s640/1.jpg

Secondly, go to the bottom of the security page where you enabled the REST API and generate a shared access key. You can adjust the Expiry date to give you more time:

https://3.bp.blogspot.com/-tPz_q2hkt68/WDuoYpNX4JI/AAAAAAAAfW4/asgB6pZrOYwaij1luaMJaOJD-7_KiOw1gCLcB/s640/2.jpg

You will use this later in Postman after we first create an Azure Event hub.

Create an Event Hub

Go into your Azure portal and create an event hub.

https://4.bp.blogspot.com/-t1-HcxgiDzs/WDupJPyvVyI/AAAAAAAAfXA/7t9kfd8vDmoCkkJCL1w9CM0h-sy04m7VgCLcB/s640/3.jpg

Ensure you create to Event Hub shared access policies. 1 for sending to the event hub and 1 for receiving. This is to allow you more granular security control over your hub.

https://1.bp.blogspot.com/-mzcv6WnbbGk/WDuqS9CjKuI/AAAAAAAAfXM/xGmOC-FY0R4Ha9FscHGPc1cKH7-_iUUCwCLcB/s320/4.jpghttps://1.bp.blogspot.com/-NRcMg4HZJiI/WDuqqi6pPaI/AAAAAAAAfXQ/DkEIQ6Zf268OH5pfym0aBUDNrt3QXpYbwCLcB/s320/5.jpg

Creating the logger in Postman

Now that you have an Event Hub, the next step is to configure a Logger in your API Management service so that it can log events to the Event Hub.

API Management loggers are configured using the API Management REST API.

To create a logger, make an HTTP PUT request using the following URL template.

https://{your service}.management.azure-api.net/loggers/{new logger name}?api-version=2014-02-14-preview

Replace {your service} with the name of your API Management service instance.
Replace {new logger name} with the desired name for your new logger. You will reference this name when you configure the log-to-eventhub policy.

Add the following headers to the request.

1

Specify the request body using the following template.

{

“type” : “AzureEventHub”,

“description” : “Sample logger description”,

“credentials” : {

“name” : “Name of the Event Hub from the Azure Classic Portal”,

“connectionString” : “Endpoint=Event Hub Sender connection string”

}

}

 Here is mine:

You will see that it returned 201 Created which means we now have a logger.

Adding an APIM Policy

Now we go into APIM and add a policy on the built in echo API.

This will send event to our event hub when the …

Product: Unlimited => API: Echo API => Operation: Retrieve resource

… API is hit.

View the Event Hub Events

If you want to view the event hub events for your own sanity check, then download Service Bus Explorer and listen to your event hub. You can see the event highlighted below has 2 params that will be used later in PowerBI. (param1 and param2)

Create Stream Analytics Job to send data to PowerBI

Next, go to your Azure portal and create a new stream analytics job.

Once it is created then create an input and a PowerBI output: Note that you will be prompted to authorize your PowerBI account.

https://3.bp.blogspot.com/-Z_7vNz73w9U/WDuxsy_43hI/AAAAAAAAfYI/zb96RDIb1FEubQfVAGHtL-t9ndk9NMpqACLcB/s640/10.jpghttps://1.bp.blogspot.com/-vxUd5j2NkoE/WDuyRTAJjzI/AAAAAAAAfYM/qHujTW8IDk8hGzXmh0giXqVvXpVTSK98gCLcB/s640/11.jpg

Also create a query to transform the data. My query doesn’t do anything special:

https://2.bp.blogspot.com/-SkU8mBwq4tM/WDu0x_Rm2JI/AAAAAAAAfYg/LJOT6yVsszw6S4tf-uoPW8icOV-s91KpgCLcB/s640/13.jpg

Then start your stream analytics job.

https://2.bp.blogspot.com/-zJvZsLmlVBI/WDu17-zTFbI/AAAAAAAAfYs/kVtnMULCdHgu_XkhhR7M2lX1GXj-KkuRwCLcB/s320/14.jpg

Now go back to your APIM instance and hit the API end point a few times. Make sure it is the API operation with the policy on it.

Also, change param1 and param2 on the operation to a few different values so we get somewhat useful data:

Now look at the trace for that operation in APIM and you will see the log to event hub event has fired:



Now log into PowerBI on the web and you will see (hopefully) a new streaming dataset:

https://2.bp.blogspot.com/-IvD9RJv8y98/WDu28ZFB3xI/AAAAAAAAfY0/xG0oN7eHbusmEnhFep18ETYlidIpgFx0wCLcB/s640/15.jpg

Create a dashboard

We will create a PowerBI dashboard to visualise our APIM data using param1 and param2

I will drag a pie chart onto the workspace and set the following: All I did was add param2 as a count. As you can see we get a great visualisation of the number of times param2 was used on the APIM operation.



So you can see that I set a request with param2=5 a lot more times that I did for other calls.

Obviously you can use your imagination as to what you can use this for.

Here we see a Tree Map, Pie Chart and Funnel displaying data from my APIM. The funnel shows distinct calls from IP address.

Leave a Reply