Sending BizTalk Tracking Data to Azure Application Insights

Microsoft BizTalk team has released its first feature pack for BizTalk 2016 with interesting new features. You can find more details on BizTalk Feature Pack 1  in Saravana Kumar’s blog article. As far as this blog article is concerned, I will be covering an exciting new feature on sending BizTalk tracking data to Azure application insights.

Why to send tracking data to Azure Application Insights?

Since BizTalk already stores the tracking data in DTADB,  question which comes to our mind is why we need to send the data to Azure Application Insights. Here are few points which I can think of.

• Popular APM (Application Performance Management) Service: Azure Applications Insights is a very popular Application performance management service especially for the web developers. It helps the developers to find performance issues with their applications and diagnose them. Since organizations are investing in streamlining application performance management, it makes sense for BizTalk applications to align with this.
• A unified approach to hybrid integration: Microsoft integration story has stretched its arms to the cloud in the form of Logic Apps, Azure functions, API management etc. It is very important that BizTalk aligns with the data tracking options in all these cloud features. At this point, I must say integrating BizTalk tracking data with Azure Applications Insights is the first step in achieving this.
• Standard Data Visualization tools: Azure has solid data visualization offerings such as Power BI and OMS portal. Already OMS portal integrated with Logic apps diagnostic logs. It makes sense to start visualizing data in such data visualization tools. At this point, BizTalk tracking data is not available in OMS, but I can clearly see it coming.

In following sections, I will explain the steps involved in sending BizTalk tracking data to Azure Application Insights.

Creating instance of Application Insights

To begin with, we need to create an instance of Application Insights in Azure portal. Follow the steps below to achieve this —

• Log on to Azure portal and add a new application insights instance

• Provide an appropriate name for the instance. I am using the name “DemoBizTalkFeaturePack”.

• For application type select General. Since there is no specific category for BizTalk we will select General. (It would have been good if we had a category for BizTalk as well)

• Select appropriate resource group and location

• Note down the instrumentation key which is needed in following steps.

Install the Feature Pack 1 on BizTalk Server

Download the BizTalk Server 2016 Feature Pack 1 from this location. Below are the steps in installing the Feature pack.

• Run the .exe and Click Next on the wizard

• When it displays a warning about all the services that will be restarted, click next

• Wait until the wizard completes installation

• You will have to restart the server after installation.

Enable Analytics in BizTalk Group

Once the feature pack is installed, we will have a new section called “Analytics” in BizTalk Group hub settings page.

• Enable the group level analytics.
• In the connection type select “Application Insights”. Currently, there is only one target type.
• In connection, parameters enter the instrumentation key of the application insight’s instance

Enable Analytics on Artifacts

Once the analytics is configured at the group level, we can go enabling tracking at individual artifacts level. In the example below, I have an orchestration for which the Analytics is enabled. This ensures that all the data tracked for this orchestration will be pushed to Application insights.

What if the internet connectivity breaks?

Sure enough, we need internet connectivity for pushing the tracking data to application insights. However, if the internet connectivity fails due to some reasons, the data will be stored temporarily in tracking tables and they will be pushed to application insights when the connectivity is back.

Run queries in Application insights

Once the data is pumped to Application Insights, different metrics will be created based on the different level of tracking enabled in BizTalk artifacts. You can explore these metrics and create graphs for data visualization. Note: Generally, Application Insights will have a lag of 20 to 30 minutes from the time they are pushed and they start appearing on the metrics explorer. You also will have the option to search the data based on various parameters such as servicename, hostname, Error Code etc.

Performance implications

As we know if we enabled the tracking on the artifacts mindfully, it will give us very helpful information to diagnose the issues without affecting performance. Same applies to enabling analytics as well. It is very important that you plan as per your performance needs.

Summary

It is very clear that the Microsoft Pro integration team is inventing a unified approach for tracking data across their offerings (BizTalk, Logic Apps, Azure Functions and API Management). BizTalk tracking data into Application insights looks to be only a beginning. They in future might integrate tracking data with different targets such as OMS (Operational Management Services), Splunk, ELK Stack etc.

DYK: You can get to see these in action?

Tord Glad Nordahl from Microsoft Product Group will be showcasing all these features in the upcoming Integration Monday session scheduled for May 8th, 2017.