Why did we expose all of our BizTalk Operations and Management REST API’s?

This blog is a part of the series of blog articles we are publishing on the topic “Why we built XYZ feature in BizTalk360”. Read the main article here.

Why do we need this feature?

On a daily base, our customers use BizTalk360 for a lot of their activities related to BizTalk operations and monitoring. However, in a number of scenarios, BizTalk administrators are faced with activities which need to be automated, for example: during the deployment of BizTalk Applications. It is easy to forget some tasks and doing them manually can be error-prone. So, it may be better to address such challenges in an automated way. Here are some example automation scenarios in that can happen in your BizTalk Server environment BizTalk Server Operations

• You might want to automatically stop certain Ports over the weekend, because of regular or temporary maintenance of internal or external systems
• External systems, like portals, might need BizTalk data/metrics on a regular basis, like information around EDI transactions, BizTalk usage or BizTalk performance metrics
• You want particular users to have easy access to download links for the messages of suspended instances, without having to give them access to BizTalk Server and/or BizTalk360
• You may want to Deploy/Publish a Business Rule Policy at a particular moment (additional discounts for the holiday season or laws to which should be complied as of a particular moment)

BizTalk Applications Maintenance and Deployment

• Before or after deployment of BizTalk Applications, you may want to automatically stop and start Host Instances
• After a deployment, you may want to set certain ports/orchestrations to a particular state
• You want to bring BizTalk down at a given date and time, due to system maintenance

Identity and Access Management

• Your organisation uses tooling for Identity and Access Management and you want to automatically create User Access Policies for accessing BizTalk360 and BizTalk
• For auditing purposes, you need to provide user lists and profiles and audit information on a regular bases

Because of the nature of this kind of scenarios, the administrators would need programmatically access to BizTalk server and rely on command files or PowerShell scripts to be able to fulfil the requirements.

What are the current options and challenges?

BizTalk Server comes with different tooling for programmatic access to BizTalk Server. This tooling is described below.

WMI and ExplorerOM

Let’s briefly discuss the purpose of both WMI and ExplorerOM:

• WMI (Windows Management Instrumentation) – an infrastructure, which comes with Windows, and which allows you to automate administrative tasks on (remote) computers. BizTalk Server comes with its own WMI management pack to do preliminary management activities like starting/stopping host instances, restarting ports etc.
• ExplorerOM – a DLL used to perform all kind of BizTalk Application related tasks like exploring the schemas, maps, orchestrations, operating on them like starting/stopping etc.

BizTalk PowerShell Provider

There is also a community initiative to provide a BizTalk PowerShell provider so you can execute some of the management tasks using PowerShell (since PS has become the core of any automated administrative tasks). However, the project is not actively maintained anymore and it’s not advisable to rely on this for any critical usage.

BizTalk Server 2016 Feature Packs

Starting with BizTalk Server 2016, Microsoft started to release Feature Packs. With the first Feature Pack (BizTalk Server 2016 Feature Pack 1). a unified API became available for BizTalk Server 2016 (not for earlier versions). This Management API enables you to perform some of the scenarios which we mentioned in the first paragraph but still lacks API’s to do extensive automation. For example, stop and start host instances. So, although BizTalk Server comes with multiple capabilities for programmatic access to BizTalk Server, writing and testing all kind of scripts is time-consuming and not all scenarios can be covered. Even when these custom scripts seem to work nicely, non-functional requirements, like security, auditing and logging, are not taken into account in these kinds of scripts. This would leave you in the dark when something unexpected happens. Definitely, not a scenario where you want to be.

How BizTalk360 solves this problem?

BizTalk360 consists of multiple components, a responsive user interface, few backend services and a rich API service layer. The API service layer consists of over 400 REST API operations. The user interface of BizTalk360 uses these extremely rich API’s to perform all the tasks to manage the BizTalk environment. A couple of years ago, we thought, exposing our API’s will hugely help BizTalk Server Administrators to automate most of their routine activities we mentioned in the beginning.  Hence we documented all the BizTalk360 API’s (which are of course the management and monitoring API’s for your BizTalk Environment) using swagger and live test within the BizTalk360 application. The API currently contains 14 services with more than 400 API operations, covering a breath of management activities you can do on your BizTalk Environment. To give you a better understanding of the capabilities, these services are:

• ActivityMonitoringService – BAM related services
• BizTalkQueryService – Execute all kind of MessageBox and Tracking oriented queries
• AlertService – All kind of Monitoring related API’s
• BizTalkGroupService – BizTalk Platform oriented API’s
• SchedulerService – Data Monitoring Scheduler related API’s
• ESBManagementService – Exception Management related API’s
• AdvancedEventViewerService  – API’s to query the Eventlog entries which are collected by the Advanced Event Viewer feature
• BizTalkApplicationService – Search for and action on BizTalk Application artifacts
• AdminService – Get and maintain all kind of settings within BizTalk360
• EDIManagementService – Get/Set EDI configuration and get EDI statistics
• EnvironmentMgmtService – API’s mainly for Knowledge Base and Dashboard management
• RulesEngineService – API’s for maintenance of Business Rules
• AnalyticsDataService – Analytics, Message Patterns and Throttling related API’s
• AzureService – Get/Operate on Logic Apps and get Integration Account artifacts

With the API you cannot just retrieve information from BizTalk via GET operations, but you can also action on BizTalk Server, BizTalk360 and even certain Azure services via POST operations. Each API can be tried from the API documentation, which comes with a Swagger definition file, by using the Try it out! feature. This helps you testing with the required parameters. All you need to do is to choose the needed API, provide the needed parameters for the API, click the Try it out! button and view the results. Let’s have a look at few examples of how you could make use of the API.

• Stop and start Host instances before and after deployment of BizTalk applications
• Disable/Enable Receive Locations and/or Stop/Start Orchestrations/Send Ports
• Get a list of currently deployed BizTalk Applications for Support purposes
• Get Event Log data of all the BizTalk servers in one go
• Get Throttling data from the BizTalk Hosts
• Get all kind of Performance metrics of your BizTalk system
• Test, Deploy, Undeploy and Publish Business Rules
• Disable/Enable BizTalk360 alarms during deployments
• Show data in an Operations Dashboard widget, for example, the state of your SQL jobs
• Automatically create BizTalk360 alarms during deployment of BizTalk applications

Conclusion

By using the BizTalk360 API and its documentation, the complexity from WMI and ExplorerOM is shielded from the user. Besides being able to act on BizTalk, the API also allows you to action on BizTalk360. You can even show the output of API calls in the Operations Dashboard! Few examples of how the BizTalk360 API`s can be used in real-world scenarios: Automate Monitoring Alarm creation in BizTalk360 If you are using BTDF (BizTalk Deployment Framework) we have a wrapper console application called BT360Deploy that can be used to automate certain tasks like creating monitoring alerts during deployment. The project can be accessed here.

Get started with a Free Trial today

Why not give BizTalk360 a try. It takes about 10 minutes to install on your BizTalk environments and you can witness the benefits of auto-healing on your own BizTalk Environments. Get started with the free 30 days trial.