Understanding the BizTalk Deployment Framework – Introduction

This is an article in a series of 2 about the BizTalk Deployment Framework (BTDF). This open source framework is often used to create installation packages for BizTalk Applications. More information about this software can be found here CodePlex. With these series of articles, we aim to provide a useful set of hands-on documentation about the framework. The series consists of the following parts:

• Part 1 – Introduction and basic setup (this article)
  • Introduction
  • BizTalk out-of-the-box capabilities
  • Top reasons to use BTDF
  • Setting up a BTDF-project
  • Conclusion
• Part 2 – Advanced setup

Introduction

In this first part, we will shortly discuss the shortcomings of the deployment packages which can be created with the out-of-the-box capabilities of BizTalk Server and introduce you to the benefits of BTDF. Further this article will show you how you can create a BTDF-project for the deployment of basic BizTalk artifacts like Schema’s, Maps and Orchestrations and how you can deploy variable settings for other DTAP environments. We will end this article with showing you how you can generate distributable MSI packages.

BizTalk out-of-the-box capabilities

Microsoft BizTalk Server 2020 comes with capabilities to create deployment packages. However, the MSI packages you create with the out-of-the-box features of BizTalk Server have some shortcomings which lead to deployments that are time consuming and error-prone. The main reasons for this are:

• The packages can only contain URI’s of Endpoints (Receive Locations and Send Ports) of a single DTAP environment. So DTAP promotion from the same package is not possible.
• Only deployment of BizTalk artifacts is possible. So if you need, for example, to create and populate database tables, this cannot be automated.

What is the Deployment Framework?

The BizTalk Deployment Framework(BTDF) is a popular free, open-source framework that addresses the limitations that come with the out-of-the-box capabilities of the BizTalk Server.

Top reasons to use BTDF

As BTDF has eliminated the before mentioned shortcomings, the framework has been widely adopted. Below you’ll find a list which contains a number of reasons why you could use this framework:

• Deploy complex BizTalk solutions containing BizTalk artifacts but also for example SSO settings, ESB itineraries and SQL Server scripts, with no human intervention
• Consolidate all of your environment-specific configuration and runtime settings into one, easy-to-use Excel spreadsheet
• Maintain a single binding file that works for all deployment environments
• Make automated deployment a native part of the BizTalk development cycle, then use the same script to deploy to your servers

How do you deploy a BizTalk project?

After you have downloaded and installed BTDF on your Development box, you can start with setting up your BTDF-project. To get that working, you need to take the following steps:

• Add a BTDF project to your BizTalk solution
• Add BizTalk artifacts to the BTDF project
• Deploy from Visual Studio
• Generate a master PortBinding file
• Apply Environment Specific Settings
• Generate a MSI

Add a BTDF project to your BizTalk solution

Firstly, you need to open a BizTalk solution in Visual Studio and perform the following steps:

• Right-click the Solution Explorer and select Add/New Project…
• In the dialog that opens, in the left hand pane, navigate to BizTalk Projects
• In the right pane, select Deployment Framework for BizTalk Project, give the project the name Deployment and click the OK button to continue the creation of the project.

In the screen that follows you can configure which artifacts you want to deploy. Perform the following:

• Double-click on the text True after Deploy Settings into SSO. It will be set to False
• Uncheck the check mark ‘Write properties to the project file only when…’
• Click the Create Project button

After the project has been created, you’ll get a small dialog box, which states that you need to edit the file Deployment.btdfproj to configure your specific deployment requirements. And that’s exactly what we are going to do in a couple of minutes! For now, just click the OK button. Although the aforementioned project file is shown in the left part of the screen, the Solution Explorer does not yet show the newly created Deployment project. To have the project shown up there, you need to manually add the project file and some other files to the solution in a Solution folder. Perform the following actions to add the Deployment project to the solution:

• Right-click on the Solution Explorer, select Add/New Solution Folder, enter Deployment and hit the Enter key
• Right-click on the Deployment folder, select Add/Existing Item… and in the dialog box that appears double-click the folder Deployment
• Select all the files and click on the Add button

The Deployment project is now added to your BizTalk solution and that project in the Solution Explorer looks like below. Note: Take care that it is important to name that solution folder Deployment, otherwise Visual Studio won’t be able to find the Deployment.btdfproj file.

Add BizTalk artifacts to the BTDF project

Now we have the Deployment project in place, we are going to add some basic BizTalk artifacts to the Deployment project. To start we’ll add the following types of artifacts:

• Schemas
• Maps
• Orchestrations

To keep the Deployment project file nicely organized, we will create an ItemGroup for each type of artifact. As we start with 3 types of artifacts, we’ll end up having 3 ItemGroups. Probably it will be clear by now that ItemGroups can be used to organize the artifacts you want to deploy. An ItemGroup is an element from MSBuild which is used by BTDF. Besides ItemGroups, you will also find PropertyGroups within the BTDF project file. PropertyGroups are used to organize all kind of parameters which can be used during deployment. Think of this for example of the name of the project, but also which kind of artifacts need to be deployed. The project file already contains an ItemGroup for deploying schemas. We will use this as a template for our own purposes. Follow below instructions to add schemas, maps and orchestrations to the deployment project.

• In the Deployment project file, search for the ItemGroup that is used for deploying Schemas. It contains the text ‘<Schemas Include=”YourSchemas.dll”>’.
• Replace ‘YourSchemas.dll’ with the name of the DLL which contains your schemas
• The line with the ‘<LocationPath>’ tag, must contain the path where the DLL resides

The ItemGroup will now look like below:

  <ItemGroup>
    <Schemas Include="Kovai.BTDF.Schemas.dll">
      <LocationPath>..\$(ProjectName).Schemas\bin\$(Configuration)</LocationPath>
    </Schemas>
  </ItemGroup>

Next we’ll prepare deployment of the BizTalk Maps.

• Below the ItemGroup to deploy Schemas, create a new ItemGroup to deploy Maps. Therefore, you can copy/paste below XML:

  <ItemGroup>
    <Transforms Include="Kovai.BTDF.Maps.dll">
      <LocationPath>..\$(ProjectName).Maps\bin\$(Configuration)</LocationPath>
    </Transforms>
  </ItemGroup>

Now we are going to prepare deployment of the BizTalk Orchestrations.

• Below the ItemGroup to deploy Maps, create a new ItemGroup to deploy Orchestrations. Therefore, you can copy/paste this XML:

<ItemGroup>
    <Orchestrations Include=" Kovai.BTDF.Orchestrations.dll ">
      <LocationPath>..\$(ProjectName).Orchestrations\bin\$(Configuration)</LocationPath>
    </Orchestrations>
  </ItemGroup>

If you have multiple projects of for example orchestrations, you can simple copy/paste the <Orchestrations>…</Orchestrations> block and provide it with the correct Include-parameter and location path. Of course, this also works the same for other types of artifacts like schemas, maps, pipelines etc.

Deploy from Visual Studio

We now have deployment for some basic artifacts in place, so let’s deploy the BizTalk solution and see what happens:

• Right-click the solution (in the solution Explorer) and select Build Solution
• When the solution is built without errors, you can deploy the solution, therefore just click the little green arrow in the upper left are of Visual Studio:

If everything runs like expected the Kovai.BTDF application is created and the schemas, the map and the orchestration will be deployed properly, but you’ll end up with the following error: Could not enlist orchestration ‘Kovai.BTDF.Orchestrations.BizTalk_Orchestration1,Kovai.BTDF.Orchestrations, Version=1.0.0.0, Culture=neutral, PublicKeyToken=72ff5decf00d3ffb’. Could not enlist orchestration ‘Kovai.BTDF.Orchestrations.BizTalk_Orchestration1’. All orchestration ports must be bound and the host must be set. This is due to the fact that the orchestration Orchestration1 contains ports which will have to be configured and bound in the BizTalk Administration Console. Let’s just do that and afterwards export the Binding file into your Deployment project.

• Open the BizTalk Server Administration Console and navigate to the BizTalk application called BTDF
• Right-click the application and click Configure. The following screen appears:

• Select a Host in the designated dropdown box
• Create a Receive Port and a Receive Location (use the FILE adapter) and bind it to rcvPort
• Create a Send Port (use the FILE adapter) and bind it to sndPort
• Click the OK button
• Right-click the application again and click .. and in the dialog box that appears, click Start again. The application should now be started properly

Generate a master PortBinding file

We now have a running BizTalk application which includes bindings between the deployed orchestration and a couple of ports. In the next step we will make these bindings available to our deployment project. Therefore, we need to export the bindings. Again Right-click the application Kovai.BTDF and now select Export/Bindings…

• Click on the ellipsis button (…), navigate to your Deployment project, select xml, click the OK button and confirm that you want to overwrite the existing file
• Switch to Visual Studio and notice that you are prompted with the following message:

• Click Yes to reload the PortBindingsMaster.xml file
• Open PortBindingsMaster.xml and notice that it contains the bindings you created earlier

Apply Environment Specific Settings

For just local deployments you now have the basics in place. However, in most scenarios you will have to deploy your BizTalk solution(s) to multiple DTAP environments. In that case, the addresses of your Receive Locations and Send Ports will differ between environments. BTDF facilitates this by providing a XML file which can be edited in Microsoft Excel. Maintaining DTAP environments That XML file, named SettingsFileGenerator.xml, can contain columns for each DTAP environment to which you have to deploy your BizTalk solution. Below you find a screen shot of how such a file looks like by default. As you can see, there are columns for multiple environments; Local Development, Shared Development, Test and Production. You are free to rename the values in the existing columns, or add/remove columns. Maintaining environment specific settings In the spreadsheet there currently are 2 settings defined; SsoAppUserGroup and SsoAppAdminGroup. You can add more settings in the first column and give them values for the different environments in the following columns. If the value is the same for two or more environments, you could enter a value in the column Default Values, as this value will be used in case no value is entered for one or more environments. The screenshot below shows a Settings file that now contains values for the addresses of the Receive Location and the Send Port. As you can see, there are Default values entered and the columns for Local Development and Shared Development are left empty, so while deploying to those environments, the Default values will be used. To be able to use for example the SP_Address variable from the settings file in the PortBindingsMaster, you need to open the PortBindingsMasterFile.xml file, navigate to the Send Port which should use the addresses and replace the current address with ${SP_Address}. So before that part of the PortBindingsMaster.xml file looks like this:

    <SendPort Name="SendPort2" IsStatic="true" IsTwoWay="false" BindingOption="1">
      ...
      <PrimaryTransport>

<Address>C:\BizTalk\Kovai.BTDF\Out\%MessageID%.xml</Address>

      ...
    </SendPort>

After the change the XML will look like below:

<SendPort Name="SendPort2" IsStatic="true" IsTwoWay="false" BindingOption="1">
      ...
      <PrimaryTransport>

<Address>${SP_Address}\%MessageID%.xml</Address>

      ...
    </SendPort>

Generate a MSI

So far, we have only done deployments through Visual Studio. When we have to deploy our BizTalk solutions to non-development servers, we do that by installing MSI’s. BTDF is also able to generate MSI’s. Besides the BizTalk artifacts which need to be deployed, these MSI’s can also contain custom components, binding files for your entire DTAP environment and all kind of custom scripts, which can also be started during deployment. Generating a MSI is done from Visual Studio, just choose one of below steps to generate a MSI:

• Click on the below button in the toolbar

• In the Visual Studio menu, select Tools/Deployment Framework for BizTalk/Build Server Deploy MSI

Both options will provide you with a MSI which you can use to deploy the BizTalk application throughout your DTAP environment.

Conclusion

In this first article we have seen how the BizTalk Deployment Framework differs from the out-of-the-box capabilities of BizTalk Server and described the benefits of BTDF. We’ve also setup a simple deployment project in an existing BizTalk solution and shown how you can add some basic BizTalk artifacts to that project. Further we have seen how we can create different settings for, for example, Receive Location addresses and how they become deployed to multiple environments. Finally, we’ve seen deployment from Visual Studio and we’ve generated a MSI based on the BTDF project. Explore best practices for deploying BizTalk Server 2016 from Sandro Pereira’s Whitepaper. In the next article, we’ll get familiar with amongst others Custom Targets, Conditions and see how we can run third party executables to automatically deploy non-standard artifacts like SQL Server scripts or even automatically create BizTalk360 alerts from within your deployment project! Check out the 2nd article of the series here.