BizTalk Server Best practices, Tips, and Tricks: #3 Creating Custom Functoids

Welcome once again to another BizTalk Server Best practices, Tips, and Tricks blog post! In my previous blog post, I talked about some essential tips and tricks for BizTalk Server administrators:

• #1 What do RosettaNet, ESB, and UDDI have in common?
• #2 BizTalk MarkLog tables

Today, as I promised, I’m going to speak about another critical Best practice, Tips and Tricks for BizTalk Server developers: Creating Custom Functoids. 

#3 Creating Custom Functoids

Custom Functoids Sandro? really? That is quite basic and easy stuff!

Yep, you are somehow right, but do you know that there is more than one type of Custom Functoids? And do you know that they have different requirements in terms of deployment? Do you know that some of them you don’t need to deploy in other environments than Development?

As you can see, there is more than you probably think about custom functoids

What are a Functoid and a Custom Functoid?

We can consider Functoids as pre-defined functions that we can use to perform complex data manipulations and transformations. Typically on a map, the data is copied from source to destination by dragging links between elements of the two schemes. Functoids stay in the middle of these operations and apply an operation on the incoming data to transform them to the requirements of the destination. The BizTalk Mapper Designer represents a functoid as a box in the middle of the link (or links) between the processing elements. 

As a point of interest, functoid is a term coined by Microsoft and is commonly described as “functions on steroids.”

The BizTalk Mapper provides an extensive set of Functoids that can be used in maps, to perform a variety of operations on data that is being mapped from a source instance message to a destination instance message.

By default, the Functoids are organized into nine categories based on their intended purpose:

• Advanced Functoids: used to create various types of data manipulation, such as implementing custom script (C#, Visual Basic .NET, XSLT), value mapping, and managing and extracting data from looping records.
• Conversion Functoids: used to convert data, such as: convert ASCII to characters or to convert numbers from one base to another (hex, decimal).
• Cumulative Functoids: used to perform various types of accumulation operations for values that occur multiple times within an instance of a message.
• Database Functoids: used to look up data from a database and to perform simple cross-referencing operations (sometimes called ID mapping).
• Date and Time Functoids: this is a set of operations applicable on dates like add date, time, date and time, or add days to a specified date in output data.
• Logical Functoids: used to control the behavior of other Functoids conditionally and to determine whether particular output data is created.
• Mathematical Functoids: used to perform specific numeric calculations such as addition, multiplication, and division.
• Scientific Functoids: used to perform specific scientific calculations such as logarithmic, exponential, and trigonometric functions.
• String Functoids: used to manipulate data strings (text alphanumeric) by using well-known string functions such as concatenation, length, find, and trim.

If you want to know more about Functoids, please check the following resources:

• Functoid Reference
• Functoids in Maps

I also recommend you to check the great Pluralsight course created by Dan Toomey: Using Functoids in the BizTalk 2013 Mapper (http://pluralsight.com/training/Courses/TableOfContents/using-Functoids-biztalk-2013-mapper). This course presents a deep-dive exploration of the BizTalk Mapper toolbox, especially the powerful built-in Functoids that enable complex message transformations via drag ‘n’ drop into the design grid.

Custom Functoids

BizTalk Server allows you to extend the range of operations available within the BizTalk Server mapping environment (aka existing out-of-the-box Functoids). To do that, we call them Custom Functoids.

Although BizTalk Server provides many Functoids to support a range of diverse operations, when you start to create many maps, you will start to realize that some of the custom and small operations or actions, many times using Scripting Functoids, are common in several of your maps. As a consequence, you will find yourself spending your time maintaining several equal code snippets that you normally copy and paste into your maps in several locations inside a map or several maps. It is a good approach to consider transforming code snippets into a custom function. As we mentioned before, Custom Functoids allow you to extend the range of operations available within the BizTalk Server mapping environment. You develop them once, and you can reuse them in several maps! Each custom Functoid is deployed as a .NET assembly using classes derived from Microsoft.BizTalk.BaseFunctoids. And one assembly can contain more than one custom Functoid.

Rather than writing the same inline code (C#, XSLT, and so on) and pasting it in multiple shapes or using a combination of multiple existing Functoids, you can combine them into a Custom Functoid. For example, in the following scenarios:

• Your functoid depends on .NET namespaces that are not available to maps.
• You need to encrypt or decrypt, encode or decode fields.
• Get data from a proprietary legacy API, for example, return a resolution internal GUID value of the lookup field in CRM.
• Basic function to generate dynamic data like generate a new GUID.
• Functoids that make date time conversions like converting a traditional date (Human Readable Date) into a UNIX date (Epoch Date) or converting the date time format.
• You want to reduce the complexity of a map by combining several related steps, using a different approach, or by using new class libraries.
• Sometimes, you do not want your business logic code exposed in clear text for all to read and possibly modify.
• Or, as I described before if more than one map uses the same script code in a script Functoid.

The advantages of using custom Functoids (or reasons to develop custom Functoids):

• Level of effort and Reusability: They are reusable. Because we are extending the BizTalk Mapper toolset of the range with more Functoids that will have the same behaviour as built-in Functoids, you can use them several times in a map (or several maps) such as built-in Functoids, and therefore you are considerably simplifying the development task at hand. But you can also reuse them on other projects, not just your current project. The idea of developing a custom Functoid instead of a Scripting Functoid is to avoid code repetition and to be able to modify it in every map that uses it simultaneously.
• Overview: Similarity to built-in Functoids, they are easier to read (visually on the map grid) than Scripting Functoids.
• Deploy and development: They don’t need to be referenced in BizTalk projects; you only need to install them on the server.

Disadvantages or precautions to take when using Custom Functoids:

• Compiler limitations: All Functoids must have a unique ID for the runtime to distinguish them.
  • Certain Functoid IDs are reserved by standard/inbuilt Mapper Functoids. Usually, the standard Mapper Functoids use IDs from 1 to 10000. While creating custom Functoids, do not use the Functoid IDs less than 10000.
• Development: it requires experience with .NET Programming.
• Maintenance: This is not about the maintenance of the map but the maintenance of the code behind the Functoid.
  • They are more difficult to debug for errors if we compare them with Scripting Functoid.
  • And more difficult to have version control of these resources.
• Testability: Like any other BizTalk artifact, testing this component is not a straightforward task. It can be accomplished but will require additional steps or approaches:
  • The first and common way is to use the custom Functoid inside a map and then validate and test the map from within Visual Studio, checking that no errors occur and manually inspecting the output to make sure it provides the expected result;
  • The second way is to use the unit testing capabilities of BizTalk maps to test and debug our custom Functoid;
  • And the last option is to debug in runtime. When debugging the functoid in runtime, make sure the latest compiled version is deployed and make sure your host instance has been restarted, so it hasn’t cached a previous version. Then inside Visual Studio, set breakpoints in your code wherever you want the code to break and when this is done, choose the BTSNTSvc.exe process in the Visual Studio main menu Debug on the Attach to Process option.

Don’t write a custom Functoid only to solve a very particular transformation problem. Instead, you should consider using and creating a custom Functoid only to solve a repeat transformation rule that you can reuse in several maps.

If you want to see some examples of custom Functoids, I developed a BizTalk Mapper Extensions UtilityPack, a set of libraries with several useful functoids to include and use in our maps. You can find more details about it here:

• BizTalk Mapper Extensions UtilityPack

Different types of Custom Functoids

Many of you don’t know, but in fact, there are two types of custom Functoids.

SetExternalFunctionName type

Normally people use the SetExternalFunctionName expression to define the action of their custom functoid, something like this:

A good example of this is the BizTalk.Configuration.Functoids available on the BizTalk Mapper Extensions UtilityPack.

In this case, we are basically saying that we will use an external assembly to perform this action, and in this case, this assembly needs to be installed in the GAC in all the environments that you will use this functoid. To add it to the toolbox inside Visual Studio, we also need to add the DLL to the Developer Tools\Mapper Extensions folder under the BizTalk installation path. 

This is the best approach when you need to use .NET namespaces that are unavailable to maps or using external DLLs.

SetScriptBuffer type

But have you already tried to find where out-of-the-box functoids are installed? And probably you didn’t find it. But if you generate the XSLT of your maps and inspect them, you will find a lot of embedded C# code.

Yes, they are SetScriptBuffer functoids. This type of functoid basically tells the BizTalk Mapper to embed the function code inside the XSLT code. They look something like this:

In terms of development effort, it is almost the same. The only differences are that:

• We need to use a different method, SetScriptBuffer, instead of SetExternalFunctionName.
• And the behaviour of the Functoid – the transformation code – instead of writing a standard function, as you can see it in the SetExternalFunctionName type. We need to transform that function as a string using the StringBuilder, and if you want the code to be displayed in the correct form, you need to add the indentation on the string as in the example above.

The huge advantage of this type of custom Funtoid is that you don’t need to deploy any assembly to your environments. The assembly of this Functoid will only need to be copied to the Developer Tools\Mapper Extensions folder in your developing environment, and you don’t need to install anything in production or other non-development environments.

A good example of this is the BizTalk.String.Functoids available on the BizTalk Mapper Extensions UtilityPack.

Stay tuned for the following BizTalk Server Best practices, Tips, and Tricks.