Announcing the ChannelAdam SOAP Library

   Submit to Reddit      
  

The Simple Object Access Protocol (SOAP) has been around for more than a decade. While HTTP-based API's and RESTful services are the current trend, SOAP web services are still the rock bed of most businesses and contemporary systems.

On-premises API gateways such as the CA API Gateway has had support for publishing SOAP web services for quite a while.

In fact, there is still such demand in the industry for integration with SOAP services that Microsoft's newer cloud-based Azure API Management service also now has the capability to publish SOAP API's.

The .NET library Nancy is an excellent lightweight web framework for building HTTP-based web services. While Nancy will not itself ship support for SOAP it still can be used for hosting SOAP web services, if you can create the SOAP payload.

Libraries such as Nancy or Mock4Net (a tiny mock HTTP server for .NET) can be used in automated Unit Tests to dynamically create mock web services that return the exact response required for each specific test case. In fact, I personally use this specific technique to perform advanced Unit Testing of black-box code artefacts such as BizTalk orchestrations.

However, creating and composing a SOAP envelope, header, and body or fault has always been a tricky undertaking, and unless you have studied the SOAP specifications in detail and use them daily, it is easy to either forget or confuse the differences between SOAP 1.1 and SOAP 1.2 - that is, until now!

Introducing the ChannelAdam SOAP Library – a fluent .NET API to make it as easy as possible to create a SOAP 1.1 or SOAP 1.2 payload.

For information on how to get started, please see the usage documentation.

You might also be interested in the ChannelAdam Nancy SOAP Library which makes using SOAP with Nancy even easier.

Modern Testing of BizTalk Maps

   Submit to Reddit      
  

Overview

In today's modern world of software development with continuous integration as a standard industry practice, what do you think would happen if you told a developer that they need to set a special Visual Studio project property in their build configuration in order for their code to "support unit testing"?

They would laugh at you in disbelief! Why does code need to be built in a special way in order to be testable?! This laughing typically would be followed by heart-ache, maybe a little quiet sobbing and an uncontrollable eye twitch as they realise that decade-old standard industry software development practices are crashing down all around them and they start reconsidering their life choices.

Sadly, the only thing that you can do is agree and try to console them by saying it is "just because it's [technology xyz]".

This is the unfortunate world for a typical BizTalk developer using the out-of-the-box functionality.

And this embarrassing situation has other negative implications:

  • As described in Unit Testing with BizTalk Server Projects and Using the Unit Testing Feature with Schemas and Maps, BizTalk developers must set the Visual Studio project property "Enable Unit Testing" (under the Deployment tab) to "True". This magically changes the base class of each BizTalk Map to "Microsoft.BizTalk.TestTools.Mapper.TestableMapBase" in order to "support unit testing". Yes, an artefact's base class is actually changed for that build – so what is being tested is not entirely the actual code that will be deployed into production. The difference between a Debug build and a Release build becomes more significant. (Admittedly for a BizTalk Map, this is less of a concern since the actual mapping is performed via XSLT under the covers and not directly by the base class, but the point is valid nevertheless – this approach is taken for other types of artefacts as well, and you really do not want to deploy Debug builds of any of those into a production environment...)

  • If you are like me and some teams I work with, you may want to develop and test the code off-premises (perhaps in your favourite cloud) and then deploy the release/deployment package to an on-premises test environment – where we also want to execute all the tests and have the test results recorded in the on-premises test and release management tools for all team members to access. However, a release/deployment package should never have that special project property set – and this prevents unit tests from compiling and executing against it.

  • When developing a BizTalk Map that uses a Scripting Functoid configured to execute a specified method of a class in an 'External Assembly', then "any assembly that you choose a script from ... must also be in the global assembly cache (GAC) for Test Map to function correctly". You need to GAC your external assemblies before running the unit tests. GAC —> Yuck!

Microsoft's Integration Roadmap clearly shows a measured commitment to and place for BizTalk in the future, alongside their Azure Platform as a Service (PaaS) technologies. With the pending release of BizTalk 2016 as a drop-in replacement for BizTalk 2013 R2 (with no code recompilation required!), I thought it was time to refresh the practice of unit testing BizTalk Maps.

The Solution

The open source ChannelAdam Test Framework Library for BizTalk addresses all of these issues and provides the following benefits and features:

  • Works with BizTalk 2010 / 2013 / 2016
  • Never again set the project property "Enable Unit Testing" to "True"!
  • BizTalk Map unit tests can be executed against a release build
  • The provided map tester classes allow you to easily unit test both XML and flat file BizTalk Maps
  • The design of tester classes follow the clean Arrange-Act-Assert (AAA) testing pattern
  • Support for arranging the input and expected output from a string, XElement, an XML serialisable object or loading from an embedded resource
  • Performs schema validation of both the input and output by default (easily overridable)
  • Support for mocking External Assembly class methods
  • Support for filtering assertions of the expected output to allow for dynamic data (such as timestamps or GUIDs)
  • Detailed information is provided allowing you to see the differences between the expected output and the actual output of the map.

With this library you now have the tools needed to easily unit test you BizTalk Maps using modern testing practices. No excuses!

For information on how to get started, please see the usage documentation.

ChannelAdam WCF Library - Version 2 Released

   Submit to Reddit      
  

Version 2 of the open source ChannelAdam WCF Library has been released.

The major changes include how retry policies for transient faults are applied, with support for plugging in any type of retry mechanism.

Please see the ChannelAdam WCF Library documentation for more information.

Logic Apps and the Service Bus Connector - The Case of the Incomplete Message!

   Submit to Reddit      
  

(Cross-posted on the Mexia Blog)

The Symptom

The other day I tracked down an issue my team was seeing with an Azure App Service Logic App. The Logic App was being triggered by an Azure Service Bus Connector that was configured to work with a Service Bus Topic. With a message published to the Service Bus Topic, an instance of the Logic App would execute successfully — as expected on the trigger's configured schedule. But then, unexpectedly, on the next scheduled time period, often a new instance of the Logic App would execute and reprocess the exact same message from the Service Bus Topic. It was as if the Service Bus Topic message was never successfully "completed" and thus removed from the Topic.

What You Need To Understand

In order to understand why the same Service Bus Topic message was being processed twice, this is what you need to know.

  • The Service Bus Connector operates in ReceiveMode.PeekLock, so it is not receiving and immediately deleting the message from the Topic in one atomic call like in ReceiveMode.ReceiveAndDelete. In PeekLock mode, two calls to the Service Bus must be made. The first call is to the Receive method, and the Service Bus puts a lock on the message for a specific lock duration. The default LockDuration for a Queue or Subscription is 1 minute, and currently that cannot be configured in the Service Bus Connector. Then a second call, to the Complete method, is required to mark that message as processed and delete it from the Service Bus.

  • After a call to a Connector's trigger method returns a result indicating that data is available to be processed and that a Logic App instance should be created and executed, the Logic App trigger mechanism immediately calls the Connector's trigger method again. This logically is being done in case there is even more data to be processed. This makes sense because if there is more data available to be processed, one typically would want that data being processed as soon as possible, and would not want to wait for the next scheduled trigger time period.

  • Connectors can have their own specified triggerState data passed from one call of their trigger method to subsequent calls of their trigger method.

  • The Service Bus Connector passes the message's unique LockToken as triggerState from the first successful trigger invocation to its next trigger call that runs immediately after it.

  • When the Service Bus Connector trigger method is executed, one of the first things that it does is to determine if the triggerState data has been provided. If the triggerState data has been provided, the Service Bus Connector calls the Complete method of the message represented by that given triggerState/LockToken. Only after doing that will it attempt to receive the next message from the configured Topic or Queue.

The Cause

Now that we understand how the Complete method is called as part of the Service Bus Connector's trigger mechanism, it is easy to understand the cause of the problem.

If the second call to the trigger fails (the call that would complete the message), for instance, because of a transient error on the gateway, then the Service Bus message will not be completed — until perhaps the next scheduled trigger.

However, after the default lock duration of 1 minute expires, the lock is lost and Service Bus makes the message available to be received again.

This is exactly what happened — the message was not completed and when the next scheduled trigger executes, the same message is processed again.

Proof — The Smoking Trigger

From the Trigger History of the Logic App, we can see the trigger that failed in the image below.

Logic App Trigger History with Failed Trigger

The call to this trigger returned a 502 "Bad Gateway" error.

502 - Web server received an invalid response while acting as a gateway or proxy server.

The full JSON of the output message for the failed trigger is:

{
    "headers": {
        "date": "Mon, 07 Sep 2015 23:28:32 GMT",
        "server": "Microsoft-IIS/8.0"
    },

    "body": "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">\r\n<html xmlns=\"http://www.w3.org/1999/xhtml\">\r\n<head>\r\n<meta http-equiv=\"Content-Type\" content=\"text/html; charset=iso-8859-1\"/>\r\n<title>502 - Web server received an invalid response while acting as a gateway or proxy server.</title>\r\n<style type=\"text/css\">\r\n<!--\r\nbody{margin:0;font-size:.7em;font-family:Verdana, Arial, Helvetica, sans-serif;background:#EEEEEE;}\r\nfieldset{padding:0 15px 10px 15px;} \r\nh1{font-size:2.4em;margin:0;color:#FFF;}\r\nh2{font-size:1.7em;margin:0;color:#CC0000;} \r\nh3{font-size:1.2em;margin:10px 0 0 0;color:#000000;} \r\n#header{width:96%;margin:0 0 0 0;padding:6px 2% 6px 2%;font-family:\"trebuchet MS\", Verdana, sans-serif;color:#FFF;\r\nbackground-color:#555555;}\r\n#content{margin:0 0 0 2%;position:relative;}\r\n.content-container{background:#FFF;width:96%;margin-top:8px;padding:10px;position:relative;}\r\n-->\r\n</style>\r\n</head>\r\n<body>\r\n<div id=\"header\"><h1>Server Error</h1></div>\r\n<div id=\"content\">\r\n <div class=\"content-container\"><fieldset>\r\n  <h2>502 - Web server received an invalid response while acting as a gateway or proxy server.</h2>\r\n  <h3>There is a problem with the page you are looking for, and it cannot be displayed. When the Web server (while acting as a gateway or proxy) contacted the upstream content server, it received an invalid response from the content server.</h3>\r\n </fieldset></div>\r\n</div>\r\n</body>\r\n</html>\r\n"
}

The Remedy

This situation should not happen often, but unfortunately it was occurring multiple times per day, and only ever on the second call to the trigger method. The only real solution is to follow Microsoft's advice and design your systems to be idempotent, because it can and occasionally will happen.

In addition to that, to reduce the frequency of the situation, hopefully Microsoft will release a change in the near future so that failed calls to the trigger method are automatically retried.