Logic App Unit testing framework

I created a simple unit testing framework for Azure Logic apps. The code is available in github – https://github.com/shankarmaha/LogicAppUnitTestManager

Logic App Test manager

Every developer has the best intention to test their code before they release. Most of them write unit-test cases to test their code in isolation. They bundle their unit tests to their CI/CD pipeline to get build issues at an early stage. As an integration developer, I would also like to add unit test for my Azure logic apps and add them to my CI/CD pipeline. Now, I believe if you are reading this, you know what Logic app is. So I would just skip to the Unit testing part. As of now, we don’t have a Unit testing framework for Logic apps. Mike Stephenson did a session on Integration Monday and showcased how someone could write an Acceptance test for a logic app using EventHub. I liked it very much, but I wanted to use the Logic apps SDK itself to unit test the app. The SDK provides a rich framework for triggerring a logic app and getting the results of each of the actions. However, you need a SPN created on the azure active directory and a contributor role should be assigned in order to connect to the azure subscription and read the workflows.

How to use

You have two options

  1. Clone this github project. Build it and add it as a reference in your C# unit test project
  2. I have packaged this project as a nuget package. You can download from nuget. PackageName: LogicAppUnitTestManager.TestFramework

Add settings in the app.confile file

Before you do this, you need to create a SPN and set the role as Contributor. Please refer to the powershell script in the below github page

https://gist.github.com/coreyasmith/29827f5d722c724a9d4a07d65da9d1f7

Add the below app settings to your C# test project app.config. Replace the xx with the correct values from your azure subscription.

<appSettings>
    <add key="SubscriptionId" value="XXXXXX" />
    <add key="TenantId" value="XXX" />
    <add key="WebApiApplicationId" value="XXX" />
    <add key="Secret" value="XXXX" />
</appSettings>

Unit Test framework

You can use any unit test framework. I used SpecFlow to define my acceptance tests. Add the below code to setup the TrackingManager.

 [Binding]
    public class Shared
    {
        [BeforeScenario("CleanTheSystem")]
        public void CleanTheSystem()
        {
            LogicAppManager.Clear();
        }

        [BeforeScenario("StartTrackingManager")]
        public void StartTracking()
        {
            LogicAppManager.Start();
        }

        [AfterScenario("StopTrackingManager")]
        public void StopTracking()
        {
            LogicAppManager.Clear();
            LogicAppManager.Stop();
        }
    }

Sample specflow feature file

Feature: BankStatementSourceCS
	In order to unit test the Bank statement import functionality 
	As an integrtion developer
	I want to be validate each of the logic app actions in the Bank Statement Source CS Logic app

@CleanTheSystem
@StartTrackingManager
@StopTrackingManager
Scenario: Import valid daily file with 1 account and valid transactions
	Given the folder location is empty
		| folderName  |
		| download    |
		| archived    |
		| bad-data    |
		| failed      |
		| in-progress |
	And the logic app is enabled
	When a BAI file is copied to the download folder
	And  a trigger message is sent to the logic app - BankStatementSourceCS
	Then the logic will receive the message
	And it will list all the files in the download folder
	And it will get the file content
	And it will copy the file to the in-progress folder
	And it will delete the file from the download folder
	And it will transform the message from BAICSV to BAIXML
	And it will transform the message from BAIXMl to Reconcilation
	And it will transform the message from Reconciliation to Canonical

Sample test code

        [Given(@"the folder location is empty")]
        public void GivenTheFolderLocationIsEmpty(Table table)
        {
            var connection = ConfigurationManager.ConnectionStrings["StorageConnectionString"].ConnectionString;
            var storageHelper = new Utilities.AzureFileShareHelper(connection);

            foreach(var row in table.Rows)
            {
                var folderName = row[0];
                storageHelper.ClearFolder(folderName, "bankstatement");
            }
        }

        [Given(@"the logic app is enabled")]
        public void GivenTheLogicAppIsEnabled()
        {
            var enabled = LogicAppManager.CheckIfLogicAppIsEnabled("resourcegroup","logicappname");
            Assert.AreEqual(true, enabled);
        }
        
        [When(@"a BAI file is copied to the download folder")]
        public void WhenABAIFileIsCopiedToTheDownloadFolder()
        {
            var connection = ConfigurationManager.ConnectionStrings["StorageConnectionString"].ConnectionString;
            var storageHelper = new Utilities.AzureFileShareHelper(connection);

            storageHelper.CopyFile(@"filepath", "bankstatement", "download", "TestFile.txt");
        }

        [When(@"a trigger message is sent to the logic app - BankStatementSourceCS")]
        public async Task WhenATriggerMessageIsSentToTheLogicApp_BankStatementSourceCS()
        {
            var status = await LogicAppManager.ExecuteLogicApp(LogicAppManager.TriggerType.Http, "manual", new StringContent(""));
            Assert.AreEqual(System.Net.HttpStatusCode.Accepted.ToString(), status);
        }


        [Then(@"the logic will receive the message")]
        public void ThenTheLogicWillReceiveTheMessage()
        {
            Assert.IsNotNull(LogicAppManager.RunId);
        }
        
        [Then(@"it will list all the files in the download folder")]
        public async Task ThenItWillListAllTheFilesInTheDownloadFolder()
        {
            var result = await LogicAppManager.CheckLogicAppAction("List_files");
            Assert.AreEqual("Succeeded", result); 
        }

        [Then(@"it will get the file content")]
        public async Task ThenItWillGetTheFileContent()
        {
            var result = await LogicAppManager.CheckLogicAppAction("Get_file_content");
            Assert.AreEqual("Succeeded", result);
        }

Advertisements

The subscription is not registered to use namespace ‘Microsoft.Logic’

You get this error when you deploy an integration account. When you are assigned an Azure subscription you would have to register a resource provider. In this case, your azure subscription has to be registered to Microsoft.Logic. Follow the steps to register this provider with your subscription.

  1. Login into Azure using Login-AzureRmAccount
  2. Register-AzureRmResourceProvider -ProviderNamespace Microsoft.Logic
  3. You should get the below message

LogicAppRegister

Sending Soap with Attachments (SwA) using BizTalk – Part 2

This is a continuation to my previous article.

Create a WCF service which can send the SOAP message with attachment.

1. Create a standard wcf service ( I guess this shouldnt be an issue )

2. Below is a very rough idea of what should go inside the service implementation

string endpoint = “”
ResponseObject resp = null;
using(var cf = new ChannelFactory<IService>(endpoint)
{
cf.Open();
var channel = cf.CreateChannel();
using(var scope = new OperationContextScope(IContextChannel)channel))
{
OperationContext.Current.OutgoingMessageProperties[SWAEncoderConstants.AttachmentProperty] = attachment.ToSimpleXmlUTF8();
resp = channel.method(request).response;
}
}

I will write a detailed post with a sample implementation later.

An error occurred while receiving the HTTP response to http://…….svc. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details

My solution is to have a wcf service which can accept a pdf/text file streamed as byte array in one of the elements.

I had a BizTalk orchestration exposed as a wcf service. The schema has a content element which is set as xs:base64binary. So the wcf service expects the service consumer to convert the text/pdf document into a byte array and send it to my service. When you expose services which expects one of the element content as byte array, the memory size of the message significanly increases.

You will get the below error when a client sends a bigger file, say 12 mb pdf file.

An error occurred while receiving the HTTP response to http://…&#8230;.svc.
This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP
request context being aborted by the server (possibly due to the service shutting down). See server logs for more
details.

To troubleshoot this, add wcf tracing in web.config

  1. In the web.config add the below under <configuration> section

    <system.diagnostics>
    <sources>
    <source name=”System.ServiceModel” 
    switchValue=”Information, ActivityTracing”
    propagateActivity=”true”>
    <listeners>
    <add name=”traceListener” 
    type=”System.Diagnostics.XmlWriterTraceListener” 
    initializeData= “c:\log\Traces.svclog” />
    </listeners>
    </source>
    </sources>
    </system.diagnostics>

    Make sure to create the “Log” folder under “C” Drive or feel free to configure a different location to save the logs.

     
  2. Use the WCF Trace viewer tool to view the Traces.

a. Open the tool from the path C:\Program files(x86)\Microsoft SDKs\Windows\v7.0A\Bin
b. Open the trace file.
c. You would be able to locate the exception details in the Description pane on the right side.
d. You would see the System.ServiceModel.CommunicationException thrown and the exception message would be “Maximum request length exceeded”

This conforms that the issue we have is related to the request length.

Two things must be done to resolve the issue.

  1. In the BizTalk receive location, set the Maximum receive message size property to 2,147,483,647. This can be found in Adapter properties > Binding section.

  2. In the web.config, add the below line under <system.web> section

<httpRuntime maxRequestLength=”2147483647″ />

Please bear in mind that setting the Maximum value for request length might solve the problem, but this can easily be expolited in case of a DOS attack.

Schematron validation using XSLT using BizTalk – Part 3

This post is a continuation to the Schematron validation using XSLT in BizTalk blog series

The source code for this blog post has been uploaded in the below location

http://code.msdn.microsoft.com/Schematron-validation-b4d892ee
Now that you know about the basic schematron XSLT implementation, we will now see how we can achieve this in BizTalk.

High Level steps for BizTalk implementation

1. Create the schematron-output schema which we discussed in the last section
2. Create a custom pipeline component( Any Category type, so that we can have the component in any of the pipeline stage, except disassembling stage)
a. Implement Custom UI functionality so that we can have the Schematron rules as property. We are also going to add a property to hold the physical path to the conformance1-5.xsl. So it is important that the conformance1-5.xsl and the skeleton1-5.xsl is placed in appropriate directories.
b. If we have any schematron errors, then we will promote the message type with the schematron output message type.
3. Apply an inbound map in the receive port if transformation is required(Optional Step)

Detailed Steps for BizTalk implementation
1. I will skip the creation of schematron-output schema .
2. Custom Pipeline component

The pipeline component has three parts
a. The component itself
b. Custom UI form
c. Custom UI Type Editor class

The custom UI is a windows form which can accept the XSLT and store it in a XML serialized format. This is pretty useful, as we can load the XSLT stylesheet from the pipeline component property itself.

The code for Custom UI and Type Editor Class are self-explanatory and I will proceed with the actual transformation logic.

public IBaseMessage Execute(IPipelineContext pc, IBaseMessage inMsg)
{
if (Enabled)
{
try
{
ReadOnlySeekableStream seekableStm = new ReadOnlySeekableStream(inMsg.BodyPart.Data);

VirtualStream vs = null;

XmlTextReader stylesheet = new XmlTextReader(new StringReader(this.SchematronXSLT));

XmlTextReader conformanceStylesheet = new XmlTextReader(this.ConformanceXSLTPath);

// Load transform
BTSXslTransform trans = new BTSXslTransform();
trans.Load(conformanceStylesheet);

XmlTextReader inputReader = new XmlTextReader(seekableStm);

//Create memory stream to hold transformed data.
vs = new VirtualStream(VirtualStream.MemoryFlag.AutoOverFlowToDisk);

//Preform transform
trans.ScalableTransform(stylesheet, null, vs, new XmlUrlResolver(), false);

vs.Seek(0, SeekOrigin.Begin);

XmlTextReader validatingXSLReader = new XmlTextReader(vs);
BTSXslTransform ValidatingTrans = new BTSXslTransform();
ValidatingTrans.Load(validatingXSLReader);

VirtualStream vs1 = new VirtualStream(VirtualStream.MemoryFlag.AutoOverFlowToDisk);

ValidatingTrans.ScalableTransform(inputReader, null, vs1, new XmlUrlResolver(), false);
vs1.Seek(0, SeekOrigin.Begin);
XmlDocument outputDoc = new XmlDocument();
outputDoc.Load(vs1);
if (outputDoc.SelectNodes(“//failed-assert”).Count > 0)
{
vs.Flush();
vs1.Flush();
vs1.Seek(0, SeekOrigin.Begin);
inMsg.BodyPart.Data = vs1;
inMsg.Context.Promote(“MessageType”, “http://schemas.microsoft.com/BizTalk/2003/system-properties&#8221;, “http://www.ascc.net/xml/schematron#schematron-output&#8221;);
inMsg.BodyPart.Data.Position = 0;
}
else
{
seekableStm.Seek(0, SeekOrigin.Begin);
inMsg.BodyPart.Data = seekableStm;
}

}
catch (Exception ex)
{
System.Diagnostics.EventLog.WriteEntry(this.Name, ex.Message);
}
}

return inMsg;

}

Key things from the above code is I have used BTSXslTransform class to do the transformation and used VirtualStream to hold the transformed data. One nice feature about VirtualStream is that it is initialized with VirtualStream.MemoryFlag.AutoOverFlowToDisk i.e. when the internal memory is not sufficient to hold the transformed data, it will be stored to the disk. This is greatly useful in case of large messages.

Other than that, the code is quite simple to follow.

Summary

So far, we have seen the development of the schematron XSLT infrastructure and how to use it with BizTalk. This part of the tutorial is only a beginner and we will see more complex schematron in the upcoming posts.