Solution:
Set the Content-type header as application/json
Write failed for table row of type ‘xxxx’. Infolog: Error: The number of dimension values specified in xxxx is different from the number set up in the active format for data entities.\r\nThings to try:\r\n • Make sure the number of dimension values specified matches the active format for data entities.\r\n • Make sure that any dimension values specified that include the chart of accounts delimiter character in the name escape the delimiter character with a backslash character ‘\\’.; Error: Error found when validating record..
I am using Logic Apps to insert a record in D365 for Operations using ODATA. The error is very much self explanatory. However, a little functional knowledge on the Dynamics side is required to undersand and make corresponding changes. There is a form on Dynamics called “Financial dimension configuration for integrating applications”
Below is the screenshot. As you can see, the format includes all the dimensions. So when you send the data via ODATA, you will have to send it as 69025— (if you are only selecting the main account). Please note that even if you are not selecting the other dimensions, the value that you feed has to match the format that you setup in this page.
Below is my sample JSON message
{
“AccountDisplayValue”: “69025-“,
“AccountType”: “Ledger”,
“CreditAmount”: 2010,
“CurrencyCode”: “GBP”,
“JournalBatchNumber”: “000023_4”,
“OffsetAccountDisplayValue”: “XXX Ltd”,
“OffsetAccountType”: “Bank”,
“TransDate”: “2017-12-01”,
“dataAreaId”: “XXXX”
}
Below is my REST endpoint
https://yourcompany.sandbox.ax.dynamics.com/data/LedgerJournalLines
After I change the AccountDisplayValue from 69025- to 69025—, the request is successful.
How to remove SoapAction from a WCF service which is exposed from BizTalk
1. Navigate to the ServiceDescripttion.xml file of this service. This would be generated by the BizTalk wizard. You can go to C:\Inetpub\wwwroot\[servicename]\App_Data\
2. Find the xml tag Operation and replace the actual value for name attribute with an empty string.
3. Recycle the app pool
4. Browse the WSDL and it should not have the soapAction value
Logic App Unit testing framework
I created a simple unit testing framework for Azure Logic apps. The code is available in github – https://github.com/shankarmaha/LogicAppUnitTestManager
Logic App Test manager
Every developer has the best intention to test their code before they release. Most of them write unit-test cases to test their code in isolation. They bundle their unit tests to their CI/CD pipeline to get build issues at an early stage. As an integration developer, I would also like to add unit test for my Azure logic apps and add them to my CI/CD pipeline. Now, I believe if you are reading this, you know what Logic app is. So I would just skip to the Unit testing part. As of now, we don’t have a Unit testing framework for Logic apps. Mike Stephenson did a session on Integration Monday and showcased how someone could write an Acceptance test for a logic app using EventHub. I liked it very much, but I wanted to use the Logic apps SDK itself to unit test the app. The SDK provides a rich framework for triggerring a logic app and getting the results of each of the actions. However, you need a SPN created on the azure active directory and a contributor role should be assigned in order to connect to the azure subscription and read the workflows.
How to use
You have two options
- Clone this github project. Build it and add it as a reference in your C# unit test project
- I have packaged this project as a nuget package. You can download from nuget. PackageName: LogicAppUnitTestManager.TestFramework
Add settings in the app.confile file
Before you do this, you need to create a SPN and set the role as Contributor. Please refer to the powershell script in the below github page
https://gist.github.com/coreyasmith/29827f5d722c724a9d4a07d65da9d1f7
Add the below app settings to your C# test project app.config. Replace the xx with the correct values from your azure subscription.
<appSettings>
<add key="SubscriptionId" value="XXXXXX" />
<add key="TenantId" value="XXX" />
<add key="WebApiApplicationId" value="XXX" />
<add key="Secret" value="XXXX" />
</appSettings>
Unit Test framework
You can use any unit test framework. I used SpecFlow to define my acceptance tests. Add the below code to setup the TrackingManager.
[Binding]
public class Shared
{
[BeforeScenario("CleanTheSystem")]
public void CleanTheSystem()
{
LogicAppManager.Clear();
}
[BeforeScenario("StartTrackingManager")]
public void StartTracking()
{
LogicAppManager.Start();
}
[AfterScenario("StopTrackingManager")]
public void StopTracking()
{
LogicAppManager.Clear();
LogicAppManager.Stop();
}
}
Sample specflow feature file
Feature: BankStatementSourceCS
In order to unit test the Bank statement import functionality
As an integrtion developer
I want to be validate each of the logic app actions in the Bank Statement Source CS Logic app
@CleanTheSystem
@StartTrackingManager
@StopTrackingManager
Scenario: Import valid daily file with 1 account and valid transactions
Given the folder location is empty
| folderName |
| download |
| archived |
| bad-data |
| failed |
| in-progress |
And the logic app is enabled
When a BAI file is copied to the download folder
And a trigger message is sent to the logic app - BankStatementSourceCS
Then the logic will receive the message
And it will list all the files in the download folder
And it will get the file content
And it will copy the file to the in-progress folder
And it will delete the file from the download folder
And it will transform the message from BAICSV to BAIXML
And it will transform the message from BAIXMl to Reconcilation
And it will transform the message from Reconciliation to Canonical
Sample test code
[Given(@"the folder location is empty")]
public void GivenTheFolderLocationIsEmpty(Table table)
{
var connection = ConfigurationManager.ConnectionStrings["StorageConnectionString"].ConnectionString;
var storageHelper = new Utilities.AzureFileShareHelper(connection);
foreach(var row in table.Rows)
{
var folderName = row[0];
storageHelper.ClearFolder(folderName, "bankstatement");
}
}
[Given(@"the logic app is enabled")]
public void GivenTheLogicAppIsEnabled()
{
var enabled = LogicAppManager.CheckIfLogicAppIsEnabled("resourcegroup","logicappname");
Assert.AreEqual(true, enabled);
}
[When(@"a BAI file is copied to the download folder")]
public void WhenABAIFileIsCopiedToTheDownloadFolder()
{
var connection = ConfigurationManager.ConnectionStrings["StorageConnectionString"].ConnectionString;
var storageHelper = new Utilities.AzureFileShareHelper(connection);
storageHelper.CopyFile(@"filepath", "bankstatement", "download", "TestFile.txt");
}
[When(@"a trigger message is sent to the logic app - BankStatementSourceCS")]
public async Task WhenATriggerMessageIsSentToTheLogicApp_BankStatementSourceCS()
{
var status = await LogicAppManager.ExecuteLogicApp(LogicAppManager.TriggerType.Http, "manual", new StringContent(""));
Assert.AreEqual(System.Net.HttpStatusCode.Accepted.ToString(), status);
}
[Then(@"the logic will receive the message")]
public void ThenTheLogicWillReceiveTheMessage()
{
Assert.IsNotNull(LogicAppManager.RunId);
}
[Then(@"it will list all the files in the download folder")]
public async Task ThenItWillListAllTheFilesInTheDownloadFolder()
{
var result = await LogicAppManager.CheckLogicAppAction("List_files");
Assert.AreEqual("Succeeded", result);
}
[Then(@"it will get the file content")]
public async Task ThenItWillGetTheFileContent()
{
var result = await LogicAppManager.CheckLogicAppAction("Get_file_content");
Assert.AreEqual("Succeeded", result);
}
Property selection is not supported on values of type ‘String’. Unable to process template language expressions in action
You get this error when you try to do a property selection on the JSON data in one of your actions. You will have to set the Content-Type to ‘application/json’ when you send the data to the logic app.
The subscription is not registered to use namespace ‘Microsoft.Logic’
You get this error when you deploy an integration account. When you are assigned an Azure subscription you would have to register a resource provider. In this case, your azure subscription has to be registered to Microsoft.Logic. Follow the steps to register this provider with your subscription.
- Login into Azure using Login-AzureRmAccount
-
Register-AzureRmResourceProvider -ProviderNamespaceMicrosoft.Logic - You should get the below message
Sending Soap with Attachments (SwA) using BizTalk – Part 2
This is a continuation to my previous article.
Create a WCF service which can send the SOAP message with attachment.
1. Create a standard wcf service ( I guess this shouldnt be an issue )
2. Below is a very rough idea of what should go inside the service implementation
string endpoint = “”
ResponseObject resp = null;
using(var cf = new ChannelFactory<IService>(endpoint)
{
cf.Open();
var channel = cf.CreateChannel();
using(var scope = new OperationContextScope(IContextChannel)channel))
{
OperationContext.Current.OutgoingMessageProperties[SWAEncoderConstants.AttachmentProperty] = attachment.ToSimpleXmlUTF8();
resp = channel.method(request).response;
}
}
I will write a detailed post with a sample implementation later.
2013 in review
The WordPress.com stats helper monkeys prepared a 2013 annual report for this blog.
Here’s an excerpt:
A New York City subway train holds 1,200 people. This blog was viewed about 7,700 times in 2013. If it were a NYC subway train, it would take about 6 trips to carry that many people.
An error occurred while receiving the HTTP response to http://…….svc. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details
My solution is to have a wcf service which can accept a pdf/text file streamed as byte array in one of the elements.
I had a BizTalk orchestration exposed as a wcf service. The schema has a content element which is set as xs:base64binary. So the wcf service expects the service consumer to convert the text/pdf document into a byte array and send it to my service. When you expose services which expects one of the element content as byte array, the memory size of the message significanly increases.
You will get the below error when a client sends a bigger file, say 12 mb pdf file.
An error occurred while receiving the HTTP response to http://…….svc.
This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP
request context being aborted by the server (possibly due to the service shutting down). See server logs for more
details.
To troubleshoot this, add wcf tracing in web.config
-
In the web.config add the below under <configuration> section
<system.diagnostics>
<sources>
<source name=”System.ServiceModel”
switchValue=”Information, ActivityTracing”
propagateActivity=”true”>
<listeners>
<add name=”traceListener”
type=”System.Diagnostics.XmlWriterTraceListener”
initializeData= “c:\log\Traces.svclog” />
</listeners>
</source>
</sources>
</system.diagnostics>Make sure to create the “Log” folder under “C” Drive or feel free to configure a different location to save the logs.
- Use the WCF Trace viewer tool to view the Traces.
a. Open the tool from the path C:\Program files(x86)\Microsoft SDKs\Windows\v7.0A\Bin
b. Open the trace file.
c. You would be able to locate the exception details in the Description pane on the right side.
d. You would see the System.ServiceModel.CommunicationException thrown and the exception message would be “Maximum request length exceeded”
This conforms that the issue we have is related to the request length.
Two things must be done to resolve the issue.
-
In the BizTalk receive location, set the Maximum receive message size property to 2,147,483,647. This can be found in Adapter properties > Binding section.
-
In the web.config, add the below line under <system.web> section
<httpRuntime maxRequestLength=”2147483647″ />
Please bear in mind that setting the Maximum value for request length might solve the problem, but this can easily be expolited in case of a DOS attack.
Schematron validation using XSLT using BizTalk – Part 3
This post is a continuation to the Schematron validation using XSLT in BizTalk blog series
The source code for this blog post has been uploaded in the below location
http://code.msdn.microsoft.com/Schematron-validation-b4d892ee
Now that you know about the basic schematron XSLT implementation, we will now see how we can achieve this in BizTalk.
High Level steps for BizTalk implementation
1. Create the schematron-output schema which we discussed in the last section
2. Create a custom pipeline component( Any Category type, so that we can have the component in any of the pipeline stage, except disassembling stage)
a. Implement Custom UI functionality so that we can have the Schematron rules as property. We are also going to add a property to hold the physical path to the conformance1-5.xsl. So it is important that the conformance1-5.xsl and the skeleton1-5.xsl is placed in appropriate directories.
b. If we have any schematron errors, then we will promote the message type with the schematron output message type.
3. Apply an inbound map in the receive port if transformation is required(Optional Step)
Detailed Steps for BizTalk implementation
1. I will skip the creation of schematron-output schema .
2. Custom Pipeline component
The pipeline component has three parts
a. The component itself
b. Custom UI form
c. Custom UI Type Editor class
The custom UI is a windows form which can accept the XSLT and store it in a XML serialized format. This is pretty useful, as we can load the XSLT stylesheet from the pipeline component property itself.
The code for Custom UI and Type Editor Class are self-explanatory and I will proceed with the actual transformation logic.
public IBaseMessage Execute(IPipelineContext pc, IBaseMessage inMsg)
{
if (Enabled)
{
try
{
ReadOnlySeekableStream seekableStm = new ReadOnlySeekableStream(inMsg.BodyPart.Data);
VirtualStream vs = null;
XmlTextReader stylesheet = new XmlTextReader(new StringReader(this.SchematronXSLT));
XmlTextReader conformanceStylesheet = new XmlTextReader(this.ConformanceXSLTPath);
// Load transform
BTSXslTransform trans = new BTSXslTransform();
trans.Load(conformanceStylesheet);
XmlTextReader inputReader = new XmlTextReader(seekableStm);
//Create memory stream to hold transformed data.
vs = new VirtualStream(VirtualStream.MemoryFlag.AutoOverFlowToDisk);
//Preform transform
trans.ScalableTransform(stylesheet, null, vs, new XmlUrlResolver(), false);
vs.Seek(0, SeekOrigin.Begin);
XmlTextReader validatingXSLReader = new XmlTextReader(vs);
BTSXslTransform ValidatingTrans = new BTSXslTransform();
ValidatingTrans.Load(validatingXSLReader);
VirtualStream vs1 = new VirtualStream(VirtualStream.MemoryFlag.AutoOverFlowToDisk);
ValidatingTrans.ScalableTransform(inputReader, null, vs1, new XmlUrlResolver(), false);
vs1.Seek(0, SeekOrigin.Begin);
XmlDocument outputDoc = new XmlDocument();
outputDoc.Load(vs1);
if (outputDoc.SelectNodes(“//failed-assert”).Count > 0)
{
vs.Flush();
vs1.Flush();
vs1.Seek(0, SeekOrigin.Begin);
inMsg.BodyPart.Data = vs1;
inMsg.Context.Promote(“MessageType”, “http://schemas.microsoft.com/BizTalk/2003/system-properties”, “http://www.ascc.net/xml/schematron#schematron-output”);
inMsg.BodyPart.Data.Position = 0;
}
else
{
seekableStm.Seek(0, SeekOrigin.Begin);
inMsg.BodyPart.Data = seekableStm;
}
}
catch (Exception ex)
{
System.Diagnostics.EventLog.WriteEntry(this.Name, ex.Message);
}
}
return inMsg;
}
Key things from the above code is I have used BTSXslTransform class to do the transformation and used VirtualStream to hold the transformed data. One nice feature about VirtualStream is that it is initialized with VirtualStream.MemoryFlag.AutoOverFlowToDisk i.e. when the internal memory is not sufficient to hold the transformed data, it will be stored to the disk. This is greatly useful in case of large messages.
Other than that, the code is quite simple to follow.
Summary
So far, we have seen the development of the schematron XSLT infrastructure and how to use it with BizTalk. This part of the tutorial is only a beginner and we will see more complex schematron in the upcoming posts.
shankars Integration works 