Schematron Validation using XSLT in BizTalk – Part 1

This blog post is a 3 part series.

The source code for this blog post has been uploaded in the below location

http://code.msdn.microsoft.com/Schematron-validation-b4d892ee

BizTalk has an out of the box XML validation pipeline component to validate the semantics of the XSD schema and return the error message. However it has a lot of drawbacks
– can’t have custom error messages for failures
– can’t have all the error messages at one shot
Enter Schematron. Schematron is a language for making assertions about patterns found in XML documents. It is a rule based validation and is an ISO/IEC Standard.
There are currently many ways for implementing schematron. If we talk about BizTalk and .Net, there are two ways.
1. XPATH based implementation
2. XSLT based implementation

XPATH based in .Net Framework

Daniel Cazzulino, Microsoft XML MVP, designed a class library called Schematron.Net which provides classes for validating XML documents against the schematron schemas. This is done by the following
1. Embed Schematron rule patterns in BizTalk XSD schemas.
2. Write a custom pipeline component to call the Schematron.Net assembly to do the validation or use the Schematron Pipeline component available in Codeplex.

The above approach has some issues
1. There is a dependency to this class library. Although this library is open source, when you build BizTalk applications for enterprises, management doesn’t really like the idea of having 3rd Software libraries in the code. In case there are issues with the library, there will be no one to support.
2. A lot of XSLT functions are not available in this implementation

XSLT based Implementation

XSLT based Implementation works in two steps (Actually its 4 steps, however the extra 2 steps are actually required in complex scenarios)
1. The Schematron schema (.sch) is first turned into a validating XSLT stylesheet by transforming it with an XSLT stylesheet provided by Academica Sinica Computing Centre. These stylesheets (schematron-basic.xsl, schematron-message.xsl, schematron-report.xsl and conformance1-5.xsl) can be found at the Schematron site and the different stylesheets generate different output.
2. This validating stylesheet is then used on the XML instance document and the result will be a report that is based on the rules and assertions in the original Schematron schema.

xslt

Advertisements

Sending Soap with Attachments (SwA) using BizTalk – Part 1

This is a continuation to my previous article. Please have a look if you hadn’t seen it.

Sending Soap with Attachments (SwA) using BizTalk

Below are the steps that we have to follow for this exercise.

1. Create a custom message encoder which can send XML as an attachment in the SOAP envelope message.

2. Create a WCF service which can send the SOAP message with attachment.

3. Create the BizTalk application which can consume the custom WCF service.

Steps are easy isn’t. In this post we will only cover Step 1.

As I have told in the previous post, this custom encoder has been developed by Austrin interoperablity council and they have published the code in Codeplex. I have taken this code and removed some hard coded stuff and placed it in configurable properties.

I have uploaded the full source code below.

http://code.msdn.microsoft.com/Send-Soap-with-Attachments-3784e5f6

Shankar

Sending Soap with Attachments (SwA) using BizTalk

Soap With attachments is a very old technique and is still used in lot of legacy systems. More new technologies has emerged like MTOM which is widely used within the WCF arena. That said, there is no out of the box support within WCF to send a SOAP message with attachments.

An Austrian Microsoft Interoperability Council had a challenge to consume a webservice which expects to send a SOAP message with attachments.

http://blogs.msdn.com/b/mszcool/archive/2009/10/19/windows-communication-foundation-and-soap-with-attachments-message-encoder-built-in-interop-lab-with-svc-sozialversicherungs-chipkarten-betriebs-und-errichtungsgesellschaft-m-b-h.aspx

They have developed a custom encoder which can add the zip message as an attachment in the outgoinh SOAP envelope.

However I had a different challenge in consuming a webservice which requires me to send a soap payload which has an XML attachment. So I had to modify the custom encoder in such a way that I can attach a XML document. I had lot more challenges. The webservice which I had to consume was written in Java and the field appears as xsd:hexBinary in the WSDL file. So there is no way I can add a service reference / web reference to my project. I did tried that, but when I tried to send the message, the server was rejecting the message. The reason for this is Visual studio when generating the client code for the service, puts the type as xsd:base64binary.

So, I had to code the datacontracts, messagecontracts and servicecontracts based on the WSDL. Then I plugged in the code to use the encoder to attach the XML message. There is no direct way to do this from BizTalk. I wrote this entire logic in a WCF Service which acts as a client to the Target web service and acts as a Server for BizTalk. BizTalk application will consume this WCF service and send the SOAP message with attachment.

I am going to present an article with walkthrough in the next post.

Shankar

Replaying failed messages from BizTalk ESB portal Part 3

This post is in continuation with the series “Replaying failed messages from BizTalk ESB Portal”

If you haven’t seen the first post of this series, please use the below link to view the post. Part 2

Until now we have seen how to use BizTalk orchestration/receive port/send port to route failed messages to the ESB. But now we will how to replay the message from within the ESB Management Console. The ESB Portal is a ASP.Net application and the whole source code is made available to us by Microsoft.

The default replaying mechanism requires us to create either HTTP or WCF-HTTP adapters for replaying. I would like to go for a FILE adapter, as I would get an oppurtunity to view the message once it is replayed. To add support for a FILE adapter, we need to customize the ESB.Portal ASP.Net application. When you would have downloaded the ESB toolkit, you would have also got the source code for the ESB Portal and the corresponding web services applications.

Once you have loaded the solution, follow the below steps.

  1. In the ESB.Portal project, expand Faults and click MessageViewer.ascx file. Right click to view code or press F7.
  2. Navigate to the method PopulateReceiveLocationList().
  3. foreach (BizTalkOperationsService.BTReceiveLocation rcvLoc in sysStatus.ReceiveLocations)
                {
                    if (rcvLoc.Handler.ProtocolName.ToUpper().Equals("HTTP"))
                    {
                        rcvLoc.Address = "http://" + rcvLoc.Handler.Host.HostInstances[0].ServerName + rcvLoc.Address;
                        httpRcvLocs.Add(rcvLoc);
                        i++;
                    }
               }
    

    Replace the above code with

    foreach (BizTalkOperationsService.BTReceiveLocation rcvLoc in sysStatus.ReceiveLocations)
                {
                    if (rcvLoc.Handler.ProtocolName.ToUpper().Equals("HTTP"))
                    {
                        rcvLoc.Address = "http://" + rcvLoc.Handler.Host.HostInstances[0].ServerName + rcvLoc.Address;
                        httpRcvLocs.Add(rcvLoc);
                        i++;
                    }
                    else if (rcvLoc.Handler.ProtocolName.ToUpper().Equals("FILE"))
                    {
                        httpRcvLocs.Add(rcvLoc);
                        i++;
                    }
                }
    
  4. The above lines of code will include the FILE receive locations to appear in the ESB Management console.
    Now we need to navigate to the ResubmitMessage() method and add the below code

    else if (resubmitUrl.Contains("*.xml"))
                {
                    string url = resubmitUrl.Replace("\\", @"\");
                    url = url.Substring(0, url.IndexOf('*'));
    
                    System.Xml.XmlDocument doc = new System.Xml.XmlDocument();
                    StringBuilder envelopeMessage = new StringBuilder();
                    envelopeMessage.Append(((TextBox)this.MessageView.FindControl("messageBodyBox")).Text);
                    doc.LoadXml(envelopeMessage.ToString());
    
                    doc.Save(url + @"\" + System.Guid.NewGuid().ToString() + ".xml");
    
                    responseCode = "202";
                    responseMessage = "Successfully sumbitted to FILE Adapter";
    
                    result = true;
    
                }
    

    The above lines of code creates a xml document and drops it into the receive location URI that you have chosen. You will understand it better when you see everything in action.

  5. compile and project. If the project location is the same as the IIS directory location for this application, then you dont need to copy anything. If you are developing in a differnt location, then you can run the setup file that is also there in the solution to get your changes into the web application.
  6. We are all done and we can ahead and do our testing. Click the fault message in the Fault viewer and go to the message details. In the message viewer, click the edit link and the Text View control will now be enabled for you to edit the message. Go ahead and add the value 1 in the Count field and click Resubmit link.
  7. You will see a confirmation status in the ReSubmission status field. Go to Biztalk administration console and you could see that our orchestration would have completed without raising any exception. If you can enable tracking on the orchestration, you could now see the altered message.
  8. Have a look at the below picture for resubmitting a message

We have now completed replaying failed message from the ESB Management console without any context properties. If you observe the context message of the new one that you received, they would have lost its original context properties. In some circumnstance, you might need to replay a message with its original context property.

We will cover that in the next post.

Shankar

Replaying failed messages from BizTalk ESB portal Part 2

This post is in continuation with the series “Replaying failed messages from BizTalk ESB Portal”

If you haven’t seen the first post of this series, please use the below link to view the post. Part 1

In this Post, we will see a sample application. This solution will comprise a schema and an orchestration. We will see how we can route messages to the ESB from an orchestration. After doing this excercise we will also see how we can route failed messages from receive port and send port to the ESB.

So, lets first see how we can route messages from Orchestration.

  1. I created a sample application to demonstrate this. I have a xml schema which will be my source schema. The schema structure looks like this.


    See the Count element promoted in the schema.

  2. I added the below dll’s as reference to the project.
    C:\Program Files (x86)\Microsoft BizTalk ESB Toolkit 2.1\Bin\Microsoft.Practices.ESB.ExceptionHandling.dll
    C:\Program Files (x86)\Microsoft BizTalk ESB Toolkit 2.1\Bin\Microsoft.Practices.ESB.ExceptionHandling.Schemas.Faults.dll
  3. Added an orchestration which will process the xml message. It receives the xml message using a receive shape and tries to assign the promoted value(Count) into a variable.
  4. As I explained in my previous post, I have added a scope that will have all my shapes. Currently there is only one shape in our business process. 7.Create a message in orchestration view and name it msgFaultMessage. Select the type as
    Microsoft.Practices.ESB.ExceptionHandling.Schemas.Faults.FaultMessage
  5. Add a construct shape and select the message that we have just added. Add a message assignment shape and the below code.
     msgFaultMessage = Microsoft.Practices.ESB.ExceptionHandling.ExceptionMgmt.CreateFaultMessage();
     msgFaultMessage.FailureCategory = “Process Questions”;
     msgFaultMessage.FaultCode = “”;
     msgFaultMessage.FaultDescription = ex.Message;
     msgFaultMessage.FaultSeverity = Microsoft.Practices.ESB.ExceptionHandling.FaultSeverity.Critical;
    Microsoft.Practices.ESB.ExceptionHandling.ExceptionMgmt.AddMessage(msgFaultMessage,msgQuestion);
    
  6. Add a send shape to send the fault message. Add a Logical send port in the orchestration and configure the binding as Direct with “Routing between ports…” option selected.
  7. Build the project and deploy the orchestration. Now to see the fault message routed to ESB, we need to create a source file for our orchestration to receive and process. Go ahead and generate an instance of the xml message. Modify the message and remove the value for the Count field. This will make the orchestration engine throw an error. That error which will caught by our exception handler and a fault message will be created. It will then be sent to Message box using direct send port.
  8. This is what you should in the BizTalk administration console.

  9. See how the message is showing up in ESB portal.

  10. You can click on the fault message to show its fault details in the fault viewer.

  11. You can find a grid at the bottom and this is the actual failed message. You can see a link for its messageid and if you click that it will give you a chance to see it message data and its context properties.

    Below is the message data

    Below is the context data

In this post I have explained how to route failed messages to ESB and see the fault messages within ESB Management console.

Messages can also be routed from receive ports and send ports. There is a check box in the properties and we can enable that to route fault messages to the message box.

In the next post, I will show you how to replay messages from the BizTalk ESB console.

Shankar

Replaying failed messages from BizTalk ESB portal Part 1

This was a new learning for me from this current contract. The ESB portal is an ASP.Net application which can be used to replay the messages. Then comes the question, can I replay a message with its original context property? Yes, you can, but hold on, you need to customize the ESB Portal application given by Microsoft and you need to create a custom pipeline component and a custom pipeline. So first let’s see how we can replay a message without context properties.

I want to split this topic into multiple parts, as one post will get rather big to accomodate all the topics.

Replaying messages without context properties

I have jotted down the steps required in bullet points, as everyone(including me) like points rather than story.

  1. When you Install Biztalk in the development environment, you would have got the ESB Portal application Installed and configured. If not, use the below link to complete that. http://msdn.microsoft.com/en-us/library/ee236731(v=bts.10).aspx
  2. AFter you have intalled and configured ESB Portal, navigate to the ESB portal website, to make sure there is no issues. It should like the below picture.
  3. Please also make sure that you have installed the ESB Biztalk application which comes with the SDK. You should have an application like this.
  4. The Above application is also required if you have to use ESB Toolkit in your development. This application has reusable orchestrations, pipelines which can be used for various scenarios. If you watch the send ports, you will see a SQL send port. By default it points to a local SQL server. If your ESBExceptionDb is installed in the local sql server, then you dont need to change this setting. However if it is a different instance then you need to change this value to the correct SQL instance. One more thing to note here is that, this send port has filters which subscribes to the ErrorReport and ESb property schema. See the below picture.
    So any messages that is sent to the BizTalk mesasge box with these fields promoted will be subscribed by the send port and it will be routed to the ESB Exception DB.
  5. The Next step is to add logic in your code to route messages to the message box, so that the messages are sent to the Exception Db.There are two ways to do it.
    A) Enable the “Failed routing for failed messages” in Receive Port for routing failed messages in the receive pipeline to the ESB. You can also tick the check box “Enable routing for failed messages” in the Send port advanced properties. When a message gets failed for any reason, it will be routed to the ESB framework. B) You can add logic in the orchestration to route messages to the ESB.
  6. To route the message to the ESB framework, the general principle is to first add a scope shape in your orchestration and add all your other shapes within the scope shape. Now add a Exception block to this shape and choose a System.Exception as the exception type.
    Add the two dlls are reference to the orchestration project.

    C:\Program Files (x86)\Microsoft BizTalk ESB Toolkit 2.1\Bin\Microsoft.Practices.ESB.ExceptionHandling.dll
    C:\Program Files (x86)\Microsoft BizTalk ESB Toolkit 2.1\Bin\Microsoft.Practices.ESB.ExceptionHandling.Schemas.Faults.dll

  7. Create a message in orchestration view and name it msgFaultMessage. Select the type as
    Microsoft.Practices.ESB.ExceptionHandling.Schemas.Faults.FaultMessage
  8. Add a construct shape and select the message that we have just added. Add a message assignment shape and the below code.

    msgFaultMessage = Microsoft.Practices.ESB.ExceptionHandling.ExceptionMgmt.CreateFaultMessage();
    msgFaultMessage.FailureCategory = “BizTalk Fault Message Testing”;
    msgFaultMessage.FaultCode = “”;
    msgFaultMessage.FaultDescription = ex.Message;
    msgFaultMessage.FaultSeverity = Microsoft.Practices.ESB.ExceptionHandling.FaultSeverity.Critical;

  9. Add a send shape to send the fault message. Add a Logical send port in the orchestration and configure the binding as Direct with “Routing between ports…” option selected.
  10. If you test your orchestration now(Add some logic to make the orchestration fail), A fault message will be created and it will be routed to the ESB framework. You can try opening the ESB portal and navigate to the Faults Menu and your fault will appear there. You can click the fault message to get the full details of the fault in the Fault Viewer page. It will give you the Exception Message.
  11. This still doesn’t show the actual message that was failed. It will gives us information about the fault. To route the failed message to the portal, you need to add one more line to the message assignment shape.

    Microsoft.Practices.ESB.ExceptionHandling.ExceptionMgmt.AddMessage(msgFaultMessage,msgInputMessage);
    where the msgInputMessage is the message that your orchestration receives from your source system.

  12. Re-build your code, deploy and see if you can now see the fault message appearing in the Faults Page.

In the next post we will see how we can replay the message with a sample application.

Shankar

Could not enlist Send Port ‘XXXXX’. Exception from HRESULT: 0xC00CE557

This happens in a typical scenario, where you have a Send port which has a filter on it(Messaging based interface). The binding file that you used had this filter section, but unfortunately it had CRLF characters in
the filter section.
There are lot of posts in the internet which explains the fact that the filter part of the binding file has to be modified. If you would have used Visual Studio to edit the binding files, then it would have alter the structure of the filter section.

I will suggest to make the entire filter section in one line.

However the mistake that I did is, even after correcting the problem and re-importing the binding file, I still saw the problem. This is because, BizTalk doesnt do a clean up of the bindings, if you import a binding file on top of it. So you need to Un-Deploy the interface, Deploy the interface again and import the binding file.

Shankar