ESB With Biztalk

• 'Enterprise Service Bus' is an architectural pattern that enables the development of 'Service Oriented Architectural' applications. That begs the question, What is 'Service Oriented Architecture'? While there is lot of depth to the concept, SOA at a very highlevel could be described as an architectural pattern that will allow you to package functionality in your software as a set of interoperable services in a loosely coupled way. It adds an additional layer to your application that consists of set of services that are provided by it. These services are exposed to consumers and they can use one or more of these services to implement business logic in their programs. For any application to fully implement SOA, it has to support these features :
  • Services should be loosely coupled with Operating Systems. Consumers had Providers should not be depended on each other or the operation system they are developed on to communicate with each other.
  • Services should be flexible. They need to be accessible through variety of transport protocols.
  • Services should be agile. Any change in business requirements should be quickly propagated and deployed without much hassle.
  • Services should be re-usable. They should be easily composed into new business processes.
  ESB is just one building block of SOA applications. Since ESB assists in the development of SOA applications, it has to align itself to the core principles of SOA namely loose coupling, flexibilty, re-usablilty and agility etc., apart from providing the following features.
  • Dynamic Resolution of endpoints for Routing.
  • Content based Routing.
  • Routing based on Rules Engine. ( In case of BizTalk, it's BRE).
  • Message Transformation using Maps ( In case of BizTalk, it's BizTalk Mapper).
  • Message Enrichment.
  • Service Aggregation.
  • Support for multiple transport protocols ( Http, Ftp, MSMQ, FILE etc.,)
  • Business Process Orchestration.
  • Business Activity Monitoring.
  • Message Logging.
  • Reports.
  • Alerts.
  • Exception Managment.
  • Support for Service Registries eg UDDI.
  • Itineraries(Mediation Policies), which consists of instructions to ESB regarding what components to use in the underlying engine, to process the message.
  
  For example, a typical BizTalk solution for consuming a webservice would be something like this. A schema would be exposed as a webservice by creating a virtual directory in IIS and creating the necessary receive ports and receive locations. Then, a send port is created and configured to subscribe to that receive location and send a WCF request to the webservice. For every webservice that needs to be consumed, you've got to create seperate receive location and a send port. Now, if the WSDL of any of the already configured webservices should change, then you've got to open the VS, import the new WSDL, compile the project, deploy it and restart the host instance. This is time consuming and ridiculous way of implementing SOA solution.
  
  ESB Toolkit comes with a bunch of webservices which helps in dynamically resolving endpoints, pullout required transformation files (maps) and process mediation policies. It also provides an 'ESB Management Console' Portal which gives various reports on Faults
  
  Envelope schema. An envelope schema is a special type of XML schema. Envelope schemas are used to define the structure of XML envelopes
  XML schema. define the structure of an XML instance message
  
  
  
  Repair and Resubmit Custom Exception Handler. This sample application demonstrates a custom exception handler that allows users to view a failed message process, edit the message, and resubmit it for processing. It includes examples of both strongly typed and loosely typed exception handlers.
  Message Persisting Custom Exception Handler. This sample application demonstrates a custom exception handler that stores the failed message in a disk file, serialized for rendering within an InfoPath template.
  BizTalk Failed Message Routing ESB Processing. This sample application demonstrates the ESB capabilities for capturing exceptions that occur during any stage of a BizTalk application. This can include failed routing or parsing messages of any type (such as PDF, flat file, binary, and XML).
  Itinerary On-Ramp. This sample application demonstrates sending messages to the Microsoft ESB Guidance Itinerary On-Ramp Web service. The sample ships with 13 predefined itineraries that describe various service executions, both in one-way and two-way (Request-Response) messaging patterns.
  JMS MQRFH2 Component. This sample application demonstrates the integration of IBM MQ Series systems with Microsoft ESB Guidance applications. This sample demonstrates how to receive JMS messages, access and edit the JMS headers within BizTalk, and send the messages back to JMS through MQ Series.
  Namespace Component. This sample application demonstrates the addition and removal of namespaces from a message. This sample implements four different uses cases that demonstrate various techniques for removing a namespace or dynamically building a namespace for the inbound document by providing various input parameters at design time.
  Transformation Service. This sample application allows external applications to submit a document to the ESB for transformation using a map deployed within BizTalk, without persisting the message in the Message Box database. Installing this sample deploys maps that the sample uses.
  Dynamic Resolution. This sample application resolves destinations and maps using various resolution methods, and it routes messages to the corresponding target. This sample demonstrates use of the Resolver and Adapter Framework through the dispatcher pipeline in both one-way and two-way (Request-Response) pure messaging scenarios. This sample implements 17 use cases demonstrating various resolution and transformation options.
  BizTalk Operations. This sample application accesses the BizTalk Operations Web service to extract information about the BizTalk environment, currently installed applications, and other BizTalk artifacts. It can also retrieve messages passing through the BizTalk orchestrations and from the BizTalk Message Box database.
  Resolver Service. This sample application executes the methods of the Resolver Web service to demonstrate how you can resolve BizTalk maps and endpoints. It shows how you can use the Resolver Web service in your own applications.
  UDDI Service. This sample application accesses the currently configured UDDI server through the UDDI Web service to extract information on registrations stored there and to publish new registrations. The sample demonstrates all of the varied methods available in the UDDI Web service.
  Scatter-Gather. This sample application illustrates how to apply ESB itineraries to send requests to services; dynamically resolve and execute multiple Web service requests and maps; aggregate all their responses; and forward the response to another endpoint defined within the itinerary.
  How the Sample Works
  The message you submit activates the EAIProcess orchestration. As the EAIProcess orchestration processes the message, it attempts to divide 1 by the unit price. Because the unit price is zero, a divide-by-zero exception occurs. Code in the event handler of the orchestration catches this exception and creates a fault message. The order quantity in the message is greater than 10, so the business logic dictates that this exception has a FaultCode field value of 1000.
  The EAIProcess orchestration then publishes the fault message to the BizTalk Message Box through a direct-bound port, and the orchestration ends.
  A custom fault handler orchestration named EAIProcessHandler, which subscribes to messages with a FaultCode field value of 1000, picks up the new fault message. The code in the orchestration creates the "Denied" message and the InfoPath file, and then it places this into the EAIProcessHandler.PostDecline and EAIProcessHandler.RepairSubmit folders ready for human intervention.
  Providing architectural and capability improvements over the previous “ESB Guidance”, the BizTalk ESB Toolkit 2.0 is a collection of tools and libraries that extend BizTalk Server 2009 capabilities of supporting a loosely coupled and dynamic messaging architecture.
  
  
  ( BizTalk ESB Toolkit 2.0 provides key building blocks that are required for implementing a comprehensive SOI including:
  · Endpoint run-time discovery and virtualization
  · Loosely coupled service composition
  · Dynamic message transformation and translation
  · Dynamic routing
  · Centralized exception management
  · Quality of service
  · Protocol transformation
  · Extensibility
  

Zombie

• What is a zombie?
  A zombie message is a message that was routed to a running orchestration from the messagebox and was "in flight" when the orchestration ended. An "in flight" message is a message that has been routed to a service instance and so is in a messagebox queue destined for the service instance. Since the message can no longer be consumed by the subscribing orchestration instance, the message is suspended and marked with a ServiceInstance/State value of "Suspended (Non-resumable)".
  A zombie service instance is an instance of an orchestration which has completed while a message that was routed to the orchestration instance from the messagebox was still "in flight". Since the orchestration instance has ended, it cannot consume the "in flight" messages and so is suspended and marked with a ServiceInstance/State value of "Suspended (Non-resumable)".
  The occurrence of zombies typically falls into one of the following categories:
  1. Terminate control messages – The orchestration engine allows the use of control messages to cancel all currently running work in a specific orchestration instance. Since the control message immediately halts the running orchestration, zombie instances are not unexpected. A number of Human Workflow related designs tend to use this mechanism as well as some other designs.
  2. Parallel listen receives – In this scenario the service instance waits for 1 of n messages and when it receives certain messages it does some work and terminates. If messages are received on a parallel branch just as the service instance is terminating, zombies are created.
  3. Sequential convoys with non-deterministic endpoints – In this scenario, a master orchestration schedule is designed to handle all messages of a certain type in order to meet some type of system design requirement. These design requirements may include ordered delivery, resource dispenser, and batching. For this scenario, the tendency is to define a while loop surrounding a listen with one branch having a receive and the other having a delay shape followed by some construct which sets some variable to indicate that the while loop should stop. This is non-deterministic since the delay could be triggered, but a message could still be delivered. Non-deterministic endpoints like this are prone to generating zombies.
  When a zombie service instance is suspended in Microsoft BizTalk Server 2006 the following error message is generated:
  Copy
  0xC0C01B4C The instance completed without consuming all of its messages. The instance and its unconsumed messages have been suspended.