Integrating MQ FTE with WebSphere Process Server/WESB

Ivan Smirnov, Senior Consultant

WebSphere MQ File Transfer Edition (known simply as MQ FTE) is a robust managed file transfer solution built on WebSphere MQ transport backbone. Product page is here: http://www-01.ibm.com/software/integration/wmq/filetransfer/index.html

This product is a recent addition to the storied and extremely stable WebSphere MQ – it came out after MQ version 7. I was excited to hear about its release because it creates new possibilities in file transfer that FTP, SCP and the like never delivered – like reliability and tolerance of transient network failures. Immediately a question arose: how can we take advantage of these new capabilities in IBM’s middleware. What is the best way to integrate IBM’s premier integration platform, WebSphere ESB (and by extension, WebSphere Process Server) with MQ FTE?

Integration between WESB and MQ FTE will occur on 2 ESB boundaries: module exports that receive files from MQ FTE and module imports that send files to a remote destination via MQ FTE. I argue that the simplest integration approach is also often the best.

On the SCA export (send files via MQ FTE) side, we will configure MQ FTE file transfer using the tool’s native interface (command line or GUI). We then will use Flat File adapter in outbound mode to create a file on local file system where it will be picked up by MQ FTE. Protection from incomplete file pickup is provided by using staging directory, which is built into Flat File adapter.

The following alternatives are possible:

1) Construct the whole MQ FTE messages on the fly in WESB (for instance, by engineering an MQ FTE import binding)

2) Put file on file system the same way, using Flat File adapter, but initiate transfer by putting a request message on MQ FTE agent queue.

The first alternative approach requires duplicating a lot of code that is already in MQ FTE while providing minimal savings in terms of disk I/O. It is not a worthy alternative.

The second alternative approach is close to the original design. However, there is no need to predefine file transfer – this will be done by sending an XML message to administrative queue of MQ FTE. Its benefit is zero administration effort, but it requires some upfront custom development.

On the SCA import (receive from MQ FTE) side, we can configure MQ FTE to transfer files to a local WESB file system. Files will be picked up by Flat File Adapter in inbound mode. MQ FTE has built in protection against picking up incomplete files.

As an alternative, it is certainly possible to receive MQ messages in WESB instead of letting MQ FTE create files on file system. This alternate solution will require complex code largely duplicating existing MQ FTE functionality to extract file payload from MQ messages and handle file-level acknowledgement and auditing. The only upside would be avoiding disk I/O by skipping writing file to file system by MQ FTE and reading it in by Flat File adapter. There is not much value in this, since at such high volume WESB will reach capacity much sooner then disk I/O becomes a bottleneck. This alternate solution is all cost and no benefit.

Conclusion: when integrating MQ FTE and WPS (WESB), simplest solution is very reasonable and will work best in many situations.

Ivan Smirnov is a Senior Consultant at Prolifics with extensive hands-on experience with the WebSphere family of products (including WebSphere Application Server and Process Server, WebSphere Studio/Rational and WebSphere MQ), Tivoli security offerings (including Tivoli Identity Manager and Tivoli Access Manager for e-business), DB2, XML and Web Services. With strong technical skills both in development and administration, as well as deep troubleshooting skills, Ivan handles aspects of implementation installation, configuration, securing and tuning/troubleshooting to development and architecture within a J2EE environment. He also possesses key Application Server migration skills and has helped several customers’ transition to the WebSphere platform from other J2EE platforms.

Human Centric Processes: A Fresh Perspective

Anant Gupta, SOA Practice Director

With BPM gaining so much attention, any workflow product worth its salt provides a lot of flexibility when it comes to definition and execution of processes specifically when it comes to human centric processes, both structured and unstructured. This flexibility however comes at a cost, cost both in terms complexity and clarity of definition of processes and performance in runtime. Also, making changes to workflows always remains a challenge. Simple changes like making steps execute in parallel instead of sequentially are a nightmare to implement and are most often simply not taken up.

It does not really need to be that complex if we take a somewhat hybrid approach to structured and unstructured processes. There are certain steps that have to happen in a particular order and then there are others that can be done at any time if certain pre-requisites are met. We have to start thinking of processes as steps where each step has a set of pre-requisites. These pre-requisites could be related to the steps in the workflow or can be content / context related. Instead of workflows, human centric processes can be defined as a set of steps with pre-requisites. For eg, a step X can be executed only when Step A and Step D are completed, when the status of a document is accepted and if the flow was initiated by a gold client.

This approach should be extended with user experience simulation to come up with the typical paths the workflow will take and this should be visually depicted. This will provide both clarity in terms of the definition and flexibility to change the workflows in an extremely simplified fashion.

Call it pre-requisites or workflow rules, the idea is to provide extreme dynamicity to human centric processes, an area that has not been addressed by most of the so called dynamic BPM products which cater almost always to system centric processes.

Anant Gupta was recently named the SOA Practice Director at Prolifics after serving as a Senior Business Integration and J2EE architect Anant has with extensive experience in IBM's SOA software portfolio and specializes in delivering business integration and business process management solutions. He has worked for major clients in the banking, insurance, telecommunications and technology industries.

Planning and Scheduling SOA/BPM Development - DO NOTS!

Jonathan Machules, Technology Director

As the momentum and understanding of BPM and SOA has increased, the projects have followed. IBM WebSphere Process Server and ESB (WPS/WESB) are common products that organizations start with when moving towards BPM/SOA/Web services. Many organizations are new to this type of SDLC. This discussion is in the context of my experience on WPS/WESB projects and certainly can be applied to other workflow and ESB products/technologies.

DO NOT:

• Delay Data Model and Data Design efforts
• Plan integration validation between systems/apps/services scheduled toward the end of the project
• Assume a Sr. Developer with no experience on WPS/WESB will design/develop a functioning application
  

DO NOT Delay Data Model and Data Design efforts. The reality here is that in development there are more than likely to be changes to the data model. Some will impact your Service Message Interfaces and create a domino effect on the Consumers of that data in design and development. Try to plan as much up front as possible and reduce the costs of changes the must happen later in the SDLC.

DO NOT plan integration validation between systems/apps/services toward the end of the development SDLC. On one recent project the customer was not familiar with WPS/WESB or integration projects in general. They planned all their integration testing toward the end of the SDLC in the Testing Phase. I am not saying integration testing shouldn’t be done in the testing phase but it should NOT be planned at the end of the SDLC. A common project plan will include a ‘Vertical Slice’, ‘Prototype’, ‘Wire-frame’ or whatever term you are familiar with, the has goal to validate the integration of the various systems early on in the SDLC.

DO NOT assume a Sr. Developer with no experience on WPS/WESB will design and develop a functioning platform. WPS/WESB are enterprise platforms that have multiple layers of technologies (e.g. Java, JEE, BPEL, WS, XML, XSLT, etc…). As a proud successful Sr Developer you may very well be able to create an application on these platforms that functions in non-production environment. However, there are number of nuisances that impact performance that should be addressed by design patterns depending on the requirements. Large business is one concern that comes to mind. Acceptable object size is dependent on business transaction volume, CPU Architecture, RAM, HEAP and other dependencies. Design patterns to deal with large business objects can be applied thus giving you better performance.

WPS/WESB is a product I’ve worked with extensively. It has seen major enhancements and improvements on usability/consumability but this doesn’t mean anyone can create a well functioning app.

Jonathan Machules first joined Prolifics as a Consultant, and is currently a Technology Director specializing in SOA, BPM, UML and IBM's SOA-related technologies. He has 12 years experience in the IT field — 2 of those years at Oracle as a Support Analyst and 10 years in Consulting. Jon is a certified IBM SOA Solution Designer, Solutions Developer, Systems Administrator and Systems Expert. Recent speaking engagements include IMPACT on SOA End-to-End Integration in 2007 and 2008, and SOA World Conference on SOA and WebSphere Process Server in 2007.

Building an Enterprise Application Integration Solution

Rajiv Ramachandran, Practice Director, Enterprise Integration / Solution Architect

Having an integration infrastructure that connects all enterprise systems to one another and provides seamless and secure access to customers, partners, and employees is the foundation of a successful enterprise. I have been involved with a lot of customers discussing their EAI architectures. More often than not, I have noticed that the approaches considered to implement such an architecture are not complete and do not provide the benefits that may be achieved with a well connected enterprise. In this blog entry, I would like to highlight aspects that need to be considered to build an end-to-end integration solution. (Note: This blog entry will not get into the details on how to implement each of these areas, which would result in me writing a book J.)

1. Connectivity – Avoiding point-to-point connectivity and ensuring that you have loosely coupled systems is key to ensuring that your EAI architecture is flexible and can scale.Use an ESB as the heart of you EAI architecture and ensure that your ESB has support for all major protocols (HTTP, SOAP, JMS, JCA, JDBC, FTP, etc.) and comes with adapters for common enterprise applications like SAP, Oracle, Siebel, PeopleSoft, etc.
   
   
2. Patterns - Build a pattern based integration solution. The following is an excellent paper that outlines some of the common patterns used in the integration space:http://www.ibm.com/developerworks/library/ws-enterpriseconnectivitypatterns/index.html
   
   
3. Data - Data is of great significance when it comes to integration. Different systems have different data formats and there are common items to consider that can help you deal with these differences:




• Define canonical data formats and ensure that you have mapping from application specific formats to canonical formats. Understand the various data formats that exist in your enterprise today and evaluate what it will take to map and manage complex data formats.


• EDI data is common in many enterprises and will require special handling.


• Define a strategy for handling reference data, how lookups can be done against this data and how reference data can be maintained.


• Define rules around both syntactic and semantic validation of messages. Do not over do validation as you will pay a price when it comes to performance. Be judicious in what you want to validate and where.




4. Monitoring - An aspect that is often overlooked when building integration solutions is monitoring. Ensure that you have the right framework(s) in place to monitor your integrations and service components. You will need the ability to monitor every protocol the ESB supports. You also need to couple monitoring with notifications and error handling. You will need a strategy for auditing your messages. Again, as in the case with validations, be judicious in what you want to audit and when. There is a price to pay. Define a centralized service for auditing requirements and ensure that all integration components make use of this service.
   


5. Security - Security is the most critical part of implementing an integration solution. I have to say that most customers realize that security is important but sometimes, because of a lack of expertise in how to correctly secure their solution, the end result is a solution that is way more costly and in fact less secure than desired. Some considerations are:




• Define your security requirements – authentication, authorization, encryption, non-repudiation, etc.


• Decide what aspect(s) can be supported by transport level security and when you need message level security.


• Decide where hardware components can be used to better implement security than software components.


• Use open standard protocols so that you can easily integrate with different systems – both internal and external.





6. Service Oriented Integration – With the adoption of SOA, one of the key architectural models used for integration is to connect to service interfaces that are exposed by various systems. With this architecture, it is also now possible to choose what service / functionality you use dynamically at runtime. Another aspect that SOA has brought to the integration world is a policy-driven approach to integration – that is, data that is being passed to a system or between systems is used to check what policy needs to be applied at runtime to determine what service to use and what actions to perform (auditing, validation, encryption etc.). Integration coupled with SOA introduces another component into the EAI architecture – a registry and repository where services and policies are cataloged and can be used to bring about dynamic, policy-driven behavior.

This confluence of pattern based connectivity, data handling, monitoring, security, and service-oriented integration can provide you with a well-connected enterprise that can respond quickly to changing business needs.

Rajiv Ramachandran first joined Prolifics as a Consultant, and is currently the Practice Director for Enterprise Integration. He has 11 years experience in the IT field — 3 of those years at IBM working as a developer at its Object Technology Group and its Component Technology Competency Center in Bangalore. He was then an Architect implementing IBM WebSphere Solutions at Fireman’s Fund Insurance. Currently, he specializes in SOA and IBM’s SOA-related technologies and products. An author at the IBM developerWorks community, Rajiv has been a presenter at IMPACT and IBM's WebSphere Services Technical Conference.

Why SOA Governance?

Rajiv Ramachandran, Practice Director, Enterprise Integration / Solution Architect

One of the buzzwords that followed the introduction of SOA was “Governance”. It was interesting to see how every aspect of a new project initiative now began to be tagged with this word. All of a sudden there were - project governance, architectural governance, infrastructure governance and so on. The real essence of what “SOA Governance” was or why “Governance” was important in the context of an SOA was lost.

I am not denying that governance is essential in every aspect of business and IT. But what I want to focus on this blog is about SOA Governance.

Services have been there all along in the technology space but the advancement in SOA and its adoption started when both customers and vendors came together to define a standard way to describe a service. It then became possible to implement this description in a programming language of choice, be able to deploy the service across diverse platforms and still be able to communicate across platform and language boundaries. With this form an SOA revolution, reusing services became much easier and with reuse came a unique set of challenges.

My business depends on service that I

• Did not write,
• Do not own,
• Cannot control who will make changes to it and when,
• Don’t know whether it will provide me with the qualities of service that I desire
  

And if do get to own the service, now I had to pay to get the service built and others get it for free.

What a SOA Governance model does, is bring uniformity and maturity in defining Service Ownership, Service Lifecycle, Service Identification & Definition, Service Funding, Service Publication & Sharing, Service Level Agreement etc. and thus provide a solution to otherwise what would have become a Service Oriented Chaos.

So next time when you talk about SOA Governance think about some of the above defined areas that pertain to an SOA and how you can align – Process, People and Products to achieve an SOA Governance solution that ensures that your SOA provides real business value

In the next set of blog entries I will focus on how IBM WebSphere Registry & Repository product helps with SOA Governance.

Rajiv Ramachandran first joined Prolifics as a Consultant, and is currently the Practice Director for Enterprise Integration. He has 11 years experience in the IT field — 3 of those years at IBM working as a developer at its Object Technology Group and its Component Technology Competency Center in Bangalore. He was then an Architect implementing IBM WebSphere Solutions at Fireman’s Fund Insurance. Currently, he specializes in SOA and IBM’s SOA-related technologies and products. An author at the IBM developerWorks community, Rajiv has been a presenter at IMPACT and IBM's WebSphere Services Technical Conference.

IBM CloudBurst Part 5: IBM CloudBurst Offering in Comparison to other Cloud Offerings

Samuel Sharaf, Solution Director West Coast

In the previous blog entries we discussed the IBM offering for establishing private clouds i.e. the CloudBurst device and its configurations. In this final part of the blog series, I will discuss the other cloud computing options, specifically open source and how they compare to the IBM offering.

Besides IBM, the other big players in the cloud computing domain are Amazon, Sun (Oracle), Google, SalesForce, etc. The Amazon EC2 (elastic compute cloud) offers a very flexible cloud computing solution for public clouds; offering both open source as well as vendor specific technologies. Before we start the comparison of the offerings; let’s revisit the architectural service layers of cloud computing. The architectural services model of cloud computing can be viewed as a set of 3 layers viz. applications, services and infrastructure.

Layer 1 – SAAS (Software as a service) e.g. Sales Force CRM application
Layer 2 – PAAS (Platform as a service) e.g. XEN image offered by Amazon
Layer 3 – IAAS (Infrastructure as a service) e.g. e.g. Amazon EC2 and S3 services

The IBM CloudBurst device basically offers all 3 layers of cloud computing in a single device for establishing a private cloud. As far as public cloud is concerned, IBM offers several SAAS services e.g. IBM Lotus Live Notes, Events, Meetings, and Lotus Connections etc.

So how does an open source cloud computing model look like? At the platform layer level, companies like Sun Microsystems are offering solutions built around open source Apache, MySQL, PHP/Perl/Python (AMP) stack. Open source communities are very actively developing solutions catered for cloud computing, in fact, cloud computing is acting as a catalyst for the development of agile new open source products like lighttpd (an open source web server), Hadoop, the free Java software framework that supports data-intensive distributed applications; and MogileFS, a file system that enables horizontal scaling of storage. However, a cloud computing solution based on open source is yet to be adopted by early adapters of cloud computing. As Tim O’Reilly, CEO of O’Reilly Media, and others have pointed out, open source is predicated on software licenses, which in turn are predicated on software distribution — and in cloud computing, software is not distributed; it’s delivered as a service over the Web. So cloud computing infrastructures and the modifications to the open-source technologies that enable them tend not to be available outside the cloud vendors’ datacenters, potentially locking their users in to a specific infrastructure.

Although the software stacks that run on top of these cloud computing infrastructures could be predominantly open source, the APIs used to control them (such as those that enable applications to provision new server instances) are not entirely open, further limiting developer choice. And cloud computing platforms that offer developers higher-level abstractions such as identity, databases, and messaging, as well as automatic scaling capabilities (often referred to as “platform as a service”), are the most likely to lock their customers in. Without open interfaces linking the variety of clouds that will exist — public, private, and hybrid — practical use cases will be difficult or impossible to deliver with open source technologies.

IBM’s offering for setting up private clouds, though completely vendor based, does offer a one stop solution which is scalable and dynamic.

Samuel Sharaf is a Solution Director at Prolifics on the West coast with real world customer expertise with Portal implementations, Dashboard, Forms and Content Management. Sam also has expertise with migrating applications from non-IBM platforms to IBM WebSphere Application and Portal Servers.

Wading Through Requirements on SOA/BPM/BAM Projects!

Jonathan Machules, Technology Director

On a recent project I was heavily involved in the requirements phase of SOA/BPM/BAM project. Requirements elicitation in its own right can be a challenge. On large projects with large teams and un-clear methodologies, approaches and tools/technologies, the issue is exacerbated.

Now throw in this whole BAM concept and things really get confusing. What I found is that BAM requires a customized approach for requirements elicitation. BAM requirements are all about the performance metrics of the business. But it is very easy for the Business Analysts and Business Owners to confuse the high level and low level requirements with design. It seems this is more of an issue with BAM due to the nature of the requirements. However, the Business Analysts and Business Owners need to focus on the scope of the requirements and the dimensions, NOT the design. When the business side elaborates on how they want to measure their business, there is a delicate balance between the calculations used to create the metrics and KPIs (Key Performance Indicators) themselves and the calculations, measures and metrics used to fulfill the requirement. Many times the business side not only gives you a description of the KPIs that they want to track, they also start to tell you where to gather data and how to calculate. This often imposes restrictions on the design that is the responsibility of the S/A and Designer.

In the end, we found that we had created a specialized approach for this project. We incorporated some specific BAM Specification and Design documents to help everyone understand the process and supply the necessary information for the BAM design team.

Some Definitions:

BAM (Business Activity Monitoring) - Gives insight into the performance and operation of your business processes real-time.

KPIs (Key Performance Indicators) - Significant measurements used to track performance against business objectives.

Monitor Dimensions - Data categories that are used to organize and select instances for reporting and analysis.

Requirements Dimensional Decomposition – The narrowing of requirements to the point of being clear, concise and unambiguous.

Metric - A holder for information, usually a business performance measurement. A metric can be used to define the calculation for a KPI, which measures performance against a business objective.


Jonathan Machules first joined Prolifics as a Consultant, and is currently a Technology Director specializing in SOA, BPM, UML and IBM's SOA-related technologies. He has 12 years experience in the IT field — 2 of those years at Oracle as a Support Analyst and 10 years in Consulting. Jon is a certified IBM SOA Solution Designer, Solutions Developer, Systems Administrator and Systems Expert. Recent speaking engagements include IMPACT on SOA End-to-End Integration in 2007 and 2008, and SOA World Conference on SOA and WebSphere Process Server in 2007.

IBM CloudBurst Appliance Part 4: Medium and Large Scale Configurations with Practical Scenarios

Samuel Sharaf, Solution Director West Coast

In part 4 of the blog series on IBM CloudBurst device we will discuss basic configurations of the CloudBurst device. Figure 1.0 below shows the CloudBurst device in a medium size configuration. Note that the key interfaces of the device are with the customer storage and Ethernet networks. As we discussed in the previous blog entries, the CloudBurst device is highly modular and can be configured according to customer requirements. The physical configuration showed here serves as an example.

The HS22 blade configuration in the middle is a good representation of the blade center system which starts at 4 blades and can grow to 14 blades to accommodate larger loads. The HS22 blades have embedded flash memory which come pre loaded with ESXi images. Besides that there is no other storage on the HS22 blades. The DS3400 (bottom of the figure) represents storage attached to the blade center and can be grown if required. To connect to their network, the customers can order the device with 1Gb switch module or can opt for 10Gb if its supported on their network. Important to note also is the Fiber connectivity (4 Gbps) offered by the device to connect with the storage blades (DS3400) and to the customer storage network. The x3650 M2 is the management server, which provides management interface to the CloudBurst device administrators.

The CloudBurst device can be easily scaled to accommodate larger practical scenarios. Figure 2.0 below provides a representation of the CloudBurst device in a larger configuration.



Note that it looks essentially the same as the medium configuration. The key difference is another blade center array added for more capacity. Note that because of the availability of high speed modular connectivity the two blade centers are connected both at the network level and storage level. The management server x3650 provides a singular management interface for both blade centers/storage units.

Let’s go into more detail on the storage part of the CloudBurst device, since this serves as the customer image repository (VM ware images) and the management software of the virtual machines. By default the device comes with the DS3400 storage unit which has a capacity of 5.4 terabytes with a 4.5 terabyte of usable space. This come fully configured with RAID 5 and 1 hot swap spare drive (in case of failure of the main storage drive). As shown in both figure 1.0 and 2.0 the storage can be expanded with EXP3000 storage expansion units, each having 5.4 terabyte of capacity. So how is the storage typically used? In practical scenarios, the VM management software takes up to 200 Gb space. A 1.2 Tb space can be taken up by customer supplied VM ware images, assuming 100 images at 12 GB each. Another 1 TB of space can be used to back up the customer supplied VM images. Because of the built in highly modular expansion storage mechanism, it is easy to scale for a larger cloud.

In the next part of this series, we will discuss how IBM offering compares to the other competitive offerings including open source cloud computing solutions…stay tuned.

Samuel Sharaf is a Solution Director at Prolifics on the West coast with real world customer expertise with Portal implementations, Dashboard, Forms and Content Management. Sam also has expertise with migrating applications from non-IBM platforms to IBM WebSphere Application and Portal Servers.

BPM Blueworks - Collaborating in the Cloud

Devi Gupta, VP of Marketing

I saw a demo of IBM’s new offering called BPM BlueWorks and was really impressed. It has many capabilities, but I was most excited by the ability to collaborate within the cloud using BPM tools.

As a systems integrator we often help our customers to assess the benefits of a BPM initiative for their organization. We pride ourselves on being able to help the business users within an organization to "simulate and visualize" their end results, dashboards, metrics, etc. before the IT team even begins implementation. A powerful and important step in the process that correctly sets expectations. With BPM BlueWorks I think we will be able to collaborate with the business analysts and users even more effectively… so to conduct workshops where we can create strategy maps, process maps and capability maps together, all within the cloud. We can import information in from, for instance, PowerPoint, modify within the cloud and export artifacts to Modeler.

I look forward to learning more as this rolls out.


Ms. Devi Gupta directs the market positioning for Prolifics and helps manage the company’s strategic alliance with IBM. Under her guidance, Prolifics has made the critical transition from a product and services company to becoming a highly reputable WebSphere service provider and winner of several awards at IBM including the Business Partner Leadership Award, Best Portal Solution, IMPACT Best SOA Solution, Rational Outstanding Solution and Overall Technical Excellence Award.

IBM CloudBurst Appliance Part 3 – Logical Architecture and Physical Topology Scenario

Samuel Sharaf, Solution Director West Coast

In part 3 of the CloudBurst blog series, I will discuss the logical architecture of the appliance and a basic physical topology demonstrating a practical scenario. In part 2, I mentioned that the CloudBurst device is modular and actually consists of multiple hardware and software components. Let’s start with the high level logical architecture of the appliance – refer to figure 1.0 below. The end users (cloud users or administrators) interact with a browser-based interface to either request virtual resources or manage the cloud.


Figure 1.0 – Logical Architecture

A quick glance at the logical architecture reveals that the key components come from the Tivoli, WebSphere and VMware stacks. The first block in the overall stack is the IBM HTTP Server which handles the user requests. The HTTP Server forwards the user requests to the Tivoli Provisioning Manager and the WebSphere software stack, which form the heart of the overall architecture and run the IBM cloud-computing software. The HTTP Server requests are handled by the TPM and the stack of Cloud computing software that is running on the WebSphere Application Server. The Tivoli Provisioning Manager makes use of the pre-built automation workflows and set of scripts which actually drive the self-service model which serves the VMs requested by the end user. The Cloud computing software also handles any errors and exceptions. The data of the TPM and the WAS Cloud computing software (blue code) is stored in the DB2, while user info is stored in the LDAP. The TPM deployment engine acts as a broker and fulfills the end user requests. Its running set of automation packages and scripts.

The TPM deployment engine makes automation and web service calls to the VMware virtual center. The VM VC manages all the hypervisors and virtual machines which are running on base hardware. The VM VC interacts with physical machines. The physical machine is running a hypervisor – a light-weight process that runs the actual VMs which the end users will be accessing. The VC also provisions and performs customization on VMs.

ITM provides monitoring of the VMs and a dashboard for Cloud Admins to monitor the VMs and the VMware stack.

Let’s now consider a physical topology demonstrating a practical usage of the CloudBurst device. Figure 2.0 below illustrates how different users interact with the ‘Cloud’ to request resources. The ‘Resource Cloud’ is an abstraction representing the CloudBurst appliance, which can be used to create a private cloud. The resources in this case are VMware images which are provided by the customer and cataloged and stored in the image repository. A typical example would be developers working on WebSphere Portal development requiring VMware images which provide a rational development environment along with a Portal Server test instance.



Figure 2.0 – Logical View/Scenario

The figure also illustrates the CloudBurst browser-based interface, which provides provisioning and management of the Cloud Resources. There are two basic categories of users: end users (software developers and quality assurance engineers) and administrators. The Resource Cloud provides a self-servicing platform for end users to request and use resources. The administrators use the browser-based interface to manage resources and users.

In the next part of this series, we will explore medium and large scale deployment topologies of the CloudBurst appliance, scalability, network interfaces and add-on storage devices.

Samuel Sharaf is a Solution Director at Prolifics on the West coast with real world customer expertise with Portal implementations, Dashboard, Forms and Content Management. Sam also has expertise with migrating applications from non-IBM platforms to IBM WebSphere Application and Portal Servers.

Service Gateway Pattern in WebSphere Enterprise Service Bus (WESB) 6.2

Rajiv Ramachandran, Practice Director, Enterprise Integration / Solution Architect

On of the new patterns that have been introduced in v6.2 of WESB is the Service Gateway Pattern. There are two implementations of this pattern – A Static Gateway and A Dynamic Gateway. In this blog entry I will focus on my experiences on using the Dynamic Service Gateway pattern.

One of the basic requirements that I expect in an ESB is the ability to introduce an ESB in the middle between a client and a service (without impacting either the client or the service) and be able to do provide value added services like logging, monitoring, auditing etc. This is exactly what we are able to accomplish with the Dynamic Service Gateway pattern. This gateway pattern is implemented as a mediation component. The interface for this mediation component has 2 operations, a request operation and a request response operation. Both these operations accept xsd:anyType (they are not bound to the type definitions exposed by any particular service). The reference partner for this mediation also maps to the same interface. The reason this pattern is dynamic, is because there is no coupling to services at compile time. You can dynamically set the service endpoint at runtime and invoke that service.

However what needs to be understood is that this pattern implementation works only for SOAP 1.2 based JAX WS Web Services. The reason why this restriction exists is because in JAX-RPC the invocation was RPC based -> a particular web service operation had to be specifically invoked. In JAX-WS the invocation is more messaging based. There is no specific operation invocation required. The following article series provide an in depth analysis of the differences between JAX-RPC and JAX-WS.
http://www.ibm.com/developerworks/library/ws-tip-jaxwsrpc4/index.html

The following diagrams show a sample implementation of this pattern for monitoring and auditing service invocations.


Assembly Diagram

Mediation Flow

Look up Table

Audit Table

Rajiv Ramachandran first joined Prolifics as a Consultant, and is currently the Practice Director for Enterprise Integration. He has 11 years experience in the IT field — 3 of those years at IBM working as a developer at its Object Technology Group and its Component Technology Competency Center in Bangalore. He was then an Architect implementing IBM WebSphere Solutions at Fireman’s Fund Insurance. Currently, he specializes in SOA and IBM’s SOA-related technologies and products. An author at the IBM developerWorks community, Rajiv has been a presenter at IMPACT and IBM's WebSphere Services Technical Conference.

Dynamic Routing using WebSphere Enterprise Service Bus (WESB)

Rajiv Ramachandran, Practice Director, Enterprise Integration / Solution Architect

One of the most common requirements I have heard from customers who are using an ESB is to say that I want to dynamically route information to several of my end systems. I don't want to change my code when i introduce a new system to the mix that requires the same data.

Careful analysis is required to implement a solution to this requirement as there are two parts to this.

1. The ability to invoke an end point dynamically (the requirement statement is crystal clear about that).
2. The second part is how do I define my routing logic? How do i describe it? How can i change it at runtime?

Setting the endpoint header dynamically on the endpoint is supported by the Service Component Architecture (SCA) framework and therefore we do have an easy solution to the first part of the requirement. However there is no standard primitive or a defined approach in the WESB product to achieve the second aspect of the dynamicity requirement.

There are multiple solutions to this

1. We can use WebSphere Business Fabric (WBSF) to provide this capability.
2. Or we can build custom solutions to achieve this goal.

One such solution approach that we have used is to use Business Rules component in WebSphere Process Server (WPS) to be a 'Routing Controller' and define business rules to describe the routing logic. The out of the box capability of business rules in WPS provides us with an editor to edit to these rules and dynamically update them at runtime. The pros of this approach are that we don't have to custom code any of the logic and we are able to use the out of the box rules component. However, since the business rules components are part of WPS, this solution is not applicable for pure WESB customers.

Rajiv Ramachandran first joined Prolifics as a Consultant, and is currently the Practice Director for Enterprise Integration. He has 11 years experience in the IT field — 3 of those years at IBM working as a developer at its Object Technology Group and its Component Technology Competency Center in Bangalore. He was then an Architect implementing IBM WebSphere Solutions at Fireman’s Fund Insurance. Currently, he specializes in SOA and IBM’s SOA-related technologies and products. An author at the IBM developerWorks community, Rajiv has been a presenter at IMPACT and IBM's WebSphere Services Technical Conference.

IBM CloudBurst Appliance Part 2 – What’s inside the Box?

Samuel Sharaf, Solution Director West Coast

In the first blog entry on this topic, I talked about IBM offerings for cloud computing (i.e. the CloudBurst appliance) and discussed its high level overview and capabilities. In this second part I will discuss the value proposition of the device and explore the technical architecture of the device - mainly what it consists of, or what you are buying when you spend more than 200k to buy one.

So again, what is CloudBurst? CloudBurst is a service delivery platform which consists of prepackaged and pre-configured servers, storage, networking, and software needed to set up a private cloud. These resources (hardware and software) can be provisioned and enabled to provide virtual server resources for application development, testing, and other activities that normally have to wait on physical hardware to be procured and deployed.

An important question to ask here is, what value this device is going to provide. In today’s market customers are largely investing in two categories of solutions, one which provides efficiency in the data centers across their IT organizations and secondly, solutions which help them differentiate from their competitors. Typically, IT data centers spend 30% to 50% of resources in developing, testing and configuring environments.

Some time it takes months to establish data center environments and configure them to be consistent with the requirements. With CloudBurst, a developer can log into a self service portal, select resources required and timeframe, select an image to provision from the service catalog, and be ready to go in minutes as opposed to months. So the idea is that if the efficiency of the data centers improve by the use of CloudBurst device, the available resources can focus and spend time on innovating products which differentiate them from competition.

So how does CloudBurst device accomplish this? To answer this, we have to look inside the box. Unlike DataPower devices which are hardware devices built for specific purposes (e.g. xml acceleration, security, integration etc), CloudBurst actually consists of several different hardware devices/components which are pre built and packaged for specific architecture needs and cloud requirements. A typical CloudBurst device (base configuration) consists of:

• 1 42U rack
• 1 3650M2 Systems Management Server
• 1 HS22 cloud management blade
• 1 BladeCenter H chassis with redundant Ethernet and Fibre Channel switch modules
• 3 managed HS22 blades
• DS3400 FC attached storage

Some important things to note here are, the 3650M2 management server hosts the pre packaged software stack (discussed below) and the HS22 blade hosts the IBM Blue Cloud computing software. The 3 managed HS22 blades hosts the client provided VM ware images, which can be cataloged for on demand provisioning. I won’t go into details of each of the individual hardware components (networking, storage etc) here in this blog as their description can be found on IBM website.

The device also comes pre packaged with IBM software, which includes:

• Systems Director 6.1.1 with BOFM, AEM; ToolsCenter 1.0; DS Storage Manager for DS4000 v10.36; VMware VirtualCenter 2.5 U4; LSI SMI-S provider for DS3400
• VMware ESXi 3.5 U4 hypervisor on all blades
• Tivoli Provisioning Manager v7.1
• DB2 ESE 9.1; WAS ND 6.1.0.13; TDS 6.1.0.1
• Special purpose customized portal and appliance wizard that enables client portal interaction
• Tivoli Monitoring v6.2.1
• OS pack
  

Note that it includes third party software from VMware (virtual center and ESX hypervisor) and IBM cloud computing software which makes use of Tivoli provisioning software components. An interesting point to note is that even though the CloudBurst device consists of several hardware and software components, it is sold, delivered and supported as a single product.

In Part 3 of this blog series, I will discuss the logical architecture of the CloudBurst device and a practical scenario which demonstrates its usage in a real client environment.

Samuel Sharaf is a Solution Director at Prolifics on the West coast with real world customer expertise with Portal implementations, Dashboard, Forms and Content Management. Sam also has expertise with migrating applications from non-IBM platforms to IBM WebSphere Application and Portal Servers.

IBM CloudBurst Appliance – Part I

Samuel Sharaf, Solution Director West Coast

Having lived and breathed in the IBM technology world for the last 10 years, I was intrigued when IBM made the announcement a few months ago (June 16th, 2009) about the CloudBurst appliance. The name of the appliance, CloudBurst, was an interesting one and what it could do almost sounded like magical. A device which can bring together hardware, software and services needed to establish a private cloud? Sounded too good to be true. Obviously, the name of the appliance suggested that It had to do something with cloud computing – a concept which is fast gaining popularity.

As part of our technology group initiative, I decided to take a deep dive into understanding the appliance, its capabilities as they relate to cloud computing and IBM technology, and ultimately how we can we position it to our customers.

In this first part of blog series on CloudBurst, I will share the device overview and its general capabilities at high level. The subsequent blogs will go in more depth in describing its practical scenarios, architecture and real world usage.

IBM CloudBurst provides everything you need to start delivering services much faster than you do today, while reducing costs and providing the benefits of a dynamic infrastructure. It is a pre-packaged private cloud offering that integrates the service management system, server, storage and services needed to establish a private cloud. This offering takes the guess work out of establishing a private cloud by pre-installing and configuring the necessary software on the hardware and leveraging services for customization to your environment. All you need to do is install your applications and start leveraging the benefits of cloud computing, like virtualization, scalability and a self server portal for provisioning new services.

Summarizing the capabilities:

• A service delivery platform that is pre-integrated at the factory
• Built-for-purpose based on the architectural requirement of specific workloads
• Delivered and supported as a single product
• Prepackaged, pre-configured servers, storage, networking, software and installation
• services needed to stand up a private cloud

IBM CloudBurst includes everything from a Self-service portal that allows end users to request their own services and improve service delivery, automation to provision the services and virtualization to make system resource available for the new services thus reducing costs significantly. This is all delivered through the integrated pre-packaged IBM CloudBurst offering which includes implementation services and a single support interface to make it easy.

In part II of this series we will go into more depth in exploring individual features of the appliance in more detail. Stay tuned…

Samuel Sharaf is a Solution Director at Prolifics on the West coast with real world customer expertise with Portal implementations, Dashboard, Forms and Content Management. Sam also has expertise with migrating applications from non-IBM platforms to IBM WebSphere Application and Portal Servers.

Database Design and the Demands of Operational Business Intelligence

Anant Gupta, SOA Practice Director

When hearing the term Business Intelligence the first thought that springs to mind is that of reporting and trend analysis based on historical data. So what exactly is Operational Business Intelligence (BI)? It almost seems to be an oxymoron. In one sentence, it is the mechanism of enabling everyone, and not just the strategy team as in the case of BI, to make decisions that improve the top-line, bottom-line or enhance customer satisfaction. These decisions are not made quarterly or yearly but on a transactional basis. A very simple example of operational intelligence is your GPS system telling you to take a different route to work because of a bad traffic condition on your regular route. To make this more interesting, let’s assume that your system not only understands the traffic conditions, but also gas prices, your schedule, tolls, etc., and needs to present you the most effective option based on all of these factors. To do this, it needs to perform a lot of analysis in real-time. Similarly, systems need to look at the product / promotion info and customer purchase history / status and determine cross-sell, up-sell opportunities.

So far we have been talking about transactional databases or data warehouses / data marts, one for OLTP and the other designed strictly for post-facto reporting and analysis. The new trend will be a middle path where we will be designing data marts that will have the capability to be queried and performing analysis in real time. There will also be a trend that will emerge to update the operational data stores with updates from the data warehouse. Essentially, data warehouses will be able to continuously analyze data, recognize and send events that operations will subscribe to. For example, I might be interested in subscribing to an event if merchandise returns for one of my product lines exceed 5% of normal returns. So, databases will need to start understanding events and perform continuous analysis to determine the occurrence of any of those events.

All of this imposes challenges to the database design as both the depth of analysis and performance will have to be delivered simultaneously.

Anant Gupta was recently named the SOA Practice Director at Prolifics after serving as a Senior Business Integration and J2EE architect Anant has with extensive experience in IBM's SOA software portfolio and specializes in delivering business integration and business process management solutions. He has worked for major clients in the banking, insurance, telecommunications and technology industries.

Messaging: JMS vs SCA

Ivan Smirnov, Senior Consultant

In SCA, bindings are the means of transporting data (messages) to and from modules. One specific application I’d like to discuss is messages between SCA modules: that is, when one SCA module calls another SCA module. There are several types of bindings that can be used in that scenario. The easiest is, I kid you not, the SCA binding. At runtime, it becomes an EJB call. SCA binding was designed specifically for inter-module calls and it is so good at this, it makes you wonder: why use any other binding type at all? Well, if you need more flexibility, JMS binding may be handy.

SCA binding is best suited for simple synchronous calls between modules. It is possible to use SCA binding for an asynchronous invocation, but it will be implemented on auto-generated JMS queues on a SCA SYSTEM bus. With little control over queues, this option does not provide all the benefits of JMS binding.

Advantages of SCA bindings over JMS bindings:

• They do not require any external configuration and thus are maintenance-free.
• Interfaces are early bound. Interface types are checked when binding is established, and the Export interface must match the Import interface. This eliminates a possibility of errors not discovered until runtime.

Advantages of JMS bindings compared to SCA:

• More flexibility: target module can be relocated (even to a different cell).
• You don’t have to use the same interface on both sides of binding. Even method names may be different on caller (module Import) and target (module Export), so long as payload type is correct.
• There is an easily accessible checkpoint between modules(a JMS destination) where messages may be inspected for troubleshooting/debugging purposes.
  

In addition to this theoretical analysis, let me share my experience. I experimented with inserting JMS binding between modules where SCA was a natural fit, just for error handling purposes. My idea was that it would be much easier to control message flow and retry logic if there was a JMS queue between modules that I could configure to my liking. As intriguing as it sounds, there was too little real benefit in that specific ESB solution. So my conclusion was that error handling is not sufficient justification for using JMS binding where SCA binding is a more natural fit.


Ivan Smirnov is a Senior Consultant at Prolifics with extensive hands-on experience with the WebSphere family of products (including WebSphere Application Server and Process Server, WebSphere Studio/Rational and WebSphere MQ), Tivoli security offerings (including Tivoli Identity Manager and Tivoli Access Manager for e-business), DB2, XML and Web Services. With strong technical skills both in development and administration, as well as deep troubleshooting skills, Ivan handles aspects of implementation installation, configuration, securing and tuning/troubleshooting to development and architecture within a J2EE environment. He also possesses key Application Server migration skills and has helped several customers’ transition to the WebSphere platform from other J2EE platforms.

SOA and Off-Shore Project Models

Jonathan Machules, Technology Director

SOA is, as it was hyped by the marketing machines years ago, the next maturity level in Architecture. Service Orientation adds another level of design, development and maintenance considerations to projects. Another trend in the industry is the off-shore model of development of these projects. You can see why project success with these maturing trends is largely dependent on the personnel involved.

At Prolifics we are leaders in both these areas. We have expertise in SOA Methodology, Design and Architecture across all functional and non-functional domains complimented by a proven off-shore methodology that delivers.

On a recent SOA project using off-shore resources I had 2 months to Architect and Design an Enterprise Integration Platform for a large financial company seeking to use a combination on-shore/off-shore model for the Design and Development of Enterprise Integration system based on SOA.

This combined model works well with the right pieces in personnel in place. Capable developers with the ability to learn new technologies, strong project management and architectural leadership. Couple this with proven communication strategies and process for corresponding tactical direction and deliverables and there is a model that can work in the face of new technologies and regionally separated personnel.

The pay-off for this model combination is reduced cost and time to market. Given the economic times I foresee we will see a fair amount of interest and subsequent implementations with such models.

-Jon

Jonathan Machules first joined Prolifics as a Consultant, and is currently a Technology Director specializing in SOA, BPM, UML and IBM's SOA-related technologies. He has 12 years experience in the IT field — 2 of those years at Oracle as a Support Analyst and 10 years in Consulting. Jon is a certified IBM SOA Solution Designer, Solutions Developer, Systems Administrator and Systems Expert. Recent speaking engagements include IMPACT on SOA End-to-End Integration in 2007 and 2008, and SOA World Conference on SOA and WebSphere Process Server in 2007.