Showing posts with label Rational. Show all posts
Showing posts with label Rational. Show all posts

Monday, May 24, 2010

Why we love using Rational Team Concert to automate the build, deploy and test of WebSphere applications

Greg Hodgkinson, Methodology Practice Leader


Team Concert continuous integration for WAS, WESB, WPS, Portal and WMB?


So you love what you’ve heard about the benefits of Rational Team Concert for improving the development performance of Java development teams, but you’re wondering if it works just as well for your WebSphere Application Server, WebSphere ESB, WebSphere Process Server, WebSphere Portal Server or WebSphere Message Broker applications?


Well wonder no more – it does!


Being better at delivering better solutions


First some background.


Here at Prolifics we specialize in helping our customers deliver business solutions running on the various WebSphere runtime platforms. In fact we’ve delivered so many that I don’t even think marketing keeps count anymore! And as we have such a focus on software development, we are especially interested in any solution that allow us to do a better job, more efficiently, and greater ease.


This explains why we love Team Concert so much.


• From the point of view of a project manager – the always-up-to-date visibility of exactly what is going on at any point in time, what issues the team is facing, the current progress trend, what has been delivered most recently, what tasks are overdue, what the current status of the build is – all combine to allow a greater level of control over the project. They say control is an illusion, but an informed project manager is more likely to make the right choices for the project.


• From the point of view of the developers – the powerful but easy to use source control with private back-up space on the server from where changes can be shared with others and accepted from others, the ability to easily pause changes and switch over to another piece of work, the feed of information of what the rest of the team is up to, the prioritizable to-do list of work items, and the powerful build automation engine that allows the build/deploy/test process to be automated and provides automated feedback on the quality of code being delivered to source control – these features allow developers to be far more productive and to operate as a team with far greater confidence that they are on the right track.


• From the point of view of testers – the ability to interact directly with the developers using the same collaboration platform that they do, with a tightly integrated round-trip from defect, to code fix, to build, to release – increasing the efficiency of not only reporting and fixing the defects, but getting fixes back out to the testers to be re-tested.


• From the point of view of the business (our customers) – they see an improved predictability in project delivery dates, increased quality in the solution delivered, reduced times to fix defects, improved visibility into the activities of the project team, and insight into what it is that is being delivered – all which mean that project budgets will be signed off with a high certainty of success.


Simply put, the tool has been revolutionary in helping us being better at delivering better solutions for our customers.


Continuous integration is vital


An important factor in our success with Team Concert is its build engine. Continuous builds are well known to be a highly effective way to improve the coordination and efficiency of a team of developers and Team Concert’s build engine does this brilliantly – along with the change sets (source control component) and work items (work item tracking component) that it traces to and from, and the dashboard that exposes its results and outputs to the team and beyond.



The process is simply and yet the effects are powerful.


1. Developers deliver their code changes, along with suitable unit tests that verify the correct operation of their code.


2. The Team Concert build engine will trigger the build either based on a continuous interval or a defined schedule.


3. The build engine fetches the code changes, the unit tests, and the automation script that will drive the build.


4. The build engine sets up a build area containing the code, the tests and the automation script, and then executes the script. Depending on the script used, it can automate a number of steps involved in creating a candidate release of the code, for example:

a. Compile the code and unit tests

b. Deploy the code to an application server (normally a dedicated continuous integration testing server)

c. Run the tests to ensure that the deployed code works correctly.


5. The build engine then publishes a build output record which has immense value to the team. This record contains the deployable unit(s) produced by the build, the logs that were produced as a result of the build activities, the results of running the unit tests, a snapshot of the exact configuration of code, tests and automation script that the build engine used, and the contents of the build both in terms of the work items and code changes that went into it.


6. The team is notified of the results of the build via a number of mechanisms: the build will appear as a notification in each team members IDE, an entry in the project feed on the dashboard, a new build in the builds section of the dashboard, a new entry in the team central view. If there has been a failure, then everyone will know about it, and the team can get straight to work on correcting the issue.


Permanent point of reference for releases



This build record forms a permanent record of the build that is immensely valuable to any future developers or release engineers that need to understand that particular build. They can very quickly determine what went into the build and what was produced by the build.


Neutralization of integration errors


A key step in the automated process steps described earlier is the “Notify team of completed build” step.


This is important for two reasons: Firstly, any further steps required to take place such as smoke tests can now take place, so notification is important.


Secondly, and most significantly, if the build failed it is important that the team is aware so they can take steps to correct the causes of this failure. Having a broken build is a bad situation because it means that the quality of any new changes that are delivered cannot be verified, so it is important that steps are taken immediately to rectify the problem.


Normally the responsibility falls with the developer whose delivery broke the build to correct things, but it is important that the whole team is aware of the situation. It doesn’t take long for developers to learn how important it is for the broken build to be fixed, and we’ve found that this detect-notify-correct process becomes very efficient at neutralizing integration errors that could otherwise require far more effort to correct once they are compounded and confused by further changes.


Zero “wait time” to produce a release


Every build is a candidate release! Or at least every build that succeeds . No more delays having to request a release engineer to create a release of the code, waiting while they gather the correct source and scripts to produce the release, and then manually deploying the release for smoke tests. Essentially every time a successful build has run you have a potential release ready for smoke tests. The “wait time” has gone down to zero.


Prolifics “project-proven” on all WebSphere platforms


To get back to the original question: Continuous integration with Team Concert is all very good and well for Java projects, but what about more advanced applications built on WAS, WESB, WPS, Portal or WMB?


Well you will be glad to know that it is exactly these kinds of projects where we have seen the most benefit out of using Team Concert’s build engines – especially those projects where there is a combination of technologies involved. We have successfully used the Team Concert build engine along with our own automation scripts to realize the benefits discussed above (permanent point of reference for releases, neutralization of integration errors, and zero “wait time” to produce a release) and many further benefits in building applications across all of the WebSphere platforms.



New starter scripts solution available on IBM Global Solutions Directory


Would you like to use your Team Concert build engine to automate the build, test and deployment of WAS, WESB, WPS, WMB or Portal applications? If so then our starter automation scripts are ideal to get you up and running quickly.


Contact us to arrange a no-obligation demonstration where one of our build engineers can show you how the solution can be used to speed up deliver on your target platform.


See the new listing in the IBM Global Solutions Directory for further information and also watch this video!


Greg Hodgkinson is the Methodology Practice Leader at Prolifics. He has worked in software architecture since 1996, initially in the field of component-based development (CBD), then seamlessly on to service-oriented architecture (SOA). His extended area of expertise is the software development process, and he assists Prolifics and IBM customers in adopting agile software development processes and SOA methods. Complementing this is his expertise in software development environment architecture. He is still very much a practitioner, and has been responsible for service architectures for a number of FTSE 100 companies. He presents on agile SOA process and methods, has co-authored a Redbook on SOA solutions, and regularly writes for DeveloperWorks.

Monday, September 21, 2009

Integrating WSRR (WebSphere Registry and Repository) and RAM (Rational Asset Manager)

Rajiv Ramachandran, Practice Director, Enterprise Integration / Solution Architect

One of the challenges we are facing as part of a current project that we are working on is to demonstrate the integration of WSRR (WebSphere Registry and Repository) and RAM (Rational Asset Manager).

First and foremost there are a lot of questions on the roles and responsibilities of both these products.

  • Do we need both products?
  • If so, what do we use RAM for and what do we use WSRR for?
  • What is an asset?
  • Is a service an asset? Can there be other assets other than services? Does that mean only service assets go into WSRR?
  • What is a service? Is it just a WSDL? What about the service code? Should that also go into WSRR?
  • How do these products work with one another?
  • When I make a change in RAM does it reflect in WSRR and vice versa?

There are more questions than answers. I don't have answers to all these questions and as we work on this project, I am sure we will learn more. The team will post more answers here on this BLOG.

Today we took the first step. How can we get WSRR and RAM connected? I should tell you to my surprise I found that it was not difficult to accomplish this.

We had a RAM instance (v7.1) running on a VM Image.

I installed WSRR v6.2 on the same image and since it was for a POC, I used the Derby DB for WSRR. (I had installed WSRR with a DB2 backend and it had worked fine.)

There were just 2 things I needed to do to get the connectivity working
  • RAM runs on a WAS instance. WSRR was running on a separate WAS instance. The communication between the two was over HTTPS. So I had to ensure that WSRR server certs were part of the trust store of the WAS server running RAM. Nothing RAM / WSRR specific here, pure WebSphere stuff.
  • Log into the RAM WAS console using the following URL (https://localhost:13043/ibm/console/logon.jsp)
  • Go to Security -> SSL certificate and key management > Trust managers > Key stores and certificates > NodeDefaultTrustStore > Signer certificates
  • Do Retrieve from Port, Provide the following values: Host -> localhost, Port 9443, Alias WSRR
  • Log out

We needed to set the configuration in the RAM Admin Console to connect to WSRR.

  • Log into the RAM UI as admin
  • Go to the admin tab, click on 'Community Name'
  • Go to Connections
  • Add a WSRR Connection
  • Call in LOCAL WSRR
    - The URL to provide is https://localhost:9443/
    - User id and password is admin / admin
    - Do 'Test Connection' and that should be successful

    - Click on Synchronize and you should see the message -> WSRR-Asset Manager synchronization started

That’s it. Now I was able to search in RAM and have visibility into assets that were in WSRR. This was a simple test. These were stand alone instances . I assume when we do have clustered configurations we may have some challenges but it was a pleasant surprise to see something work the first time you try it.

Rajiv Ramachandran first joined Prolifics as a Consultant, and is currently the Practice Director for Enterprise Integration. He has 11 years experience in the IT field — 3 of those years at IBM working as a developer at its Object Technology Group and its Component Technology Competency Center in Bangalore. He was then an Architect implementing IBM WebSphere Solutions at Fireman’s Fund Insurance. Currently, he specializes in SOA and IBM’s SOA-related technologies and products. An author at the IBM developerWorks community, Rajiv has been a presenter at IMPACT and IBM's WebSphere Services Technical Conference.

Monday, June 29, 2009

Collaboration and Rational Tooling

Greg Hodgkinson, Methodology Pratice Leader

Running effective software development projects requires a certain level of tool support. For most organizations, the tools they have come from multiple vendors, and may also include a mix of both licensed and open-source tooling. As tools tend to be acquired to meet point problems, it can be easy to lose sight of the big picture – how your team and tools fit together to deliver the business solutions needed. Integration (SOA) projects bring with them a new set of challenges which may require tooling of a type that you don’t currently have. Do you find yourself asking any of the following?
  • How do I drive my team’s work in a way that allows me to track progress?
  • How do I figure out what work and changes went into a build of the code created 6 months ago?
  • How do I keep my distributed team aware of each other’s actions and progress?
  • How do I ensure that my best practices are being adhered to?

These are questions we hear on a regular basis from our customers. Having tested and used various tooling, we have had great success with Rational Team Concert which is highly effective for team collaboration. Below are some slides I've put together which illustrate what I'm referring to...
















Greg Hodgkinson is the Methodology Practice Leader at Prolifics. He has worked in software architecture since 1996, initially in the field of component-based development (CBD), then seamlessly on to service-oriented architecture (SOA). His extended area of expertise is the software development process, and he assists Prolifics and IBM customers in adopting agile software development processes and SOA methods. Complementing this is his expertise in software development environment architecture. He is still very much a practitioner, and has been responsible for service architectures for a number of FTSE 100 companies. He presents on agile SOA process and methods, has co-authored a Redbook on SOA solutions, and regularly writes for DeveloperWorks.

Thursday, June 18, 2009

Think Fast! Rational AppScan using a SaaS Model

A quick thought for you...

With all the buzz around SaaS these days there's a cool application of Rational AppScan for Web Application Security that may be of interest. You can now purchase Rational AppScan using a SaaS model. It basically is an outsourced version that is hosted and managed by security experts at IBM. You buy a subscription service so there is no infrastructure cost and setup time. Could be a good way to get started...

Tuesday, June 2, 2009

The Curious Case of Web Services Migration - Part II

Vladimir Serebryany, Senior Consultant

As I mentioned in the previous blog entry, the final goal was to develop a Maven-powered environment in which the deployable unit - in our case the EAR file - would be generated by a set of scripts starting from checking out versioned code from PVCS. I would lie if I tell you that I liked Maven at first sight. My initial feelings were anger and frustration as Maven-enabled RAD essentially diminished RAD to be just a fancy code editor - and nothing more. Gone were round-trip interactive development, and my productivity as a developer really suffered. Only by the end of the project I found out how to keep RAD effective and fully engaged and still be able to build using Maven. At the very end of the project I started to really appreciate the effectiveness of Maven as a build tool especially when it comes to building deployable units for different environments in a very uniform and reliable way. Its ability to manage versioned dependencies is just outstanding given the relative simplicity with which this is achieved.

Anyway, for regular Dynamic Web Projects the task would be trivial - Maven already has a plug-in to generate all deployable artifacts. But for Web projects with Web services there was nothing I could use because there were a number of generated artifacts which simply did not fit into Maven’s rigid default directory structure. I faced a task of developing a new Maven plug-in just for the tasks on hand (as long as I am mentioning this, you may guess that plug-in was successfully implemented, but read below to find out at what cost).

To assess the scope of the effort involved, let me just list the tasks accomplished.
Given: set of RAD projects fresh from PVCS each with Maven pom.xml. One specialized project had all WSDL and XSD files. Parent folder had pom.xml with our custom Maven Web services plug-in properties and configurations. Note that only scripts and libraries available with WAS 6.1 run-time are used for code generation. No part of RAD is used for Maven plug-in (in fact our customer uses AIX to run build scripts).

Below is the overview of custom plug-in functionality:
  • For each WSDL in parent pom.xml run WSDL2Java script, create temporary folder structure with all Java and XML configuration files.
  • For each generated webservices.xml go in and replace generated placeholder with actual servlet class name (this is because WSDL2Java script just does not do that by design).
  • Combine all webservices.xml files together and create single webservices.xml to be put in the target project.
  • Collect all mapping xml files and put them together in the target project.
  • Collect all servlets names and classes names for each WSDL and put them as servlet/servlet-mapping entries in the Web Deployment Descriptor in the target project.
  • Copy WSDL and XSD files into the target project WEB-INF/wsdl folder.
  • Copy all generated Java sources and put them together with existing source code.
  • Pass resulting project to the standard Maven Web project plug-in to be compiled and built.

At the end the Prolifics team had accomplished everything the client asked us to do. and we left the site with application up and running in production with no problems. The only thing they asked with amazement was: "How did you do that, guys?"

Vladimir Serebryany is a Senior Consultant at Prolifics with in-depth knowledge and broad hands-on experience with the J2EE environment as well as expertise in EJBs, Servlets, JSP and JSF. Excelling at migrations, Vladimir has over 9 years of experience with a wide range of complementary skills including WebSphere, WebSphere MQ, WebSphere Business Integration Message Broker, UNIX, C/C++, Java, HTML/ASP, JavaScript, and Visual Basic. He has served as team leader/senior developer roles in large, complex projects and configurations with clients in the financial, insurance and telecommunication industries - among others.

Monday, May 25, 2009

The Curious Case of Web Services Migration - Part I

Vladimir Serebryany, Senior Consultant

During one of my recent BEA WebLogic to IBM WebSphere migration assignments at a major insurance company, I encountered an interesting problem.

The client had a large number of Web services which were running on BEA WebLogic and consumed by a .NET front-end. In the course of migration, our team had to not only migrate code to WebSphere but also help the customer refactor the source code repository and create a set of Maven scripts to provide a fully scripted build/deploy process for all environments - from unit test to production.

We started with the typical set of issues:
  • WSDL files and corresponding XSD files had relative namespaces in them - and the spec strongly recommends absolute ones
  • Source code was stored in typical WebLogic structures and deployable components were created by running four (!) different Ant scripts
  • Code and WSDL files were stored in PVCS and the revision history had to be preserved
  • WSDL2Java wizard was invoked every time scripts ran so binding Java code would be generated on the fly and not stored in PVCS - wise decision but binding implementation classes with client's code in them had to be preserved from being overwritten.

The first issue was easy to deal with – a couple sed/awk scripts took care of the namespaces. We gave WSDL files back to client; they validated them against their .NET environment and found no problems with new namespaces.

Item #2 was also relatively easy to solve - we manually rearranged code into a Rational Application Developer (RAD) structure and that was it. RAD is really good at that sort of thing. While doing that we also extracted PVCS archives as files from the central repository and rearranged these archives in the file system according to the new directory structure. We sent back these directories to the client’s PVCS team, and they imported these archives back into a new PVCS project. That way all the previous generations of source code have been preserved. We now were able to work with PVCS and track a file’s revision history right back to Noah's Ark.

Now we had to deal with WSDL to Java files generation. There were three sorts of files: custom code with business logic, binding code and classes generated from data types definitions in XSD files, and binding implementation classes. We did not have to worry about the first two types: the first set was permanent code which we shifted to another utility’s Java project for simplicity of maintenance. Generated Java was what it was - generated code which the wizard took care of, only the binding implementation classes was something we had to take care of. On one hand all the methods stubs are generated by the WSDL2Java wizard - on the other hand these methods already contained the customer's code. If we allowed the wizard to run "freely," this code would be overwritten.

Here we needed a trick and we found one. As it is well-known, RAD may designate one or more folders in the project to be the Source folders - meaning it will compile all Java code in these and only these folders into binary form. It happens that built-in RAD wizards are very sensitive to the order in which Source folders are listed in "Order and Export" tab of "Java Build Path" configuration page. If the WSDL2Java wizard for bottom-down Web services generation is invoked either from the pop-up menu or from the wsgen.xml script, it always places all generated files into the folder listed first in "Order and Export" tab. That "observation" provided us with opportunity to solve the problem. We would define two Source folders: “src/main/java” and “src/main/java2.” “src/main/java” would be listed first in "Order and Export" tab, “src/main/java2” would be the second. The single "target/bin" folder was designed for binary files. We placed all existing binding implementation files into “src/main/java2” folder, and we put WSDL and XSD files into a separate project. (Note that by using “src/main/java” folder structure we were shooting to use Maven.)

From that point on, if a WSDL definition changed, we would clean “src/main/java” folder and run the Web services wizard/script against new WSDL/XSD files. Newly generated binding files would go into “src/main/java” folder. As long as binding implementation files were also on the classpath, WSDL2Java wizard was smart enough to not regenerate *BindingImpl.java files, but rather pick them up from “src/main/java2” folder. IBM provided ws_gen Ant task, script and sample properties files were customized to run the whole task at any time for all 20 WSDL's we had in the service definition (see http://publib.boulder.ibm.com/infocenter/radhelp/v7r0m0/topic/com.ibm.etools.webservice.was.creation.core.doc/ref/rtdwsajava.html). We didn't include “src/main/java” folder into PVCS and never had to check-in transient generated code.

The remaining issue to solve was to reproduce in RAD the style in which WebLogic's WSDL2Java script generated the code. By tweaking WSDL2Java options on Window->Preferences->Web services->Websphere->JAX-RPC Code Generation->WSDL2Java tab in RAD, we matched the style of generated code so that RAD-generated code was almost drop-in replacement for WebLogic-generated code. The problem was solved - but only for development phase so far.

Coming in Part II: “My initial feelings were anger and frustration as Maven-enabled RAD essentially diminished RAD to be just a fancy code editor - and nothing more. Gone were round-trip interactive development, and my productivity as a developer really suffered. Only by the end of the project I found out how to keep RAD effective and fully engaged and still be able to build using Maven…”

Vladimir Serebryany is a Senior Consultant at Prolifics with in-depth knowledge and broad hands-on experience with the J2EE environment as well as expertise in EJBs, Servlets, JSP and JSF. Excelling at migrations, Vladimir has over 9 years of experience with a wide range of complementary skills including WebSphere, WebSphere MQ, WebSphere Business Integration Message Broker, UNIX, C/C++, Java, HTML/ASP, JavaScript, and Visual Basic. He has served as team leader/senior developer roles in large, complex projects and configurations with clients in the financial, insurance and telecommunication industries - among others.