SOA11 and Coherence integration case study

In one of the project where I've worked on, the customer was facing issues due to the many web services calls triggered by their portal. Basically the Service Bus was crashing since it was reaching its limit in terms of web services transactions supported.

The initial architecture

The logged in users in Web Center portal were triggering calls trough the ADF framework to the OSB, and those calls were validated and enriched and then routed to the target third party services. All those operation were synchronous.

The analysis

We decided to take a deeper look to understand which services were being called, how many times and with which frequency, etc.
The tool we used here, was the Statistics in the OSB. We enabled them on all the Web Services, we run the Portal, and played with the functionality, which were calling the WS under analysis.

After 10 minutes we got the following result, where we noted 2 main things:

1. There were some WS operations called very frequently (1 each 20 msecs for each logged users) like GetTask and GetCurrentState, while all the others were being called only few times.
2.  The WS calls were getting information only for the specified WC Portal user which was triggering theggering thise calls.

The Solution

We spoke with the developers of the third party WS, proposing them to move to a "bundle" approach. Basically we asked them to expose two more operations, GetBundleTask and GetBundleState, which respectively were like GetTask and GetCurrentState, but with a list of usersId/taskId in input.
We configured a new Coherence server, thinking about using an asynchronous approach and the caching of the data. The architecture now looks like this:

Benefits:

• The WebCenter/ADF layer did not need any modification since we did not touch the OSB WS interfaces.
• The SOA Async process was getting data in bundle for all the Users logged in in one call (or very few) with much better results.
• The services exposed on the OSB were getting the data 99% of the time from the local cache, so the network latency was now almost zero and the average response time was 30 times less.

Here are the new statistics after applying those changes. The number of calls to getTask and getCurrent state external WebServices from the OSB were dropped of the 99%, all the data were coming from the cache.

Here the Async BPEL service was facing the same latency calling the bundle services then the singe OSB services, but at least this time we were calling them with a list of users.

Here are some sequence diagram which explains now the sequence of the operations. The OSB Service now try to get the data from Coherence, if it is  not found it get the same from the OLD external services, and "subscribe" the users for update.

The SOA Business service was calling the external bundle services based on the "subscribed" users list.  

SOA Testing in Maven

Currently there are few tools that can support testing SOAP interfaces.

For instance Jmeter and SoapUI are suited for testing soap interfaces.

• SoapUI is explicitly created for testing SOAP interfaces
• Jmeter has a SOAP support since version 2.3.x. 

SoapUI, has an intuitive user interface and is flexible.
You can run SoapUI stand alone and it can be to integrated within an automated process.

Below you will find instructions for running SoapUI as a part of a maven build. This makes it possible to run your automated SOAP tests in Maven with a build process like Jenkins. Combined with automatic deployment it is possible to support an agile software development process that supports frequent delivery of versions and continuous integration and testing.

Maven2 supports SoapUI with the Maven SoapUI plugin.

Usage:

Add the eviware plugin repository to your repository list.

 
<pluginRepositories>
  <pluginRepository>
    <id>eviwarePluginRepository</id>
    <url>http://www.eviware.com/repository/maven2/</url>
  </pluginRepository>
</pluginRepositories>

Attach to verify phase

By attaching the SoapUI maven plugin to the verify phase your build process runs it automatically in the integration-test phase. The “iso-soapui-project.xml” is the reference to the SoapUI xml file.

 
<plugin>
  <groupId>eviware</groupId>
  <artifactId>maven-soapui-plugin</artifactId>
  <version>2.0.2</version>
  <executions>
    <execution>
      <phase>verify</phase>
      <id>soapui-tests</id>
      <configuration>
        <projectFile>${basedir}/src/test/soapui/iso-soapui-project.xml</projectFile>
        <outputFolder>${basedir}/target/soapui</outputFolder>
        <junitReport>true</junitReport>
        <exportwAll>true</exportwAll>
        <printReport>false</printReport>
      </configuration>
      <goals>
        <goal>test</goal>
      </goals>
    </execution>
  </executions>
</plugin>

Convert log to a report

The log export of SoapUI can be interpreted like a normal Surefire unit report. By just adding this part to your maven reports section you can generate a nice overview of your test results.

 
<reporting>
  <plugins>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-surefire-report-plugin</artifactId>
      <configuration>
         <outputDirectory>target/site/soapui</outputDirectory>
         <reportsDirectories>
           <reportsDirectories>target/soapui/</reportsDirectories>
         </reportsDirectories>
      </configuration>
    </plugin>
  </plugins>
</reporting>

Thanks to the AMIS blog for this information!!!

Oracle Code Compliance Inspector

The Code Compliance Inspector uses a pre-defined set of assertions that are based on SOA AIA Integration Developer guidelines and the Web Services Interoperability Organization Basic Profile (WS-I BP) to check SOA projects for design consistency and good coding and documentation practices. CCI qualifies code as Compliant, Conformant, or Fully Conformant to be in sync The Open Group Architecture Framework (TOGAF) standard guidelines based on the pass criteria of the highest priority assertions.

• CCI is available as a JDeveloper extension
• As a command-line utility
• Oracle Enterprise Repository (OER) utility

Developers will typically use the JDeveloper extension and will continuously check compliance on JDeveloper projects as they develop.

The CCI command line utility can be used to incorporate CCI code check as part of our build/deployment/continuous integration process. So every time someone checks code into main branch, the Continuous Integration tool can automatically checkout the code, do a CCI code compliance test and then build and deploy. Of course if the code is not compliant it can send an error notification to a distribution list, without deploying.
Here are some examples for invoking the Code Compliance Inspector from a command line:

• Windows: checkCompliance.bat -inputDir D:\AIA\demo -outputDir D:\ComplianceOut
• Linux: sh checkCompliance.sh -inputDir /AIA/demo -outputDir /ComplianceOut

CCI provides optional integration to Oracle Enterprise Repository (OER). When OER is present, CCI can synchronize results to the repository, enabling users to access the report from the OER console. Integrating compliance data into OER provides repository users with information about whether composites are compliant into the repository reports and individual asset metadata.

To make it working in JDEV11 is quite easy, just download and install the extensions in JDeveloper

Then right-click the project and select Check Code Compliance.

Note: when the project is selected, the code compliance inspector can also be launched
from the toolbar.

A highlighted Compliance Results tab will appear in the log panel, go ahead and select it
to see the project status

In this environment, we are simply checking compliance against WS-I standards, but there are also other standards delivered with Oracle Application Integration Architecture Foundation Pack that can check much more as well as additional checks that can be added by your team