Showing posts with label SOA. Show all posts
Showing posts with label SOA. Show all posts

Thursday, 14 April 2016

How to create a SOA 12.2.1 docker image on OracleLinux


Introduction

What is docker?

Unless you’ve been living without internet access for the last two years, it would be hard not to at least heard of Docker. But, as an emerging technology not everyone has taken the time to work out what Docker is, where it fits in and how it can benefit you.

So, what exactly is Docker? Here’s what Docker themselves describe it as:

    Docker is an open platform for developers and sysadmins of distributed applications.

Essentially, Docker is a container based system for your applications. If you’re used to the concept of virtual servers, Docker provides further levels of abstraction for your application. Here’s a visual representation of how it differs:

VM vs Containers - Docker


Rather than just being one part of the puzzle, Docker provides a number of components and tools to assist with the entire lifecycle management of the application. This includes the container environment, image management and orchestration.

Docker started it’s life as an internal project within a hosting company called dotCloud, but quickly took off once they open sourced it in early 2013. Since then, it's benefited from over 15,000 software commits from over 900 contributors.

Why use Docker?

Now that you have a basic understanding of Docker, there are a number of great reasons to start using it.
  • It’s very fast. Start a Docker container can be complete in as little as 50ms. That’s not a typo, it really can be this quick! This is the advantage of having such high levels of abstraction, you reduce the number of components you need to run. This also means that there's very little to no overhead in it's implementation.
  • One command deployments . It really is as simple as installing an application with one line. Want to install MySQL? One command. Want WordPress, MySQL, Nginx and Memcache all installed and configured? Yep, it’s one command.
  • Pre-configured Apps. At last count, there were over 13,000 applications already packaged as a Docker image. Chances are, if you’re using a common application then most of the initial work has already been done for you. But, it doesn't end there. You can take the existing image, make your own changes and push it to your own repository for ease of re-deployment.
  • Resource Isolation. Previously, if you ran all of your services on the one server then there’s a chance one of them could exhaust all of the server resources. Docker allows you to set, monitor and adjust these on a per application or service basis.
  • Consistency. Docker really is the “write once deploy anywhere” type environment. It removes all of the hassles going from a development to a production environment or similar. Each set of libraries is very tightly coupled to the docker image to ensure the consistency.
  • A complete platform. Rather than just being one part of the puzzle, Docker is shaping up to be a complete platform. There’s the base Engine for the containers, the Registry for image management, Compose to orchestrate complex deployments, Swarm for clustering and Machine for provisioning. This is what’s made Docker different from other container implementations, you can manage the entire lifecycle quite easily.
  • Scale. This is is one area where Docker really shines, especially if you have a micro-service based application. Compose and Swarm assist with deploying scalable applications and there’s then third party tools like Kubernetes and Mesos, which both take it to the next level. We’re talking the ability to manage the entire lifecycle with up to millions of containers, so scale isn’t a problem!
Moreover Oracle is now quite focused on certifying its Oracle Fusion Middleware for Docker. Details are here. Also below is the current status of certification Docker related:

Oracle Linux x64 Oracle Linux 6(UL5+)
Oracle Linux 7(UL0+)
Red Hat Enterprise Linux 7(UL0+)
Docker Containers
  • Oracle WebLogic Server 12c (12.2.1) (Oracle Linux 6 (UL6+) is required. Refer to Note 6)
  • Oracle WebLogic Server 12c (12.1.3) (Refer to Note 4)


How can docker be useful to us?

Docker could  be  particularly useful for the following reasons;
  • More reliable releases
  • Enable continuous delivery
  • Recreate environments
  • Internal tools

Docker Installation

This guide below assumes the operating system is Oracle Linux 7.
Oracle provides a very good guide available here on how to install and configure Docker on OL7 (it is also available for OL6). I will just post here the main steps.

Operating systems pre-requirement

Docker version 1.9 and later require that you configure the system to use the Unbreakable Enterprise Kernel Release 4 (UEK R4) and boot the system with this kernel.

  • Disable access to the ol7_x86_64_UEKR3 channel and enable access to the ol7_x86_64_UEKR4 channel.

If you use Oracle Public Yum, disable the ol7_UEKR3 repository and enable the ol7_UEKR4 repository in the /etc/yum.repos.d/public-yum-ol7.repo file, for example:

[ol7_UEKR3]
name=Latest Unbreakable Enterprise Kernel Release 3 for Oracle Linux $releasever ($basearch)
baseurl=http://public-yum-qa.oracle.com/repo/OracleLinux/OL7/UEKR3/$basearch/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=0

[ol7_UEKR4]
name=Latest Unbreakable Enterprise Kernel Release 4 for Oracle Linux $releasever ($basearch)
baseurl=http://public-yum-qa.oracle.com/repo/OracleLinux/OL7/UEKR4/$basearch/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1

You can download the latest version of this file from http://public-yum.oracle.com/public-yum-ol7.repo.

  • Run the following command to upgrade the system to UEK R4:

# yum update

It is also recommended to make UEK R4 the default boot kernel, see Section 4.3, “About the GRUB 2 Boot Loader” at https://docs.oracle.com/cd/E52668_01/E54669/html/ol7-grub2_bootloader.html.

  • Reboot the system, selecting the UEK R4 kernel (version 4.1.12) if this is not the default boot kernel.

# systemctl reboot

  • Enable the ol7_addons repository in the /etc/yum.repos.d/public-yum-ol7.repo file, for example:

[ol7_addons]
name=Oracle Linux $releasever Add ons ($basearch)
baseurl=http://public-yum.oracle.com/repo/OracleLinux/OL7/addons/$basearch/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1

Docker engine Installation

  • Install the docker-engine package.

# yum install docker-engine

By default, the Docker Engine uses the device mapper as a storage driver to manage Docker containers. 
As with LXC, there are benefits to using the snapshot features of btrfs instead.

  • Configure Docker images to be saved into btrfs  filesystem (optional but strongly recommended)
Oracle recommends using btrfs because of the stability and maturity of the technology. If a new device for btrfs is not available, you should use overlay as the storage driver instead of device-mapper for performance reasons. You can configure overlay by adding the --storage-driver=overlay option to DOCKER_STORAGE_OPTIONS in /etc/sysconfig/docker-storage. The overlayfs file system is available with UEK R4.
For more information, see https://docs.docker.com/engine/userguide/storagedriver/overlayfs-driver/.

To configure the Docker Engine to use btrfs instead of the device mapper:

Use yum to install the btrfs-progs package.

# yum install btrfs-progs

If the root file system is not configured as a btrfs file system, create a btrfs file system on a suitable device such as /dev/sdb in this example:

In the case you are using Virtual Box (or any virtual machines tool) this is quite easy, just follow the steps below (details are available here):

1. Add Disk Storage to Oracle Virtual Box using the tool (make sure it's big! I used a 50 GB one)

2. Turn on the VirtualBox OL7 machine and switch to the root user.

3. Then use fdisk utility to get list of hard drives.
$ fdisk -l

Disk /dev/sda: 107.4 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000e4833

Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          64      512000   83  Linux Partition 1 does not end on cylinder boundary.
/dev/sda2              64       13055   104344576   8e  Linux LVirtualBox

Disk /dev/sdb: 12.9 GB, 12884901888 bytes
255 heads, 63 sectors/track, 1566 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/mapper/VolGroup-lv_root: 53.7 GB, 53687091200 bytes
255 heads, 63 sectors/track, 6527 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

.....

From the first block you can clearly see that we have two hard drives i.e. /dev/sda and /dev/sdb. 
You can also see that /dev/sda has been formatted and mounted using logical volume management. 
This was surely done when you created the VirtualBox or when the template was created, as it is the default way Linux formats the hard drives.

4. We need to partition the /dev/sdb device. Use the following command to partition the device.

fdisk /dev/sdb 

Follow the screen instructions as shown below.
# fdisk /dev/sdb
[/sociallocker] Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x2e3c77cd.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').
Command (m for help): n
Command action
e   extended
p   primary partition (1-4)
p

Partition number (1-4): 1
First cylinder (1-1566, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1566, default 1566):
Using default value 1566

Command (m for help): w

The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks. 
We have used ‘n’ to create a new partition, p to create the partition as primary, 1 to specify the partition number and used defaults for begin and end range for partition size.

Finally ‘w’ to make changes permanent.

5. Query the partition table again to see now that /dev/sdb1 partition has been created into /dev/sdb disk.

# fdisk -l

6. Format the Partition using the file system (btrfs) recommended

mkfs.btrfs /dev/sdb1

7. Use the blkid command to display the UUID of the device and make a note of this value, for example:

# blkid /dev/sdb1
/dev/sdb: UUID="a7dc508d-5bcc-4112-b96e-f40b19e369fe" \
  UUID_SUB="1aa666eb-0861-4dc8-a37e-f3c87c7003b8" TYPE="btrfs" 

8. Create the file /etc/systemd/system/var-lib-docker.mount with the following contents:

[Unit]
Description = Docker Image Store

[Mount]
What = UUID=REPLACE_HERE_THE_UUID_value
Where = /var/lib/docker
Type = btrfs

[Install]
WantedBy = multi-user.target


This file defines the target that systemd uses to mount the file system on /var/lib/docker.

9. Enable the var-lib-docker.mount target.

# systemctl enable var-lib-docker.mount
ln -s '/etc/systemd/system/var-lib-docker.mount' \
  '/etc/systemd/system/multi-user.target.wants/var-lib-docker.mount'

This command enables systemd to mount the file system when required for use with the docker service. It does not mount the file system. If you need to mount the file system independently of Docker, use the following command:

# systemctl start var-lib-docker.mount

Create the drop-in file /etc/systemd/system/docker.service.d/var-lib-docker-mount.conf, which contains the following lines:

[Unit]
Requires=var-lib-docker.mount
After=var-lib-docker.mount

These entries tell systemd to mount the /var/lib/docker file system by using the var-lib-docker.mount target before starting the docker service.

After doing this restart the OS and make sure Docker restart correctly.

Moreover running the command docker info you can check the file system configuration is now applied:
Containers: 0
Images: 0
Server Version: 1.9.1
Storage Driver: btrfs
 Build Version: Btrfs v3.19.1
 Library Version: 101
Execution Driver: native-0.2

Problems:

In my test, Docker did not restart, so I checked at the log using the command below
more /var/log/messages | grep docker

I saw this:
SELinux is not supported with the BTRFS graph driver!

Googling the issue I just found a workaround to this problem (not sure for PROD environment if acceptable)

Edit the file 
vi /etc/sysconfig/docker

Removing the selinux-enabled from the options as below:
OPTIONS=''
#'--selinux-enabled'

Restart Docker!
# systemctl daemon-reload
# systemctl restart docker

Docker Post-Installation steps

  • Start the docker service and configure it to start at boot time.
# systemctl start docker
    # systemctl enable docker
      ln -s '/etc/systemd/system/docker.service' \
          '/etc/systemd/system/multi-user.target.wants/docker.service'
          systemctl status docker
            Useful commands to check Docker status:

            # docker info
            # docker version
            • Enabling Non-root Users to Run Docker Commands

            Create the docker group:

            # groupadd docker
            Restart the docker service:

            # service docker restart
            The UNIX socket /var/run/docker.sock is now readable and writable by members of the docker group.

            Add the users that should have Docker access to the docker group:

            # usermod -a -G docker user1

            SOA Image Creation

            To create the Docker SOA 12.2.1 image I used this blog post here, also the DockerFile I used are available here.
            Therefore make sure you execute the following steps:
            1. open a terminal in the VM and suso as a Root, then
              1. mkdir /fmw/soa
              2. cd /fmw/soa
              3. mkdir installers
            2. Copy into installers the following files:
              1. fmw_12.2.1.0.0_infrastructure_Disk1_1of1.zip
              2. fmw_12.2.1.0.0_soaqs_Disk1_1of2.zip
              3. fmw_12.2.1.0.0_soaqs_Disk1_2of2.zip
              4. jdk-8u77-linux-x64.gz
              5. silent.rsp (download it from the blog page as above and customize)
            3. Copy into fmw/soa the following file:
              1. Dockerfile.12.2.1.0.0 (download it from the blog page as above and customize)
            4. Change directory to /fmw/soa and execute the below command:
              1. docker build -t oracle/fmw:12.2.1.0.0 -f ./Dockerfile.12.2.1.0.0 .
            5. Run the container just created with the folowing
              1. docker run -i -P --name=oracle-soa-1 -t oracle/fmw:12.2.1.0.0
            6. Accessing to the machine please verify everything has been installed successfully (you could tsart Jdev following the procedure here below at export via X11)

            Oracle XE Image Creation

            To create the Docker Oracle XE 11 image I used the Docker file here.
            Therefore make sure you execute the following steps:
            1. open a terminal in the VM and suso as a Root, then
              1. mkdir /database/
              2. cd /database/
              3. mkdir xe 
            2. Copy into xe the following files:
              1. Dockerfile11.2.0.1.0 (download it from the blog post above)
              2. oracle-xe-11.2.0-1.0.x86_64.rpm.zip 
            3. Change directory to /database and execute the below command:
              1. docker build -t oracle/db/xe:11.2.0.1.0 -f ./xe/Dockerfile.11.2.0.1.0 .
            4. Run docker
              1. docker run -i -P --name=oracle-db-xe-1 -t oracle/db/xe:11.2.0.1.0
            5. Test the Database is up and running connecting via SQL Developer o similar tools (you can connect from any place is the same subnet, check the external port by running docker ps -a)

            Possible Problems

            If you have tried to create Docker containers for Oracle Database XE, you might not be able to configure/start a database instance successfully. The installation may complete successfully but when you try to configure or start an instance, you might notice the following error in the logs.

            ORA-01034: ORACLE not available
            ORA-27101: shared memory realm does not exist
            Linux-x86_64 Error: 2: No such file or directory
            ORA-00845: MEMORY_TARGET not supported on this system

            Reason might be related to the Docker version used. Oracle Database needs a shared memory [/dev/shm] available of at least 1 GB, while any Docker container (up to 1.9 version) is 64MB.

            Some blog post suggest to get the latest of docker (10.X) and make use of the new option --shm-size=2g:

            cat >/etc/yum.repos.d/docker.repo <<-EOF
            [dockerrepo]
            name=Docker Repository
            baseurl=https://yum.dockerproject.org/repo/main/oraclelinux/7
            enabled=1
            gpgcheck=1
            gpgkey=https://yum.dockerproject.org/gpg
            EOF

            Then make sure docker engine is update running the below
            yum install docker-engine

            Other approach (like the one I used just avoid this issue adding the following in the Docker file:

            # Work around sysctl limitation of docker
            RUN sed -i -e 's/^\(memory_target=.*\)/#\1/' /u01/app/oracle/product/11.2.0/xe/config/scripts/initXETemp.ora \
                && sed -i -e 's/^\(memory_target=.*\)/#\1/' /u01/app/oracle/product/11.2.0/xe/config/scripts/init.ora

            Also creating the XE server database FROM oraclelinux:7 (6 does not work)

            SOA Domain creation

            In order to create a domain, first of all the SOA docker container needs to be able to talk with the Database one. Docker by default allows containers to see each other however it is recommended to have different network.
            Good practice is to create sub-network in the Docker engine. This can be performed with the following steps:
            1. docker network create my-soa-network
            2. Run the container just created with the following (here adding -u=0 the container will be accessed as a ROOT user, which will allow to install telnet for testing the connection to the database "yum install telnet") 
              1. docker run -i -P --net=my-soa-network --name=oracle-soa-1 -t oracle/fmw:12.2.1.0.0
            3. Run in another terminal
              1. docker run -i -P --net=my-soa-network --name=oracle-db-xe-1 -t oracle/db/xe:11.2.0.1.0
            Second step is to create the SOA Db Schemas using the RCU, to do that use ifconfig to get the DB IP, then:
            1. In the SOA Docker container creates a file "passwd.txt" with 1 password for each database user that RCU is going to create. For the SOA Domain 7 password are needed.
            2. Then execute the following:
              1. rcu -silent -createRepository -connectString <DB IP>:1521:XE -databaseType ORACLE -dbUser SYS -dbRole SYSDBA -schemaPrefix <YOUR PREFIX> -variables SOA_PROFILE_TYPE=SMALL -component SOAINFRA -component OPSS -component IAU -component IAU_APPEND -component IAU_VIEWER -component MDS -component WLS -component UCSUMS -f < passwd.txt
            Last step is to create the domain using WLST script
            1. Run the following wlst Script using you r enrionment details
              1. WLST

            Tips

            Display export via X11

            Sometimes it is nice to have the UI.
            Docker and the OracleLinux template by default does not support any UI as it supposed to be a lightweight system.
            However exporting the display it is easy and can be done with the following steps:
            1. Run the image with the option -u (user root) as below
              1.  docker run -u 0 -it --name soa1221 oracle/fmw:12.2.1.0.0
            2. Install wget and download and install the X11 rpm packege
              1. yum install wget
              2. wget http://vault.centos.org/6.2/os/x86_64/Packages/xorg-x11-server-Xvfb-1.10.4-6.el6.x86_64.rpm
              3. yum localinstall xorg-x11-server-Xvfb-1.10.4-6.el6.x86_64.rpm
            3. Change user to oracle, export the DISPLAY and run Jdev
              1. su oracle
              2. export DISPLAY=192.168.1.220:1.0   (assuming this is the target IP where you'd like to UI to appear, of course this reuqires an X11 server, in my case I used MobaXTerm, a very simple one)
              3. Run Jdeveloper or any other tool with UI

            SSH Server

            In case the SSH Serer is needed the below command can be added to the DockerFile creation, also the container needs to be started with this option -p 22

            RUN yum -y install openssh-server

            However opening the ssh communication is not recommended while it is much more secure to use the docker EXEC <container> bash command.

            Saturday, 22 November 2014

            SOA11 and Coherence integration case study

            In one of the project where I've worked on, the customer was facing issues due to the many web services calls triggered by their portal. Basically the Service Bus was crashing since it was reaching its limit in terms of web services transactions supported.

            The initial architecture



            The logged in users in Web Center portal were triggering calls trough the ADF framework to the OSB, and those calls were validated and enriched and then routed to the target third party services. All those operation were synchronous.

            The analysis

            We decided to take a deeper look to understand which services were being called, how many times and with which frequency, etc.
            The tool we used here, was the Statistics in the OSB. We enabled them on all the Web Services, we run the Portal, and played with the functionality, which were calling the WS under analysis.

            After 10 minutes we got the following result, where we noted 2 main things:

            1. There were some WS operations called very frequently (1 each 20 msecs for each logged users) like GetTask and GetCurrentState, while all the others were being called only few times.
            2.  The WS calls were getting information only for the specified WC Portal user which was triggering theggering thise calls.




            The Solution

            We spoke with the developers of the third party WS, proposing them to move to a "bundle" approach. Basically we asked them to expose two more operations, GetBundleTask and GetBundleState, which respectively were like GetTask and GetCurrentState, but with a list of usersId/taskId in input.
            We configured a new Coherence server, thinking about using an asynchronous approach and the caching of the data. The architecture now looks like this:


            Benefits:
            • The WebCenter/ADF layer did not need any modification since we did not touch the OSB WS interfaces.
            • The SOA Async process was getting data in bundle for all the Users logged in in one call (or very few) with much better results.
            • The services exposed on the OSB were getting the data 99% of the time from the local cache, so the network latency was now almost zero and the average response time was 30 times less.


            Here are the new statistics after applying those changes. The number of calls to getTask and getCurrent state external WebServices from the OSB were dropped of the 99%, all the data were coming from the cache.



            Here the Async BPEL service was facing the same latency calling the bundle services then the singe OSB services, but at least this time we were calling them with a list of users.

            Here are some sequence diagram which explains now the sequence of the operations. The OSB Service now try to get the data from Coherence, if it is  not found it get the same from the OLD external services, and "subscribe" the users for update.

            The SOA Business service was calling the external bundle services based on the "subscribed" users list.  





            Thursday, 13 November 2014

            SOA Testing in Maven

            Currently there are few tools that can support testing SOAP interfaces.

            For instance Jmeter and SoapUI are suited for testing soap interfaces.

            • SoapUI is explicitly created for testing SOAP interfaces
            • Jmeter has a SOAP support since version 2.3.x. 


            SoapUI, has an intuitive user interface and is flexible.
            You can run SoapUI stand alone and it can be to integrated within an automated process.

            Below you will find instructions for running SoapUI as a part of a maven build. This makes it possible to run your automated SOAP tests in Maven with a build process like Jenkins. Combined with automatic deployment it is possible to support an agile software development process that supports frequent delivery of versions and continuous integration and testing.

            Maven2 supports SoapUI with the Maven SoapUI plugin.

            Usage:

            Add the eviware plugin repository to your repository list.

             
            <pluginRepositories>
              <pluginRepository>
                <id>eviwarePluginRepository</id>
                <url>http://www.eviware.com/repository/maven2/</url>
              </pluginRepository>
            </pluginRepositories>
            
            

            Attach to verify phase

            By attaching the SoapUI maven plugin to the verify phase your build process runs it automatically in the integration-test phase. The “iso-soapui-project.xml” is the reference to the SoapUI xml file.

             
            <plugin>
              <groupId>eviware</groupId>
              <artifactId>maven-soapui-plugin</artifactId>
              <version>2.0.2</version>
              <executions>
                <execution>
                  <phase>verify</phase>
                  <id>soapui-tests</id>
                  <configuration>
                    <projectFile>${basedir}/src/test/soapui/iso-soapui-project.xml</projectFile>
                    <outputFolder>${basedir}/target/soapui</outputFolder>
                    <junitReport>true</junitReport>
                    <exportwAll>true</exportwAll>
                    <printReport>false</printReport>
                  </configuration>
                  <goals>
                    <goal>test</goal>
                  </goals>
                </execution>
              </executions>
            </plugin>
            
            


            Convert log to a report

            The log export of SoapUI can be interpreted like a normal Surefire unit report. By just adding this part to your maven reports section you can generate a nice overview of your test results.

             
            <reporting>
              <plugins>
                <plugin>
                  <groupId>org.apache.maven.plugins</groupId>
                  <artifactId>maven-surefire-report-plugin</artifactId>
                  <configuration>
                     <outputDirectory>target/site/soapui</outputDirectory>
                     <reportsDirectories>
                       <reportsDirectories>target/soapui/</reportsDirectories>
                     </reportsDirectories>
                  </configuration>
                </plugin>
              </plugins>
            </reporting>
            
            



            Thanks to the AMIS blog for this information!!!

            Oracle Code Compliance Inspector


            The Code Compliance Inspector uses a pre-defined set of assertions that are based on SOA AIA Integration Developer guidelines and the Web Services Interoperability Organization Basic Profile (WS-I BP) to check SOA projects for design consistency and good coding and documentation practices. CCI qualifies code as Compliant, Conformant, or Fully Conformant to be in sync The Open Group Architecture Framework (TOGAF) standard guidelines based on the pass criteria of the highest priority assertions.


            • CCI is available as a JDeveloper extension
            • As a command-line utility
            • Oracle Enterprise Repository (OER) utility


            Developers will typically use the JDeveloper extension and will continuously check compliance on JDeveloper projects as they develop.

            The CCI command line utility can be used to incorporate CCI code check as part of our build/deployment/continuous integration process. So every time someone checks code into main branch, the Continuous Integration tool can automatically checkout the code, do a CCI code compliance test and then build and deploy. Of course if the code is not compliant it can send an error notification to a distribution list, without deploying.
            Here are some examples for invoking the Code Compliance Inspector from a command line:

            • Windows: checkCompliance.bat -inputDir D:\AIA\demo -outputDir D:\ComplianceOut
            • Linux: sh checkCompliance.sh -inputDir /AIA/demo -outputDir /ComplianceOut


            CCI provides optional integration to Oracle Enterprise Repository (OER). When OER is present, CCI can synchronize results to the repository, enabling users to access the report from the OER console. Integrating compliance data into OER provides repository users with information about whether composites are compliant into the repository reports and individual asset metadata.

            To make it working in JDEV11 is quite easy, just download and install the extensions in JDeveloper




            Then right-click the project and select Check Code Compliance.


            Note: when the project is selected, the code compliance inspector can also be launched
            from the toolbar.

            A highlighted Compliance Results tab will appear in the log panel, go ahead and select it
            to see the project status



            In this environment, we are simply checking compliance against WS-I standards, but there are also other standards delivered with Oracle Application Integration Architecture Foundation Pack that can check much more as well as additional checks that can be added by your team

            Wednesday, 29 October 2014

            Oracle SOA 12c Templates


            SOA Composite Templates

            One of the best new feature in this SOA 12.1.3 release is this about the Template.
            The SOA template can really increase the productivity of the Developers team and Oracle is right saying this..
            Here I just want to recap the different type of template and main features, across  SOA and OSB, hope this might help.

            SOA ‘Starter’ Templates

            • ‘Starter’ Template is a re-usable part of a SOA project
            • Fully editable
            • Store in MDS
            • Three types:
              • SOA Project Template
              • Component Template
              • Custom Activity Template



            • Project template accessible at the time of new project creation
            • Automatically discover in the component palette
              • Component Template
              • Custom Activity Template
            • Share and re-use from MDS



            BPEL Sub-processes: standalone and inline

            Very useful for groups of activities that are reused within one (inline) BPEL process or the whole Composite (standalone).
            • Allows business logic to be modularized and reused
            • Permits access of data in parent process
            • Improves performance and manageability
            • Compensation and fault handling inherited from calling process
            • Faster rendering as only the entity in question is rendered
            The main difference between Standalone and Inline sub processes is that the firsts are visible within the Composite the seconds only within the Bpel Process in which it's been generated.


            Service Bus Templates


            Pipeline ‘Inherited’ Templates
            • New artifact type.
            • Two ways to create a Template
              • Generate template from existing pipeline
              • Create template from scratch.
            • New pipelines are generated from the Template.
            • Derived pipelines are considered concrete
              • Concrete pipelines can be linked (default) or unlinked
              • If linked, all changes in Template are reflected in derived concretes
            • Templates actions can be ‘Locked’

            • Placeholders in Pipeline template to define where modifications may be placed

            Share artifacts within and across applications

            SOA Design-Time MDS Repository
            • Wizard to share design time artifacts such as WSDLs and Schemas
            • Moves the artifacts and its dependencies to Design-Time MDS (viz. schema with WSDL)
            • Updates all the references to the transferred artifacts with MDS URLs on transfer


            Monday, 16 June 2014

            Mocking your services

            Almost all the developers which are using developing WebServices are well aware about SOAP UI, however I've always used only to send manual messages to some WebServices for manual testing.

            SOAP UI much more than that and not even a licensed version is needed for this. A free one is enough.

            Mocking your services is very important if you want to test your own services but you want to do it in isolation, basically without any of the external services (used by your own service to test) to cause of a less determined behavior. Another reason might be because you might want to execute performance test on your services and none of the external has to slow down the test itself.

            Step by Step process:

            • Let's suppose you've got a service already available, if you don't have it just create a new simple one and deploy it somewhere.

            The service in the following example accepts a String parameter which will be used for the mocking response, in your service you can add a String parameters or later you an use a similar approach with a different parameter type.


            • Create in Soap UI a new project and add the WSDL of the service. Here is my service which I am going to mock.





            • Right click at the port type in SOA UI of the imported WSDL and "Generate MockService". Port and Path can be customized. Select at least the operation you are going to mock.
            • As I did, create as many response as you need to verify your business rules. For instance

            Some variable might be used in the response as showed below. SOAP UI uses Groovy as a script language then you might take a look of it.


            • After defining your answers, you have to define how these will be served to the caller; SoapUI provides several strategies:
              • RANDOM – serve a random answer
              • SEQUENCE – iterate through the available answers
              • QUERY_MATCH/XPATH – set up a mapping for each answer based on some queried request value
              • SCRIPT – create a Groovy script to determine the answer The most interesting (and easiest to set up) is the Groovy Script. I have to admit that it took me some time to come up with the script in a programming language I did not know before, but Groovy has a steep learning curve.


            Above is my example, as you can see I've created a mapping

            def symbolMap = [
            "S":"Success" ,
            "F":"Invalid parameters Fault" ,
            "L":"Invalid login"
            ]

            And in the script I get the first letter of the "Password" field and I use it as a key for the MAP above.
            The result , which is the name of the Response, is returned back and SoapUI returns the correct Response to the client.

             
            def xmlHolder = new com.eviware.soapui.support.XmlHolder( mockRequest.requestContent )
            
            // extract the parameters from the request def
            
            def string = xmlHolder["//*[local-name()='Password']"]
            
            def reqSymbol = string.substring(0,1)
            
            
            
            // request by symbol - lookup the appropriate response
            
            log.info "The requested symbol (by symbol)=" + reqSymbol
            
            // if mapping not found: null will be returned
            
            def response = symbolMap[reqSymbol]
            
            return response
            
            
            • Last step for you is to run and test your Mock Service. You've got two options:
              • Start the SOAP UI mock service. This can be started by right clicking at the mock port type and then Start mock service.
              • Deploy as a war from the Project context menu.


            Thursday, 13 February 2014

            Configuring Oracle BPM and SOA Suite 11g with a SQL Authentication Provider

            Login to integrated weblogic server with admin user (weblogic) and password

            click on security realms and select my realm option

            click on new provider to create a new SQLAuthenticator

            provide the name of the SQLAuthenticator and set the control flag as sufficient.

            Provide the data source to the database.
            The table USERS, GROUPS and GROUPS_MEMBER have to be already created
            into the database. For instance this script uses the default table and column
            names.
            Using the attached script the query to access to the users and groups don't need to be
            modified. Otherwise, using a custom DB the query needs to be changed.


            Reorder the providers and keep the SQL one at the top
            Change the control flag of default authenticator from required to sufficient.

            Restart WLS and check that users and groups from the database have been
            imported into WLS.
            For the SOA/BPM environment there is a known issue (Logon To Oracle Workspace Failed With Users Of SQL Authenticator (Doc ID 1499679.1)). I followed this approach as a workaround as suggested by oracle to being able to see the accounts in BPM. Here are the steps to apply it

            • Download the DBProvider
            • Copy dbprovider.jar to MW_HOME/user_projects/domains/{soa_domain}/lib
            • Shutdown SOA and Admin Server
            • Navigate to MW_HOME/user_projects/domains/{domain_name}/config/fmwconfig
            • Take a backup of jps-config.xml
            • Edit jps-config.xml and make the following modifications
            In the section starting with <jpsContext name="default" 
            modify <serviceInstanceRef ref="idstore.ldap"/> to 
            <serviceInstanceRef ref="idstore.custom"/>

            Within the <serviceInstances>...</serviceInstances> section, 
            add the following (change the db_* parameters as per the environment): 
            <serviceInstance name="idstore.custom" provider="custom.provider" location="dumb"> 
            <description>Custom Identity Store Service Instance</description> 
            <property name="idstore.type" value="CUSTOM"/> 
            <property name="ADF_IM_FACTORY_CLASS" value="org.sample.providers.db.DBIdentityStoreFactory"/> 
            <property name="DB_SERVER_NAME" value="db_host_name"/> 
            <property name="DB_DATABASE_NAME" value="db_sid"/> 
            <property name="ST_SECURITY_PRINCIPAL" value="db_user"/> 
            <property name="ST_SECURITY_CREDENTIALS" value="db_passwd"/>
            </serviceInstance>
            

            Within the <serviceProviders>...</serviceProviders> section,
            add the following 
            <serviceProvider type="IDENTITY_STORE" name="custom.provider" 
            class="oracle.security.jps.internal.idstore.generic.GenericIdentityStoreProvider"> 
            <description>Custom IdStore Provider</description> </serviceProvider>
            

            • Start SOA/Admin server





            Oracle Case Management 11g EURENT Configuration


            Case Management is a new addition to Oracle BPM in release 11.1.1.1.7 (PS6). This new release contains the Case Management engine, see blog Léon for more details.
            However, currently this release does not contain a case portal.
            The case management API's, just like the already existing Oracle BPM API's, help in developing a portal page with relative ease.

            In this post I will show how to install, configure and fix common issues for the EURent case management application and portal application developed by Oracle.

            Downloads all the files:
            Case MGM UI
            EurRent
            Demo Community
            Patches for Case Mgm

            Then starts unzipping the EURENT dowloaded file Sample PDF and follow the steps indicated within.

            • Pre-requirements
              • Patch installation for WLS and Java if needed
            • Depoly EURENT
              • Make sure in the property the admin has to point to the ADMIN server and the SERVER has to point to the SOA_SERVER
              • If needed the workflow-001-DemoCommunitySeedApp can be found here
            • Running ANT to create and configure the users and groups, it may happen:
            BUILD FAILED
            C:\oracle\Middleware\11.1.1.7-JDEV\workspace\EURent\case-permission-grant-build.xml:28: The following error occurred while executing this line:
            C:\oracle\Middleware\11.1.1.7-JDEV\workspace\workflow-001-DemoCommunitySeedApp\build.xml:38: The following error occurred while executing this line:
            C:\oracle\Middleware\11.1.1.7-JDEV\workspace\workflow-001-DemoCommunitySeedApp\build.xml:42: Problem: failed to create task or type if
            Cause: The name is undefined.
            Action: Check the spelling.
            Action: Check that any custom tasks/types have been declared.
            Action: Check that any / declarations have taken place.
            

            This only happens because some Target used in this script are not part of the default ANT installation, which then needs to be imported using the following:
            <taskdef resource="net/sf/antcontrib/antlib.xml">
              <classpath>
                <pathelement location="your/path/to/ant-contrib-${version}.jar">
              </pathelement></classpath>
            </taskdef>
            


            Another issue which may happen is that the groups are not correctly associated to the users because the users are not already defined in the LDAP. At the end of the ANT script you may note the following:

            seedDemoCommunity:
                 [echo] Invoking seeding servlet to start seeding demo community...
                 [echo] seed.server.url : http://localhost:8001, seed.admin.url : t3://localhost:7001 seed.admin.name : weblogic 
                 [java] URL : http://localhost:8001/integration/SOADemoCommunity/DemoCommunitySeedServlet
                 [java] inputFile=C:\oracle\Middleware\11.1.1.7-JDEV\workspace\EURent\CasePermissionRoleGrants.xml&adminServer=localhost&adminPort=7001&adminUser=weblogic
                 [java] <html>
                 [java] <head>
                 [java] <title>Demo User Community</title>
                 [java] <link rel="stylesheet" type="text/css" href="stylesheet.css"/>
                 [java] </head>
                 [java] <body>
                 [java] <table border="1" style="background-color:#C8C8C8" align="center" width="70%">
                 [java] <tr>
                 [java] <td>
                 [java] <H1 ALIGN="CENTER">Seed Demo Community</H1>
                 [java] <BR><B>App-Role Grants : </B>
                 [java] </td>
                 [java] </tr>
                 [java] </table>
                 [java] </body>
                 [java] </html>
            


            In "App-Role Grants :" we should see the list of Users-Roles granted. As you can see in the Server log:

            user:jcooper User (jcooper) specified as grantee for app role EURent.CONFIDENTIA
            L.READ.Role does not exist. Please modify the XML input file to create the user
            and rerunuser:jstein User (jstein) specified as grantee for app role EURent.CONF
            IDENTIAL.READ.Role does not exist. Please modify the XML input file to create th
            e user and rerunuser:sfitzger User (sfitzger) specified as grantee for app role
            EURent.CONFIDENTIAL.READ.Role does not exist. Please modify the XML input file t
            o create the user and rerunJMXAppRoleGranter.getMembersForApplicationRole() : IN
             : appStripe : OracleBPMProcessRolesApp, appRole : EURent.CONFIDENTIAL.UPDATE.Ro
            le
            


            Too fix it the users defined in the CasePermissionRoleGrants.xml have to be manually created in the WLS realm, then the script ANT can be re-run again

            the result has to be like this:


            seedDemoCommunity:
                 [echo] Invoking seeding servlet to start seeding demo community...
                 [echo] seed.server.url : http://localhost:8001, seed.admin.url : t3://localhost:7001 seed.admin.name : weblogic
                 [java] URL : http://localhost:8001/integration/SOADemoCommunity/DemoCommunitySeedServlet
                 [java] inputFile=C:\oracle\Middleware\11.1.1.7-JDEV\workspace\EURent\CasePermissionRoleGrants.xml&adminServer=localhost&adminPort=7001&adminUser=weblogic
                 [java] <html>
                 [java] <head>
                 [java] <title>Demo User Community</title>
                 [java] <link rel="stylesheet" type="text/css" href="stylesheet.css"/>
                 [java] </head>
                 [java] <body>
                 [java] <table border="1" style="background-color:#C8C8C8" align="center" width="70%">
                 [java] <tr>
                 [java] <td>
                 [java] <H1 ALIGN="CENTER">Seed Demo Community</H1>
                 [java] <BR><B>App-Role Grants : </B>
                 [java] EURent.PUBLIC.READ.Role is granted to user mmitch.
                 [java] EURent.PUBLIC.READ.Role is granted to user jausten.
                 [java] EURent.PUBLIC.READ.Role is granted to user wfaulk.
                 [java] EURent.PUBLIC.READ.Role is granted to user wshake.
                 [java] EURent.PUBLIC.READ.Role is granted to user jcoope.
                 [java] EURent.PUBLIC.READ.Role is granted to user jstein.
                 [java] EURent.PUBLIC.READ.Role is granted to user sfitzger.
                 [java] EURent.PUBLIC.UPDATE.Role is granted to user mmitch.
                 [java] EURent.PUBLIC.UPDATE.Role is granted to user jausten.
                 [java] EURent.PUBLIC.UPDATE.Role is granted to user wfaulk.
                 [java] EURent.PUBLIC.UPDATE.Role is granted to user wshake.
                 [java] EURent.PUBLIC.UPDATE.Role is granted to user jcooper.
                 [java] EURent.PUBLIC.UPDATE.Role is granted to user jstein.
                 [java] EURent.PUBLIC.UPDATE.Role is granted to user sfitzger.
                 [java] EURent.PUBLIC.INVOKE.Role is granted to user mmitch.
                 [java] EURent.PUBLIC.INVOKE.Role is granted to user jausten.
                 [java] EURent.PUBLIC.INVOKE.Role is granted to user wfaulk.
                 [java] EURent.PUBLIC.INVOKE.Role is granted to user wshake.
                 [java] EURent.PUBLIC.INVOKE.Role is granted to user jcooper.
                 [java] EURent.PUBLIC.INVOKE.Role is granted to user jstein.
                 [java] EURent.PUBLIC.INVOKE.Role is granted to user sfitzger.
                 [java] EURent.CONFIDENTIAL.READ.Role is granted to user jcooper.
                 [java] EURent.CONFIDENTIAL.READ.Role is granted to user jstein.
                 [java] EURent.CONFIDENTIAL.READ.Role is granted to user sfitzger.
                 [java] EURent.CONFIDENTIAL.UPDATE.Role is granted to user jcooper.
                 [java] EURent.CONFIDENTIAL.UPDATE.Role is granted to user jstein.
                 [java] EURent.CONFIDENTIAL.UPDATE.Role is granted to user sfitzger.
                 [java] EURent.CONFIDENTIAL.INVOKE.Role is granted to user jcooper.
                 [java] EURent.CONFIDENTIAL.INVOKE.Role is granted to user jstein.
                 [java] EURent.CONFIDENTIAL.INVOKE.Role is granted to user sfitzger.
                 [java] </td>
                 [java] </tr>
                 [java] </table>
                 [java] </body>
                 [java] </html>
            
            

            • At this stage the UI has to be deployed. Just configure the deploy script with your environment variable and run it. It creates the EAR which can be manually deployed through the WLS console. Deploy it to the SOA_SERVER, if different than the ADMIN. The console can be tested at this point accessing as a granted user (mitch jstein, ...)
            • Now all has been deployed and configured, it is time to start a case! Open the SampleInput XML and call the EURENT.service (note that in some case it may be good to comment "<ns2:updatedDate></ns2:updatedDate>", I got some issue because of this field). The service can be called:
              • through the EM (accessing to the service in SOA-infra)
              • accessing to http://<host>:<port>/soa-infra selecting EURENT.service


            • Now connecting to the sample UI as a mitch, in case you don't see any record it maybe because the Groups in the BPM/workspace have not been configured correctly

            Access to the BPM Console and check if the highlighted groups have been granted to someone.
            In my case they weren't, then I granted all of them to weblogic.



            Then accessing to the CMSampleUI again as weblogic user I see the created records.


            • asas