XPath - Terra Ignota

There are more things in heaven and earth, Horatio,

Than are dreamt of in your philosophy.

As usual before  holidays we've got another issue with timeouts,  adapters errors and database performance.

I found that DB Adapter used several queries and then joins result sets. This way get us right result (very simplified version you can see  bellow) with pretty far from optimal performance. 

<off:OfferActionCollection xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                           xmlns:off="http://xmlns.oracle.com/offer">
  <off:OfferAction>
    <off:id>2121</off:id>
    <off:offerPositionId>
      <off:id>2133</off:id>
      <off:positionNum>3</off:positionNum>
    </off:offerPositionId>
    <off:offerActionDetalCollection>
      <off:OfferActionDetal>
        <off:id>2262</off:id>
      </off:OfferActionDetal>
      <off:OfferActionDetal>
        <off:id>2263</off:id>
      </off:OfferActionDetal>
    </off:offerActionDetalCollection>
  </off:OfferAction>
</off:OfferActionCollection>

Option  'Use Outer Join ..' corrects  query issues , but returns non-unique collection. Like this one:

<off:OfferActionCollection xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                           xmlns:off="http://xmlns.oracle.com/offer">
  <off:OfferAction>
    <off:id>2121</off:id>
    <off:offerPositionId>
      <off:id>2133</off:id>
      <off:positionNum>3</off:positionNum>
    </off:offerPositionId>
    <off:offerActionDetalCollection>
      <off:OfferActionDetal>
        <off:id>2262</off:id>
      </off:OfferActionDetal>
      <off:OfferActionDetal>
        <off:id>2263</off:id>
      </off:OfferActionDetal>
    </off:offerActionDetalCollection>
  </off:OfferAction>

  <off:OfferAction>
    <off:id>2121</off:id>
    <off:offerPositionId>
      <off:id>2133</off:id>
      <off:positionNum>3</off:positionNum>
    </off:offerPositionId>
    <off:offerActionDetalCollection>
      <off:OfferActionDetal>
        <off:id>2262</off:id>
      </off:OfferActionDetal>
      <off:OfferActionDetal>
        <off:id>2263</off:id>
      </off:OfferActionDetal>
    </off:offerActionDetalCollection>
  </off:OfferAction>
</off:OfferActionCollection>

I realised that I don't know much about DB Adapter, so we decide to create XQuery/XPath (or anything else)  to filer collection. 

Well, in fact now I realise that I know about X* almost nothing. By the way, after several hours googling, writing and trying  I found an acceptable message transformation expression.

It loops thru distinct values of OfferAction/id (for clause) and returns first node of Xpath result ( /OfferAction[off:id=$ActionId][1]).

The hardest thing for me was - how to combine XPath filters and expressions.

It works just perfect, but I think that next time I'll play on my own field - 

 queries, indexes and hints.

Happy New Year to everyone! Hope to see you in the next year.

Oracle BAM 11g. Report optimizations

I'm trying to keep in touch with our SOA development team. So I've been surprised when I know that they aren't going to maintain and develop any BAM reports for new composites. During discussion they mentioned very poor  reports performance, they tired to optimize it, create several indexes, but without any noticeable result.
Errare humanum est - said Ancient Romans, they leaved to us a lot of such phrases. And  I also have several rules. Meet with one of these:

" The most important part of performance tuning is the database design and optimization"

You can tune heap and garbage collection, you can try to split threads and assign it to separate processor cores. But you always can crate such "great" design, that your efforts will give you almost nothing, because all performance will be limited by disks IO.
I know what you could say, and I have to notice that I'm talking about old-school relational databases. Well lets go back to the BAM case.

After short inspection, I've noticed that there are  only two big data objects, and several lookup tables, so reports were created against classic star - schema.
In fact, I was able to get report with aggregate composites information, but I was not able to open detailed process report.
It's sounds quite strange for me, so I took a look to the  layout of  main data objects.

Lookup fields in the BAM data object

If you are not familiar with BAM, I should explain what BAM data object is.

Using  BAM Architect you can manipulate with data representation objects. Then  you can build reports against your objects in  BAM Studio.  From the other side BAM server publishes  data objects with a set of web services and you can populate it with data. The most common using of BAM is a SOA monitoring  dashboards.

By the way, from the performance tuning perspective you should know two things:

•  You can build indexes against Data Objects fields. 
• BAM Data Objects use database tables and indexes to persist information.

Hmmm (I like this sound so much), is there any indexes for lookup fields and in the lookup data objects? The short answer was - NO, and I've immediately created them. 
Actually you can use Data Architect capabilities and don't ask DBA to do this.

Add Index to the data object

Just open Indexes part (1) of data object description and click to Add Index (2). Select object fileds (database columns) and name it.

When I've done it, against all objects and all source and lookup fields. The BAM reports start works, like a charm.   




 

Siebel & SOA misscommunication.

Quite confusing situation had resolved couple minutes ago.

The history was started by Sibel implementation team, they have noticed strange hangs for 5-7 seconds while user  process orders. So they started  investigation and founded the reason (They think so). There were detected strange Siebel EAI behavior.

1. EAI  receive request from a Siebel user
2. Send it to our LBR-OHS and OSB cluster process it.
3. Then OSB send response back to EAI
4. EAI waits for a 5 second 
5. And then send respond to user.

Siebel team points to wrong  OHS configuration particularly to keepAlive and keepAliveTimeout. It is enabled by default and timeout is 5 seconds. Ping !

The next step was mine. I've create the simplest WSDL for a echoing Proxy Service, and ran several tests with different scenarios, using soapUI.

Well there is no timeouts at all. Actually there were several really long responses, but the min response time was around 5-9 ms, and any average less 100 ms.

I've sent porject, report and explanation to Siebel team. Pong!

Guys  from other side reproduced the test and found the root of the issue.
soapUI uses header user-agent with a value Jakarta Commons-HttpClient/3.0.1, meanwhile Siebel EAI set user-agent to  Mozilla/4.0. Ping !


Hmm, quite strange. When I set user-agent to Mozilla/4.0 I get the same result, the client (soapUI in my case), waits for a keepAliveTimeout and only then responses.
So i don't know how the system behavior will be changed if I switch off keepAlive.

But what I know, that you can change only one parameter in Siebel Enterprise Application Integration and avoid such cumbersome cases.

Oracle Portal and WSRP Producer

If you are developing WSRP portlet provider with JDeveloper/Weblogic, and going to implement some security, you may have some issue like we had.

Or you have not if you wouldn't  use Oracle Portal.

The simplest thing you should  get user name and make some decision or just greet the user. 

So the standard way is:

 .....

PortletRenderRequest portletRequest = (PortletRenderRequest)

                            request.getAttribute("oracle.portal.PortletRenderRequest");

String name = portletRequest.getRemoteUser();

......

Most likely that name will be null. We wasted some time, trying to find the solution. Thanks Google, The Wonderful and Manoj we find it. Here you are:

Add this option to your portlet.xml and you will get user name from Oracle Portal or Webcenter application.

...

<container-runtime-option>

     <name>com.oracle.portlet.useWsrpUserContextForUserAuthenticationInfo</name> 

     <value>true</value>

</container-runtime-option>

.....

P.S. It definitively works with Oracle Portal  11.1.1.4 and WebCenter 11.1.1.5

WebCache 11g. Name based balance

We used to use WebCache as load balancer for SOA 10/11g cluster from the first prototypes. I have to notice that it's the only one component that has no issues for ages.

Soб for some reasons we should to reconfigure OiD/OIDDAS server and change the access url for sso and oiddas services. I decided to configure WebCache as a proxy to pass user's request to identity server and vice versa. I've created additional server definitions for ports 80 and 443. Network admins, changed for us DNS and allowed Webcache access identity system by application ports (they are defaults 7777 and 4443).

Site definitions

Then, I've created site to server mappings, and voila, the system is ready to manage requests soa11 for a cluster and ldap for user management.  After a while, SOA applications admin noticed me that they couldn't see some services, so the process couldn't receive callbacks. 

The results of the short investigation  was - If some process calls web service by the server name for example - http://soa11.telecom/osb/someService  - it works fine, if it uses IP address it fails.

Gosh, when they start using names instead of addresses! By the way, I have to manage this situation; because even developers immediately redeploy the new versions, there are still be callback messages with direct IP.

So the problem is on the top, before reconfiguration, WebCache sent any request with any name to only one destination (actually balance to the two HTTP servers) and now it has to decide who will be receiver for the message. The solution is quite easy; I’ve added aliases to the SOA site definitions, so the final site definitions table is

Final Site definitions table

In the last column you can see couple aliases for the site definition. Apply configuration, restart WebCache and now it definitely works.

OSB 11g - Siebel communication

Introduction

For a last five years I'm involved in a huge SOA project in a telco company.
So there were a quite interesting case.

We are migrating some helper processes from BPEL 10g to OSB 11g.
The main purpose for these utilities to mediate between Siebel services and BPEL process and composites.

So during process migration and tests we've got Business Service exception :
Error Code: 10944629 Error Message: Error: Missing body length in response to SWE request!
Metalink has a note ID 1301243.1the main idea of this document that SOAP message is sending to the Siebel service with  parameter Transfer-Encoding: chunked, so you should change the way of message delivery.

Solution
Oracle Service Bus  has a wide range of parameters, and you able to control almost any aspect of communication between business service and service provider. To do it, i just remove mark in Advanced section of HTTP Transport configuration.

Nothing interesting in there, but there are findings, that could be useful for you, my guests. I had to be sure, that we are the source of the issues, and fully realize that.

Utility
The OSB 11g Cluster runs on two OEL 64bit boxes, on the customer site and I don't want to install JDeveloper or any other IDE only for SOAP over HTTP check.
The second point is that I don't want to harvest TCP/IP messages in a log files.
Googling for tcp tunnel for Linux points me to Apache's utility TcpTunnelGui
Gotcha !! Another small investigation and I know where to find it on my server, and on your's too. I've found classes org.apache.soap.util.net.TcpTunnel & org.apache.soap.util.net.TcpTunnelGui in the library $FMW_HOME/osb/harvester/soap-2.2.jar


Proof

1. Create a clones of original proxy and business services
2. Point the Business Service to host with Apache's utility and one of free ports. Let's assume it as 8080.
3. Modify proxy clone, to route on the new business serivce.
4. Run the tunneler :
    $ . $DOMAIN_HOME/bin/setDomainEnv.sh

   $ java -classpath $FMW_HOME/osb/harvester/lib/soap-2.2.jar  org.apache.soap.util.net.TcpTunnelGui 8080 10.6.4.26 80

To run tunneler we should pass three parameters
- port to listen (8080)
- origin host (10.6.4.26)
- origin port (80)
5. Test cloned service.

We easily can see the transport information and SOAP messages.

Regards,

How to train your bear.

If you want to be scare, you should do it like me!