GridControl 12c switch repository database to RAC

This post may be useful if you consider scale out Oracle Enterprise Manager Cloud Control 12c with RAC database. Migration to the cluster is quite simple with rconfig utility. You can start with Oracle Database documentation or google for numerous posts about that process. I omit database migration by itself and describe the final steps only. In an ideal world, you always have a source instance and new shiny RAC, but in reality, you may not have enough resources and you have converted database in-place, as I've done?
Well, it means that you can't have both databases up and running at the same time. For my case original single instance repository has become a cluster node, which means OMS is down and emctl utility throws errors and doesn't change the configuration.Here you may find a few steps how to address this issue:
1. Open a new terminal connection and start AdminServer from the command line:

[oracle@host]$$MW_HOME/gc_inst/user_projects/domains/GCDomain/bin/startWebLogic.sh

It will throw a l-o-o-o-t of error messages about JPS database availability. Just keep it going until you 'd have your RUNNING state. Now you should be able to run $OMS_HOME/bin/emctl  normally nd update database connection descriptor using documentation steps. Just use another terminal window to keep AdminServer up and running.
2. OMS configuration doesn't understand format host:port:service, which means you have the only choice for cluster databases: SQL*Net service descriptors. Example for SCAN listeners would look like the sample command below:

[oracle@host]$ $OMS_HOME/bin/emctl config oms -store_rpos_details -repos_conndesc \
"(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=scan-listener.your.domain.com)(PORT=1521)))(CONNECT_DATA=(SERVICE=omsservice.yourdomain.com)))"  -repos_user sysman

3. Service Descriptor is the standard SQL*Net format with some limitations (at least on Linux platforms):

• The descriptor is enclosed in double quotes. 
• Descriptor shouldn't contain any spaces. 

Return to the first session and stop AdminServer with Ctrl-Break. Now you can start Grid Control as usual, repository connection points to the clustered repository.

WebCache 11g. Name based balance

We used to use WebCache as load balancer for SOA 10/11g cluster from the first prototypes. I have to notice that it's the only one component that has no issues for ages.

Soб for some reasons we should to reconfigure OiD/OIDDAS server and change the access url for sso and oiddas services. I decided to configure WebCache as a proxy to pass user's request to identity server and vice versa. I've created additional server definitions for ports 80 and 443. Network admins, changed for us DNS and allowed Webcache access identity system by application ports (they are defaults 7777 and 4443).

Site definitions

Then, I've created site to server mappings, and voila, the system is ready to manage requests soa11 for a cluster and ldap for user management.  After a while, SOA applications admin noticed me that they couldn't see some services, so the process couldn't receive callbacks. 

The results of the short investigation  was - If some process calls web service by the server name for example - http://soa11.telecom/osb/someService  - it works fine, if it uses IP address it fails.

Gosh, when they start using names instead of addresses! By the way, I have to manage this situation; because even developers immediately redeploy the new versions, there are still be callback messages with direct IP.

So the problem is on the top, before reconfiguration, WebCache sent any request with any name to only one destination (actually balance to the two HTTP servers) and now it has to decide who will be receiver for the message. The solution is quite easy; I’ve added aliases to the SOA site definitions, so the final site definitions table is

Final Site definitions table

In the last column you can see couple aliases for the site definition. Apply configuration, restart WebCache and now it definitely works.