BEA-141196 - The managed server could not update the configuration files during the registration with the deployment service

Issue: When starting manager server Instance from the Admin console sometimes the remote managed server instances do not come up due to the below errors noticed in std out file

 <Error> <Management> <BEA-141196> <The managed server could not update the configuration files during the registration with the deployment service. The update failed due to an exception:

weblogic.management.DeploymentException: Exception occured while downloading files
        at weblogic.deploy.internal.targetserver.datamanagement.ConfigDataUpdate.doDownload(ConfigDataUpdate.java:83)
        at weblogic.deploy.internal.targetserver.datamanagement.DataUpdate.download(DataUpdate.java:56)
        at weblogic.deploy.internal.targetserver.datamanagement.Data.prepareDataUpdate(Data.java:97)
        at weblogic.management.provider.internal.RuntimeAccessDeploymentReceiverService.downloadFiles(RuntimeAccessDeploymentReceiverService.java:921)
        at weblogic.management.provider.internal.RuntimeAccessDeploymentReceiverService.updateFiles(RuntimeAccessDeploymentReceiverService.java:879)
        at weblogic.management.provider.internal.RuntimeAccessDeploymentReceiverService.handleRegistrationResponse(RuntimeAccessDeploymentReceiverService.java:728)
        at weblogic.management.provider.internal.RuntimeAccessDeploymentReceiverService.registerHandler(RuntimeAccessDeploymentReceiverService.java:700)
        at weblogic.management.provider.internal.RuntimeAccessDeploymentReceiverService.start(RuntimeAccessDeploymentReceiverService.java:169)
        at weblogic.t3.srvr.ServerServicesManager.startService(ServerServicesManager.java:461)
        at weblogic.t3.srvr.ServerServicesManager.startInStandbyState(ServerServicesManager.java:166)
        at weblogic.t3.srvr.T3Srvr.initializeStandby(T3Srvr.java:882)
        at weblogic.t3.srvr.T3Srvr.startup(T3Srvr.java:572)
        at weblogic.t3.srvr.T3Srvr.run(T3Srvr.java:469)
        at weblogic.Server.main(Server.java:71)
Caused By: java.io.IOException: Posted content doesn't set it's Content-Length
        at 

Resolution:
Copy the entire config directory from your domain admin server in to the remote managed server config directory and restart the server. 

 

BEA-110482 error when extending an existing SOA Domain to include OSB

Issue:
When you extend an existing soa_domain to include OSB servers and use the same SOAINFRA schema for   wlsbjmsrpDataSource then you might see the below errors


<Critical> <JTA> <BEA-110482> <A logging last resource failed during initialization. The server cannot boot unless all configured logging last resources (LLRs) initialize. Failing reason:
weblogic.transaction.loggingresource.LoggingResourceException: java.sql.SQLException: JDBC LLR, table verify failed for table 'WL_LLR_OSB_SERVER1', row 'JDBC LLR Domain//Server' record had unexpected value 'osb_domain//osb_server1' expected 'soa_domain//osb_server1'*** ONLY the original domain and server that creates an LLR table may access it ***

weblogic.transaction.loggingresource.LoggingResourceException: java.sql.SQLException: JDBC LLR, table verify failed for table 'WL_LLR_OSB_SERVER1', row 'JDBC LLR Domain//Server' record had unexpected value 'osb_domain//osb_server1' expected 'soa_domain//osb_server1'*** ONLY the original domain and server that creates an LLR table may access it ***
        at weblogic.jdbc.wrapper.JTSLoggableResourceImpl.recoverXARecords(JTSLoggableResourceImpl.java:248)
        at weblogic.transaction.internal.ServerTransactionManagerImpl.registerLoggingResourceTransactions(ServerTransactionManagerImpl.java:747)
        at weblogic.jdbc.common.internal.RmiDataSource.recoverLoggingResourceTransactions(RmiDataSource.java:302)
        at weblogic.jdbc.common.internal.DataSourceManager.createAndStartDataSource(DataSourceManager.java:148)
        at weblogic.jdbc.common.internal.DataSourceManager.createAndStartDataSource(DataSourceManager.java:98)
        Truncated. see log file for complete stacktrace
Caused By: java.sql.SQLException: JDBC LLR, table verify failed for table 'WL_LLR_OSB_SERVER1', row 'JDBC LLR Domain//Server' record had unexpected value 'osb_domain//osb_server1' expected 'soa_domain//osb_server1'*** ONLY the original domain and server that creates an LLR table may access it ***
        at weblogic.jdbc.wrapper.JTSLoggableResourceTable.newVerifyException(JTSLoggableResourceTable.java:244)
        at weblogic.jdbc.wrapper.JTSLoggableResourceTable.findOrCreateTable(JTSLoggableResourceTable.java:327)
        at weblogic.jdbc.wrapper.JTSLoggableResourceImpl.findOrCreateBackingTable(JTSLoggableResourceImpl.java:528)
        at weblogic.jdbc.wrapper.JTSLoggableResourceImpl.recoverXARecords(JTSLoggableResourceImpl.java:241)
        at weblogic.transaction.internal.ServerTransactionManag

Resolution:

This exception is a result of a wrong domain name in WL_LLR_ADMINSERVER table and WL_LLR_OSB_SERVER1 tables. You can fix it by running the below queries 

update WL_LLR_OSB_SERVER1

set RECORDSTR = 'soa_domain//osb_server1'

where XIDSTR = 'JDBC LLR Domain//Server';

update WL_LLR_ADMINSERVER
set RECORDSTR = 'soa_domain//Adminserver'
where XIDSTR = 'JDBC LLR Domain//Server';

In my example, my domain name is soa_domain and I have a separate Admin Server (Adminserver is the name of the admin) and osb_server1 (OSB Managed server)

Java Errors when using the Test Connection feature in Primavera P6 Administrator

Issue:

Java Errors when using the Test Connection feature in Primavera P6 Administrator while testing connection from Directory Services or Database Eventing (Primavera Deployed on Weblogic)

Sample Screenshots of errors:

Resolution:

While Configuring Directory Services for Database Eventing, most likely you want to test the connection using the “Test Connection” feature in the P6 Administrator Thick client (Right click and select Test Connection).  Oracle Documentation is pretty clear in terms of how to configure the Directory Services and Eventing but they simply forgot to mention that if we need to use the Test Connection feature in the P6 Administrator we need to copy the wlfullclient.jar in to $PRIMAVERA_HOME/lib directory and update the CLASSPATH in adminpv.sh or adminpv.cmd file.

If you already have a wlfullclient.jar in any other environments running on the same version of weblogic, you can copy it otherwise follow the below instructions to generate your own wlfullclient.jar and then copy it to the $PRIMAVERA_HOME/lib directory

Steps to generate the wlfullclient.jar  (Oracle Documentation)

Use the following steps to create a wlfullclient.jar file for a client application:

1.       Change directories to the server/lib directory.

cd WL_HOME/server/lib

2.       Use the following command to create wlfullclient.jar in the server/lib directory:

java -jar ../../../modules/com.bea.core.jarbuilder_X.X.X.X.jar

where X.X.X.X is the version number of the jarbuilder module in the WL_HOME/server/lib directory. For example:

java -jar ../../../modules/com.bea.core.jarbuilder_1.0.1.0.jar

3.       You can now copy and bundle the wlfullclient.jar with client applications.

4.       Add the wlfullclient.jar to the client application’s classpath.

After you generate the wlfullclient.jar, copy it to the $PRIMAVERA_HOME/lib directory and update the classpath to include the jar.

 Your updated classpath should like below

"%JAVA_COMMAND%\java" -classpath "%PVLIBDIR%/prm-console.jar;%PVLIBDIR%/prm-bpm-common.jar;%PVLIBDIR%/prm-common.jar;%PVLIBDIR%/resource_strings.jar;%PVLIBDIR%/prm-infr.jar;%PVLIBDIR%/prm-bo-admin.jar;%PVLIBDIR%/prm-initializer.jar;%PVLIBDIR%/ojdbc6.jar;%PVLIBDIR%/spring.jar;%PVLIBDIR%/sqljdbc.jar;%PVLIBDIR%/xml-apis.jar;%PVLIBDIR%/xercesImpl.jar;%PVLIBDIR%/xalan.jar;%PVLIBDIR%/serializer.jar;%PVLIBDIR%/commons-logging.jar;%PVLIBDIR%/log4j.jar;%PVLIBDIR%/mail.jar;%PVLIBDIR%/jide-common.jar;%PVLIBDIR%/jide-grids.jar;%PVLIBDIR%/prm-enm.jar;%PVLIBDIR%/wlfullclient.jar" -Dadmin.workingDir="%PV_HOME%" -Dadmin.console="n" %PRIMAVISION_OPTS% com.primavera.admintool.AdminApp

After the Copy, the Test Connection Feature should work fine and you should see the below screens.

References:

http://docs.oracle.com/cd/E17266_01/English/Technical_Documentation/Event_Handling/Using%20Events%20with%20P6.pdf

http://docs.oracle.com/cd/E11035_01/wls100/client/jarbuilder.html

BAM-01257 Error while drilling down to reports in BAM Active Viewer

Issue:
Users reporting BAM-01257 Error while drilling down to reports in BAM Active Viewer

Resolution:

1. Login to the BAM as administrator ( http://hostname:<bam_server_port>/OracleBAM )
2. Click on the Architect View
3. On the Left Hand panel select the DataObject (Default is system) that the user is referencing for the Reports

    4.  On the right hand panel select the permissions

    5. Click on the edit permissions and selct the user that got the above permissions error and give them atleast the read privileges. Save

    6. Request the user to retry it again


References:

http://www.oracle.com/technetwork/middleware/bam/documentation/bam-trouble-133227.pdf

BEA-000386 - oracle.security.jps.JpsException: [PolicyUtil] Exception while getting default policy Provider

Issue: AdminServer doesnt startup due to the below JPS errors in 10.3.4 or 10.3.5

####<Jul 31, 2012 1:35:49 PM EDT> <Critical> <WebLogicServer>  <AdminServer> <main> <<WLS Kernel>> <> <> <1343756149538> <BEA-000386> <Server subsystem failed. Reason: weblogic.security.SecurityInitializationException: The loading of OPSS java security policy provider failed due to exception, see the exception stack trace or the server log file for root cause. If still see no obvious cause, enable the debug flag -Djava.security.debug=jpspolicy to get more information. Error message: oracle.security.jps.JpsException: [PolicyUtil] Exception while getting default policy Provider
weblogic.security.SecurityInitializationException: The loading of OPSS java security policy provider failed due to exception, see the exception stack trace or the server log file for root cause. If still see no obvious cause, enable the debug flag -Djava.security.debug=jpspolicy to get more information. Error message: oracle.security.jps.JpsException: [PolicyUtil] Exception while getting default policy Provider
        at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.loadOPSSPolicy(CommonSecurityServiceManagerDelegateImpl.java:1398)
        at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initialize(CommonSecurityServiceManagerDelegateImpl.java:1018)
        at weblogic.security.service.SecurityServiceManager.initialize(SecurityServiceManager.java:873)
        at weblogic.security.SecurityService.start(SecurityService.java:141)
        at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
        at weblogic.work.ExecuteThread.execute(ExecuteThread.java:207)
        at weblogic.work.ExecuteThread.run(ExecuteThread.java:176)
Caused By: oracle.security.jps.JpsRuntimeException: oracle.security.jps.JpsException: [PolicyUtil] Exception while getting default policy Provider
        at oracle.security.jps.internal.policystore.PolicyDelegationController.<init>(PolicyDelegationController.java:291)
        at oracle.security.jps.internal.policystore.PolicyDelegationController.<init>(PolicyDelegationController.java:282)
        at oracle.security.jps.internal.policystore.JavaPolicyProvider.<init>(JavaPolicyProvider.java:261)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at java.lang.Class.newInstance0(Class.java:355)
        at java.lang.Class.newInstance(Class.java:308)
        at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.loadOPSSPolicy(CommonSecurityServiceManagerDelegat

Resolution:
Actually, I have seen this issue twice in my career. Two completly different clients and projects :-(
Resolution for the issue was very similar both times but not the same. Root cause is similar i.e. possible corruption of policy stores or Embedded LDAP due to disk space full or some other crap

First time, i was able to solve the problem by simply copying the following files from a working domain or a backup of the existing domain:
Directory: $DOMAIN_HOME/config/fmwconfig
Files: system-jazn-data.xml and cwallet.sso

Second time when I faced the similar issue, I had to copy the entire $DOMAIN_HOME/config/fmwconfig from a different working domain and delete the $DOMAIN_HOME/servers/AdminServer directory and restart the AdminServer. I had to let it recreate the AdminServer directory again and that fixed the problem.

BEA-141281 Issue in Weblogic 11g

Issue:
If you get the below errors when starting the admin server or a managed server, follow the below steps in resolution
<Jul 31, 2012 10:16:15 AM EDT> <Info> <Management> <BEA-141281> <unable to get file lock, will retry ...>
<Jul 31, 2012 10:16:25 AM EDT> <Info> <Management> <BEA-141281> <unable to get file lock, will retry ...>
<Jul 31, 2012 10:16:35 AM EDT> <Info> <Management> <BEA-141281> <unable to get file lock, will retry ...>
<Jul 31, 2012 10:16:45 AM EDT> <Info> <Management> <BEA-141281> <unable to get file lock, will retry ...>
<Jul 31, 2012 10:16:55 AM EDT> <Info> <Management> <BEA-141281> <unable to get file lock, will retry ...> 


Resolution:

Kill the hanging server start process (Kill -9 on the process ID)
Delete the edit.lok under the $DOMAIN_HOME and AdminServer.lok under the $DOMAIN_HOME/servers/<Server_Name>/tmp directory.

Restart the server

ConfigFwk:390105- OSB 11g Administration Issues

OSB 11g Administration Issue: 

Issue: 

If you encounter error ConfigFwk:390105  while importing new configuration in OSB console, follow the below steps to resolve it

Solution: Shutdown your OSB server instance and go to the OSB Domain directory and delete the edit.lok file. After you delete the file, restart the OSB Instance.