How to Change the OPSS Schema Password

1- Stop all WLS servers

2- Change the DATA source password with WLST (offline)

2.1- Run  $MW_HOME/oracle_common/common/bin/wlst.sh to set the wlst environment.

2.2- Use the readDomain WLST command to read the configuration of the domain in offline mode,
wls:/offline>readDomain (‘<WLS_DOMAIN_HOME_PATH>’)
Example:

wls:/offline> readDomain('/opt/Middleware/user_projects/domains/soa_domain')
2.3- Use cd(‘JDBCSystemResource/<OPSSDS name>/JdbcResource/<OPSSDS name>/JDBCDriverParams/NO_NAME_0’) 
Where "OPSSDS name" is data source name. The data source name can be found in DOMAIN_HOME/config/jdbc/opss-jdbc.xml.
Example:

wls:/offline>cd('JDBCSystemResource/OPSSDS/JdbcResource/OPSSDS/JDBCDriverParams/NO_NAME_0')
2.4- The WLST set(‘<parameter_name>’,’<parameter_value>’) is used to change a parameter offline. To change the password ,  set(‘PasswordEncrypted’,’<password_value>’)
Example:

wls:/offline/soa_domain/JDBCSystemResource/OPSSDS/JdbcResource/OPSSDS/JDBCDriverParams/NO_NAME_0>set('PasswordEncrypted','welcome2')
2.5- After all the changes to the parameters are done, the updateDomain() command needs to run to save them to the domain configuration:
Example:

wls:/offline/soa_domain/JDBCSystemResource/OPSSDS/JdbcResource/OPSSDS/JDBCDriverParams/NO_NAME_0>updateDomain()
2.6- Run  exit() to exit WLST. 
Example:

wls:/offline/soa_domain/JDBCSystemResource/OPSSDS/JdbcResource/OPSSDS/JDBCDriverParams/NO_NAME_0>exit()
2.7- Run exit() to exit WLST.
Example:

wls:/offline/soa_domain/JDBCSystemResource/OPSSDS/JdbcResource/OPSSDS/JDBCDriverParams/NO_NAME_0>exit()
Later:

3- Change the OPSS schema password in the database

4- The next steps are required if there is a bootstrap credential map specified in the "props.db.1" section in jps-config.xml
Go to the oracle_common\common\bin>wlst.sh

5- Run modifyBootStrapCredential(jpsConfigFile='./jps-config.xml', username='<schema_owner>', password='<db user password>')
Example:

modifyBootStrapCredential(jpsConfigFile='/opt/Middleware/user_projects/domains/soa_domain/config/fmwconfig/jps-config.xml', username='DEV_OPSS', password='welcome1')

Enable coherence for SOA Servers

Please follow the below steps to enable coherence for SOA Servers.

ms01- 

hostname=soaserver1.us.com
portnumber=50561

ms02- 

hostname=soaserver2.us.com
portnumber=50562

Please add the fololwoiing enttires into server start arugemtns 

ms01--> configuration --> ServerStart --> Arguments 


ms01 

-Dtangosol.coherence.wka1=soaserver1.us.com -Dtangosol.coherence.wka2=soaserver2.us.com -Dtangosol.coherence.localhost=soaserver1.us.com -Dtangosol.coherence.wka1.port=50561 -Dtangosol.coherence.wka2.port=50562 -Dtangosol.coherence.localport=50561 -Dtangosol.coherence.log=/opt/domains/soa_domain/coherence_ms01.log -Dtangosol.coherence.log.level=9 

ms02--> configuration --> ServerStart --> Arguments 

ms02 

-Dtangosol.coherence.wka1=soaserver1.us.com -Dtangosol.coherence.wka2=soaserver2.us.com -Dtangosol.coherence.localhost=soaserver2.us.com -Dtangosol.coherence.wka1.port=50561 -Dtangosol.coherence.wka2.port=50562 -Dtangosol.coherence.localport=50561 -Dtangosol.coherence.log=/opt/domains/soa_domain/coherence_ms02.log -Dtangosol.coherence.log.level=9 

Reset the AdminServer Password in WebLogic 11g and 12c

If you forget the AdminServer password for your WebLogic 11g domain, you can reset it from the command line using the following process.

Set up the following environment variables. They are not necessary for the process itself, but will help you navigate. In this case my domain is called "ClassicDomain". Remember to change the value to match your domain.

export MW_HOME=/u01/app/oracle/middleware
export DOMAIN_HOME=$MW_HOME/user_projects/domains/ClassicDomain
Shut down the WebLogic domain.

$ $DOMAIN_HOME/bin/stopWebLogic.sh
Rename the data folder.

$ mv $DOMAIN_HOME/servers/AdminServer/data $DOMAIN_HOME/servers/AdminServer/data-old
Set the environment variables.

$ . $DOMAIN_HOME/bin/setDomainEnv.sh
Reset the password using the following command. Remember to substitute the appropriate username and password.

$ cd $DOMAIN_HOME/security
$ java weblogic.security.utils.AdminAccount <username> <password> .
Update the "$DOMAIN_HOME/servers/AdminServer/security/boot.properties" file with the new username and password. The file format is shown below.

username=<username>
password=<password>
Start the WebLogic domain.

$ $DOMAIN_HOME/bin/startWebLogic.sh

SOA 12c RCU Installation or SOA 12.1.1.3.0 RCU installation

In SOA 12c RCU binary file will come with installation itself.

Here is the RCU location.

ORACLE_HOME\oracle_common\bin>rcu.bat

i.e C:\Oracle\MiddlewareSOA_1213\oracle_common\bin>rcu.bat 

Open the command prompt with Administrator right navigate to rcu location.

C:\Oracle\MiddlewareSOA_1213\oracle_common\bin>rcu.bat

I

In the welcome screen click on Next

In 12c we have new option.

1       We can create repository if we have sys access

2      We can create scripts and provide that scripts to DBA to run

    

Provide DB information and click on Next 

Select Ignore if you are using XE Database.

Select the schema and click on Next 

Click on Ok

Provide the common password for all schemas or you can provide different password for each schema

click on Next

Select database profile (small/medium/large) and click on Next 

Create table spaces

Click on create 

how to drop SOA RCU schemas in oracle

Using the below command we can drop the RCU schemas with out running RCU.

Connect System User.And run the following commands.

DROP USER test_mds CASCADE; 

DROP USER test_soainfra CASCADE; 

DROP USER test_orabam CASCADE; 

DROP USER test_orasdpm CASCADE; 

MDS Export using WLST

1. Launch Weblogic Scripting Tool:

Weblogic_HOME/common/bin>wslt.cmd

2. Connect to weblogic:

connect('weblogic','welcome1','t3://localhost:7001')

3. Verify the list of all available commands:

help(all)

4. List all applications:

listApplications()

5. Check the syntax of the export Metadata command:

help('exportMetadata')

6. Execute the export

exportMetadata(application='soa-infra', server='soa_server1', toLocation='c:\\export')

Please note that on Windows both

toLocation='c:\\export') and toLocation='c:\export') (where the difference is one or two slashes after 'c:') will work.

However, toLocation='c:\export\my_export') will fail with the following error:

Traceback (innermost last):
  File "<console>", line 1, in ?
  File "C:\WLS103~1\ORACLE~1\common\wlst\mdsWLSTCommands.py", line 237, in exportMetadata
  File "C:\WLS103~1\ORACLE~1\common\wlst\mdsWLSTCommands.py", line 733, in executeAppRuntimeMBeanOperation
  File "C:\WLS103~1\ORACLE~1\common\wlst\mdsWLSTCommands.py", line 996, in saveStackAndRaiseException
  WLSTException: MDS-90035: The target directory or archive "c:   ar\export" for exporting metadata is
 invalid.MDS-91009: Operation "exportMetadata" failure. Use dumpStack() to view the full stacktrace.

But, toLocation='c:\\export\\my_export') will work fine

Not able to start AdminServer,Caused By: oracle.security.jps.JpsRuntimeException: JPS-00056: Failed to create identity store service instance idstore.ldap.provider:idstore.ldap. Reason: oracle.security.jps.JpsRuntimeException: JPS-00027: internal error You configured a generic WLS LDAPAuthenticator. The identity store type cannot be determined. Please choose an LDAP Authentication provider that matches your LDAP server.

Issue 

Not able to start AdminServer due to 

Caused By: oracle.security.jps.JpsRuntimeException: JPS-00056: Failed to create identity store service instance idstore.ldap.provider:idstore.ldap. Reason: oracle.security.jps.JpsRuntimeException: JPS-00027: internal error You configured a generic WLS LDAPAuthenticator.
The identity store type cannot be determined. Please choose an LDAP Authentication provider that matches your LDAP server.

####<Feb 3, 2015 12:51:27 PM CST> <Critical> <WebLogicServer> <localhost> <adminserver> <Main Thread> <<WLS Kernel>> <> <> <1422989487383> <BEA-000386> <Server subsystem failed. Reason: weblogic.security.SecurityInitializationException: The loading of OPSS java security policy provider failed due to exception, see the exception stack trace or the server log file for root cause. If still see no obvious cause, enable the debug flag -Djava.security.debug=jpspolicy to get more information. Error message: JPS-00056: Failed to create identity store service instance idstore.ldap.provider:idstore.ldap. Reason: oracle.security.jps.JpsRuntimeException: JPS-00027: internal error You configured a generic WLS LDAPAuthenticator.
The identity store type cannot be determined. Please choose an LDAP Authentication provider that matches your LDAP server.
weblogic.security.SecurityInitializationException: The loading of OPSS java security policy provider failed due to exception, see the exception stack trace or the server log file for root cause. If still see no obvious cause, enable the debug flag -Djava.security.debug=jpspolicy to get more information. Error message: JPS-00056: Failed to create identity store service instance idstore.ldap.provider:idstore.ldap. Reason: oracle.security.jps.JpsRuntimeException: JPS-00027: internal error You configured a generic WLS LDAPAuthenticator.
The identity store type cannot be determined. Please choose an LDAP Authentication provider that matches your LDAP server.
        at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.loadOPSSPolicy(CommonSecurityServiceManagerDelegateImpl.java:1402)
        at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initialize(CommonSecurityServiceManagerDelegateImpl.java:1022)
        at weblogic.security.service.SecurityServiceManager.initialize(SecurityServiceManager.java:873)
        at weblogic.security.SecurityService.start(SecurityService.java:141)
        at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
        at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
        at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)
Caused By: oracle.security.jps.JpsRuntimeException: JPS-00056: Failed to create identity store service instance idstore.ldap.provider:idstore.ldap. Reason: oracle.security.jps.JpsRuntimeException: JPS-00027: internal error You configured a generic WLS LDAPAuthenticator.
The identity store type cannot be determined. Please choose an LDAP Authentication provider that matches your LDAP server.

        at oracle.security.jps.internal.policystore.PolicyDelegationController.<init>(PolicyDelegationController.java:170)



Solution: 

Please chekc the config.xml file and change the 

<sec:authentication-provider xsi:type="wls:ldap-authenticatorType">
        <sec:name>AD_Authenticator_1</sec:name>

to 

<sec:authentication-provider xsi:type="wls:active-directory-authenticatorType">
        <sec:name>AD_Authenticator_1</sec:name>

Manually checking the MQ Version information in the WebSphere MQ JAR files

Manually checking the information in the WebSphere MQ JAR files 
Alternatively, you can check a specific JAR installed as part of WebSphere MQ to find out the version information. The JAR to check is shown below: 

WebSphere MQ classes for Java: com.ibm.mq.jar 
WebSphere MQ classes for JMS : com.ibm.mqjms.jar 

Both of these JARs are located in the directory MQ_INSTALLATION_PATH/java/lib. 

Once the JARs have been located, follow the procedure below: 

1) Unjar the appropriate JAR using one of the commands shown: 

WebSphere MQ classes for Java: 
#jar -xvf com.ibm.mq.jar 

WebSphere MQ classes for JMS: 
#jar -xvf com.ibm.mqjms.jar 

2) Enter the uncompressed jar directory. This will contain a subdirectory called META-INF, which has a file in it called MANIFEST.MF. 

3) Open this file, and look for a line similar to the ones shown below. These lines indicate which version of the WebSphere MQ classes for Java or classes for JMS are being used by your application: 

WebSphere MQ classes for Java - com.ibm.mq.jar 
Specification-Title: WebSphere MQ classes for Java 
Specification-Version: 7.1.0.0 

WebSphere MQ classes for JMS - com.ibm.mqjms.jar 
Implementation-Title: WebSphere MQ classes for Java Message Service 
Implementation-Version: 7.1.0.0 - k000-L111005.1 

Mark-and-Sweep Garbage Collection

 mark-and-sweep  garbage collection algorithm

 The mark-and-sweep algorithm was the first garbage collection algorithm to be developed that is able to reclaim cyclic data structures.gif Variations of the mark-and-sweep algorithm continue to be among the most commonly used garbage collection techniques.

When using mark-and-sweep, unreferenced objects are not reclaimed immediately. Instead, garbage is allowed to accumulate until all available memory has been exhausted. When that happens, the execution of the program is suspended temporarily while the mark-and-sweep algorithm collects all the garbage. Once all unreferenced objects have been reclaimed, the normal execution of the program can resume.

The mark-and-sweep algorithm is called a tracing garbage collector because is traces out the entire collection of objects that are directly or indirectly accessible by the program. The objects that a program can access directly are those objects which are referenced by local variables on the processor stack as well as by any static variables that refer to objects. In the context of garbage collection, these variables are called the roots . An object is indirectly accessible if it is referenced by a field in some other (directly or indirectly) accessible object.

com.bea.wli.sb.transports.TransportException: [OSB FLOW Transport:2030702]Unexpected error, the Split-Join transport configuration descriptor cannot be loaded. Root cause: "File not found

Issue : 

when deploying OSB project

Caused by: com.bea.wli.sb.transports.TransportException: [OSB FLOW Transport:2030702]Unexpected error, the Split-Join transport configuration descriptor cannot be loaded. Root cause: "File not found
Solution:
Please restart the All Domain Servers and redeploy the application.

BPM roles are not updating Properly

Issue: BPM roles are not showing properly for all BPM servers.

Solution: 

Please validate system-jazn-data.xml (location: $DOMAIN_HOME/config/fmwconfig) 

into all Managed Servers and Admin Server.And update 

Oracle Fusion Middleware: Undeploy the composite When soa_infra in down

Oracle Fusion Middleware: Undeploy the composite When soa_infra in down: Check the SOA logs and determine which composite is causing the problem and then follow the below process to undeploy the composite by edit...

Undeploy the composite When soa_infra in down

Check the SOA logs and determine which composite is causing the problem and then follow the below process to undeploy the composite by editing deployed-composites.xml:

1. Download and copy the ShareSoaInfraPartition.ear file to $MIDDLEWARE_HOME/oracle_common/common/bin

2. cd to $MIDDLEWARE_HOME/oracle_common/common/bin

and

run wlst.sh

3.

connect('weblogic','weblogic1','t3://localhost:7001')


provide the username, password and server URL

4. run the below command to deploy ShareSoaInfraPartition.ear

deploy('ShareSoaInfraPartition','ShareSoaInfraPartition.ear',upload='true')


5. Now run the below command by changing the "toLocation" ('/fmw11g/fmw1115/Middleware' is some location path on SOA machine)

exportMetadata(application='ShareSoaInfraPartition',server='AdminServer',toLocation='/fmw11g/fmw1115/Middleware',docs='/deployed-composites/deployed-composites.xml')


6. A deployed-composites folder will be created at "toLocation" path with deployed-composites.xml in it

7. Delete the composite which is causing the problem and save the file

For example, the MediatorTest composite:

<composite-series name="default/MediatorTest" default="default/MediatorTest!1.0">
<composite-revision dn="default/MediatorTest!1.0" state="on" mode="active" location="dc/soa_58b98be8-9ec8-41af-bb83-590f6004d1aa">
<composite dn="default/MediatorTest!1.0*soa_58b98be8-9ec8-41af-bb83-590f6004d1aa" deployedTime="2011-11-17T09:01:54.750+05:30"/>


8. Now run the below command by changing the "fromLocation" (this should be the same location as previous)

importMetadata(application='ShareSoaInfraPartition',server='AdminServer',fromLocation='/fmw11g/fmw1115/Middleware',docs='/deployed-composites/deployed-composites.xml')


9. Now bounce your server and the composite will not be deployed by SOA when it comes up and hence that should bring your soa-infra up.

Connection test failed with the following exception: weblogic.common.resourcepool.ResourceDeadException: 0:weblogic.common.ResourceException: Could not create pool connection. The DBMS driver exception was: Listener refused the connection with the following error: ORA-12514, TNS:listener does not currently know of service requested in connect descriptor

Error


 Connection test failed with the following exception: weblogic.common.resourcepool.ResourceDeadException: 0:weblogic.common.ResourceException: Could not create pool connection. The DBMS driver exception was: Listener refused the connection with the following error: ORA-12514, TNS:listener does not currently know of service requested in connect descriptor


Solution

The DB Service went down abrublty as This exception is generated when an application request to get a connection fails because the connection selected to be returned to the application from the pool fails the configured tests.

Weblogic tests a connection before it is given to a client.

DB needs to restart.

Find the Hostname with IPAddress

[test@srinfosoa01 logs]$ nslookup srinfosoa01
Server:         128.166.33.44
Address:        128.166.33.44#53



nslookup 208.77.188.166 
...
Non-authoritative answer:

166.188.77.208.in-addr.arpa     name = www.example.com.

oracle.bam.adc.common.exceptions.CacheException: BAM-01265

Error 

Caused by: oracle.bam.adc.common.exceptions.CacheException: BAM-01265: An error (weblogic.common.resourcepool.ResourceDeadException: 0:weblogic.common.ResourceException: Could not create pool connection. The DBMS driver exception was: Listener refused the connection with the following error:
ORA-12514, TNS:listener does not currently know of service requested in connect descriptor
 ) has occurred in the Active Data Cache storage system.

When you found the above issue 

Cause: 

Message icon - Warning Test of BAMDataSource on server bam_ms01 failed, reason:

Message icon - Error Connection test failed with the following exception: weblogic.common.resourcepool.ResourceDeadException: 0:weblogic.common.ResourceException: Could not create pool connection. The DBMS driver exception was: Listener refused the connection with the following error: ORA-12514, TNS:listener does not currently know of service requested in connect descriptor

Solution


Please check with BAMDataSource, there is an issue with DB connectivity .

Data source should be available 

HUDSON CONFIGURATION

Create a folder Called HUDSON and download the latest Hudson .war file from hudson site
C:\HUDSON

 

Weblogic 12.1.2 Update

Note:

As of WebLogic Server 12.1.2, Oracle no longer provides upgrade installers. You must install WebLogic Server 12.1.2 to a new directory location. You cannot install it over an existing installation.

stuck threads

Hogging threads are candidates for stuck threads.

Threads that “might” get stuck. These threads will be declared “stuck” after StuckThreadMaxTimeout seconds which usually is 600secs.

If the thread gets released by the request before this timeout, it wont be called hogging thread anymore and will be released to the thread pool.

Hogging Threads that have taken the too much time and we can assume that they are never going to come back.
Hogging threads help us take some decisions, lets say many threads are hogging, we may take a decision to create new threads for next cycle.

Understanding about Thread States in WebLogic Server:
ACTIVE
STUCK
STANDBY

A live thread which is ready to process the request, which is known as ACTIVE state. That is indicated when thread newly created. WebLogic Server start the server instance with 1 ACTIVE thread and the thread count grows as per the min size if specified other wise it will do self-tune as per the request.

Threads might wait for other thread to release resource. This might happen due to application variables. The variables are 2 types thread-safe other is risk for thread. All local variables in the methods are thread-safe. The variable defined in class level are unsafe. which causes memory leak, this state of threads are known as hogging. WebLogic identify a thread as hog by the time interval. If thread is waiting more than 600 sec will be treated as hog. STUCK thread interval we can tune as per the project need.
If the number of HoggingThreadCount increases then the server health is in dangerous. That time you can take the ThreadDump
After Threads increase to a max utilization then the thread will be in STANDBY state.

WebLogic Server Health Status can be one of the following:

HEALTH_OK
HEALTH_WARN
HEALTH_FAILED
HEALTH_CRITICAL
LOW_MEMORY_REASON
HEALTH_OVERLOADED

OK is indicates everything fine, no worries!!

WARN raised when there is few stuck threads in the server instance.

LOW_MEMORY_REASON is going to tell you about JVM crash expected. You can configure to ‘Exit’ the managed server on low memory conditions with the help of NodeManager and WorkManager.

CRITICAL when multiple number of stuck threads happening and the threadpool count reaching unsual number. This case you need to suspect Network, JDBC or back-end connectivity has trouble.

FAILED happen when the new deployments fails. The NodeManager should not restart this managed server.

OVERLOADED Change the server health state to OVERLOADED on overload. The Nodemanager need to work at this state and bounce such WebLogic instance. This is a new feature of WebLogic 9.x and later versions, for detecting, avoiding and recovering from an overload condition of a WebLogic managed server. Overload protection can be used to throttle Work Managers and thread pools for performance. You can configure Shutdown the Work Manager or application on stuck threads when it crosss more than 5 or you can set threshold.

Server Monitoring script

def reportDomainHealth(usern, passw, url):
print ""
print "===================================================================================="
connect(usern,passw,url)
domainRuntime()

print "Found Servers: "
serverList=ls('ServerRuntimes');
serverList=serverList.split()
print "%15s %15s %20s %15s %15s %40s" % ("Server","Threads","HoggingThreads","ServerState","Heap_Free","HealthState ")
print "----------------------------------------------------------------------------------------------------------------------------"
for i in range(len(serverList)):
if serverList[i] != 'dr--':
server_st=get('ServerRuntimes/' + serverList[i] + '/HealthState')
server_tc=get('ServerRuntimes/' + serverList[i] + '/ThreadPoolRuntime/ThreadPoolRuntime/ExecuteThreadTotalCount')
server_hog=get('ServerRuntimes/' + serverList[i] + '/ThreadPoolRuntime/ThreadPoolRuntime/HoggingThreadCount')
server_ql=get('ServerRuntimes/' + serverList[i] + '/State')
server_hpfp=get('ServerRuntimes/' + serverList[i] + '/JVMRuntime/' + serverList[i] + '/HeapFreePercent')
print "%15s %15s %20s %15s %15s %40s" % (serverList[i],str(server_tc),str(server_hog),str(server_ql),str(server_hpfp)+"%",str(server_st))
print "===================================================================================="


reportDomainHealth('weblogic','weblogic1','t3://localhost:7001')