How to resolved adminserver startup failure.

Introduction:- In this article we have resolved adminserver startup failure. While starting the WLS Admin Server, unable to do that, after connecting to node manager showing error as ERROR: Invalid credentials passed even we passed correct password.

Reproduced the error as below. You can start the adminserver on the run file system. After that we Resolved adminserver startup failure. 

Reproduced the error as below. You can you start the adminserver on the run file system. 

$ADMIN_SCRIPTS_HOME/adstartall.sh.

Executing service control script:

/oracle/PROD/EBSFS/fs2/inst/apps/PROD_prdsrv/admin/scripts/adadminsrvctl.sh start -nopromptmsg

Timeout specified in context file: 4000 second(s)

script returned:

****************************************************

You are running adadminsrvctl.sh version 120.10.12020000.11

Starting WLS Admin Server…

Invalid credentials passed.

Refer /oracle/PROD/EBSFS/fs2/inst/apps/PROD_prdsrv/logs/appl/admin/log/adadminsrvctl.txt for details

AdminServer logs are located at /oracle/PROD/EBSFS/fs2/FMW_Home/user_projects/domains/EBS_domain/servers/AdminServer/logs

adadminsrvctl.sh: exiting with status 1

adadminsrvctl.sh: check the logfile /oracle/PROD/EBSFS/fs2/inst/apps/PROD_prdsrv/logs/appl/admin/log/adadminsrvctl.txt for more information …

Connecting to Node Manager …

ERROR: Invalid credentials passed.

ERROR: Unable to connect to the Node Manager. The Admin Server cannot be started up.

Successfully disconnected from Node Manager.

AdminServer logs are located at /oracle/PROD/EBSFS/fs2/FMW_Home/user_projects/domains/EBS_domain/servers/AdminServer/logs

08/21/21-15:02:48 :: adadminsrvctl.sh: exiting with status 1

To resolve the error we have done many more  as below. Here we have decrypted the weblogic password to findout correct password. Finally we got same password , earlier we passed during startup.

Please find below the way to decrypt WebLogic password as requested:

1> First we need to get the encrypted password from boot.properties file :

cat $EBS_DOMAIN_HOME/servers/AdminServer/security/boot.properties

username={AES}X55JvJKC9Fqw2wPrj/7D0NGZonC8ES/qQTCMY0qb/TI=

password={AES}bVOPrUkmVNlnYMpIIVddRKn+rUGLb46yJOjAT5KO2n8=

2> Get the domain path :

echo $EBS_DOMAIN_HOME

/u01/oracle/fs1/FMW_Home/user_projects/domains/EBS_domain_VIS

3> Run below WLST script :

$FMW_HOME/oracle_common/common/bin/wlst.sh

wls:/offline> domain = “/u01/oracle/fs1/FMW_Home/user_projects/domains/EBS_domain_VIS “

wls:/offline> service = weblogic.security.internal.SerializedSystemIni.getEncryptionService(domain)

wls:/offline> encryption = weblogic.security.internal.encryption.ClearOrEncryptedService(service)

wls:/offline> print encryption.decrypt(“{AES}bVOPrUkmVNlnYMpIIVddRKn+rUGLb46yJOjAT5KO2n8=”)

welcome1

– The boot.properties file is locate at $EBS_DOMAIN_HOME/servers/AdminServer/security

– Replace the encrypted username and password with clear text

Example:

username=weblogic

password=password

– Start the Admin Server

– Once the Admin Server starts the username and password will get encrypted again

also please check below In run and patch fs.

$FMW_HOME/user_projects/domains/EBS_domain_<host>/edit.lock

Remove the lok from patch and retest

Checking if the Node Manager is already up..

Connecting to Node Manager …

ERROR: Invalid credentials passed.

ERROR: Unable to connect to the Node Manager

After done above steps still showing same error.

Finally we have decided to recreate the weblogic domain.  Before going to recreate the domain,we need to know how to recreate the domain.

1. In the Database Tier do the following:

1.1. Log into the database tier.

1.2. Source the environment.

1.3. Ensure that database and listener are up and running.

2. In the Applications Tier do the following:

2.1. Source the RUN file system environment.

2.2. Stop all middle tier services using $ADMIN_SCRIPTS_HOME/adstpall.sh.

2.3. Ensure that no process is running from RUN file system.

2.4. Take a backup of the context file. Update s_adminservertimeout context variable value to 4000 and run AutoConfig.

2.5. Execute below command:

$ $FND_TOP/bin/txkrun.pl -script=ChkEBSDependecies -server=ALL_SERVERS

2.6. Recreate the domain by following below steps:

$ cd $FND_TOP/patch/115/bin

$ perl txkEBSDomainConfig.pl

Domain recreation is completed successfully.

2.7. Start the services and check the WLS domain is fine. For example, you might want to start the WLS console and check all the expected managed servers are up and running.

Once domain recreation completed successfully. Now we need to recreate the patch fs as below.

Below are the needed steps to recreate the patch fs from the scratch, You will need to make sure to have a valid backup of your system before you execute it as this plan will imply to remove information from your system and we might need to go back to the point the action plan is executed in case anything may not go as expected.

[1] adpreclone

——————-

On the RUN File-System, source RUN File-System:

$ source EBSapps.env run

# RUN Admin server MUST be up for this step

# check status:

$ $ADMIN_SCRIPTS_HOME/adadminsrvctl.sh status

# start if needed:

$ $ADMIN_SCRIPTS_HOME/adadminsrvctl.sh start

# execute adpreclone:

$ cd $INST_TOP/admin/scripts

$ perl adpreclone.pl appsTier