JAR Resources in JNLP file are not signed by same certificate.

Dear Folks,

We have some issue with JWS when attempting to launch form with below navigation in 12.2 instance.

                    Workflow Administrator
                    Status Monitor
                    Search: PO Approval
                    Click on Status Diagram

It throws below execption.

Solution (
*******

1) Control panel and click on Java, Click on the "Settings.." button:

2) Click on the "Delete files.." button:

It should solve the issue. Now able to see status diagram.

If above solution doesn't work. Please follow below plan.

Regenerate JAR files with FORCE option set to Yes

Do you wish to force regeneration of all jar files? [No] ? Yes

Thanks.

How to change login page logo for 12.2 EBS

Dear Folks,

Today, i am giving simple steps how we change login page logo for 12.2 EBS.

**cat $FND_TOP/mds/sso/login/webui/MainLoginPG.xml|grep -i corporateBrandingImage

 <oa:image id="corporateBrandingImage" source="/OA_MEDIA/oracle_white_logo.png"/>

You may find FNDSSCORP.gif as well.

cat $FND_TOP/mds/sso/login/webui/MainLoginPG.xml|grep -i corporateBrandingImage

      <oa:image id="corporateBrandingImage" source="/OA_MEDIA/FNDSSCORP.gif"/>

Steps

*****

cd $OA_MEDIA

[****@****media]$ pwd

/u01/app/****/KIYRUT/fs1/EBSapps/comn/java/classes/oracle/apps/media

[username@servername media]$ cp oracle_white_logo.png oracle_white_logo.png_Org

[username@servername media]$ cp oracle_logo.png oracle_logo.png_org

[username@servername media]$ cp TEST.gif oracle_logo.png

[username@servername media]$ cp TEST.gif oracle_white_logo.png

[username@servername media]$ cp FNDSSCORP.gif FNDSSCORP.gif_bkp

[username@servername media]$ cp TEST.gif FNDSSCORP.gif

[username@servername media]$ cd $JAVA_TOP/oracle/apps/media/

[username@servername media]$ pwd

/u01/app/username/KIYRUT/fs1/EBSapps/comn/java/classes/oracle/apps/media

[username@servername media]$ mv oracle_white_logo.png oracle_white_logo.png_Org

[username@servername media]$ mv oracle_logo.png oracle_logo.png_Org

[username@servername media]$ cp FNDSSCORP.gif oracle_logo.png

[username@servername media]$ cp FNDSSCORP.gif oracle_white_logo.png

adapcctl.sh stop

admanagedsrvctl.sh stop oacore_server1

adapcctl.sh start

admanagedsrvctl.sh start oacore_server1

gif may vary depends upon environment. Our case it is TEST instance, hence it is TEST.gif

Thanks.

Re-link fails on target "client_sharedlib while applying Grid Patch 29252164 (April CPU).

Hi All,

We are experienced a issue with April CPU Grid patch failing with below error when we were trying to apply on to Oracle Home.


==Following patches FAILED in apply:

Patch: /home/****/29252164/29176115/29141015
Log: /u02/app/*****/****/db/tech_st/12.1.0/cfgtoollogs/opatchauto/core/opatch/opatch2019-07-31_02-43-34AM_1.log
Reason: Failed during Patching: oracle.opatch.opatchsdk.OPatchException: Re-link fails on target "client_sharedlib".
Re-link fails on target "ldapaddmt".
Re-link fails on target "ldapadd".
Re-link fails on target "ldapmodify".
Re-link fails on target "ldapmodifymt".
Re-link fails on target "client_sharedlib".

After fixing the cause of failure Run opatchauto resume

]
OPATCHAUTO-68061: The orchestration engine failed.
OPATCHAUTO-68061: The orchestration engine failed with return code 1
OPATCHAUTO-68061: Check the log for more details.
OPatchAuto failed.

Further drill down issue by checking opatch log file and found below error.

[Jul 31, 2019 2:44:58 AM] [INFO]    8) OUI-67200:Make failed**** to invoke "/usr/bin/make -f ins_net_client.mk client_sharedlib ORACLE_HOME=/u02/app//**/db/tech_st/12.1.0"....'genclntsh: genclntsh: Could not locate /u02/app//**/db/tech_st/12.1.0/network/admin/shrept.lst

                                    make: *** [client_sharedlib] Error 1

Hence we created shrept.lst file at $ORACLE_HOME/network/admin

**Go to $ORACLE_HOME/network/admin

** Create file shrept.lst 

**Provide 644 permissions.

**Retry patching.

Thanks.

How to change Guest user password in Oracle E-Business Suite.

Dear Folks,

Here are the steps to change GUEST user password in Oracle-EBS.

**Backup contextfile on all nodes.

** Update new guest password - s_guest_pass in context file.

Note :: Password should be case sensitive. You must use CAPS else need to change below profile at site level.

              Password Case Option from  "mixed"  to "insensitive".

** Bring down the application services.

** Run Autoconfig across all the nodes.

** Just bring up OPMN Services.

**Login to CRM HTML Administration responsibility --> set up ---> settings ---> system ---> properties ---> Update new password.

**Start All Application Services.

Thanks.
Sreeharsha.

Cronjob for every month last Saturday.

Hi All,

Client have a requirement that they want to execute a script last Saturday of every month. So we keep as below.

30 01 * * 6 [ $(date +"\%m") -ne $(date -d 7days +"\%m") ] && sh /home/appeprd1/scripts/Interface_tables_purge.sh > /home/appeprd1/scripts/error.log 2>&1

This works as expected.You can modify according your requirement says last thursday/friday. Just change 4(for thursday) and 5 ( For friday). It works.

Thanks.

ORA-02304: invalid object identifier literal

Dear All,

We have a requirement that client need to import one schema data (Our case XXEXT) to another newly created schema(in our case XXEXTNEW)

While importing, facing below error.

ORA-39083: Object type TYPE:"XXEXTNEW"."RULEDATAELEMENT" failed to create with error:

ORA-02304: invalid object identifier literal

Failing sql is:

CREATE EDITIONABLE TYPE "XXEXTNEW"."RULEDATAELEMENT"   OID '8299F41C5D6807A3E0531CFC6A0AFB72' AS OBJECT (name VARCHAR2(255), value VARCHAR2(255));

ORA-39083: Object type TYPE:"XXEXTNEW"."RULEDATA" failed to create with error:

ORA-02304: invalid object identifier literal

Failing sql is:

CREATE EDITIONABLE TYPE "XXEXTNEW"."RULEDATA"   OID '829A2D8D350019C4E0531CFC6A0A3062' AS TABLE OF ruleDataElement;

It is because Object ID(OID) Value of type (XXEXTNEW) should be unique in database. If we try to import TYPE with Same Object ID: It throws above error.

Check whether OID present or not in database. Our case, OID exists.

OID '8299F41C5D6807A3E0531CFC6A0AFB72'

Since OID already exists, we need to import TYPE with new OID. So we have to use TRANSFORM=oid:n parameter in impdp command.

Solution
++++++

impdp system/<pwd>  dumpfile=XXEXT.dmp directory=<directory_Name> logfile=XXEXTNEW_SCHEMA.log remap_schema=XXEXT:XXEXTNEW parallel=4 TRANSFORM=oid:n

Then Import successfully Completed.

Thanks
Sreeharsha.

The Server is not able to service this request: [Socket:000445]Connection rejected, filter blocked Socket, weblogic.security.net.FilterException: [Security:090220]rule 2

Dear Folks,

We came accross weblogic issue where we couldn't able to access the console,getting following error.

The Server is not able to service this request: [Socket:000445]Connection rejected, filter blocked Socket, weblogic.security.net.FilterException: [Security:090220]rule 2 

This is the issue started after we applied April 2019 CPU patches on one of our 12.2.8 EBS environment.

From April2019 patches, Oracle allows only trusted hosts to access Weblogic console. Refer below document for more details and refer section Configuring Oracle WebLogic Server Connection Filter

https://docs.oracle.com/cd/E26401_01/doc.122/e22953/T174296T589913.htm#T672703

Fix
---

** Log in to the primary node of the Oracle E-Business Suite instance.

** Start the Oracle WebLogic Admin Server from the run file system.

** Take a backup of the run file system context file.

** Edit the run file system context file to set the value for the s_wls_admin_console_access_nodes context variable to the list of trusted hosts that are allowed to access the Admin Server. For each host,
    specify either the FQDN or the IP address. Use commas to separate the hosts in the list. For example:

<s_wls_admin_console_access_nodes oa_var="s_wls_admin_console_access_nodes">admin-ws1.example.com,admin-ws2.example.com</s_wls_admin_console_access_nodes>

Note: When you add the fully qualified domain name or the IP address for a host to the list in thes_wls_admin_console_access_nodes context variable, ensure that the host name is resolvable from all application tier nodes of the Oracle E-Business Suite instance.

** Run AutoConfig.

** Stop and restart the Oracle WebLogic Admin Server.

We had followed the above fix and updated  hostnames (Application,DB)  in context files. However issue still remains.

The Admin server log shows below IP address. So we'd added IP to context file to resolve issue.

                         10.70.200.201( This IP address belongs to NAT'd IP )

However it seems to be specific to one VPN.

So we approached Oracle to get clarity about the issue. They finally came up with below alternative workaround.

In order to fix this issue, please comment or remove the following lines in domain config.xml:

go to $EBS_DOMIAN_HOME/config


 <connection-filter>oracle.apps.ad.tools.configuration.wls.filter.EBSConnectionFilterImpl</connection-filter>
 <connection-filter-rule><host>.<domain> * * allow</connection-filter-rule>
 <connection-filter-rule>0.0.0.0/0 * * deny</connection-filter-rule>
 <connection-logger-enabled>true</connection-logger-enabled>

 Once done, save and restart admin server.

 Now access the console and set the filter rules as per the following document:

NOTE :: Even in JULY CPU Oracle didn't fix this issue. As we were following with them. They are going to raise a BUG.

Fix for anove one

Alternative Methods to Allow Access to Oracle WebLogic Server Administration Console from Trusted Hosts for Oracle E-Business Suite Release 12.2 (Doc ID 2542826.1)

Please add IP address of your internal node in config.xml file

<s_wls_admin_console_access_nodes oa_var="s_wls_admin_console_access_nodes">192.0.2.0/24</s_wls_admin_console_access_nodes>

Thanks.
Sreeharsha

ERRORMSG: Adsplice action did not go through successfully.

Hello every one,

We have encountered a issue with adop prepare phase where it was failing with below. This issue happened when we added custom application post 12.2.8 upgrade with adsplice.

Inside evalADPATCHStatus()...

=============================

message_status: ERROR

Adsplice action did not go through successfully.

*******FATAL ERROR*******

PROGRAM : (/u01/app/***/KLNMRY/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSynchronize.pl)

TIME  : Fri Apr 19 12:49:55 2019

FUNCTION: main::execADSPLICE [ Level 1 ]

ERRORMSG: Adsplice action did not go through successfully.

Solution

======

We've below custom applications in our environment installed. we'd got this custom products in a report from the script ADZDPSUM.sql at Enable online patching.

SQL> select ORACLE_ID,ORACLE_USERNAME,ENABLED_FLAG from FND_ORACLE_USERID WHERE ORACLE_USERNAME in ('KAAPO','KAAGL','KKKAA','KAAAP','KKE4A','KKE4ARW');

ORACLE_ID ORACLE_USERNAME        E

---------- ------------------------------ -

   20043 KAAAP

   20044 KAAGL

   20045 KAAPO

   50505 KKE4A             N

   50506 KKE4ARW            N

   20046 KKKAA

**Take backup of table "FND_ORACLE_USERID"

SQL> create table FND_ORACLE_USERID_bkp as select * from FND_ORACLE_USERID;

Table created.

**Run below command for custom application products.

UPDATE FND_ORACLE_USERID SET ENABLED_FLAG = 'N' WHERE ORACLE_ID in (20043,20044,20045,20046);

SQL> select ORACLE_ID,ORACLE_USERNAME,ENABLED_FLAG from FND_ORACLE_USERID WHERE ORACLE_USERNAME in ('KAAPO','KAAGL','KKKAA','KAAAP','KKE4A','KKE4ARW');

ORACLE_ID ORACLE_USERNAME        E

---------- ------------------------------ -

   20043 KAAAP             N

   20044 KAAGL             N

   20045 KAAPO             N

   50505 KKE4A             N

   50506 KKE4ARW            N

   20046 KKKAA             N

6 rows selected.

Then kick start adop prepare phase. It will compete with no issues.

ML :: 1932207.1

Thanks.

Sreeharsha

Find rdf file from program name

Hi,

Steps to find the RDF file from program name given.

1) Go to system administrator > Concurrent > Program > Define.

Search for the program name and copy the executable name.

2) Use below query and paste the executable name.

SELECT APPLICATION_NAME,'$'||BASEPATH||'/'||'reports/US' Reports_Path,EXECUTION_FILE_NAME FROM APPS.FND_EXECUTABLES_VL A, APPS.FND_APPLICATION_VL B WHERE EXECUTION_METHOD_CODE='P' AND A.APPLICATION_ID=B.APPLICATION_ID AND EXECUTION_FILE_NAME like '%&RDF_NAME%';

It shows below output.

APPLICATION_NAME
--------------------------------------------------------------------------------
REPORTS_PATH
--------------------------------
EXECUTION_FILE_NAME
-------------------------------------------------------------
Receivables
$AR_TOP/reports/US
RAXCUS

Receivables
$AR_TOP/reports/US
RAXCUSLR

Thanks.
Sreeharsha

libXi.so.6: cannot open shared object file: No such file or directory

Hi,

We are faced issue when we are running rapidwiz for 12.2.0 installation and failed with below error.

rapidwiz]$ Exception in thread "main" java.lang.UnsatisfiedLinkError: /backup/*****/R12_2_stage/startCD/Disk1/rapidwiz/jre/Linux_x64/1.6.0/lib/i386/xawt/libmawt.so: libXi.so.6: cannot open shared object file: No such file or directory
    at java.lang.ClassLoader$NativeLibrary.load(Native Method)
    at java.lang.ClassLoader.loadLibrary0(Unknown Source)
    at java.lang.ClassLoader.loadLibrary(Unknown Source)
    at java.lang.Runtime.load0(Unknown Source)
    at java.lang.System.load(Unknown Source)
    at java.lang.ClassLoader$NativeLibrary.load(Native Method)
    at java.lang.ClassLoader.loadLibrary0(Unknown Source)
    at java.lang.ClassLoader.loadLibrary(Unknown Source)
    at java.lang.Runtime.loadLibrary0(Unknown Source)
    at java.lang.System.loadLibrary(Unknown Source)
    at sun.security.action.LoadLibraryAction.run(Unknown Source)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.awt.NativeLibLoader.loadLibraries(Unknown Source)
    at sun.awt.DebugHelper.<clinit>(Unknown Source)
    at java.awt.Component.<clinit>(Unknown Source)
Could not find the main class: oracle.apps.ad.rapidwiz.RIWizard.  Program will exit.


Solution (ML 2168094.1)
====

Copy a 64 bit version of the JDK from any 12c Oracle Home.

To resolve the issue test the following steps in a development instance and then migrate accordingly:

1. Backup the existing JDK located in startCD/Disk1/rapidwiz/jre/Linux_x64/

2. Copy a 64 bit version of the JDK to a zip file from any 12c Oracle Database or Grid Home located in $ORACLE_HOME/ via the following command:
  $ zip -ryT 1.6.0.zip jdk

3. Move the zip file 1.6.0.zip (via scp or other) into startCD/Disk1/rapidwiz/jre/Linux_x64

4. Unzip file 1.6.0.zip in directory startCD/Disk1/rapidwiz/jre/Linux_x64

5. mv jdk  1.6.0

5. Retest Rapidwiz and confirm the error is resolved.



Thanks.

ORA-20000: ORA-24033: no recipients for message Failed to call the procedure fnd_apps_mode_util.set_to_maintenance_mode while enabling maintenance mode.

Hi Guys,

I'm bringing one issue into your notice where we had faced issue while  enable maintenance mode.

SQL> @$AD_TOP/patch/115/sql/adsetmmd.sql ENABLE
 declare
*
ERROR at line 1:
ORA-20000: ORA-24033: no recipients for message Failed to call the procedure
fnd_apps_mode_util.set_to_maintenance_mode while enabling
maintenance mode.
ORA-06512: at line 38

Solution
=======

SQL> create table fnd_profile_option_values_bak1 as select * from fnd_profile_option_values ;

Table created.

SQL> select profile_option_id from FND_PROFILE_OPTIONS where PROFILE_OPTION_NAME ='APPS_MAINTENANCE_MODE';

PROFILE_OPTION_ID
-----------------
            5823
SQL> update fnd_profile_option_values set profile_option_value ='MAINT' where profile_option_id=5823;

1 row updated.

SQL> commit;

Commit complete.

Thanks
Sreeharsha.

WF_WS_JMS_OUT_R , WF_WS_SAMPLE_R , WF_JAVA_DEFERRED_R invalids after Database upgrade from 11g to 12c

Dear Folks,

We have observed few invalids related to workflow after DB upgrade to 12c from 11G.


OWNER           OBJECT_NAME                    CREATED   OBJECT_TYPE                    LAST_DDL_ STATUS
--------------- ------------------------------ --------- ------------------------------ --------- -------
APPLSYS         WF_WS_JMS_OUT_R                09-NOV-13 RULE SET                       03-APR-19 INVALID
APPLSYS         WF_WS_SAMPLE_R                 09-NOV-13 RULE SET                       03-APR-19 INVALID
APPLSYS         WF_JMS_IN_R                    09-NOV-13 RULE SET                       03-APR-19 INVALID
APPLSYS         WF_JMS_OUT_R                   09-NOV-13 RULE SET                       03-APR-19 INVALID
APPLSYS         WF_JAVA_DEFERRED_R             09-NOV-13 RULE SET                       03-APR-19 INVALID
APPLSYS         WF_WS_JMS_IN_R                 09-NOV-13 RULE SET                       03-APR-19 INVALID
APPLSYS         WF_JMS_JMS_OUT_R               09-NOV-13 RULE SET                       03-APR-19 INVALID
APPLSYS         WF_JAVA_ERROR_R                09-NOV-13 RULE SET                       03-APR-19 INVALID

Solution
======

As per ML 337294.1 we executed below procedure for each object listed above.

declare
l_wf_schema varchar2(200);
lagent sys.aq$_agent;
l_new_queue varchar2(30);
begin
l_wf_schema := wf_core.translate('WF_SCHEMA');
l_new_queue := l_wf_schema||'.WF_WS_JMS_OUT_R';
lagent := sys.aq$_agent('WF_WS_JMS_OUT_R',null,0);
dbms_aqadm.add_subscriber(queue_name=>l_new_queue, subscriber=>lagent,rule=>'1=1');
end;
/
commit;

declare

l_wf_schema varchar2(200);

lagent sys.aq$_agent;

l_new_queue varchar2(30);

begin

l_wf_schema := wf_core.translate('WF_SCHEMA');

l_new_queue := l_wf_schema||'.WF_WS_SAMPLE_R';

lagent := sys.aq$_agent('WF_WS_SAMPLE_R',null,0);

dbms_aqadm.add_subscriber(queue_name=>l_new_queue, subscriber=>lagent,rule=>'1=1');

end;

/

commit;

declare

l_wf_schema varchar2(200);

lagent sys.aq$_agent;

l_new_queue varchar2(30);

begin

l_wf_schema := wf_core.translate('WF_SCHEMA');

l_new_queue := l_wf_schema||'.WF_JMS_IN_R';

lagent := sys.aq$_agent('WF_JMS_IN_R',null,0);

dbms_aqadm.add_subscriber(queue_name=>l_new_queue, subscriber=>lagent,rule=>'1=1');

end;

/

commit;

declare

l_wf_schema varchar2(200);

lagent sys.aq$_agent;

l_new_queue varchar2(30);

begin

l_wf_schema := wf_core.translate('WF_SCHEMA');

l_new_queue := l_wf_schema||'.WF_JMS_OUT_R';

lagent := sys.aq$_agent('WF_JMS_OUT_R',null,0);

dbms_aqadm.add_subscriber(queue_name=>l_new_queue, subscriber=>lagent,rule=>'1=1');

end;

/

commit;

declare

l_wf_schema varchar2(200);

lagent sys.aq$_agent;

l_new_queue varchar2(30);

begin

l_wf_schema := wf_core.translate('WF_SCHEMA');

l_new_queue := l_wf_schema||'.WF_JAVA_DEFERRED_R';

lagent := sys.aq$_agent('WF_JAVA_DEFERRED_R',null,0);

dbms_aqadm.add_subscriber(queue_name=>l_new_queue, subscriber=>lagent,rule=>'1=1');

end;

/

commit;

declare

l_wf_schema varchar2(200);

lagent sys.aq$_agent;

l_new_queue varchar2(30);

begin

l_wf_schema := wf_core.translate('WF_SCHEMA');

l_new_queue := l_wf_schema||'.WF_WS_JMS_IN_R';

lagent := sys.aq$_agent('WF_WS_JMS_IN_R',null,0);

dbms_aqadm.add_subscriber(queue_name=>l_new_queue, subscriber=>lagent,rule=>'1=1');

end;

/

commit;

declare

l_wf_schema varchar2(200);

lagent sys.aq$_agent;

l_new_queue varchar2(30);

begin

l_wf_schema := wf_core.translate('WF_SCHEMA');

l_new_queue := l_wf_schema||'.WF_JMS_JMS_OUT_R';

lagent := sys.aq$_agent('WF_JMS_JMS_OUT_R',null,0);

dbms_aqadm.add_subscriber(queue_name=>l_new_queue, subscriber=>lagent,rule=>'1=1');

end;

/

commit;

declare

l_wf_schema varchar2(200);

lagent sys.aq$_agent;

l_new_queue varchar2(30);

begin

l_wf_schema := wf_core.translate('WF_SCHEMA');

l_new_queue := l_wf_schema||'.WF_JAVA_ERROR_R';

lagent := sys.aq$_agent('WF_JAVA_ERROR_R',null,0);

dbms_aqadm.add_subscriber(queue_name=>l_new_queue, subscriber=>lagent,rule=>'1=1');

end;

/

commit;

Then compile invalid objects. Issue solves.

Thanks.

Sreeharsha

APP-PAY-06882: Assertion failure detected at location py3vntsetup:2

Dear Folks,

Recently we have faced issue in Vertex monthly data upgrade where the program "Quantum Data Update installer" had failed with below error.

LOCATION py3vntsetup:2
APP-PAY-06882: Assertion failure detected at location py3vntsetup:2.

Cause:        an internal error has occurred at location py3vntsetup:2.  

Action:        please contact your Oracle customer representative.

Work Around
+++++++++++

1) Check TAX_DATA & TAX_LIBRARIES paths ,if they are pointing to the wrong path.

SQL> select * from pay_action_parameters where parameter_name like 'TAX%';

PARAMETER_NAME
----------------------------------------------------------------------
PARAMETER_VALUE
--------------------------------------------------------------------------------
TAX_DATA
/u01/app/appki/****/apps/apps_st/appl/pay/12.0.0/vendor/quantum/data

TAX_LIBRARIES
/u01/app/***/******/apps/apps_st/appl/pay/12.0.0/vendor/quantum/lib

From Above, TAX_LIBRARIES path pointed to the production. Hence we've corrected it to right path.

SQL> update pay_action_parameters set parameter_value = '/u01/app/***/*****/apps/apps_st/appl/pay/12.0.0/vendor/quantum/lib' where parameter_name = 'TAX_LIBRARIES';

1 row updated.

SQL> commit;

Commit complete.

2) Run Quantum Data Update installer program. It should be completed normal.

Thanks.

Sreeharsha

How to change APPS password in EBS 11i

***************How to change APPS Password in EBS 11i******************

Here are the steps to change apps password in 11i EBS

1) Shut down the application on all nodes
2) Take the backup below files on all application nodes.

$IAS_ORACLE_HOME/Apache/modplsql/cfg/wdbsvr.app
$ORACLE_HOME/reports60/server/CGIcmd.dat  (8.0.6 Oracle_home)
$FND_TOP/resource/wfmail.cfg

3) Backup below tables
=====================

SQL> create table FND_USER_BK_<YYYYMMSS> as select * from FND_USER;
SQL> create table FND_ORACLE_USERID_BK_<YYYYMMSS> as select * from FND_ORACLE_USERID;

4) Change apps password using below command.

FNDCPASS apps/<appspwd> 0 Y system/<pwd> SYSTEM APPLSYS <new password>

 **Changing APPLSYS user password will automatically changes APPS user password.

5) Change APPS password in the following files.

$IAS_ORACLE_HOME/Apache/modplsql/cfg/wdbsvr.app
$ORACLE_HOME/reports60/server/CGIcmd.dat  (8.0.6 Oracle_home)
$FND_TOP/resource/wfmail.cfg

6) Start Application on all the nodes.

7 ) Validate services.

8) Recreate DB links if any needed.

9) Check any cron job that have apps password hardcoded.

Thanks.

Sreeharsha

ICM log file location in R12 oracle EBS

                      ****ICM log file location in EBS R12****


All concurrent log files stay at the location $APPLCSF/$APPLLOG

1) Login into Apps environment and source the env file.

2) Go to $APPLCSF/$APPLLOG location

3) ls -lrt *$TWO_TASK*

[****@*****log]$ ls -lrt *$TWO_TASK*
-rw-r--r-- 1 24852359 Mar 20 01:07 KAPI_0801.mgr

4) Identify the internal concurrent manager log file (SID_MMDR.mgr)

5) If you want to identify other manager log files.

******Standard manager log:  wxxxx.mgr.   (xxxx represents Process ID )

                      EX :: -rw-rw-r-- 1 ****** 624100 Mar 20 01:09 w19983.mgr

***** Conflict resolution manager log :: cxxx.mgr  

                   EX: [***@*****log]$ ls -ltr c*.mgr
                              -rw-rw-r-- 1 **** ***1021 Nov  4 01:00 c19947.mgr

***** Transaction manager log:  txxxx.mgr

                  EX :: -rw-rw-r-- 1 *** *** 2672 Feb 28 13:24 t19939.mgr

Option 2

======

You can identify ICM log file using below query.

SELECT 'ICM_LOG_NAME=' || fcp.logfile_name

FROM fnd_concurrent_processes fcp, fnd_concurrent_queues fcq

WHERE fcp.concurrent_queue_id = fcq.concurrent_queue_id

AND fcp.queue_application_id = fcq.application_id

AND fcq.manager_type = '0'

AND fcp.process_status_code = 'A';

EX ::

SQL> SELECT 'ICM_LOG_NAME=' || fcp.logfile_name

FROM fnd_concurrent_processes fcp, fnd_concurrent_queues fcq

WHERE fcp.concurrent_queue_id = fcq.concurrent_queue_id

AND fcp.queue_application_id = fcq.application_id

AND fcq.manager_type = '0'

AND fcp.process_status_code = 'A';

  2    3    4    5    6  

'ICM_LOG_NAME='||FCP.LOGFILE_NAME

--------------------------------------------------------------------------------

ICM_LOG_NAME=/u01/app/****/KLSPI/inst/apps/KLSPI_servername/logs/appl/conc/lo

g/KLSPI_0801.mgr

Option 3

======

We can identify ICM log file from frontend.

Log in to Sysadmin Responsibility

Go to ->Concurrent -> Manager -> Administer -Processes -> Click button Internal Manager Log.

Thanks

Sreeharsha.

Workflow mailer log file location Oracle EBS R12

Workflow log location
++++++++++++++

cd $APPLCSF/$APPLLOG

ls -ltr FNDCPGSC*.txt

OR

Below query shows active workflow log file.

SELECT fcp.logfile_name

FROM fnd_concurrent_queues fcq, fnd_concurrent_processes fcp, fnd_lookups flkup

WHERE concurrent_queue_name in (‘WFMLRSVC’)

AND fcq.concurrent_queue_id = fcp.concurrent_queue_id

AND fcq.application_id = fcp.queue_application_id

AND flkup.lookup_code=fcp.process_status_code

AND lookup_type =’CP_PROCESS_STATUS_CODE’

AND meaning=’Active’;

--------------------------------------------------------------------------------
/u01/app/****/****/inst/apps/PSSPI_servername/logs/appl/conc/log/FNDCPGSC200
49.txt

Thanks
Sreeharsha

ORA-01403: No Data Found Has Been Detected In FND_CONC_RAC_UTILS

Dear Folks,

In one of our EBS customer running on 12.2.4. We've noticed few errors in internal concurrent manager log after user terminated a custom request.

Below observed in concurrent request log file.

--------------------------------------------------------------
An error occured in client-side routine afpsmckp for Service Manager FNDSM_*****_EBSPROD. The routine returned code 1.
Check for preceding errors and as well as the service manager log file for further details."
18-MAR-2019 11:03:10 - Could not submit job to kill concurrent process 188273: Oracle error 100: ORA-01403: no data found has been detected in FND_CONC_RAC_UTILS.SUBMIT_MANAGER_KILL_SESSION.

Solution

====

Apply Patches  24408550 and Patch 19479047 as per ML 1988419.1. Then issue resolved. No errors can see in ICM log file.

Concurrent Programs Fails With ORA-01403: No Data Found Has Been Detected In FND_CONC_RAC_UTILS (Doc ID 1988419.1)

Thanks.

Sreeharsha.

Patch Impact Analysis

col patch_name format a8 heading "Patch|Name"
col instance_name format a10 heading "Instance|Name"
col name format a10 heading "Node Name"
col APPL_TOP format a10
col subdir format a20
col filename format a18
col latest format a20
col driver_file_name format a13 heading "Driver|File|Name"
col action format a10
set lines 150
set pages 1000
Select
J.PATCh_NAME,
H.APPLICATIONS_SYSTEM_NAME Instance_Name,
H.NAME,
I.DRIVER_FILE_NAME,
D.APP_SHORT_NAME appl_top,
D.SUBDIR,
D.FILENAME,
max(F.VERSION) latest,
E.ACTION_CODE action
from
AD_BUGS A,
AD_PATCH_RUN_BUGS B,
AD_PATCH_RUN_BUG_ACTIONS C,
AD_FILES D,
AD_PATCH_COMMON_ACTIONS E,
AD_FILE_VERSIONS F,
AD_PATCH_RUNS G,
AD_APPL_TOPS H,
AD_PATCH_DRIVERS I,
AD_APPLIED_PATCHES J
where
A.BUG_ID = B.BUG_ID and
B.PATCH_RUN_BUG_ID = C.PATCH_RUN_BUG_ID and
C.FILE_ID = D.FILE_ID and
E.COMMON_ACTION_ID = C.COMMON_ACTION_ID and
D.FILE_ID = F.FILE_ID and
G.APPL_TOP_ID = H.APPL_TOP_ID and
G.PATCH_DRIVER_ID = I.PATCH_DRIVER_ID and
I.APPLIED_PATCH_ID = J.APPLIED_PATCH_ID and
B.PATCH_RUN_ID = G.PATCH_RUN_ID and
C.EXECUTED_FLAG = 'Y' and
G.PATCH_DRIVER_ID in
 (select PATCH_DRIVER_ID
 from AD_PATCH_DRIVERS
 where APPLIED_PATCH_ID in
 (select APPLIED_PATCH_ID
 from AD_APPLIED_PATCHES
 where PATCH_NAME in('22284589')) )
GROUP BY
J.PATCH_NAME,
H.APPLICATIONS_SYSTEM_NAME,
H.NAME,
I.DRIVER_FILE_NAME,
D.APP_SHORT_NAME,
D.SUBDIR,
D.FILENAME,
E.ACTION_CODE;

Unable To Generate Fnd Debug Logfiles And No Messages Are Writing To Fnd_log_messages

Dear Folks,

One of 12.2.4 customer facing issue several times to generate fnd log messages after the FND Diagnostics Profile is Set to 'Yes' at the Site Level.

It returns no rows in fnd_log_messages.

SQL> select count(*) from apps.fnd_log_messages;

 COUNT(*)
----------
     0

SQL> select max(LOG_SEQUENCE) from fnd_log_messages;

MAX(LOG_SEQUENCE)
-----------------


As per ML:: 2007068.1, We applied patch 22563940. Then issue resolved.

Expected Behavior
Expect FND Debugging to write to the fnd_log_messages tables.

The issue can be reproduced at will with the following steps:
1. Set FND Debugging

The issue has the following business impact:
Due to this issue, dba/appsdba cannot enable fnd debugging.

Now they are able to see rows in fnd_log_messages table.

Thanks.
Sreeharsha.

Relink of module "FEMCCE" failed after CPU applied.

Dear Folks,

Recently, we've observed relink error while applying CPU (Q1-2019) Patches on to one of non-prod instance.

Relinking module 'FEMCCE' in product fem ...

make -f

/u01/app/***/$TWO_TASK/apps/apps_st/appl/admin/$TWO_TASK/out/link_fem_28563.mk

/u01/app/*****/$TWO_TASK/apps/apps_st/appl/fem/12.0.0/bin/FEMCCE

Starting  link of fem executable 'FEMCCE' on Mon Feb  4 22:21:45 EST 2019

 ORDER BY 2;

make: *** [/u01/app/****/$TWO_TASK/apps/apps_st/appl/fem/12.0.0/bin/FEMCCE]

Error 1

Done with link of fem executable 'FEMCCE' on Mon Feb  4 22:21:45 EST 2019

Relink of module "FEMCCE" failed.

See error messages above (also recorded in log file) for possible

reasons for the failure.  Also, please check that the Unix userid

running adrelink has read, write, and execute permissions

on the directory /u01/app/****/$TWO_TASK/apps/apps_st/appl/fem/12.0.0/bin,

and that there is sufficient space remaining on the disk partition

containing your Oracle E-Business Suite installation.

Done with link of product 'fem' on Mon Feb  4 22:21:45 EST 2019

adrelink is exiting with status 1

Verified, We don't have file exists at OS level.

[****@Kouuda scripts]$ ls -lrt /u01/app/****/$TWO_TASK/apps/apps_st/appl/fem/12.0.0/bin/FEMCCE

ls: cannot access /u01/app/****/$TWO_TASK/apps/apps_st/appl/fem/12.0.0/bin/FEMCCE: No such file or directory

Hence, we copied binary file from production and relinked it manually.

[****@ scripts]$ make -f /u01/app/****/$TWO_TASK/apps/apps_st/appl/admin/$TWO_TASK/out/link_fem_28563.mk /u01/app/****/$TWO_TASK/apps/apps_st/appl/fem/12.0.0/bin/FEMCCE

make: `/u01/app/****/$TWO_TASK/apps/apps_st/appl/fem/12.0.0/bin/FEMCCE' is up to date.

Now Patch completed successfully.

Thanks.

Sreeharsha

APP-FND-01372 : Please register System printer with Application object library.

Dear Folks,

In one of our EBS customer running on 11i. Customer facing issue while switching responsibilities. Below pop-up error window does appear while doing.

APP-FND-01372 : Please register System printer with Application object library.

Solution
=======

Verified, The profile value for Printer is set to "Null"

Hence Change profile value of "Printer" to "noprint" and save changes.

Now,  customer able to switch responsibilities.

Thanks.
Sreeharsha

txkrun.pl (forms deplyoment) failed with TXK::Common::setError & TXK::Process::run

Dear Folks,

In one of our EBS customer, we came across an issue with forms deployment post 10.1.2 patches as part of CPU.

forms deployment failing with below error.

Command error: <rc> = 38400, <command> = /u01/app/****/apps/tech_st/10.1.3/opmn/bin/opmnctl startproc instancename=forms

STACK TRACE

    TXK::Error::abort('TXK::Error','HASH(0x830c890)') called at /u01/app/****/apps/apps_st/appl/au/12.0.0/perl/TXK/Common.pm line 299

    TXK::Common::doError('TXK::Process=HASH(0x8b112d0)','Command error: <rc> = 38400, <command> = /u01/app/****/apps/...','undef') called at /u01/app/****/apps/apps_st/appl/au/12.0.0/perl/TXK/Common.pm line 314

    TXK::Common::setError('TXK::Process=HASH(0x8b112d0)','Command error: <rc> = 38400, <command> = /u01/app/****/apps/...') called at /u01/app/****/apps/apps_st/appl/au/12.0.0/perl/TXK/Process.pm line 449

    TXK::Process::run('TXK::Process=HASH(0x8b112d0)','HASH(0x8add484)') called at /u01/app/****/apps/apps_st/appl/fnd/12.0.0/patch/115/bin/txkCfgOC4JApp.pl line 1587

    TXK::RunScript::execOPMNControl('HASH(0x8ce7fb8)') called at /u01/app/****/apps/apps_st/appl/fnd/12.0.0/patch/115/bin/txkCfgOC4JApp.pl line 606

    require /u01/app/****/apps/apps_st/appl/fnd/12.0.0/patch/115/bin/txkCfgOC4JApp.pl called at /u01/app/****/apps/apps_st/appl/au/12.0.0/perl/TXK/RunScript.pm line 105

    TXK::RunScript::require('TXK::RunScript','/u01/app/****/apps/apps_st/appl/fnd/12.0.0/patch/115/bin/txk...') called at /u01/app/****/apps/apps_st/appl/au/12.0.0/perl/TXK/Script.pm line 177

Steps to deploy forms (397174.1).

==============

*** cp $INST_TOP/ora/10.1.3/j2ee/forms/config/system-jazn-data.xml $INST_TOP/ora/10.1.3/j2ee/forms/config/system-jazn-data.xml.orig_11Aug2018

*** vi $INST_TOP/ora/10.1.3/j2ee/forms/config/system-jazn-data.xml

Add OC4J Administrator credentials.

<name>oc4jadmin</name>

    <display-name>OC4J Administrator</display-name>

    <description>OC4J Administrator</description>

    <credentials>!welcome123</credentials>

   </user>

***

[****@jauauaua ~]$ cp $IAS_ORACLE_HOME/opmn/bin/opmnctl $IAS_ORACLE_HOME/opmn/bin/opmnctl.org

[****@jauauaua ~]$ cp $INST_TOP/ora/10.1.3/opmn/bin/opmnctl $IAS_ORACLE_HOME/opmn/bin/opmnctl

[****@jauauaua ~]$ cp $IAS_ORACLE_HOME/opmn/conf/opmn.xml $IAS_ORACLE_HOME/opmn/conf/opmn.xml.org

[****@jauauaua ~]$ cp $INST_TOP/ora/10.1.3/opmn/conf/opmn.xml $IAS_ORACLE_HOME/opmn/conf/opmn.xml

*** Stop the application /adstpall.sh

4.[****@jauauaua ~]$ $FND_TOP/bin/txkrun.pl -script=CfgOC4JApp

*** ALL THE FOLLOWING FILES ARE REQUIRED FOR RESOLVING RUNTIME ERRORS

*** Log File = /u01/app/k****/inst/apps/JAJAJ_jauauaua/logs/appl/rgf/TXK/txkCfgOC4JApp_Sat_Aug_11_16_55_58_2018.log

Program : /u01/app/****/k****/apps_st/appl/fnd/12.0.0/patch/115/bin/txkCfgOC4JApp.pl started @ Sat Aug 11 16:55:58 2018

*** Log File = /u01/app/****/inst/apps/JAJAJ_jauauaua/logs/appl/rgf/TXK/txkCfgOC4JApp_Sat_Aug_11_16_55_58_2018.log

Enter Application name for re-deployment ? forms

Enter Oc4j Instance password for re-deployment ?   <<Enter password welcome123>>

Run Autoconfig <Yes/No> ? No

6.Run AutoConfig on the instance by running the command:

$ $ADMIN_SCRIPTS_HOME/adautocfg.sh

7.Restart the application tier services by running the command:

$ $ADMIN_SCRIPTS_HOME/adstrtal.sh <apps user/apps password>

Solution when form deployment failing with above errors.

======================================

Add -Djdk.crypto.KeyAgreement.legacyKDF=true to oacore,forms,oafm start/stop properties in contextfile.

Ex::

****@jaidkalala ~]$ grep -i oacore $CONTEXT_FILE

                        <forms_jvm_start_options oa_var="s_forms_jvm_start_options">-server -verbose:gc -Xmx256M -Xms64M -XX:MaxPermSize=128M -XX:NewRatio=2 -XX:+PrintGCTimeStamps -XX:+UseTLAB -XX:+UseParallelGC -XX:ParallelGCThreads=2 -Djava.security.policy=$ORACLE_HOME/j2ee/oacore/config/java2.policy -Djava.awt.headless=true -Dhttp.webdir.enable=false -Doracle.security.jazn.config=/u01/app/****/inst/apps/JUAUa_JAUAUAdkd/ora/10.1.3/j2ee/forms/config/jazn.xml -Djdk.crypto.KeyAgreement.legacyKDF=true</forms_jvm_start_options>

                        <forms_jvm_stop_options oa_var="s_forms_jvm_stop_options">-server -verbose:gc -Xmx256M -Xms64M -XX:MaxPermSize=128M -XX:NewRatio=2 -XX:+PrintGCTimeStamps -XX:+UseTLAB -XX:+UseParallelGC -XX:ParallelGCThreads=2 -Djava.security.policy=$ORACLE_HOME/j2ee/oacore/config/java2.policy -Djava.awt.headless=true -Dhttp.webdir.enable=false -Djdk.crypto.KeyAgreement.legacyKDF=true</forms_jvm_stop_options>

                <oacore_server>

                        <oacore_jvm_start_options oa_var="s_oacore_jvm_start_options">-server -verbose:gc -Xmx1024M -Xms256M -XX:MaxPermSize=160M -XX:NewRatio=2  -XX:+PrintGCTimeStamps -XX:+UseTLAB -XX:+UseParallelGC  -XX:ParallelGCThreads=2  -Dcom.sun.management.jmxremote -Djava.security.policy=$ORACLE_HOME/j2ee/oacore/config/java2.policy -Djava.awt.headless=true -Dhttp.webdir.enable=false -Doracle.security.jazn.config=/u01/app/***/inst/apps/JUAUa_JAUAUAdkd/ora/10.1.3/j2ee/oacore/config/jazn.xml -Doracle.mds.custcache.maxentries=1000 -Djdk.crypto.KeyAgreement.legacyKDF=true</oacore_jvm_start_options>

                        <oacore_oc4j_options oa_var="s_oacore_oc4j_options">-out /u01/app/***/inst/apps/JUAUa_JAUAUAdkd/logs/ora/10.1.3/opmn/oacorestd.out -err /u01/app/***/inst/apps/JUAUa_JAUAUAdkd/logs/ora/10.1.3/opmn/oacorestd.err</oacore_oc4j_options>

                        <oacore_jvm_stop_options oa_var="s_oacore_jvm_stop_options">-server -verbose:gc -Xmx512M -Xms128M -XX:MaxPermSize=160M -XX:NewRatio=2  -XX:+PrintGCTimeStamps -XX:+UseTLAB -XX:+UseParallelGC  -XX:ParallelGCThreads=2  -Djava.security.policy=$ORACLE_HOME/j2ee/oacore/config/java2.policy -Djava.awt.headless=true -Dhttp.webdir.enable=false -Djdk.crypto.KeyAgreement.legacyKDF=true</oacore_jvm_stop_options>

                        <oafm_jvm_start_options oa_var="s_oafm_jvm_start_options">-server -verbose:gc -Xmx256M -Xms64M -XX:MaxPermSize=128M -XX:NewRatio=2 -XX:+PrintGCTimeStamps -XX:+UseTLAB -XX:+UseParallelGC -XX:ParallelGCThreads=2  -Djava.security.policy=$ORACLE_HOME/j2ee/oacore/config/java2.policy  -Djava.awt.headless=true  -Dhttp.webdir.enable=false -Doracle.security.jazn.config=/u01/app/***/inst/apps/JUAUa_JAUAUAdkd/ora/10.1.3/j2ee/oafm/config/jazn.xml -Djdk.crypto.KeyAgreement.legacyKDF=true</oafm_jvm_start_options>

                        <oafm_oc4j_options oa_var="s_oafm_oc4j_options">-out  /u01/app/***/inst/apps/JUAUa_JAUAUAdkd/logs/ora/10.1.3/opmn/oafmstd.out -err  /u01/app/***/inst/apps/JUAUa_JAUAUAdkd/logs/ora/10.1.3/opmn/oafmstd.err</oafm_oc4j_options>

                        <oafm_jvm_stop_options oa_var="s_oafm_jvm_stop_options">-server -verbose:gc -Xmx256M -Xms64M -XX:MaxPermSize=128M -XX:NewRatio=2 -XX:+PrintGCTimeStamps -XX:+UseTLAB -XX:+UseParallelGC -XX:ParallelGCThreads=2  -Djava.security.policy=$ORACLE_HOME/j2ee/oacore/config/java2.policy  -Djava.awt.headless=true  -Dhttp.webdir.enable=false -Djdk.crypto.KeyAgreement.legacyKDF=true</oafm_jvm_stop_options>

** And ran autoconfig

**Forms deployment completed successfully.

Program : /u01/app/****/apps/apps_st/appl/fnd/12.0.0/patch/115/bin/txkCfgOC4JApp.pl completed @ Wed Nov 7 21:39:02 2018

End of /u01/app/****/apps/apps_st/appl/fnd/12.0.0/patch/115/bin/txkCfgOC4JApp.pl : No Errors encountered

ML :: 2353710.1

Thanks

Sreeharsha

License A new product/Module in EBS using OAM.

Hello every one,

Let's see how we can do license a new module/product in EBS using Oracle Application Manager.

Here are the steps
==========

**Login as sysadmin user

**Go to System Administrator

**Go to Oracle Applications Manager

**Click on License Manager

**In license section select products.

**Click on Show more options and select License Applications Product

**Select required module and click next and submit.

Post steps

=======

3. Now using adadmin and choose first option 1.

            AD Administration Main Menu

   --------------------------------------------------

   1.    Generate Applications Files menu

   2.    Maintain Applications Files menu

   3.    Compile/Reload Applications Database Entities menu

   4.    Maintain Applications Database Entities menu

   5.    Change Maintenance Mode

   6.    Exit AD Administration

   

Select Generate Application Files menu

Enter your choice [6] : 1

         Generate Applications Files

   ----------------------------------------

   1.    Generate message files

   2.    Generate form files

   3.    Generate report files

   4.    Generate product JAR files

   5.    Return to Main Menu

Run below.

   ***Generate message files

   ***Generate form files

   ***Generate report files

Back to Main Menu

Select Compile/Reload Applications Database Entities menu

Enter your choice [6] : 3

        Compile/Reload Applications Database Entities

   ---------------------------------------------------

   1.    Compile APPS schema

   2.    Compile menu information

   3.    Compile flexfields

   4.    Reload JAR files to database

   5.    Return to Main Menu

Run following.

Compile APPS schema

Compile menu information

Compile flexfields

4. Run autoconfig 

5. Verify Product installed successfully or not using below query.

select fa.APPLICATION_SHORT_NAME, fpi.PATCH_LEVEL, DECODE(fpi.STATUS, 'I','Installed',

'S','Shared', 'N', 'Inactive', fpi.STATUS) Status , fpi.DB_STATUS

from fnd_product_installations fpi,FND_APPLICATION fa

where fpi.APPLICATION_ID in (

select APPLICATION_ID from FND_APPLICATION where APPLICATION_SHORT_NAME like 'AME')

and fa.APPLICATION_ID=fpi.APPLICATION_ID; 

6) Make sure we verify related TOPS available on File system level. 

Thanks.

Sreeharsha

UNNAMED data file in standby after adding new data file in primary – ORA-01111, ORA-01110, ORA-01157 , ORA-01274

Dear Folks,

We came across missing data file in standby after adding new data file in primary which causes MRP job terminate. This could be due to following reasons.

**Either we don't have sufficient  disk space/ASM available at standby host.
**standby_file_management parameter set to MANUAL.

i.e, if standby_file_management parameter is MANUAL on standby, MRP processes couldn't apply archives due to this wrong parameter settings, a UNNAMED data file being created at $ORACLE_HOME/dbs location.

From alert log
==========

File #59 added to control file as 'UNNAMED00059' because
the parameter STANDBY_FILE_MANAGEMENT is set to MANUAL
The file should be manually created to continue.
Errors with log +JDUAUA_FRA/xcmrc_stby/archivelog/2019_02_13/thread_1_seq_9619.319.1000088461
MRP0: Background Media Recovery terminated with error 1274
Errors in file /
ORA-01274: cannot add datafile '+JDUAUA_DATA/xcmrc/datafile/apps_ts_tx_data.299.1000076027' - file could not be created
Managed Standby Recovery not using Real Time Apply

Recovery stopped due to failure in applying recovery marker (opcode 17.30).

So from above log, we clearly identified missing data file=59

SQL> select file#,name from v$datafile where file#=59;

     FILE#

----------

NAME

--------------------------------------------------------------------------------

        59

+JDUAUA_DATA_DATA/xcmrc/datafile/apps_ts_tx_data.299.1000076027

Identify the missing datafile from standby

     FILE#

----------

NAME

--------------------------------------------------------------------------------

        59

/u01/app/oraxx/JDUAUA_DATA/db/tech_st/11.2.0/dbs/UNNAMED00059

Hence created missing data file in Standby.

SQL > alter database create datafile '/u01/app/oraxx/JDUAUA_DATAdb/tech_st/11.2.0/dbs/UNNAMED00059' as '+JDUAUA_DATA_DATA' size 5576m

Database altered.

Enabled standby_file_management to AUTO

SQL> alter system set standby_file_management=AUTO sid='*';

System altered.

Start MRP process.

===========

SQL> recover managed standby database disconnect from session;

Media recovery complete.

Thanks.

Sreeharsha

RMAN-6091: no channel allocated for maintenance (of an appropriate type)

Hello Every one,

Recently, In one of our EBS customer, we came across issue with RMAN backup post CPU Q3 patches  on to 12.1.0.2 instance. The backup failing with below error.

RMAN> delete noprompt archivelog until time 'sysdate -3' backed up 1 times to sbt;
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of delete command at 02/03/2019 20:17:05
RMAN-06091: no channel allocated for maintenance (of an appropriate type)

Hence, we've implemented workaround  as per ML 2428682.1

allocate disk channel and delete the datafile copies and archivelog copies.


run {
allocate channel ch1 type 'sbt_tape''";
backup database
include current controlfile ;
}
allocate channel for maintenance device type disk;
crosscheck archivelog all;
delete force noprompt archivelog all;

Workaround::2

============

Apply patch  28432129.

Expected behavior
============

Event after applying the patch 28432129, RMAN may fail with an Error RMAN-6091.

Enhancement Request will be implemented in the future release:" Bug 28593955 : RMAN-6091 ERRORS SEEN EVEN AFTER APPLYING THE PATCH 28432129 "

Note:: This bug 28593955 still yet to implemented by oracle. So we need to apply patch 28432129

Bug fix 28432129 only to allow deletion of archive log and image copies (Not backupsets) , So having some maintenance command mail fail if appropriate channels are not allocated.

Rman never deleted disk backups from SBT channels.

Suggested changes to RMAN scripts to avoid an error.

1) Move all maintenance commands into separate RUN{} or outside RUN{} block as shown in above command and allocate required channels (DISK and TAPE, if RMAN needs to operate on backup files residing on both device types).

2) Have backup commands by allocating whatever device type channels the user wants to create backup files

We should still stick to requirement to have correct device types allocated for maintenance commands operations to work correctly.

Thanks.

Sreeharsha

Oracle XML Database invalid, could not find xdbconfig

Hello Guys,


In one of our EBS customer have many invalids in database and we try to compile them using utrp.sql and eventually failed with below error.


DECLARE
*
ERROR at line 1:
ORA-04063: package body "SYS.DBMS_SQLTUNE" has errors
ORA-06508: PL/SQL: could not find program unit being called: "SYS.DBMS_SQLTUNE"
ORA-06512: at "SYS.UTL_RECOMP", line 875
ORA-06512: at line 4


We've found few components in dba_registry shows INVALID.


COMP_NAME           COMP_ID  VERSION  STATUS
------------------------------ ---------- ---------- ----------
Oracle XML Database      XDB    12.1.0.2.0 INVALID
Oracle Database Packages and T CATPROC  12.1.0.2.0 INVALID
Oracle OLAP API        XOQ    12.1.0.2.0 INVALID


To over come the issue. Please follow below steps.


For CATPROC component:
If there has been no recent database upgrade taking place, the following set of commands may help validating them:
$ sqlplus / as sysdba
@$ORACLE_HOME/rdbms/admin/catalog.sql
@$ORACLE_HOME/rdbms/admin/catproc.sql
@$ORACLE_HOME/rdbms/admin/utlrp.sql
--
-- usually, <utlrp.sql> has to be executed couple of more times for dependency reasons
--
@$ORACLE_HOME/rdbms/admin/utlrp.sql
@$ORACLE_HOME/rdbms/admin/utlrp.sql


for Rest of component:
start up the database in UPGRADE mode and connect ‘/as sysdba’ to execute any of the following scripts.


Oracle XML Database
SQL> @?/rdbms/admin/xdbpatch.sql


OLAP API
SQL> @?/olap/admin/xoqpatch.sql




How to Diagnose Components with NON VALID Status in DBA_REGISTRY after an Upgrade (Doc ID 753041.1)
IF: Invalid Catalog and/or Catproc Database Registry Components (Doc ID 2358487.1)


Thanks.
Sreeharsha

MWA Startup Issue

Hello every one,


Recently we came across MWA bounce in which one of port couldn't able to start. This issue due to  MWA wrapper script which used to stop/start MWA. In our environment we have lower version of mwactlwrpr.sh script. Hence we had to apply 15868470 patch  as per ML DOC :: 1515211.1 to get latest version.


DOC ID:Mobile Web Applications MWA Server Processes Not Starting Using Wrapper Script mwactlwrpr.sh start (Doc ID 1515211.1)

From log file
==========

Stopping MWA Servers and the dispatcher ....
Stopping MWA Server on Port number: 10218
Stopping MWA Server on Port number: 10220
Stopping MWA Server on Port number: 10222
Stopping MWA Dispatcher on Port number: 10809
Killing mwa Dispatcher, if still running..........
Starting MWA Wrapper script. Checking if all ports are free to start the servers and dispatcher...
Starting MWA Server on Port number: 10218
Starting MWA Server on Port number: 10220
PortNumber: 10222 is not free to start the server. Please specify a different port in the configuration file (or) end the process using this port
Starting MWA Dispatcher on Port number: 10809


Before apply patch
===============
$Header mwactlwrpr.sh 120.6.12010000.2
After apply patch
============
$INST_TOP/admin/scripts/mwactlwrpr.sh 120.6.12010000.3

Thanks.
Sreeharsha

ORA-00600: internal error code, arguments: [krbvalmrange_metacnt_mismatch]

Hello every one,


Recently, one of our customer requested us to clone a test environment with mile stone backup which taken 2 weeks back in produciton. Hence we copied backup pieces from source to target using FTP.


But, when we start restoring, it fails with below error.


RMAN-03002: failure of restore command at 02/06/2019 22:19:58
ORA-19870: error while restoring backup piece /backup/JAJAUA/07022019/df_8dto8quo_1_12019%mJAJAJA
ORA-00600: internal error code, arguments: [krbvalmrange_metacnt_mismatch], [400], [13562], [34116], [], [], [], [], [], [], [], []


It is because we copied pieces using FTP and pieces are corrupted.


Solution
=====


Copy backup pieces using SCP command from source to Target.. Make sure not to use FTP command.

Thanks.
Sreeharsha

Javacache.log Flooded With Java.lang.OutOfMemoryError Error Messages

Hello every one,


In one of our customer, a file Javacache.log growing rapidly around 250GB and consuming most  space on mount. By looking into the log file, we've seen  it filled with "Java.lang.OutOfMemoryError Error Messages" . The issue is caused by JVM debug log level was not be set.

we came across one ML doc 1604874.1. However we had to check with oracle for further action plan.


Solution
====


1) Take backup of oc4j.properties


2) Add below parameter in below files.
IASCACHELOGLEVEL=0   >> Value is Zero.

$ORA_CONFIG_HOME/10.1.3/j2ee/oacore/config/oc4j.properties


[appdi@Jauaua1 config]$ diff oc4j.properties_1303470_bkp oc4j.properties
129a130,131
> #Parameter for JavaCachelog_TKT_1303470
> IASCACHELOGLEVEL=0


If autoconfig runs, it overwrite the file and value will be diminish. Hence we changed in template file which associated with the autoconfig generated file oc4j.properties.


To review which configuration file changed as part of next  autoconfig run. Please use below command. It will generate a HTML file and review it. It shows which files will be overwrite in next autoconfig run.


adchkcfg.sh contextfile=<CONTEXT_FILE>


Reviewed generated HTML file and found template file oc4j_properties_1013.tmp associated with oc4j.properties file. Hence we modified in template file. So next autoconfig run wouldn't diminish the value.


$FND_TOP/admin/template/custom/oc4j_properties_1013.tmp


[appdi@uauauacustom]$ cp oc4j_properties_1013.tmp oc4j_properties_1013.tmp_1303470


Add below parameter in template file.


IASCACHELOGLEVEL=0   >>Value is Zero.


[appdi@uauauauacustom]$ diff oc4j_properties_1013.tmp oc4j_properties_1013.tmp_1303470
122,123d121
< ##Parameter for JavaCachelog_TKT_1303470
< IASCACHELOGLEVEL=0
3) Take backup of javacache.log file and nullified it.
4) Bounce OPMN Services.


Thanks.
Sreeharsha

oracle.apps.fnd.framework.OAException: Application: FND, Message Name: FND_NO_DATABASE_CONNECTION

Hello Every one,


Recently one of our customer facing issue with PROD application login. They getting exception when they tried to login.


So further checked oacore logs (application.log)  and identify below error.


Location :: /prod/oracle/PROD/inst/apps/PROD_pchklpp01/logs/ora/10.1.3/j2ee/oacore/oacore_default_group_1


Errors from log file
=============


19/02/08 05:31:39.733 html: chain failed
javax.servlet.ServletException: Application: FND, Message Name: FND_NO_DATABASE_CONNECTION.
        at com.evermind[Oracle Containers for J2EE 10g (10.1.3.5.0) ].server.http.EvermindPageContext.handlePageThrowable(EvermindPageContext.java:899)
        at com.evermind[Oracle Containers for J2EE 10g (10.1.3.5.0) ].server.http.EvermindPageContext.handlePageException(EvermindPageContext.java:816)
        at _OALogout._jspService(_OALogout.java:243)
        at com.orionserver[Oracle Containers for J2EE 10g (10.1.3.5.0) ].http.OrionHttpJspPage.service(OrionHttpJspPage.java:59)
        at oracle.jsp.runtimev2.JspPageTable.service(JspPageTable.java:473)
        at oracle.jsp.runtimev2.JspServlet.internalService(JspServlet.java:594)
        at oracle.jsp.runtimev2.JspServlet.service(JspServlet.java:518)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
        at com.evermind[Oracle Containers for J2EE 10g (10.1.3.5.0) ].server.http.ResourceFilterChain.doFilter(ResourceFilterChain.java:64)
        at oracle.apps.jtf.base.session.ReleaseResFilter.doFilter(ReleaseResFilter.java:26)
        at com.evermind[Oracle Containers for J2EE 10g (10.1.3.5.0) ].server.http.EvermindFilterChain.doFilter(EvermindFilterChain.java:15)
        at oracle.apps.fnd.security.AppsServletFilter.doFilter(AppsServletFilter.java:318)
        at com.evermind[Oracle Containers for J2EE 10g (10.1.3.5.0) ].server.http.ServletRequestDispatcher.invoke(ServletRequestDispatcher.java:642)
        at com.evermind[Oracle Containers for J2EE 10g (10.1.3.5.0) ].server.http.ServletRequestDispatcher.forwardInternal(ServletRequestDispatcher.java:391)
        at com.evermind[Oracle Containers for J2EE 10g (10.1.3.5.0) ].server.http.HttpRequestHandler.doProcessRequest(HttpRequestHandler.java:908)
        at com.evermind[Oracle Containers for J2EE 10g (10.1.3.5.0) ].server.http.HttpRequestHandler.processRequest(HttpRequestHandler.java:458)
        at com.evermind[Oracle Containers for J2EE 10g (10.1.3.5.0) ].server.http.AJPRequestHandler.run(AJPRequestHandler.java:313)
        at com.evermind[Oracle Containers for J2EE 10g (10.1.3.5.0) ].server.http.AJPRequestHandler.run(AJPRequestHandler.java:199)


For immediate fix, we bounced opmn and it sets every thing. However the issue still repeats. Hence we applied patch 11832737  as per ML DOC :: 1298103.1  for Permanant fix.


R12: Login Fails Intermittently With Error "oracle.apps.fnd.framework.OAException: Application: FND, Message Name: FND_NO_DATABASE_CONNECTION" (Doc ID 1298103.1)

This patch delivers below two files with higher version. Finally issue fixed.


fnd java/framework/webui OARESTUtil.java 120.0.12010000.12
fnd java/framework/webui/rest OABaseRESTHandler.java 120.0.12010000.13


Thanks,
Sreeharsha

ORA-04030: out of process memory when trying to allocate 123416 bytes (QERHJ hash-joi,kllcqas:kllsltba)

Hello Every one,


Recently we'd faced ORA-04030 errors in one of our customer alert log. Customer running on 11.2.0.4 database with 12.1.3 application. So it is expected few bugs in it. We've identified many ML Docs and didn't find exact hint. Hence we had to raise an SR with oracle and finally they came with solution.

There could be several reasons why these errors triggered. Generally  ORA-04030 can be caused by RAM  shortage on non shared server environment, that means oracle processes couldn't get enough amount of RAM to complete it operation  and more over instance having small PGA set , So we need to look at memory structure and set correct parameters as per ML DOC (396009.1) for 11g instance  and also it cause  by not setting enough kernel parameters  to allow enough RAM for processes when running.. 

From alert log
========


ORA-04030: out of process memory when trying to allocate 123416 bytes (QERHJ hash-joi,kllcqas:kllsltba)
ORA-04030: out of process memory when trying to allocate 96 bytes (kkqcs.c.kgght,kkqcs.c.kgght)
ORA-19202: Error occurred in XML processing
ORA-06503: PL/SQL: Function returned without value
ORA-06512: at "APPS.XXAS_COM_HIER_PKG", line 2767


By looking above alert log errors, we dig further and identified a custom package execution taking more PGA memory and finally failed with errors. We can involve developer to tune that package.


We checked same with oracle by providing them a trace and incident file and solutions given as


ACTION PLAN:
==============
1. On OS side edit /etc/sysctl.conf. Add or edit the following line:
vm.max_map_count = 262120
Save the file and load sysctl configuration
$ sysctl -p
Log out and log in again.
2. On database, set hidden parameter _realfree_heap_pagesize_hint:
alter system set "_realfree_heap_pagesize_hint" = 262144 scope=spfile;
Bounce database to reflect changes.


Thanks,
Sreeharsha

Users Receiving Larger Number of Emails For Already Closed Notifications

Hello All,

Recently we'd an issue with one of EBS user who receiving high volume of Emails from workflow around 4000 Emails. Hence we dig the issue deeper by logged in to workflow accout and identified bunch of Undeliverable emails in process folder. Below we could see the message.

Undeliverable: Closed: Aacksonville Aviation Sector- Standard Purchase Order 39535, 0"

So We opened email attachments and found few emails are invalid as below and and NID::442082 mail_status is INVALID and and status is CLOSED.The issue is caused by the mailer processing INVALID responses for CLOSED notifications.

Your message to samathar.b.kaakkt@ciecouldn't be delivered

Your message to mike.c.kakkat@cie.com couldn't be delivered.

Your message to pallu.p.deckler@cie.com couldn't be delivered.

SQL> select RECIPIENT_ROLE,STATUS,MAIL_STATUS,FROM_USER,TO_USER from wf_notifications where NOTIFICATION_ID =442082;

RECIPIENT_ROLE                   STATUS  MAIL_STATUS
-------------------------------------------------- -------- --------------------
FROM_USER           TO_USER
------------------------------ ----------------------------------------
ADHOC245759830598227883PO             CLOSED  NULL
fraier , monter Join Controver INC

The supplier "Join Controver INC" having 4-Emails to which notifications are being sent.

The EBS user who having valid Email and other 3 emails are invalid. So workflow is retrying 2-3 minutes to resend the undeliverable  emails and hence volume. Since one EBS user email is correct. He/She receiving bunch of undelierable email notifications every couple of minutes.

To over come the issue, we set mail_status to NULL. However we need to ask client those emails should be corrected by payable / Procurement team. Otherwise any further Purchase Orders that are communicated to that supplier without invalid emails to be corrected will cause this issue again.

Solutions 

=====

Mark Mail_status to NULL in wf_notifications for particular NID which having issue.

update wf_notifications set mail_status='NULL' where notification_id=442082

OR please follow solutions that refers in below document.

Users Receiving Larger Number of Emails For Already Closed Notifications (Doc ID 1293792.1) Thanks, Sreeharsha