Dear Folks.
Recently, we have faced an issue with workflow notification mailer which it wouldn't come up and shows "Service container Not running" when it was started. We have checked all components and all were up and running fine.
Checked workflow mailer log and found below errors. It was due to wf_control queue corrupt.
177.129.7:30240:1652094015861:1:UNEXPECTED:[fnd.wf.bes.ConnectionManager$1]:Failed to establish Java Business Event System control connection: databaseId = PAMPI, ownerName = APPLSYS, queueName = WF_CONTROL, subscriberName = WFBESCL_5204, selector = null, exception = oracle.apps.fnd.wf.bes.BusinessEventException: A JMS error occurred; see the linked exception for details., linkedException = oracle.jms.AQjmsException: ORA-06512: at "SYS.DBMS_AQADM_SYS", line 9306
ORA-06512: at "SYS.DBMS_PRVTAQIS", line 1873
ORA-06512: at "SYS.DBMS_AQADM_SYS", line 8231
ORA-06512: at "SYS.DBMS_PRVTAQIM", line 855
ORA-00928: missing SELECT keyword
ORA-06512: at "SYS.DBMS_AQADM_SYSCALLS", line 1138
ORA-06512: at "SYS.DBMS_PRVTAQIM", line 850
ORA-06512: at "SYS.DBMS_AQADM_SYS", line 8307
ORA-06512: at "SYS.DBMS_AQADM_SYS", line 8222
ORA-06512: at "SYS.DBMS_PRVTAQIS", line 1835
ORA-06512: at "SYS.DBMS_PRVTAQIS", line 1516
ORA-06512: at "SYS.DBMS_AQADM_SYS", line 9900
ORA-06512: at "SYS.DBMS_AQADM_SYS", line 9269
ORA-06512: at line 1
ORA-06512: at "SYS.DBMS_AQJMS", line 134
ORA-06512: at "SYS.DBMS_AQJMS", line 186
ORA-06512: at line 1
Since, we have no direct ML found for above error. We had to approach the Oracle for solution and here it is.
Below fix helped to bring back workflow mailer up.
=====
1- Stop the Concurrent Managers
2- Drop/Recreate queeu wf_control :
sqlplus apps/<apps_pwd> @$FND_TOP/patch/115/sql/wfctqrec.sql APPLSYS <applsys_pwd>
3- Start the Conc Managers
4- Verify the issue
