GoldenGate Extract Pump process keeps ABENDing – OGG-01031

I encountered an odd situation recently where the extract pump processes kept ABENDing. The report file listed the OGG-01031 error message.

GGSCI (exadb04.new.fastrieve.com) 23> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING                                           
EXTRACT     RUNNING     CBOA        00:00:00      00:00:02    
EXTRACT     RUNNING     CGGMONX     00:00:01      00:00:05    
EXTRACT     ABENDED     PBOA        00:00:00      01:46:40    
EXTRACT     ABENDED     PGGMONX     00:00:00      01:43:14    
REPLICAT    RUNNING     RGGMONY     00:00:00      00:03:24    


GGSCI (exadb04.new.fastrieve.com) 1> view report pggmonx


***********************************************************************
                 Oracle GoldenGate Capture for Oracle
    Version 11.2.1.0.1 OGGCORE_11.2.1.0.1_PLATFORMS_120423.0230_FBO
   Linux, x64, 64bit (optimized), Oracle 11g on Apr 23 2012 08:42:16
 
Copyright (C) 1995, 2012, Oracle and/or its affiliates. All rights reserved.


                    Starting at 2013-08-14 16:15:12
***********************************************************************
.
.
.

2013-08-14 16:15:28  ERROR   OGG-01031  There is a problem in network communication, a remote file problem,
encryption keys for target and source do not match (if using ENCRYPT) or an unknown error. (Reply received
is Unable to open file "./dirdat/gX000017" (error 11, Resource temporarily unavailable)).

I wasn’t able to find very much on this issue, but I did run across MOS note 1318980.1, and it turned out to have the fix in my case. By renaming the associated $GG_HOME/dirchk/*.cps and $GG_HOME/dirchk/*.cpe files, I was able to get the processes running again. Note that in this case the $GG_HOME/dirchk directory resides on an NFS mount. Apparently this type of error can occur on shared file systems causing the $GG_HOME/dirchk entries to be non-updatable.

Extract/Replicat Shows Already Running When Started and Already Stopped When Stopped from GGSCI, OGG-01031, Reply received is Unable to open file (Doc ID 1318980.1)

[exadb04:oracle:FTRVXP4] /u01/app/oracle/product/golden_gate/dirchk 
> ls -lhF
total 696K
-rw-rw-r-- 1 oracle dba  72K Aug 14 16:28 PGGMONX.cpe
-rw-rw-r-- 1 oracle dba 2.2K Aug 14 16:27 PGGMONX.cps

[exadb04:oracle:FTRVXP4] /u01/app/oracle/product/golden_gate/dirchk 
> mv PGGMONX.cpe PGGMONX.cpe_new

[exadb04:oracle:FTRVXP4] /u01/app/oracle/product/golden_gate/dirchk 
> mv PGGMONX.cps PGGMONX.cps_new

[exadb04:oracle:FTRVXP4] /u01/app/oracle/product/golden_gate/dirchk 
> ls -lhF
total 696K
-rw-rw-r-- 1 oracle dba  72K Aug 14 16:28 PGGMONX.cpe_new
-rw-rw-r-- 1 oracle dba 2.2K Aug 14 16:27 PGGMONX.cps_new

[exadb04:oracle:FTRVXP4] /u01/app/oracle/product/golden_gate/dirchk 
> mv PGGMONX.cpe_new PGGMONX.cpe

[exadb04:oracle:FTRVXP4] /u01/app/oracle/product/golden_gate/dirchk 
> mv PGGMONX.cps_new PGGMONX.cps

[exadb04:oracle:FTRVXP4] /u01/app/oracle/product/golden_gate/dirchk 
> ls -lhF
total 696K
-rw-rw-r-- 1 oracle dba  72K Aug 14 16:28 PGGMONX.cpe
-rw-rw-r-- 1 oracle dba 2.2K Aug 14 16:27 PGGMONX.cps

After restart the processes, everything was caught up in a couple of minutes.

GGSCI (exadb04.new.fastrieve.com) 11> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING                                           
EXTRACT     RUNNING     CBOA        00:00:07      00:00:04    
Description "***** Capture for Bank of America *****"
EXTRACT     RUNNING     CGGMONX     00:02:21      00:00:01    
Description "***** Capture for GGMON X *****"
EXTRACT     RUNNING     PBOA        00:00:00      00:00:05    
EXTRACT     RUNNING     PGGMONX     00:00:00      00:00:08    
Description "***** Pump for GGMON X *****"
REPLICAT    RUNNING     RGGMONY     00:00:00      00:00:02    

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s