oraagent.bin High Memory Usage

Cause : Grid Infrastructure oraagent.bin process using huge amount of memory and  forking huge number of threads due to missing "ora.LISTENER.lsnr" listener.

Solution: Perform the following steps if you found in oraagent alert 

1. Check te log file of oraagent service and if you really missing to register listener perform the following.

cat $GRID_HOME/log/hostname/agent/ohasd/oraagent_grid/oraagent_grid.log

2017-01-15 19:03:22.679: [ USRTHRD][3433169]{0:237:2} CrsCmd::ClscrsCmdData::stat entity 1 statflag 33 useFilter 0
2017-01-15 19:03:22.745: [ USRTHRD][3433169]{0:237:2} checkCrsStat 2 CLSCRS_STAT ret: 200
2017-01-15 19:03:22.745: [ USRTHRD][3433169]{0:237:2} checkCrsStat 2 clscrs_res_get_op_status CLSCRS_STAT status 210 err_msg CRS-0210: Could not find resource 'ora.LISTENER.lsnr'.
2017-01-15 19:03:22.745: [ USRTHRD][3433169]{0:237:2} AsmCommonAgent::setLocalListener cls::Exception CRS-0210: Could not find resource 'ora.LISTENER.lsnr'.
2017-01-15 19:03:22.745: [ USRTHRD][3433169]{0:237:2} ASM Dedicated Thread }
2017-01-15 19:03:22.745: [ USRTHRD][3433169]{0:237:2} Thread:ASM DedicatedThreadisRunning is reset to false here
2017-01-15 19:03:23.584: [ora.asm][3599]{0:237:2} [check] AsmProxy StartDependeeRes = ora.LISTENER.lsnr
2017-01-15 19:03:23.584: [ USRTHRD][3599]{0:237:2} Thread:ASM DedicatedThreadstart {
2017-01-15 19:03:23.584: [ USRTHRD][3599]{0:237:2} Thread:ASM DedicatedThreadstart }
2017-01-15 19:03:23.585: [ USRTHRD][3433428]{0:237:2} ASM Dedicated Thread {
2017-01-15 19:03:23.594: [ USRTHRD][3433428]{0:237:2} CrsCmd::ClscrsCmdData::stat entity 1 statflag 32 useFilter 0
bash-4.3$

2. Check the cluster resource 

bash-4.3$ crsctl status resource -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DG_BACKUP.dg
               ONLINE  ONLINE       erpdb01
ora.DG_DATA01.dg
               ONLINE  ONLINE       erpdb01
ora.DG_FRA.dg
               ONLINE  ONLINE       erpdb01
ora.DG_OCR_VOT.dg
               ONLINE  ONLINE       erpdb01
ora.DG_REDO.dg
               ONLINE  ONLINE       erpdb01
ora.asm
               ONLINE  ONLINE       erpdb01                Started
ora.ons
               OFFLINE OFFLINE      erpdb01
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
      1        ONLINE  ONLINE       erpdb01
ora.diskmon
      1        OFFLINE OFFLINE
ora.evmd
      1        ONLINE  ONLINE       erpdb01
bash-4.3$

3. Check the status of listener 

bash-4.3$ lsnrctl status

LSNRCTL for IBM/AIX RISC System/6000: Version 11.2.0.4.0 - Production on 15-JAN-2017 19:08:21

Copyright (c) 1991, 2013, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC1522)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER
Version                   TNSLSNR for IBM/AIX RISC System/6000: Version 11.2.0.4.0 - Production
Start Date                02-DEC-2016 22:16:23
Uptime                    43 days 20 hr. 51 min. 58 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      ON
Listener Parameter File   /u01/app/11.2.0/grid/network/admin/listener.ora
Listener Log File         /u01/app/grid/diag/tnslsnr/erpdb01/listener/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1522)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=erpdb01.sonalibank.com.bd)(PORT=1521)))
Services Summary...
Service "sbldcdbXDB" has 1 instance(s).
  Instance "erpdrdb", status READY, has 1 handler(s) for this service...
Service "erpdrdb" has 1 instance(s).
  Instance "erpdrdb", status READY, has 1 handler(s) for this service...
Service "slbdrdb" has 1 instance(s).
  Instance "erpdrdb", status UNKNOWN, has 1 handler(s) for this service...
The command completed successfully

4. Add the listener

bash-4.3$ srvctl add listener -l LISTENER -p 1522
bash-4.3$
bash-4.3$ crsctl status resource -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DG_BACKUP.dg
               ONLINE  ONLINE       erpdb01
ora.DG_DATA01.dg
               ONLINE  ONLINE       erpdb01
ora.DG_FRA.dg
               ONLINE  ONLINE       erpdb01
ora.DG_OCR_VOT.dg
               ONLINE  ONLINE       erpdb01
ora.DG_REDO.dg
               ONLINE  ONLINE       erpdb01
ora.LISTENER.lsnr
               OFFLINE OFFLINE      erpdb01
ora.asm
               ONLINE  ONLINE       erpdb01                Started
ora.ons
               OFFLINE OFFLINE      erpdb01
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
      1        ONLINE  ONLINE       erpdb01
ora.diskmon
      1        OFFLINE OFFLINE
ora.evmd
      1        ONLINE  ONLINE       erpdb01
bash-4.3$

5. Stop the listener

bash-4.3$ lsnrctl stop

LSNRCTL for IBM/AIX RISC System/6000: Version 11.2.0.4.0 - Production on 15-JAN-2017 19:26:45

Copyright (c) 1991, 2013, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC1522)))
The command completed successfully
bash-4.3$

6. Start the listener

bash-4.3$ srvctl start listener
bash-4.3$
bash-4.3$ srvctl status listener
Listener LISTENER is enabled
Listener LISTENER is running on node(s): erpdb01