aix+Oracle10g环境,同一主机上有多套数据库,发现有两套库的AWR快照不能定时生成;
现象是:手动执行exec dbms_workload_repository.create_snapshot;可以生成快照,或者kill mmon进程后,MMON进程启动时能生成一个快照,之后无法生成,查看mmnl进程trace,可以发现里面是有数据的;
参考MOS文档Troubleshooting: Missing Automatic Workload Repository (AWR) Snapshots and Other Collection Issues (文档 ID 1301503.1),按文档中的排查顺序也未查到原因; 通过查看进程的等待事件,发现等待事件是:wait for EMON to process ntfns。
最终查到是原因是OEM Agent,关闭OEM Agent后,MMON进程不再出现此等待事件,后续AWR快照生成恢复正常。
参考MOS文档:Slow Response OEM Agent Causes MMON Hung with "EMON to process ntnfs" (文档 ID 2308245.1) 其它AWR未生成排查思路参考以前写的
主要排查过程信息:
1.MMON进程的异常等待事件:
EVENT PROGRAM SID
------------------------------ ------------------------- ----------
EMON idle wait (EMN0) 1629
wait for EMON to process ntfns (MMON) 1645
2.关闭oem agent
ABC123:/u01/oracle/agent12c/agent_inst/bin> ./emctl stop agent
Oracle Enterprise Manager Cloud Control 12c Release 5
Copyright (c) 1996, 2015 Oracle Corporation. All rights reserved.
Stopping agent ... stopped.
-rw-r----- 1 oracle oinstall 995984 Jan 09 17:18 emdctlj.log
-rw-r----- 1 oracle oinstall 3326 Jan 09 17:18 emagent.nohup
ABC123:/u01/oracle/agent12c/agent_inst/sysman/log>
3.查看MMON进程是否有异常等待事件
SQL> select event,PROGRAM,sid from v$session where event like '%EMON%';
EVENT PROGRAM SID
------------------------------ ------------------------- ----------
EMON idle wait (EMN0) 1629
4.查看AWR快照生成情况
SQL> @?/rdbms/admin/awrrpt.sql
Current Instance
~~~~~~~~~~~~~~~~
DB Id DB Name Inst Num Instance
----------- ------------ -------- ------------
4112577184 test123 1 test123
………………
Enter value for num_days:
Listing all Completed Snapshots
Snap
Instance DB Name Snap Id Snap Started Level
------------ ------------ --------- ------------------ -----
test123 test123 58503 08 Jan 2018 12:40 1
58504 09 Jan 2018 16:21 1
58505 09 Jan 2018 17:18 1