数据库异常宕机导致的UNDO坏块修复-CFANZ编程社区

问题： 又一个夜半加班后的早上，刚睡着不久，客户来电有一个数据库环境异常宕机，之后无法启动。这时候顾不上睡意，抓紧连上环境，检查数据库ALERT日志，发现报错数据文件坏块，从ALERT日志中看是undotbs01.dbf上的坏块，报错ORA-01578及ORA-00600: internal error code, arguments: [2032], [12584268], [12584268], [8192]等；抓紧用RMAN命令校验了一下，只是UNDO数据文件上有一个坏块，算是比较幸运的。
问题处理： 这个环境有DG备机，有RMAN备份，但是备份在备份软件(备份软件厂商及磁带库)，恢复可能涉及连磁带库的参数会比较麻烦。因此决定从备机上备份UNDO数据文件并进行块恢复方式来修复坏块。
整个过程从7:41开始登陆系统到确定方案，用时4分钟，之后10分钟内备机做好备份并传输，1分钟内做好块恢复，7:58分数据再次校验完成，总体用时20分钟内，在8点钟工厂上班前完成了故障处理。

关于坏块的处理MOS文档及思路，也可以参考上一篇"一次ORA-01578 NOLOGGING 坏块的处理"。如下是本次相关操作日志及步骤：

1.数据库ALERT日志查看
Sun Dec 22 07:40:29 2019
Errors in file /oracle/app/oracle/diag/rdbms/aaadb/aaadb/trace/aaadb_ora_3777.trc  (incident=9619):
Sun Dec 22 07:40:29 2019   熷?(鏂囦欢宸 3, 鍧楀?1356)
Sun Dec 22 07:4枃浠?3: '/oracle/app/oracle/oradata/aaadb/undotbs01.dbf'
Sun Dec 22 07:40:50 2019潡鎜racle/diag/rdbms/aaadb/aaadb/trace/aaadb_ora_3777.trc  (incident=9619):
Errors in file /oracle/app/oracle/diag/rdbms/aaadb/aaadb/trace/aaadb_j000_9435.trc  (incident=8562):
ORA-01578: ORACLE data block corrupted (file # 3, block # 1356)
ORA-01110: data file 3: '/oracle/app/oracle/oradata/aaadb/undotbs01.dbf'
Sun Dec 22 07:41:07 2019
Sweep [inc][9619]: completed
Sweep [inc][8562]: completed
Sun Dec 22 07:41:43 2019
Errors in file /oracle/app/oracle/diag/rdbms/aaadb/aaadb/trace/aaadb_smon_3426.trc  (incident=9770):
ORA-00600: internal error code, arguments: [2032], [12584268], [12584268], [8192], [2], [255], [0], [767], [], [], [], []
Use ADRCI or Support Workbench to package the incident.

############
2.数据库坏块校验
RMAN> backup validate datafile 3;
Starting backup at 2019-12-22 07:45:55
………………
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:15
List of Datafiles
=================
File Status Marked Corrupt Empty Blocks Blocks Examined High SCN
---- ------ -------------- ------------ --------------- ----------
3    OK     1              639          37760           162273790 
  File Name: /oracle/app/oracle/oradata/aaadb/undotbs01.dbf
  Block Type Blocks Failing Blocks Processed
  ---------- -------------- ----------------
  Data       0              0               
  Index      0              0               
  Other      0              37121           
Finished backup at 2019-12-22 07:46:11

查看坏块信息
SQL> select * from v$database_block_corruption ;
     FILE#     BLOCK#     BLOCKS CORRUPTION_CHANGE# CORRUPTIO
---------- ---------- ---------- ------------------ ---------
         3       1356          1                  0 FRACTURED

##############
3.备份及恢复坏块命令
RMAN> backup datafile 3 format '/home/oracle/undotbs.bak';
RMAN> catalog start with '/home/oracle/undotbs.bak';
RMAN>  blockrecover datafile 3 block 1356;

##############
4.数据库OPEN及坏块校验
SQL> alter database open;
Database altered.
SQL> alter system checkpoint;
System altered.
SQL> alter system switch logfile;
System altered.
此时ALERT日志无异常信息。

[oracle@aaa-APPDATA ~]$ rman target /
Recovery Manager: Release 11.2.0.4.0 - Production on Sun Dec 22 07:58:20 2019
Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.
connected to target database: aaaDB (DBID=1758763282)

RMAN>  backup validate check logical database;
…………………………

Recovery Manager complete.
[oracle@aaa-APPDATA ~]$ sqlplus  / as sysdba
SQL*Plus: Release 11.2.0.4.0 Production on Sun Dec 22 08:01:14 2019
Copyright (c) 1982, 2013, Oracle.  All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> select * from v$database_block_corruption ;

no rows selected