RAC异常小结

前几天在VMWare上安装了RAC，安装过程中以及今天在启动时都出了几个错误，简要记录下：

1.配置 Oracle 集群文件系统 (OCFS2)出错:

# ocfs2console
Configure Nodes --> Add --> 输入NODE名和IP --> OK --> Apply
出现如下错误：
    o2cb_ctl: Unable to access cluster service while creating node
        Could not add node node1

解决办法:先把/etc/ocfs2/下的cluster.conf文件删掉，重新配置就可以了。

2.DBCA创建数据库过程中进行到拷贝安装文件至远程节点时出错:

WARNING: Error while copying directory /u01/oracle/product/10.2.0/db_1 with exclude file list '/tmp/OraInstall2008-12-27_01-48-46PM/installExcludeFile.lst' to nodes 'rac1'. [PRKC-1002 : All the submitted commands did not execute successfully]
----------------------------------------------------------------------------------
rac1:
    /bin/tar: ./assistants/rconfig/sampleXMLs: time stamp 2008-12-27 13:25:06 is 768 s in the future
    /bin/tar: ./assistants/rconfig: time stamp 2008-12-27 13:24:34 is 736 s in the future
    /bin/tar: ./assistants/jlib: time stamp 2008-12-27 13:25:19 is 781 s in the future
    ……
----------------------------------------------------------------------------------
Refer to '/u01/oracle/oraInventory/logs/installActions2008-12-27_01-48-46PM.log' for details. You may fix the errors on the required remote nodes. Refer to the install guide for error recovery. Click 'Yes' if you want to proceed. Click 'No' to exit the install. Do you want to continue?

解决办法:这种错误是因为节点间时间不同步造成的。这种情况下可以选择"Yes"忽略此错误，继续往下。

3.启动RAC过程中出错:

rac2-> srvctl start nodeapps -n rac1
CRS-1028: Dependency analysis failed because of:
CRS-0223: Resource 'ora.rac1.gsd' has placement error.
CRS-1028: Dependency analysis failed because of:
CRS-0223: Resource 'ora.rac1.ons' has placement error.
CRS-1028: Dependency analysis failed because of:
CRS-0223: Resource 'ora.rac1.LISTENER_RAC1.lsnr' has placement error.

查看CRS状态:

rac2-> crs_stat -t
Name           Type           Target    State     Host       
------------------------------------------------------------
ora.....crm.cs application    OFFLINE   OFFLINE              
ora....db1.srv application    OFFLINE   OFFLINE              
ora.devdb.db   application    ONLINE    UNKNOWN   rac1       
ora....b1.inst application    ONLINE    OFFLINE              
ora....b2.inst application    ONLINE    OFFLINE              
ora....SM1.asm application    ONLINE    UNKNOWN   rac1       
ora....C1.lsnr application    ONLINE    UNKNOWN   rac1       
ora.rac1.gsd   application    ONLINE    UNKNOWN   rac1       
ora.rac1.ons   application    ONLINE    UNKNOWN   rac1       
ora.rac1.vip   application    ONLINE    ONLINE    rac1       
ora....SM2.asm application    ONLINE    UNKNOWN   rac2       
ora....C2.lsnr application    ONLINE    UNKNOWN   rac2       
ora.rac2.gsd   application    ONLINE    UNKNOWN   rac2       
ora.rac2.ons   application    ONLINE    UNKNOWN   rac2       
ora.rac2.vip   application    ONLINE    ONLINE    rac2

发现很多服务都不正常，尝试使用crs_start -all 启动:

rac2-> crs_start -all
CRS-1028: Dependency analysis failed because of:
'Resource in UNKNOWN state: ora.rac1.ASM1.asm'
CRS-1028: Dependency analysis failed because of:
'Resource in UNKNOWN state: ora.rac2.ASM2.asm'

Attempting to start `ora.devdb.crm.cs` on member `rac1`
Attempting to start `ora.devdb.crm.devdb1.srv` on member `rac1`
Start of `ora.devdb.crm.devdb1.srv` on member `rac1` failed.
Attempting to start `ora.devdb.crm.devdb1.srv` on member `rac2`
Start of `ora.devdb.crm.devdb1.srv` on member `rac2` failed.
CRS-1006: No more members to consider
Start of `ora.devdb.crm.cs` on member `rac1` failed.
Attempting to start `ora.devdb.crm.cs` on member `rac2`
Start of `ora.devdb.crm.cs` on member `rac2` failed.
CRS-1006: No more members to consider
CRS-0215: Could not start resource 'ora.devdb.crm.cs'.
CRS-0215: Could not start resource 'ora.devdb.crm.devdb1.srv'.
CRS-0223: Resource 'ora.devdb.devdb1.inst' has placement error.
CRS-0223: Resource 'ora.devdb.devdb2.inst' has placement error.

竟然不成功，于是尝试一个个重新启动:

rac2-> crs_start ora.rac1.gsd
CRS-1028: Dependency analysis failed because of:
'Resource in UNKNOWN state: ora.rac1.gsd'
CRS-0223: Resource 'ora.rac1.gsd' has placement error.

靠，还是不成功，先STOP:

rac2-> crs_stop ora.rac1.gsd
Attempting to stop `ora.rac1.gsd` on member `rac1`
Stop of `ora.rac1.gsd` on member `rac1` succeeded.
rac2-> crs_start ora.rac1.gsd
Attempting to start `ora.rac1.gsd` on member `rac1`
Start of `ora.rac1.gsd` on member `rac1` succeeded.

耶，成功，于是逐个先stop后start。不过发现"ora.rac1.ASM1.asm"服务竟然有大小写区分:

rac2-> srvctl start asm -n rac1
PRKS-1009 : Failed to start ASM instance "+ASM1" on node "rac1", [PRKS-1009 : Failed to start ASM instance "+ASM1" on node "rac1", [CRS-1028: Dependency analysis failed because of:
CRS-0223: Resource 'ora.rac1.ASM1.asm' has placement error.]]
  [PRKS-1009 : Failed to start ASM instance "+ASM1" on node "rac1", [CRS-1028: Dependency analysis failed because of:
CRS-0223: Resource 'ora.rac1.ASM1.asm' has placement error.]]
rac2-> crs_stop ora.rac1.asm1.asm
CRS-0210: Could not find resource 'ora.rac1.asm1.asm'.

rac2-> crs_start ora.rac1.asm1.asm
CRS-0210: Could not find resource 'ora.rac1.asm1.asm'.
rac2-> crs_start ora.rac1.ASM1.asm
CRS-1028: Dependency analysis failed because of:
'Resource in UNKNOWN state: ora.rac1.ASM1.asm'
CRS-0223: Resource 'ora.rac1.ASM1.asm' has placement error.
rac2-> crs_stop ora.rac1.ASM1.asm
Attempting to stop `ora.rac1.ASM1.asm` on member `rac1`
Stop of `ora.rac1.ASM1.asm` on member `rac1` succeeded.
CRS-1016: Resources depending on 'ora.rac1.ASM1.asm' are running
CRS-0223: Resource 'ora.rac1.ASM1.asm' has placement error.
rac2-> crs_start ora.rac1.ASM1.asm
Attempting to start `ora.rac1.ASM1.asm` on member `rac1`
Start of `ora.rac1.ASM1.asm` on member `rac1` succeeded.

最后检查crs状态,都正常:

rac2-> crs_stat -t
Name           Type           Target    State     Host       
------------------------------------------------------------
ora.....crm.cs application      ONLINE    OFFLINE              
ora....db1.srv application      ONLINE    OFFLINE              
ora.devdb.db   application      ONLINE    OFFLINE              
ora....b1.inst application      ONLINE    OFFLINE              
ora....b2.inst application      ONLINE    OFFLINE              
ora....SM1.asm application      ONLINE    ONLINE    rac1       
ora....C1.lsnr application      ONLINE    ONLINE    rac1       
ora.rac1.gsd   application      ONLINE    ONLINE    rac1       
ora.rac1.ons   application      ONLINE    ONLINE    rac1       
ora.rac1.vip   application      ONLINE    ONLINE    rac1       
ora....SM2.asm application      ONLINE    ONLINE    rac2       
ora....C2.lsnr application      ONLINE    ONLINE    rac2       
ora.rac2.gsd   application      ONLINE    ONLINE    rac2       
ora.rac2.ons   application      ONLINE    ONLINE    rac2       
ora.rac2.vip   application      ONLINE    ONLINE    rac2