OS 환경 : Oracle Linux 8.4 (64bit)
DB 환경 : Oracle Database 19.27.0.0
방법 : ORA-30012: undo tablespace 'UNDOTBS3' does not exist or of wrong type
2node rac에서 3번노드를 delnode 한 뒤 다시 add node 함
참고 : 오라클 19c RAC 노드 제거 방법(silent) ( https://positivemh.tistory.com/1244 )
참고 : 오라클 19c RAC 노드 추가 방법(silent) ( https://positivemh.tistory.com/1243 )
이때 발생한 에러들을 게시글로 정리함
내용이 많아 글을 나눠서 업로드함
메인 게시글 : 오라클 19c RAC 제거한 노드 다시 추가 시 발생한 에러들 ( https://positivemh.tistory.com/1312 )
에러1. ORA-01613: instance ORA19DB3 (thread 3) only has 0 logs - at least 2 logs required to enable. ( https://positivemh.tistory.com/1308 )
에러2. ORA-30012: undo tablespace 'UNDOTBS3' does not exist or of wrong type ( https://positivemh.tistory.com/1309 )
에러3. [INS-43042] The cluster nodes [ora19rac3] specified for addnode is already part of a cluster. ( https://positivemh.tistory.com/1310 )
에러4. PRCR-1079 : Failed to start resource ora.cvu ( https://positivemh.tistory.com/1311 )
이전 게시글에서 redo 추가 및 thread 3을 enable로 변경한 뒤 다시 db를 기동시도했지만 실패하는 상황임
참고 : ORA-01613: instance ORA19DB3 (thread 3) only has 0 logs - at least 2 logs required to enable. ( https://positivemh.tistory.com/1308 )
다시 3번 인스턴스 기동 시도
|
1
2
3
4
5
6
7
8
9
10
11
12
|
$ srvctl start instance -d ora19db -i ORA19DB3
PRCR-1013 : Failed to start resource ora.ora19db.db
PRCR-1064 : Failed to start resource ora.ora19db.db on node ora19rac3
CRS-5017: The resource action "ora.ora19db.db start" encountered the following error:
ORA-00603: ORACLE server session terminated by fatal error
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-30012: undo tablespace 'UNDOTBS3' does not exist or of wrong type
Process ID: 73679
Session ID: 237 Serial number: 14511
. For details refer to "(:CLSN00107:)" in "/oracle/app/oracle/diag/crs/ora19rac3/crs/trace/crsd_oraagent_oracle.trc".
CRS-2674: Start of 'ora.ora19db.db' on 'ora19rac3' failed
|
thread 문제는 해결이 되었지만 UNDOTBS3를 못찾는듯함
undo 확인
|
1
2
3
4
5
6
7
8
9
10
11
12
13
|
SQL>
set lines 200 pages 1000
col tablespace_name for a10
col file_name for a50
select tablespace_name, file_name, bytes/1024/1024 mb
from dba_data_files
where tablespace_name like '%UNDO%'
order by 1;
TABLESPACE FILE_NAME MB
---------- -------------------------------------------------- ----------
UNDOTBS1 +DATANEW/ORA19DB/DATAFILE/undotbs1.276.1201188463 1024
UNDOTBS2 +DATANEW/ORA19DB/DATAFILE/undotbs2.278.1201188471 1024
|
3번 언두가 존재하지 않음
undo 파라미터 확인
|
1
2
3
4
5
6
7
8
9
10
11
|
SQL> show spparameter undo
SID NAME TYPE VALUE
-------- ----------------------------- ----------- ----------------------------
* temp_undo_enabled boolean
* undo_management string
* undo_retention integer
* undo_tablespace string UNDOTBS1
ORA19DB1 undo_tablespace string UNDOTBS1
ORA19DB2 undo_tablespace string UNDOTBS2
ORA19DB3 undo_tablespace string UNDOTBS3
|
3번 인스턴스에 UNDOTBS3이 필요함
UNDOTBS3 생성
|
1
2
3
|
SQL> create tablespace undotbs3 datafile '+DATANEW' size 1g;
Tablespace created.
|
생성됨
undo 재확인
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
|
SQL>
set lines 200 pages 1000
col tablespace_name for a10
col file_name for a50
select tablespace_name, file_name, bytes/1024/1024 mb
from dba_data_files
where tablespace_name like '%UNDO%'
order by 1;
TABLESPACE FILE_NAME MB
---------- -------------------------------------------------- ----------
UNDOTBS1 +DATANEW/ORA19DB/DATAFILE/undotbs1.276.1201188463 1024
UNDOTBS2 +DATANEW/ORA19DB/DATAFILE/undotbs2.278.1201188471 1024
UNDOTBS3 +DATANEW/ORA19DB/DATAFILE/undotbs3.343.1218471741 1024
|
생성됨
다시 3번 인스턴스 기동 시도
|
1
2
3
4
5
6
7
8
9
10
11
12
|
$ srvctl start instance -d ora19db -i ORA19DB3
PRCR-1013 : Failed to start resource ora.ora19db.db
PRCR-1064 : Failed to start resource ora.ora19db.db on node ora19rac3
CRS-5017: The resource action "ora.ora19db.db start" encountered the following error:
ORA-00603: ORACLE server session terminated by fatal error
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-30012: undo tablespace 'UNDOTBS3' does not exist or of wrong type
Process ID: 73679
Session ID: 237 Serial number: 14511
. For details refer to "(:CLSN00107:)" in "/oracle/app/oracle/diag/crs/ora19rac3/crs/trace/crsd_oraagent_oracle.trc".
CRS-2674: Start of 'ora.ora19db.db' on 'ora19rac3' failed
|
여전히 UNDOTBS3를 못찾는듯함
원인은 undo를 일반 ts처럼 생성해서 안된것
테이블스페이스 속성 확인
|
1
2
3
4
5
6
7
8
9
10
|
SQL>
select tablespace_name, contents, status
from dba_tablespaces
where tablespace_name like '%UNDO%';
TABLESPACE CONTENTS STATUS
---------- --------------------- ---------
UNDOTBS1 UNDO ONLINE
UNDOTBS2 UNDO ONLINE
UNDOTBS3 PERMANENT ONLINE
|
3번을 일반 테이블스페이스로 만듬, 언두 형식으로 만들어야함
삭제후 재생성
|
1
2
3
4
5
6
7
8
9
|
#잘못된 UNDOTBS3 삭제
SQL> drop tablespace undotbs3 including contents and datafiles;
Tablespace dropped.
#정상 UNDOTBS3 생성
SQL> create UNDO tablespace undotbs3 datafile '+DATANEW' size 1g;
Tablespace created.
|
생성됨
테이블스페이스 속성 재확인
|
1
2
3
4
5
6
7
8
9
10
|
SQL>
select tablespace_name, contents, status
from dba_tablespaces
where tablespace_name like '%UNDO%';
TABLESPACE CONTENTS STATUS
---------- --------------------- ---------
UNDOTBS1 UNDO ONLINE
UNDOTBS2 UNDO ONLINE
UNDOTBS3 UNDO ONLINE
|
undo로 생성됨
다시 3번 인스턴스 기동 시도
|
1
2
3
4
5
6
7
8
|
$ srvctl start instance -d ora19db -i ORA19DB3
SQL> select instance_name, status from gv$instance;
INSTANCE_NAME STATUS
---------------- ------------
ORA19DB1 OPEN
ORA19DB2 OPEN
ORA19DB3 OPEN
|
정상적으로 기동됨
grid 상태 확인
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
|
$ crsctl stat res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
ONLINE ONLINE ora19rac1 STABLE
ONLINE ONLINE ora19rac2 STABLE
ONLINE ONLINE ora19rac3 STABLE
ora.chad
ONLINE ONLINE ora19rac1 STABLE
OFFLINE OFFLINE ora19rac2 STABLE
ONLINE ONLINE ora19rac3 STABLE
ora.net1.network
ONLINE ONLINE ora19rac1 STABLE
ONLINE ONLINE ora19rac2 STABLE
ONLINE ONLINE ora19rac3 STABLE
ora.ons
ONLINE ONLINE ora19rac1 STABLE
ONLINE ONLINE ora19rac2 STABLE
ONLINE ONLINE ora19rac3 STABLE
ora.proxy_advm
OFFLINE OFFLINE ora19rac1 STABLE
OFFLINE OFFLINE ora19rac2 STABLE
OFFLINE OFFLINE ora19rac3 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
1 ONLINE ONLINE ora19rac1 STABLE
2 ONLINE ONLINE ora19rac2 STABLE
3 ONLINE ONLINE ora19rac3 STABLE
ora.DATANEW.dg(ora.asmgroup)
1 ONLINE ONLINE ora19rac1 STABLE
2 ONLINE ONLINE ora19rac2 STABLE
3 ONLINE ONLINE ora19rac3 STABLE
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE ora19rac2 STABLE
ora.OCRVOTE.dg(ora.asmgroup)
1 ONLINE ONLINE ora19rac1 STABLE
2 ONLINE ONLINE ora19rac2 STABLE
3 ONLINE ONLINE ora19rac3 STABLE
ora.RECONEW.dg(ora.asmgroup)
1 ONLINE ONLINE ora19rac1 STABLE
2 ONLINE ONLINE ora19rac2 STABLE
3 ONLINE ONLINE ora19rac3 STABLE
ora.asm(ora.asmgroup)
1 ONLINE ONLINE ora19rac1 Started,STABLE
2 ONLINE ONLINE ora19rac2 Started,STABLE
3 ONLINE ONLINE ora19rac3 Started,STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
1 ONLINE ONLINE ora19rac1 STABLE
2 ONLINE ONLINE ora19rac2 STABLE
3 ONLINE ONLINE ora19rac3 STABLE
ora.cvu
1 ONLINE ONLINE ora19rac2 STABLE
ora.ora19db.db
1 ONLINE ONLINE ora19rac1 Open,HOME=/oracle/ap
p/oracle/product/19c
,STABLE
2 ONLINE ONLINE ora19rac2 Open,HOME=/oracle/ap
p/oracle/product/19c
,STABLE
3 ONLINE ONLINE ora19rac3 Open,HOME=/oracle/ap
p/oracle/product/19c
,STABLE
ora.ora19rac1.vip
1 ONLINE ONLINE ora19rac1 STABLE
ora.ora19rac2.vip
1 ONLINE ONLINE ora19rac2 STABLE
ora.ora19rac3.vip
1 ONLINE ONLINE ora19rac3 STABLE
ora.qosmserver
1 ONLINE ONLINE ora19rac2 STABLE
ora.scan1.vip
1 ONLINE ONLINE ora19rac2 STABLE
--------------------------------------------------------------------------------
|
정상임
참조 :
https://positivemh.tistory.com/284
https://positivemh.tistory.com/1244
Database fails to start on due to ORA-01618: redo thread 2 is not enabled - cannot mount (Doc ID 1677362.1)
Top Issues When Adding Node for Grid Infrastructure via GridSetup.sh (Doc ID 2955583.1)
https://pat98.tistory.com/731
https://docs.oracle.com/en/database/oracle/oracle-database/19/cwadd/cluster-verification-utility-reference.html#GUID-B445A858-9F00-4423-990E-109545AC11C3
https://dbmentors.blogspot.com/2013/11/clusterware-resource-oracvu.html
Clusterware resource ora.cvu FAQ (Doc ID 1524235.1)
Cluster Verification Utility (CLUVFY) FAQ (Doc ID 316817.1)
"Roottfa.sh: Not Found" after executing root.sh script. (Doc ID 2960836.1)
root.sh Hanging During Relink on AIX (Doc ID 3091869.1)
Root.sh Failed During Rac Installation Due To Antivirus (Doc ID 3058530.1)
https://dbacentrals.blogspot.com/2017/08/srvm1337crs-10051-cvu-found-following.html