OS환경 : Oracle Linux 7.6 (64bit)
DB 환경 : Oracle Database 19.6.0.0 RAC
방법 : 오라클 19c RAC network interface 변경 후 grid 기동불가 시 해결방법
network interface 이름이 변경되었을 때는 oifcfg 명령의 setif delif 로 grid 에 저장된 network interface 정보를 변경시켜주어야함
하지만 db, grid를 종료한 뒤 network interface 이름이 변경되었을 때는 grid 프로세스가 정상적으로 올라오지 못하기 때문에 oifcfg 명령을 사용하지 못함
이 경우 network interface 를 기존 값으로 변경하면 되지만 상황에 따라 network interface는 변경하지 못하는 경우가 발생할 수 있음
이런 상황이 발생했을 경우 해결하는 두가지 방법이 존재함
방법1. exclusive mode로 grid 기동 후 변경(grid가 기동중일때만 변경가능)
방법2. gpnptool 이용 변경(grid가 기동중이지 않아도 변경가능)
이번에는 방법1. exclusive mode로 grid 기동 후 변경하는 방법에 대해 설명함
시나리오
방법1. exclusive mode로 grid 기동 후 변경
기존 Public IP의 네트워크 인터페이스 ens32를 db, grid 종료 후 ens36으로 변경 후 grid를 기동해보고
기동되지 않을 때 해결방법을 설명함
기존 아이피 대역
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
|
# vi /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
##Public
192.168.200.10 oel19db1
192.168.200.20 oel19db2
##Private
10.10.200.10 oel19db1-priv
10.10.200.20 oel19db2-priv
##Virtual
192.168.200.100 oel19db1-vip
192.168.200.200 oel19db2-vip
##SCAN
192.168.200.150 oel19db-scan
|
기존 네트워크 인터페이스 확인
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
|
1번 노드
# ifconfig
ens32: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.200.10 netmask 255.255.255.0 broadcast 192.168.200.255
inet6 fe80::20c:29ff:fed3:6d16 prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:d3:6d:16 txqueuelen 1000 (Ethernet)
RX packets 6204 bytes 1748209 (1.6 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 6655 bytes 2162880 (2.0 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens32:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.200.100 netmask 255.255.255.0 broadcast 192.168.200.255
ether 00:0c:29:d3:6d:16 txqueuelen 1000 (Ethernet)
ens32:2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.200.150 netmask 255.255.255.0 broadcast 192.168.200.255
ether 00:0c:29:d3:6d:16 txqueuelen 1000 (Ethernet)
ens35: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.10.200.10 netmask 255.255.255.0 broadcast 10.10.200.255
inet6 fe80::20c:29ff:fed3:6d20 prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:d3:6d:20 txqueuelen 1000 (Ethernet)
RX packets 209889 bytes 235403103 (224.4 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 149915 bytes 140239088 (133.7 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
2번 노드
# ifconfig
ens32: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.200.20 netmask 255.255.255.0 broadcast 192.168.200.255
inet6 fe80::250:56ff:fe3d:54c6 prefixlen 64 scopeid 0x20<link>
ether 00:50:56:3d:54:c6 txqueuelen 1000 (Ethernet)
RX packets 6625 bytes 1746422 (1.6 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 5962 bytes 1977799 (1.8 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens32:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.200.200 netmask 255.255.255.0 broadcast 192.168.200.255
ether 00:50:56:3d:54:c6 txqueuelen 1000 (Ethernet)
ens35: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.10.200.20 netmask 255.255.255.0 broadcast 10.10.200.255
inet6 fe80::250:56ff:fe39:a3bc prefixlen 64 scopeid 0x20<link>
ether 00:50:56:39:a3:bc txqueuelen 1000 (Ethernet)
RX packets 164161 bytes 152180472 (145.1 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 235133 bytes 267400380 (255.0 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
|
현재 Public IP의 네트워크 인터페이스가 ens32임
Vip 도 Public IP의 대역을 함께 사용하기 때문에 ens32:1로 표시되어있음
Scan IP 도 Public IP의 대역을 함께 사용하기 때문에 ens32:2로 표시되어있음
기존 grid 상태 확인
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
|
# crsctl stat res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
ONLINE ONLINE oel19db1 STABLE
ONLINE ONLINE oel19db2 STABLE
ora.chad
ONLINE ONLINE oel19db1 STABLE
ONLINE ONLINE oel19db2 STABLE
ora.net1.network
ONLINE ONLINE oel19db1 STABLE
ONLINE ONLINE oel19db2 STABLE
ora.ons
ONLINE ONLINE oel19db1 STABLE
ONLINE ONLINE oel19db2 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNEWLSNR_ASM.lsnr(ora.asmgroup)
1 ONLINE ONLINE oel19db1 STABLE
2 ONLINE ONLINE oel19db2 STABLE
3 ONLINE OFFLINE STABLE
ora.DATA.dg(ora.asmgroup)
1 ONLINE ONLINE oel19db1 STABLE
2 ONLINE ONLINE oel19db2 STABLE
3 OFFLINE OFFLINE STABLE
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE oel19db1 STABLE
ora.OCR_VOTE.dg(ora.asmgroup)
1 ONLINE ONLINE oel19db1 STABLE
2 ONLINE ONLINE oel19db2 STABLE
3 OFFLINE OFFLINE STABLE
ora.asm(ora.asmgroup)
1 ONLINE ONLINE oel19db1 Started,STABLE
2 ONLINE ONLINE oel19db2 Started,STABLE
3 OFFLINE OFFLINE STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
1 ONLINE OFFLINE STABLE
2 ONLINE OFFLINE STABLE
3 OFFLINE OFFLINE STABLE
ora.cvu
1 ONLINE ONLINE oel19db1 STABLE
ora.oel19db1.vip
1 ONLINE ONLINE oel19db1 STABLE
ora.oel19db2.vip
1 ONLINE ONLINE oel19db2 STABLE
ora.oradb.db
1 ONLINE ONLINE oel19db1 Open,HOME=/ORA19/app
/oracle/product/19.3
.0/db_1,STABLE
2 ONLINE ONLINE oel19db2 Open,HOME=/ORA19/app
/oracle/product/19.3
.0/db_1,STABLE
ora.qosmserver
1 ONLINE ONLINE oel19db1 STABLE
ora.scan1.vip
1 ONLINE ONLINE oel19db1 STABLE
--------------------------------------------------------------------------------
|
모두 정상
grid 종료(1번, 2번 노드 모두)
|
1
|
# crsctl stop crs
|
grid 자동 기동 설정 해제(1번, 2번 노드 모두)
|
1
2
|
# crsctl disable crs
CRS-4621: Oracle High Availability Services autostart is disabled.
|
네트워크 인터페이스 설정
네트워크 인터페이스 추가를 위해 OS 종료(1번, 2번 노드 모두)
|
1
|
# poweroff
|
vmware에서 네트워크 Adapter 추가(1번, 2번 노드 모두)
Edit virtual machine settings 선택
Network Adapter 선택
NAT 선택
1번 노드, 2번 노드 동일하게 1개씩 추가됨
OS 기동 후 확인
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
|
1번 노드
# ifconfig
ens32: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.200.10 netmask 255.255.255.0 broadcast 192.168.200.255
inet6 fe80::20c:29ff:fed3:6d16 prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:d3:6d:16 txqueuelen 1000 (Ethernet)
RX packets 35 bytes 4071 (3.9 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 41 bytes 6233 (6.0 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens35: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.10.200.10 netmask 255.255.255.0 broadcast 10.10.200.255
inet6 fe80::20c:29ff:fed3:6d20 prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:d3:6d:20 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 10 bytes 766 (766.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens36: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 00:0c:29:d3:6d:2a txqueuelen 1000 (Ethernet)
RX packets 5 bytes 300 (300.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
2번 노드
# ifconfig
ens32: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.200.20 netmask 255.255.255.0 broadcast 192.168.200.255
inet6 fe80::250:56ff:fe3d:54c6 prefixlen 64 scopeid 0x20<link>
ether 00:50:56:3d:54:c6 txqueuelen 1000 (Ethernet)
RX packets 36 bytes 4105 (4.0 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 44 bytes 6427 (6.2 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens35: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.10.200.20 netmask 255.255.255.0 broadcast 10.10.200.255
inet6 fe80::250:56ff:fe39:a3bc prefixlen 64 scopeid 0x20<link>
ether 00:50:56:39:a3:bc txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 14 bytes 1046 (1.0 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens36: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 00:0c:29:62:6e:b6 txqueuelen 1000 (Ethernet)
RX packets 5 bytes 300 (300.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
|
정상적으로 ens36번이 생김
ens32의 ip정보를 ens36번으로 수정(/etc/udev/rules.d/)(1번, 2번 노드 모두)
MAC Address는 ifconfig에 나온 ens36 정보(ether 부분)를 넣음
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
|
1번 노드
# cd /etc/udev/rules.d/
# vi 70-persistent-net.rules
기존
SUBSYTEM=="net", ACTION=="add" , ATTR{address}=="00:0C:29:D3:6D:16" , NAME=={ens32}
SUBSYTEM=="net", ACTION=="add" , ATTR{address}=="00:0C:29:D3:6D:20" , NAME=={ens35}
변경
SUBSYTEM=="net", ACTION=="add" , ATTR{address}=="00:0C:29:D3:6D:20" , NAME=={ens35}
SUBSYTEM=="net", ACTION=="add" , ATTR{address}=="00:0c:29:d3:6d:2a" , NAME=={ens36}
2번 노드
# cd /etc/udev/rules.d/
# vi 70-persistent-net.rules
기존
SUBSYTEM=="net", ACTION=="add" , ATTR{address}=="00:0C:29:D3:6D:16" , NAME=={ens32}
SUBSYTEM=="net", ACTION=="add" , ATTR{address}=="00:0C:29:D3:6D:20" , NAME=={ens35}
변경
SUBSYTEM=="net", ACTION=="add" , ATTR{address}=="00:0C:29:D3:6D:20" , NAME=={ens35}
SUBSYTEM=="net", ACTION=="add" , ATTR{address}=="00:0c:29:62:6e:b6" , NAME=={ens36}
|
ens36용 uuid 생성(1번, 2번 노드 모두)
|
1
2
3
4
5
6
7
|
1번 노드
# uuidgen
b49aac4e-24f9-4fe7-bdd5-b9c6973f38f0
2번 노드
# uuidgen
094d500f-f61b-4328-b618-3755b4571ec0
|
ens32의 ip정보를 ens36번으로 수정(/etc/sysconfig/network-scripts/)(1번, 2번 노드 모두)
위에서 생성한 uuid 값 넣어주고 NAME, DEVICE부분을 ens36으로 변경
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
|
1번 노드
# cd /etc/sysconfig/network-scripts/
# cp ifcfg-ens32 ifcfg-ens36
# vi ifcfg-ens36
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=no
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ens36
DEVICE=ens36
ONBOOT=yes
IPADDR=192.168.200.10
PREFIX=24
GATEWAY=192.168.200.2
DNS1=168.126.63.1
IPV6_PRIVACY=no
UUID=b49aac4e-24f9-4fe7-bdd5-b9c6973f38f0
2번 노드
# cd /etc/sysconfig/network-scripts/
# cp ifcfg-ens32 ifcfg-ens36
# vi ifcfg-ens36
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=no
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ens36
DEVICE=ens36
ONBOOT=yes
IPADDR=192.168.200.20
PREFIX=24
GATEWAY=192.168.200.2
DNS1=168.126.63.1
IPV6_PRIVACY=no
UUID=094d500f-f61b-4328-b618-3755b4571ec0
|
ens32의 ip 정보 삭제(1번, 2번 노드 모두)
IPADDR, GATEWAY삭제 후 ONBOOT=no 설정
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
|
# cd /etc/sysconfig/network-scripts/
# vi ifcfg-ens32
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=no
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ens32
DEVICE=ens32
ONBOOT=no
PREFIX=24
DNS1=168.126.63.1
IPV6_PRIVACY=no
UUID=152beb06-47c5-c5e8-95a9-385590654382
|
설정 변경후 network restart(1번, 2번 노드 모두)
|
1
|
# systemctl restart network
|
ifconfig로 네트워크 정보 확인
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
|
1번 노드
# ifconfig
ens32: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 00:0c:29:d3:6d:16 txqueuelen 1000 (Ethernet)
RX packets 1231 bytes 103637 (101.2 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 786 bytes 112765 (110.1 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens35: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.10.200.10 netmask 255.255.255.0 broadcast 10.10.200.255
inet6 fe80::20c:29ff:fed3:6d20 prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:d3:6d:20 txqueuelen 1000 (Ethernet)
RX packets 19 bytes 2256 (2.2 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 28 bytes 2092 (2.0 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens36: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.200.10 netmask 255.255.255.0 broadcast 192.168.200.255
inet6 fe80::20c:29ff:fed3:6d2a prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:d3:6d:2a txqueuelen 1000 (Ethernet)
RX packets 45 bytes 4432 (4.3 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 25 bytes 2588 (2.5 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
2번 노드
# ifconfig
ens32: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 00:50:56:3d:54:c6 txqueuelen 1000 (Ethernet)
RX packets 625 bytes 52519 (51.2 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 456 bytes 64333 (62.8 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens35: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.10.200.20 netmask 255.255.255.0 broadcast 10.10.200.255
inet6 fe80::250:56ff:fe39:a3bc prefixlen 64 scopeid 0x20<link>
ether 00:50:56:39:a3:bc txqueuelen 1000 (Ethernet)
RX packets 22 bytes 2532 (2.4 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 24 bytes 1812 (1.7 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens36: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.200.20 netmask 255.255.255.0 broadcast 192.168.200.255
inet6 fe80::20c:29ff:fe62:6eb6 prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:62:6e:b6 txqueuelen 1000 (Ethernet)
RX packets 50 bytes 4626 (4.5 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 21 bytes 2096 (2.0 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
|
1번, 2번 노드 모두 ens32는 ip가 없어지고 ens36에 Public IP 가 들어감
grid 기동 시도
이상태로 grid 기동 시도(1번, 2번 노드 모두)
|
1
2
|
# crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
|
grid 상태 확인(1번, 2번 노드 모두)
|
1
2
3
|
# crsctl stat res -t
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Status failed, or completed with errors.
|
grid가 제대로 올라가지 않아 상태가 나오지 않음
crs alert log 확인
|
1
2
3
4
5
6
7
8
9
10
11
12
|
# su - oracle
$ cd $ORACLE_BASE/diag/crs/oel19db1/crs/trace
$ vi alert.log
2021-08-05 06:07:26.605 [GIPCD(14256)]CRS-42216: No interfaces are configured on the local node for interface definition
ens35(:.*)?:10.10.10.0: available interface definitions are [ens32(:.*)?:192.168.200.0][ens32:1(:.*)?:192.168.200.0]
[ens35(:.*)?:10.10.200.0][ens35:1(:.*)?:169.254.0.0][ens32(:.*)?:[fe80:0:0:0:0:0:0:0]][ens35(:.*)?:[fe80:0:0:0:0:0:0:0]].
2021-08-05 06:07:26.607 [GIPCD(14256)]CRS-42216: No interfaces are configured on the local node for interface definition
ens35(:.*)?:10.10.10.0: available interface definitions are [ens32(:.*)?:192.168.200.0][ens32:1(:.*)?:192.168.200.0]
[ens35(:.*)?:10.10.200.0][ens35:1(:.*)?:169.254.0.0][ens32(:.*)?:[fe80:0:0:0:0:0:0:0]][ens35(:.*)?:[fe80:0:0:0:0:0:0:0]].
2021-08-05 06:07:26.607 [GIPCD(14256)]CRS-42216: No interfaces are configured on the local node for interface definition
ens35(:.*)?:10.10.10.0: available interface definitions are [ens32(:.*)?:192.168.200.0][ens32:1(:.*)?:192.168.200.0]
[ens35(:.*)?:10.10.200.0][ens35:1(:.*)?:169.254.0.0][ens32(:.*)?:[fe80:0:0:0:0:0:0:0]][ens35(:.*)?:[fe80:0:0:0:0:0:0:0]].
|
network interface 관련 메세지가 발생중임
oifcfg getif 명령 확인
|
1
2
|
# oifcfg getif
PRIF-10: failed to initialize the cluster registry
|
grid가 비정상이어서 oifcfg 명령이 제대로 되지 않음
비정상적으로 올라간 grid 강제종료(1번, 2번 노드 모두)
|
1
|
# crsctl stop crs -f
|
gpnp 프로파일 백업($GRID_HOME/gpnp/{hostname}/profiles/peer)
|
1
2
3
4
5
6
7
8
9
10
|
# cd $GRID_HOME/gpnp/oel19db1/profiles/peer
# cp -p profile.xml profile.xmlbak
# ls -al
total 16
drwxr-x--- 2 oracle dba 90 Jul 28 04:04 .
drwxr-x--- 3 oracle dba 18 Mar 4 2020 ..
-rw-r--r-- 1 oracle dba 1863 Jul 27 23:27 profile.old
-rw-r--r-- 1 oracle dba 1873 Mar 4 2020 profile_orig.xml
-rw-r--r-- 1 oracle dba 1936 Jul 27 23:29 profile.xml
-rw-r--r-- 1 oracle dba 1936 Jul 27 23:29 profile.xmlbak
|
* grid 11.2 부터 Private IP는 OCR 뿐만 아니라 gpnp 프로필에도 저장됨
Private 네트워크를 사용할 수 없거나 정의가 잘못된 경우 CRSD 프로세스가 시작되지 않고 이후에 OCR을 변경할 수 없음
따라서 Private 네트워크 구성을 수정할 때 주의가 필요함
올바른 순서로 변경을 수행하는 것이 중요하고 gpnp 프로필의 수동 변경은 지원되지 않음(gpnptool을 이용해야함)
grid exclusive mode로 기동(1번 노드만)
|
1
|
# crsctl start crs -excl
|
grid 상태 확인(1번 노드만)
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
|
# crsctl stat res -t
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Status failed, or completed with errors.
# crsctl stat res -t -init
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.asm
1 ONLINE ONLINE oel19db1 Started,STABLE
ora.cluster_interconnect.haip
1 ONLINE ONLINE oel19db1 STABLE
ora.crf
1 ONLINE ONLINE oel19db1 STABLE
ora.crsd
1 ONLINE INTERMEDIATE oel19db1 EXCLUSIVE,STABLE
ora.cssd
1 ONLINE ONLINE oel19db1 STABLE
ora.cssdmonitor
1 ONLINE ONLINE oel19db1 STABLE
ora.ctssd
1 ONLINE ONLINE oel19db1 ACTIVE:0,STABLE
ora.diskmon
1 OFFLINE OFFLINE STABLE
ora.evmd
1 ONLINE ONLINE oel19db1 STABLE
ora.gipcd
1 ONLINE ONLINE oel19db1 STABLE
ora.gpnpd
1 ONLINE ONLINE oel19db1 STABLE
ora.mdnsd
1 ONLINE ONLINE oel19db1 STABLE
ora.storage
1 ONLINE ONLINE oel19db1 STABLE
--------------------------------------------------------------------------------
|
crsctl stat res -t 로는 결과가 나오지 않지만
crsctl stat res -t -init 명령시 crsd가 exclusive mode 로 표시됨
oifcfg getif 명령 확인(1번 노드만)
|
1
2
3
|
# oifcfg getif
ens32 192.168.200.0 global public
ens35 10.10.200.0 global cluster_interconnect,asm
|
명령어가 정상적으로 사용됨
oifcfg setif, delif 명령으로 ens36 추가 및 ens32 제거(1번 노드만)
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
|
ens36 추가
# oifcfg setif -global ens36/192.168.200.0:public
확인
# oifcfg getif
ens32 192.168.200.0 global public
ens35 10.10.200.0 global cluster_interconnect,asm
ens36 192.168.200.0 global public
ens32 제거
# oifcfg delif -global ens32
# oifcfg getif
ens35 10.10.200.0 global cluster_interconnect,asm
ens36 192.168.200.0 global public
|
grid 종료 후 재기동(1번 노드만)
|
1
2
|
# crsctl stop crs
# crsctl start crs
|
grid 상태 확인
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
|
# crsctl stat res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
ONLINE OFFLINE oel19db1 STABLE
ora.chad
ONLINE OFFLINE oel19db1 STABLE
ora.net1.network
ONLINE OFFLINE oel19db1 STABLE
ora.ons
ONLINE OFFLINE oel19db1 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNEWLSNR_ASM.lsnr(ora.asmgroup)
1 ONLINE OFFLINE STABLE
2 ONLINE OFFLINE STABLE
3 ONLINE OFFLINE STABLE
ora.DATA.dg(ora.asmgroup)
1 ONLINE ONLINE oel19db1 STABLE
2 OFFLINE OFFLINE STABLE
3 OFFLINE OFFLINE STABLE
ora.LISTENER_SCAN1.lsnr
1 ONLINE OFFLINE STABLE
ora.OCR_VOTE.dg(ora.asmgroup)
1 ONLINE ONLINE oel19db1 STABLE
2 OFFLINE OFFLINE STABLE
3 OFFLINE OFFLINE STABLE
ora.asm(ora.asmgroup)
1 ONLINE INTERMEDIATE oel19db1 Started,STABLE
2 ONLINE OFFLINE STABLE
3 OFFLINE OFFLINE STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
1 ONLINE OFFLINE STABLE
2 ONLINE OFFLINE STABLE
3 OFFLINE OFFLINE STABLE
ora.cvu
1 ONLINE OFFLINE STABLE
ora.oel19db1.vip
1 ONLINE OFFLINE STABLE
ora.oel19db2.vip
1 ONLINE OFFLINE STABLE
ora.oradb.db
1 ONLINE OFFLINE Instance Shutdown,ST
ABLE
2 ONLINE OFFLINE STABLE
ora.qosmserver
1 ONLINE OFFLINE STABLE
ora.scan1.vip
1 ONLINE OFFLINE STABLE
--------------------------------------------------------------------------------
|
몇가지 리소스가 ONLINE 이지만 VIP가 OFFLINE임
vip nodeapps 정보 확인
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
|
# srvctl config nodeapps -a
Network 1 exists
Subnet IPv4: 192.168.200.0/255.255.255.0/ens32, static
Subnet IPv6:
Ping Targets:
Network is enabled
Network is individually enabled on nodes:
Network is individually disabled on nodes:
VIP exists: network number 1, hosting node oel19db1
VIP Name: oel19db1-vip
VIP IPv4 Address: 192.168.200.100
VIP IPv6 Address:
VIP is enabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:
VIP exists: network number 1, hosting node oel19db2
VIP Name: oel19db2-vip
VIP IPv4 Address: 192.168.200.200
VIP IPv6 Address:
VIP is enabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:
|
vip는 아직 ens32정보로 저장되어 있음
vip 정보 ens36으로 업데이트 후 확인
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
|
vip 정보 변경
# srvctl modify nodeapps -n oel19db1 -A oel19db1-vip/255.255.255.0/ens36
# srvctl modify nodeapps -n oel19db2 -A oel19db2-vip/255.255.255.0/ens36
확인
# srvctl config nodeapps -a
Network 1 exists
Subnet IPv4: 192.168.200.0/255.255.255.0/ens36, static
Subnet IPv6:
Ping Targets:
Network is enabled
Network is individually enabled on nodes:
Network is individually disabled on nodes:
VIP exists: network number 1, hosting node oel19db1
VIP Name: oel19db1-vip
VIP IPv4 Address: 192.168.200.100
VIP IPv6 Address:
VIP is enabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:
VIP exists: network number 1, hosting node oel19db2
VIP Name: oel19db2-vip
VIP IPv4 Address: 192.168.200.200
VIP IPv6 Address:
VIP is enabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:
|
정상적으로 ens36으로 변경됨
2번 노드 기동 후 grid 상태 확인
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
|
2번 노드 기동
# crsctl start crs
grid 상태 확인
# crsctl stat res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
ONLINE ONLINE oel19db1 STABLE
ONLINE ONLINE oel19db2 STABLE
ora.chad
ONLINE ONLINE oel19db1 STABLE
ONLINE ONLINE oel19db2 STABLE
ora.net1.network
ONLINE ONLINE oel19db1 STABLE
ONLINE ONLINE oel19db2 STABLE
ora.ons
ONLINE ONLINE oel19db1 STABLE
ONLINE ONLINE oel19db2 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNEWLSNR_ASM.lsnr(ora.asmgroup)
1 ONLINE ONLINE oel19db1 STABLE
2 ONLINE ONLINE oel19db2 STABLE
3 ONLINE OFFLINE STABLE
ora.DATA.dg(ora.asmgroup)
1 ONLINE ONLINE oel19db1 STABLE
2 ONLINE ONLINE oel19db2 STABLE
3 OFFLINE OFFLINE STABLE
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE oel19db1 STABLE
ora.OCR_VOTE.dg(ora.asmgroup)
1 ONLINE ONLINE oel19db1 STABLE
2 ONLINE ONLINE oel19db2 STABLE
3 OFFLINE OFFLINE STABLE
ora.asm(ora.asmgroup)
1 ONLINE ONLINE oel19db1 Started,STABLE
2 ONLINE ONLINE oel19db2 Started,STABLE
3 OFFLINE OFFLINE STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
1 ONLINE ONLINE oel19db1 STABLE
2 ONLINE ONLINE oel19db2 STABLE
3 OFFLINE OFFLINE STABLE
ora.cvu
1 ONLINE ONLINE oel19db1 STABLE
ora.oel19db1.vip
1 ONLINE ONLINE oel19db1 STABLE
ora.oel19db2.vip
1 ONLINE ONLINE oel19db2 STABLE
ora.oradb.db
1 ONLINE ONLINE oel19db1 Open,HOME=/ORA19/app
/oracle/product/19.3
.0/db_1,STABLE
2 ONLINE ONLINE oel19db2 Open,HOME=/ORA19/app
/oracle/product/19.3
.0/db_1,STABLE
ora.qosmserver
1 ONLINE ONLINE oel19db1 STABLE
ora.scan1.vip
1 ONLINE ONLINE oel19db1 STABLE
--------------------------------------------------------------------------------
|
모두 정상 상태
이렇게 네트워트 인터페이스가 변경되어 grid가 제대로 기동되지 않을때는
grid를 exclusive mode로 기동된다면
exclusive mode 로 기동 시킨후 oifcfg 명령으로 grid에 등록된 네트워크 정보를 변경해 준뒤
grid 재기동 후 vip 정보도 변경시켜주면 정상적으로 사용이 가능함
(오라클 19c RAC network interface 변경 후 grid 기동불가 시 해결방법 https://positivemh.tistory.com/743 )
exclusive mode로 기동 되지 않는다면
gpnp에 등록된 정보를 변경하는 gpnptool을 이용하면됨
(오라클 19c RAC network interface 변경 후 grid 기동불가 시 해결방법2(gpnptool 이용) https://positivemh.tistory.com/755)
참조 :
https://positivemh.tistory.com/755
https://positivemh.tistory.com/742
오라클 19c RAC IP 변경 방법(IP Change)
OS환경 : Oracle Linux 7.6 (64bit) DB 환경 : Oracle Database 19.6.0.0 방법 : 오라클 19c RAC IP 변경 방법(IP Change) 오라클 19c 2node RAC 환경에서 Public IP, Virtual IP, Scan IP, Private IP 를..
positivemh.tistory.com
https://pat98.tistory.com/m/718
https://community.oracle.com/mosc/discussion/3593716/starting-crs-in-exclusive-mode-to-restore-ocr
'ORACLE > Rac' 카테고리의 다른 글
| 오라클 19c RAC network interface 변경 후 grid 기동불가 시 해결방법2(gpnptool 이용) (0) | 2021.09.27 |
|---|---|
| 오라클 19c RAC 리스너 포트 변경 방법 (0) | 2021.08.14 |
| 오라클 19c RAC IP 변경 방법(IP Change) (8) | 2021.07.27 |
| 오라클 19c RAC에서 ASM 리소스 제거 방법 (0) | 2021.05.10 |
| Oracle 10g RAC 운영시 rcp, rsh, rlogin 이 필요한지 여부 (0) | 2020.07.16 |