오라클 19c Hugepages 설정 방법

OS환경 : Oracle Linux 7.5 (64bit)

 

DB 환경 : Oracle Database 19.3.0.0

 

방법 : 오라클 19c Hugepages 설정 방법

HugePage 란?

일반적인 컴퓨팅 시스템은 물리적 메모리 크기를 극복하기 위해 가상메모리 기법을 사용하며, 

상이한 두 메모리를 매핑하기 위해 Page Table이 존재하고 Page 단위로 관리됨

리눅스 시스템의 경우 보통 4k로 되어있음

대용량 메모리를 장착한 시스템의 효율적인 Page Table 관리를 위해 Kernel 2.6부터는 HugePage 기술이 도입되어 더 큰 Page 크기(예 : 2M ~ 256M)로 관리 할 수 있게 되었음

 

 

HugePage 사용 시 장점

1. Normal Page와는 다르게 HugePage는 Swap되지 않음. 따라서 Page-in/.Page-Out의 Overhead가 없음

2. HugePage는 상대적으로 적은 수의 Meta 정보로 좀 많은 물리적 메모리를 Cover하므로 CPU내의 Buffer(=Translation Lookaside Buffer, TLB)좀 더 많은 Page Entry를 보관할 수 있어 Page Hit율이 높아짐

3. Page Table에서 관리하는 물리적 메모리의 Entry 개수가 줄어들고 더 적은 메모리 공간을 차지함

따라서 오라클 관점에서 보면 오라클 프로세스의 계산이나 PGA에 좀 더 많은 메모리가 제공됨

4. Normal Page의경우, 50GB의 메모리를 관리하기 위해서는 

1,300 만개의 page(13,107,200 = 50 * 1024* 1024 KB / 4 KB)가 유지되어야함

HugePages를 사용할 경우 25,600 개의 Page가 관리되므로 

메모리 관리을 위한 Overhead가 줄어들고 자원사용율도 절약됨

5. 평균적으로 20~30% 성능향상이 있을수 있음

 

 

HugePage 사용 시 단점

Oracle DB가 갑자기 종료된 경우 HugePage에서 잡고 있던 메모리를 천천히 반환하기 때문에

DB 종료 직후 다시 기동하려 할때 메모리 부족으로 안올라 갈 수도있음

이때는 메모리 반환을 기다리는 방법과 서버 재기동하는 방법이 있음

 

 

Oracle 11.2.0.2 이상부터는 use_large_pages = ( TRUE | FALSE | ONLY ) 파라미터를 사용하여 

Oracle SGA 의 HugePage 사용을 관리하고 문제의 발생을 예방함 

Oracle 11.2.0.3 부터는 HugePages가 SGA에 비해 작을 경우, HugePages와 Normal Pages를 동시에 사용함

 

 

HugePage 설정 방법

1. 서버에 있는 모든 인스턴스 OPEN

SQL> startup
ORACLE instance started.
 
Total System Global Area 1677718256 bytes
Fixed Size            8897264 bytes
Variable Size          536870912 bytes
Database Buffers     1124073472 bytes
Redo Buffers            7876608 bytes
Database mounted.
Database opened.

 

 

2. DB가 AMM을 사용하고 있는지 확인(AMM 사용시 HugePage 적용불가)

SQL> show parameter memory_target
 
NAME                     TYPE     VALUE
------------------------------------ ----------- ------------------------------
memory_target                 big integer 0

현재 AMM 미사용

 

 

3. Doc. 401749.1 에 있는 권장 HugePage 값을 확인하는 스크립트 저장

#!/bin/bash
#
# hugepages_settings.sh
#
# Linux bash script to compute values for the
# recommended HugePages/HugeTLB configuration
# on Oracle Linux
#
# Note: This script does calculation for all shared memory
# segments available when the script is run, no matter it
# is an Oracle RDBMS shared memory segment or not.
#
# This script is provided by Doc ID 401749.1 from My Oracle Support
# http://support.oracle.com
 
# Welcome text
echo "
This script is provided by Doc ID 401749.1 from My Oracle Support
(http://support.oracle.com) where it is intended to compute values for
the recommended HugePages/HugeTLB configuration for the current shared
memory segments on Oracle Linux. Before proceeding with the execution please note following:
 * For ASM instance, it needs to configure ASMM instead of AMM.
 * The 'pga_aggregate_target' is outside the SGA and
   you should accommodate this while calculating the overall size.
 * In case you changes the DB SGA size,
   as the new SGA will not fit in the previous HugePages configuration,
   it had better disable the whole HugePages,
   start the DB with new SGA size and run the script again.
And make sure that:
 * Oracle Database instance(s) are up and running
 * Oracle Database 11g Automatic Memory Management (AMM) is not setup
   (See Doc ID 749851.1)
 * The shared memory segments can be listed by command:
     # ipcs -m
 
 
Press Enter to proceed..."
 
read
 
# Check for the kernel version
KERN=`uname -r | awk -F. '{ printf("%d.%d\n",$1,$2); }'`
 
# Find out the HugePage size
HPG_SZ=`grep Hugepagesize /proc/meminfo | awk '{print $2}'`
if [ -z "$HPG_SZ" ];then
    echo "The hugepages may not be supported in the system where the script is being executed."
    exit 1
fi
 
# Initialize the counter
NUM_PG=0
 
# Cumulative number of pages required to handle the running shared memory segments
for SEG_BYTES in `ipcs -m | cut -c44-300 | awk '{print $1}' | grep "[0-9][0-9]*"`
do
    MIN_PG=`echo "$SEG_BYTES/($HPG_SZ*1024)" | bc -q`
    if [ $MIN_PG -gt 0 ]; then
        NUM_PG=`echo "$NUM_PG+$MIN_PG+1" | bc -q`
    fi
done
 
RES_BYTES=`echo "$NUM_PG * $HPG_SZ * 1024" | bc -q`
 
# An SGA less than 100MB does not make sense
# Bail out if that is the case
if [ $RES_BYTES -lt 100000000 ]; then
    echo "***********"
    echo "** ERROR **"
    echo "***********"
    echo "Sorry! There are not enough total of shared memory segments allocated for
HugePages configuration. HugePages can only be used for shared memory segments
that you can list by command:
 
    # ipcs -m
 
of a size that can match an Oracle Database SGA. Please make sure that:
 * Oracle Database instance is up and running
 * Oracle Database 11g Automatic Memory Management (AMM) is not configured"
    exit 1
fi
 
# Finish with results
case $KERN in
    '2.4') HUGETLB_POOL=`echo "$NUM_PG*$HPG_SZ/1024" | bc -q`;
           echo "Recommended setting: vm.hugetlb_pool = $HUGETLB_POOL" ;;
    '2.6') echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;;
    '3.8') echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;;
    '3.10') echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;;
    '4.1') echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;;
    '4.14') echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;;
    '4.18') echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;;
    '5.4') echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;;
    *) echo "Kernel version $KERN is not supported by this script (yet). Exiting." ;;
esac
 
# End

hugepages_settings.sh

0.00MB

 

 

4. 실행 권한 부여

$ chmod +x hugepages_settings.sh

 

 

5. 권장 HugePage 값 확인 쉘 실행

$ ./hugepages_settings.sh 
 
This script is provided by Doc ID 401749.1 from My Oracle Support
(http://support.oracle.com) where it is intended to compute values for
the recommended HugePages/HugeTLB configuration for the current shared
memory segments on Oracle Linux. Before proceeding with the execution please note following:
 * For ASM instance, it needs to configure ASMM instead of AMM.
 * The 'pga_aggregate_target' is outside the SGA and
   you should accommodate this while calculating the overall size.
 * In case you changes the DB SGA size,
   as the new SGA will not fit in the previous HugePages configuration,
   it had better disable the whole HugePages,
   start the DB with new SGA size and run the script again.
And make sure that:
 * Oracle Database instance(s) are up and running
 * Oracle Database 11g Automatic Memory Management (AMM) is not setup
   (See Doc ID 749851.1)
 * The shared memory segments can be listed by command:
     # ipcs -m
 
 
Press Enter to proceed...
 
Recommended setting: vm.nr_hugepages = 808

권장값이 808로 나옴

 

 

해당 값으로 hugapages 설정

# sysctl -w vm.nr_hugepages=808
vm.nr_hugepages = 808

 

 

서버 재기동 이후에도 변경되지 않도록 /etc/sysctl.conf 에 기록

# vi /etc/sysctl.conf
vm.nr_hugepages=808

 

 

오라클 use_large_pages 파라미터 변경 전 Huge Page사용량 확인

$ grep Huge /proc/meminfo
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
HugePages_Total:     808
HugePages_Free:        7
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

Huge Page Total이 808이고 Free가 7임을 확인 할 수 있음

 

 

오라클 접속 후 파라미터 변경

SQL> show parameter use_large_pages
 
NAME                     TYPE     VALUE
------------------------------------ ----------- ------------------------------
use_large_pages              string     TRUE
 
SQL> alter system set use_large_pages=only scope=spfile;
 
System altered.

기존 값 TRUE에서 ONLY로 변경

 

 

재기동

SQL> shutdown immediate
Database closed.
Database dismounted.
ORACLE instance shut down.
 
SQL> startup
 
Total System Global Area 1677718256 bytes
Fixed Size            8897264 bytes
Variable Size          536870912 bytes
Database Buffers     1124073472 bytes
Redo Buffers            7876608 bytes
Database mounted.
Database opened.

 

 

*DB 기동시 ORA-27106 발생 시 서버 재기동 후 다시 시도

SQL> startup
ORA-27106: system pages not available to allocate memory
Additional information: 6122
Additional information: 1

 

 

오라클 use_large_pages 파라미터 변경 후 Huge Page사용량 확인

$ grep Huge /proc/meminfo
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
HugePages_Total:     808
HugePages_Free:        6
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

Huge Page Total이 808이고 Free가 6임을 확인 할 수 있음

오라클 use_large_page =only 설정 및 재기동으로 HugePages_Free가 더 줄어듦 확인 ( 7에서 6으로 변경됨  ) 

 

 

 

참조 : https://positivemh.tistory.com/602

361323.1, 401749.1, 361468.1