iscsi(target)를 이용한 공유 디스크 구성
iscsi 를 이용한 공유 디스크를 구성한다. TAC 와 같은 Active-Active DB Cluster 를 구성하는 경우, 스토리지 서버 (iscsi target server) 1 대와 DB 서버 2 대 총 3대 의 linux 서버가 필요하다.
스토리지 서버(target server)에서 아래와 같이 /dev/targetcli/lun01 로 이름지어진 block type device 인 lv 를 생성한다. DB 서버(iscsi client) 각각에서 하나의 디스크 장치(/dev/sdd)로 보여질 예정.
lvscan
ACTIVE '/dev/targetcli/lun01' [931.50 GiB] inherit
ACTIVE '/dev/centos/home' [20.00 GiB] inherit
ACTIVE '/dev/centos/swap' [<8.00 GiB] inherit
ACTIVE '/dev/centos/app' [40.00 GiB] inherit
ACTIVE '/dev/centos/root' [<50.05 GiB] inherit
ACTIVE '/dev/data/hdd01' [<3.64 TiB] inherit
스토리지 서버(target server) 설정
target 설치 및 실행
스토리지 서버에서 아래와 같이 yum으로 간단하게 target 패키지를 설치한다.
yum -y install targetcli iscsi-initiator-utils
systemctl enable target
systemctl start target
systemctl status target
target server 설정
targetcli 명령을 이용해 구성을 진행하며, targetcli 프롬프트가 나오면 아래와 같이 진행한다.
target block 지정
/> cd /backstores/block
/backstores/block> create lun01_target /dev/targetcli/lun01
iqn 생성
/backstores/block> cd /iscsi
/iscsi> create iqn.2017-10.node1.elk.target:lun01
initiator 생성
/iscsi> cd iqn.2017-10.node1.elk.target:lun01/tpg1/acls
/iscsi/iqn.20...n01/tpg1/acls> create iqn.2017-10.node1.elk.target:initiator01
/iscsi/iqn.20...n01/tpg1/acls> create iqn.2017-10.node1.elk.target:initiator02
initiator id/pass 설정
/iscsi/iqn.20...n01/tpg1/acls> cd iqn.2017-10.node1.elk.target:initiator01
/iscsi/iqn.20...t:initiator01> set auth userid=root
/iscsi/iqn.20...t:initiator01> set auth password=tmaxtmax
/iscsi/iqn.20...t:initiator01> cd ..
/iscsi/iqn.20...n01/tpg1/acls> cd iqn.2017-10.node1.elk.target:initiator02
/iscsi/iqn.20...t:initiator02> set auth userid=root
/iscsi/iqn.20...t:initiator02> set auth password=tmaxtmax
lun 생성
/iscsi/iqn.20...t:initiator02> cd /iscsi/iqn.2017-10.node1.elk.target:lun01/tpg1/luns
/iscsi/iqn.20...n01/tpg1/luns> create /backstores/block/lun01_target
설정 내용 저장
/iscsi/iqn.20...n01/tpg1/luns> cd /
/> saveconfig
target 생성 작업 로그
[root@node1 ~]# targetcli
targetcli shell version 2.1.fb46
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'.
/> cd /backstores/block
/backstores/block> create lun01_target /dev/targetcli/lun01
Created block storage object lun01_target using /dev/targetcli/lun01.
/backstores/block> cd /iscsi
/iscsi> create iqn.2017-10.node1.elk.target:lun01
Created target iqn.2017-10.node1.elk.target:lun01.
Created TPG 1.
Global pref auto_add_default_portal=true
Created default portal listening on all IPs (0.0.0.0), port 3260.
/iscsi> cd iqn.2017-10.node1.elk.target:lun01/tpg1/acls
/iscsi/iqn.20...n01/tpg1/acls> create iqn.2017-10.node1.elk.target:initiator01
Created Node ACL for iqn.2017-10.node1.elk.target:initiator01
/iscsi/iqn.20...n01/tpg1/acls> create iqn.2017-10.node1.elk.target:initiator02
Created Node ACL for iqn.2017-10.node1.elk.target:initiator02
/iscsi/iqn.20...n01/tpg1/acls> cd iqn.2017-10.node1.elk.target:initiator01
/iscsi/iqn.20...t:initiator01> set auth userid=root
Parameter userid is now 'root'.
/iscsi/iqn.20...t:initiator01> set auth password=tmaxtmax
Parameter password is now 'tmaxtmax'.
/iscsi/iqn.20...t:initiator01> cd ..
/iscsi/iqn.20...n01/tpg1/acls> cd iqn.2017-10.node1.elk.target:initiator02
/iscsi/iqn.20...t:initiator02> set auth userid=root
Parameter userid is now 'root'.
/iscsi/iqn.20...t:initiator02> set auth password=tmaxtmax
Parameter password is now 'tmaxtmax'.
/iscsi/iqn.20...t:initiator02> cd /iscsi/iqn.2017-10.node1.elk.target:lun01/tpg1/luns
/iscsi/iqn.20...n01/tpg1/luns> create /backstores/block/lun01_target
Created LUN 0.
Created LUN 0->0 mapping in node ACL iqn.2017-10.node1.elk.target:initiator02
Created LUN 0->0 mapping in node ACL iqn.2017-10.node1.elk.target:initiator01
/iscsi/iqn.20...n01/tpg1/luns>
/iscsi/iqn.20...n01/tpg1/luns> cd /
/> saveconfig
Last 10 configs saved in /etc/target/backup/.
Configuration saved to /etc/target/saveconfig.json
/> exit
Global pref auto_save_on_exit=true
Last 10 configs saved in /etc/target/backup/.
Configuration saved to /etc/target/saveconfig.json
[root@node1 ~]#
DB 서버 (iscsi client) 설정
target 설치 및 실행
DB 서버에서도 마찬가지로 아래와 같이 yum으로 간단하게 target 패키지를 설치한다.
yum -y install targetcli iscsi-initiator-utils
systemctl enable target
systemctl start target
systemctl status target
iscsi 설정
initiatorname.iscsi
vi /etc/iscsi/initiatorname.iscsi
InitiatorName=iqn.2017-10.node1.elk.target:initiator01
iscsid.conf
vi /etc/iscsi/iscsid.conf
node.session.auth.username = root
node.session.auth.password = tmaxtmax
위 파일 수정 완료 후에 systemctl restart iscsid 명령을 수행하여 iscsid 서비스를 재실행한다.
target 서버 조회
iscsiadm -m discovery -t st -p target.node1.elk 192.168.179.80:3260,1
iscsiadm -m discovery -t st -p target.node1.elk 192.168.179.80:3260,1 iqn.2017-10.node1.elk.target:lun01
target 서버 로그인
iscsiadm -m node -T iqn.2017-10.node1.elk.target:lun01 -p target.node1.elk -l
디스크 파티션 확인
cat /proc/partitions
8 16 117220824 sdb
8 17 204800 sdb1
8 18 1048576 sdb2
8 19 115965952 sdb3
8 0 488386584 sda
8 1 488385560 sda1
8 32 1953514584 sdc
8 33 1953513560 sdc1
8 48 976752640 sdd
(참고) target 서버 로그 아웃
iscsiadm -m node -u -T iqn.2017-10.node1.elk.target:lun01 -p 192.168.179.80:3260
taget 서버 로그인 작업 로그
[root@node3 ~]# iscsiadm -m discovery -t st -p target.node1.elk 192.168.179.80:3260,1
192.168.179.80:3260,1 iqn.2017-10.node1.elk.target:lun01
[root@node3 ~]# iscsiadm -m discovery -t st -p target.node1.elk 192.168.179.80:3260,1 iqn.2017-10.node1.elk.target:lun01
192.168.179.80:3260,1 iqn.2017-10.node1.elk.target:lun01
[root@node3 ~]# iscsiadm -m node -T iqn.2017-10.node1.elk.target:lun01 -p target.node1.elk -l
Logging in to [iface: default, target: iqn.2017-10.node1.elk.target:lun01, portal: 192.168.179.80,3260] (multiple)
Login to [iface: default, target: iqn.2017-10.node1.elk.target:lun01, portal: 192.168.179.80,3260] successful.
[root@node3 ~]#
raw device 생성
block device (logical volume) 생성
fdisk /dev/sdd # fdisk 프롬프트에서 sdd1 primary partition 생성
pvcreate /dev/sdd1
vgcreate raws /dev/sdd1
lvcreate -L 1G -n lv1g00 raws
lvcreate -L 1G -n lv1g01 raws
lvcreate -L 1G -n lv1g02 raws
lvcreate -L 1G -n lv1g03 raws
lvcreate -L 1G -n lv1g04 raws
lvcreate -L 1G -n lv1g05 raws
lvcreate -L 1G -n lv1g06 raws
lvcreate -L 1G -n lv1g07 raws
lvcreate -L 1G -n lv1g08 raws
lvcreate -L 1G -n lv1g09 raws
lvcreate -L 1G -n lv1g10 raws
lvcreate -L 1G -n lv1g11 raws
lvcreate -L 1G -n lv1g12 raws
lvcreate -L 1G -n lv1g13 raws
lvcreate -L 1G -n lv1g14 raws
lvcreate -L 1G -n lv1g15 raws
lvcreate -L 1G -n lv1g16 raws
lvcreate -L 1G -n lv1g17 raws
lvcreate -L 1G -n lv1g18 raws
lvcreate -L 1G -n lv1g19 raws
lvcreate -L 1G -n lv1g20 raws
lvcreate -L 1G -n lv1g21 raws
lvcreate -L 1G -n lv1g22 raws
lvcreate -L 1G -n lv1g23 raws
lvcreate -L 1G -n lv1g24 raws
lvcreate -L 1G -n lv1g25 raws
lvcreate -L 1G -n lv1g26 raws
lvcreate -L 1G -n lv1g27 raws
lvcreate -L 1G -n lv1g28 raws
lvcreate -L 1G -n lv1g29 raws
lvcreate -L 5G -n lv5g00 raws
lvcreate -L 5G -n lv5g01 raws
lvcreate -L 5G -n lv5g02 raws
lvcreate -L 5G -n lv5g03 raws
lvcreate -L 5G -n lv5g04 raws
lvcreate -L 5G -n lv5g05 raws
lvcreate -L 5G -n lv5g06 raws
lvcreate -L 5G -n lv5g07 raws
lvcreate -L 5G -n lv5g08 raws
lvcreate -L 5G -n lv5g09 raws
lvcreate -L 5G -n lv5g10 raws
lvcreate -L 5G -n lv5g11 raws
lvcreate -L 5G -n lv5g12 raws
lvcreate -L 5G -n lv5g13 raws
lvcreate -L 5G -n lv5g14 raws
lvcreate -L 5G -n lv5g15 raws
lvcreate -L 5G -n lv5g16 raws
lvcreate -L 5G -n lv5g17 raws
lvcreate -L 5G -n lv5g18 raws
lvcreate -L 5G -n lv5g19 raws
lvcreate -L 10G -n lv10g00 raws
lvcreate -L 10G -n lv10g01 raws
lvcreate -L 10G -n lv10g02 raws
lvcreate -L 10G -n lv10g03 raws
lvcreate -L 10G -n lv10g04 raws
lvcreate -L 10G -n lv10g05 raws
lvcreate -L 10G -n lv10g06 raws
lvcreate -L 10G -n lv10g07 raws
lvcreate -L 10G -n lv10g08 raws
lvcreate -L 10G -n lv10g09 raws
lvcreate -L 30G -n lv30g00 raws
lvcreate -L 30G -n lv30g01 raws
lvcreate -L 30G -n lv30g02 raws
lvcreate -L 30G -n lv30g03 raws
lvcreate -L 30G -n lv30g04 raws
lvcreate -L 30G -n lv30g05 raws
lvcreate -L 30G -n lv30g06 raws
lvcreate -L 30G -n lv30g07 raws
lvcreate -L 30G -n lv30g08 raws
lvcreate -L 30G -n lv30g09 raws
raw
raw /dev/raw/raw1 /dev/raws/lv1g00
raw /dev/raw/raw2 /dev/raws/lv1g01
raw /dev/raw/raw3 /dev/raws/lv1g02
raw /dev/raw/raw4 /dev/raws/lv1g03
raw /dev/raw/raw5 /dev/raws/lv1g04
raw /dev/raw/raw6 /dev/raws/lv1g05
raw /dev/raw/raw7 /dev/raws/lv1g06
raw /dev/raw/raw8 /dev/raws/lv1g07
raw /dev/raw/raw9 /dev/raws/lv1g08
raw /dev/raw/raw10 /dev/raws/lv1g09
raw /dev/raw/raw11 /dev/raws/lv1g10
raw /dev/raw/raw12 /dev/raws/lv1g11
raw /dev/raw/raw13 /dev/raws/lv1g12
raw /dev/raw/raw14 /dev/raws/lv1g13
raw /dev/raw/raw15 /dev/raws/lv1g14
raw /dev/raw/raw16 /dev/raws/lv1g15
raw /dev/raw/raw17 /dev/raws/lv1g16
raw /dev/raw/raw18 /dev/raws/lv1g17
raw /dev/raw/raw19 /dev/raws/lv1g18
raw /dev/raw/raw20 /dev/raws/lv1g19
raw /dev/raw/raw21 /dev/raws/lv1g20
raw /dev/raw/raw22 /dev/raws/lv1g21
raw /dev/raw/raw23 /dev/raws/lv1g22
raw /dev/raw/raw24 /dev/raws/lv1g23
raw /dev/raw/raw25 /dev/raws/lv1g24
raw /dev/raw/raw26 /dev/raws/lv1g25
raw /dev/raw/raw27 /dev/raws/lv1g26
raw /dev/raw/raw28 /dev/raws/lv1g27
raw /dev/raw/raw29 /dev/raws/lv1g28
raw /dev/raw/raw30 /dev/raws/lv1g29
raw /dev/raw/raw31 /dev/raws/lv5g00
raw /dev/raw/raw32 /dev/raws/lv5g01
raw /dev/raw/raw33 /dev/raws/lv5g02
raw /dev/raw/raw34 /dev/raws/lv5g03
raw /dev/raw/raw35 /dev/raws/lv5g04
raw /dev/raw/raw36 /dev/raws/lv5g05
raw /dev/raw/raw37 /dev/raws/lv5g06
raw /dev/raw/raw38 /dev/raws/lv5g07
raw /dev/raw/raw39 /dev/raws/lv5g08
raw /dev/raw/raw40 /dev/raws/lv5g09
raw /dev/raw/raw41 /dev/raws/lv5g10
raw /dev/raw/raw42 /dev/raws/lv5g11
raw /dev/raw/raw43 /dev/raws/lv5g12
raw /dev/raw/raw44 /dev/raws/lv5g13
raw /dev/raw/raw45 /dev/raws/lv5g14
raw /dev/raw/raw46 /dev/raws/lv5g15
raw /dev/raw/raw47 /dev/raws/lv5g16
raw /dev/raw/raw48 /dev/raws/lv5g17
raw /dev/raw/raw49 /dev/raws/lv5g18
raw /dev/raw/raw50 /dev/raws/lv5g19
raw /dev/raw/raw51 /dev/raws/lv10g00
raw /dev/raw/raw52 /dev/raws/lv10g01
raw /dev/raw/raw53 /dev/raws/lv10g02
raw /dev/raw/raw54 /dev/raws/lv10g03
raw /dev/raw/raw55 /dev/raws/lv10g04
raw /dev/raw/raw56 /dev/raws/lv10g05
raw /dev/raw/raw57 /dev/raws/lv10g06
raw /dev/raw/raw58 /dev/raws/lv10g07
raw /dev/raw/raw59 /dev/raws/lv10g08
raw /dev/raw/raw60 /dev/raws/lv10g09
raw /dev/raw/raw61 /dev/raws/lv30g00
raw /dev/raw/raw62 /dev/raws/lv30g01
raw /dev/raw/raw63 /dev/raws/lv30g02
raw /dev/raw/raw64 /dev/raws/lv30g03
raw /dev/raw/raw65 /dev/raws/lv30g04
raw /dev/raw/raw66 /dev/raws/lv30g05
raw /dev/raw/raw67 /dev/raws/lv30g06
raw /dev/raw/raw68 /dev/raws/lv30g07
raw /dev/raw/raw69 /dev/raws/lv30g08
raw /dev/raw/raw70 /dev/raws/lv30g09
symbolic link 생성
ln -s /dev/raw/raw1 /dev/tbdata/rlv1g00
ln -s /dev/raw/raw2 /dev/tbdata/rlv1g01
ln -s /dev/raw/raw3 /dev/tbdata/rlv1g02
ln -s /dev/raw/raw4 /dev/tbdata/rlv1g03
ln -s /dev/raw/raw5 /dev/tbdata/rlv1g04
ln -s /dev/raw/raw6 /dev/tbdata/rlv1g05
ln -s /dev/raw/raw7 /dev/tbdata/rlv1g06
ln -s /dev/raw/raw8 /dev/tbdata/rlv1g07
ln -s /dev/raw/raw9 /dev/tbdata/rlv1g08
ln -s /dev/raw/raw10 /dev/tbdata/rlv1g09
ln -s /dev/raw/raw11 /dev/tbdata/rlv1g10
ln -s /dev/raw/raw12 /dev/tbdata/rlv1g11
ln -s /dev/raw/raw13 /dev/tbdata/rlv1g12
ln -s /dev/raw/raw14 /dev/tbdata/rlv1g13
ln -s /dev/raw/raw15 /dev/tbdata/rlv1g14
ln -s /dev/raw/raw16 /dev/tbdata/rlv1g15
ln -s /dev/raw/raw17 /dev/tbdata/rlv1g16
ln -s /dev/raw/raw18 /dev/tbdata/rlv1g17
ln -s /dev/raw/raw19 /dev/tbdata/rlv1g18
ln -s /dev/raw/raw20 /dev/tbdata/rlv1g19
ln -s /dev/raw/raw21 /dev/tbdata/rlv1g20
ln -s /dev/raw/raw22 /dev/tbdata/rlv1g21
ln -s /dev/raw/raw23 /dev/tbdata/rlv1g22
ln -s /dev/raw/raw24 /dev/tbdata/rlv1g23
ln -s /dev/raw/raw25 /dev/tbdata/rlv1g24
ln -s /dev/raw/raw26 /dev/tbdata/rlv1g25
ln -s /dev/raw/raw27 /dev/tbdata/rlv1g26
ln -s /dev/raw/raw28 /dev/tbdata/rlv1g27
ln -s /dev/raw/raw29 /dev/tbdata/rlv1g28
ln -s /dev/raw/raw30 /dev/tbdata/rlv1g29
ln -s /dev/raw/raw31 /dev/tbdata/rlv5g00
ln -s /dev/raw/raw32 /dev/tbdata/rlv5g01
ln -s /dev/raw/raw33 /dev/tbdata/rlv5g02
ln -s /dev/raw/raw34 /dev/tbdata/rlv5g03
ln -s /dev/raw/raw35 /dev/tbdata/rlv5g04
ln -s /dev/raw/raw36 /dev/tbdata/rlv5g05
ln -s /dev/raw/raw37 /dev/tbdata/rlv5g06
ln -s /dev/raw/raw38 /dev/tbdata/rlv5g07
ln -s /dev/raw/raw39 /dev/tbdata/rlv5g08
ln -s /dev/raw/raw40 /dev/tbdata/rlv5g09
ln -s /dev/raw/raw41 /dev/tbdata/rlv5g10
ln -s /dev/raw/raw42 /dev/tbdata/rlv5g11
ln -s /dev/raw/raw43 /dev/tbdata/rlv5g12
ln -s /dev/raw/raw44 /dev/tbdata/rlv5g13
ln -s /dev/raw/raw45 /dev/tbdata/rlv5g14
ln -s /dev/raw/raw46 /dev/tbdata/rlv5g15
ln -s /dev/raw/raw47 /dev/tbdata/rlv5g16
ln -s /dev/raw/raw48 /dev/tbdata/rlv5g17
ln -s /dev/raw/raw49 /dev/tbdata/rlv5g18
ln -s /dev/raw/raw50 /dev/tbdata/rlv5g19
ln -s /dev/raw/raw51 /dev/tbdata/rlv10g00
ln -s /dev/raw/raw52 /dev/tbdata/rlv10g01
ln -s /dev/raw/raw53 /dev/tbdata/rlv10g02
ln -s /dev/raw/raw54 /dev/tbdata/rlv10g03
ln -s /dev/raw/raw55 /dev/tbdata/rlv10g04
ln -s /dev/raw/raw56 /dev/tbdata/rlv10g05
ln -s /dev/raw/raw57 /dev/tbdata/rlv10g06
ln -s /dev/raw/raw58 /dev/tbdata/rlv10g07
ln -s /dev/raw/raw59 /dev/tbdata/rlv10g08
ln -s /dev/raw/raw60 /dev/tbdata/rlv10g09
ln -s /dev/raw/raw61 /dev/tbdata/rlv30g00
ln -s /dev/raw/raw62 /dev/tbdata/rlv30g01
ln -s /dev/raw/raw63 /dev/tbdata/rlv30g02
ln -s /dev/raw/raw64 /dev/tbdata/rlv30g03
ln -s /dev/raw/raw65 /dev/tbdata/rlv30g04
ln -s /dev/raw/raw66 /dev/tbdata/rlv30g05
ln -s /dev/raw/raw67 /dev/tbdata/rlv30g06
ln -s /dev/raw/raw68 /dev/tbdata/rlv30g07
ln -s /dev/raw/raw69 /dev/tbdata/rlv30g08
ln -s /dev/raw/raw70 /dev/tbdata/rlv30g09
