LinuCエヴァンジェリストの鯨井貴博@opensourcetechです。
今回は、etcdを使ってみます。
etcdとは？
etcdといえば、kubernetesで使われているkey value store(クラスタの全情報を持っている)として有名です。
https://kubernetes.io/ja/docs/concepts/overview/components/
本家サイトは、以下。
https://etcd.io/
key value storeとは?
キーバリューストア、色々なところで聞きますが、いまいちピンときづらいですね。
なので、表(RDBやExcel)と比較すると、以下のようになります。
RBDの場合
| Name | Age | Location | Remarks |
|---|---|---|---|
| Takahiro | 39 | Tokyo | |
| Daisuke | 43 | Osaka |
key value storeの場合
| key | value |
|---|---|
| Name | Takahiro |
| Age | 39 |
| Location | Tokyo |
| Remarks |
etcdのインストール
CentOS7でやってみます。
yumを使って簡単にインストール可能です。
[root@9a7df59dc26d ~]# yum search etcd Loaded plugins: fastestmirror, langpacks Determining fastest mirrors epel/x86_64/metalink | 32 kB 00:00:00 * base: mirror.softaculous.com * epel: mirror.23m.com * extras: mirror.softaculous.com * updates: ftp.rz.uni-frankfurt.de base | 3.6 kB 00:00:00 docker-ce-stable | 3.5 kB 00:00:00 epel | 4.7 kB 00:00:00 extras | 2.9 kB 00:00:00 updates | 2.9 kB 00:00:00 vscode | 3.0 kB 00:00:00 (1/10): base/7/x86_64/group_gz | 153 kB 00:00:00 (2/10): epel/x86_64/group_gz | 96 kB 00:00:00 (3/10): base/7/x86_64/primary_db | 6.1 MB 00:00:00 (4/10): epel/x86_64/updateinfo | 1.0 MB 00:00:00 (5/10): docker-ce-stable/x86_64/primary_db | 66 kB 00:00:00 (6/10): extras/7/x86_64/primary_db | 243 kB 00:00:00 (7/10): docker-ce-stable/x86_64/updateinfo | 55 B 00:00:00 (8/10): epel/x86_64/primary_db | 7.0 MB 00:00:00 (9/10): vscode/primary_db | 1.5 MB 00:00:00 (10/10): updates/7/x86_64/primary_db | 12 MB 00:00:00 ======================================= N/S matched: etcd ======================================= netcdf-cxx.x86_64 : Legacy netCDF C++ library netcdf-cxx-devel.x86_64 : Development files legacy netCDF C++ library netcdf-cxx-static.x86_64 : Static libraries for legacy netCDF C++ library netcdf-devel.x86_64 : Development files for netcdf netcdf-fortran.x86_64 : Fortran libraries for NetCDF-4 netcdf-fortran-devel.x86_64 : Development files for Fortran NetCDF API netcdf-fortran-mpich.x86_64 : NetCDF Fortran mpich libraries netcdf-fortran-mpich-devel.x86_64 : NetCDF Fortran mpich development files netcdf-fortran-mpich-static.x86_64 : NetCDF Fortran mpich static libraries netcdf-fortran-openmpi.x86_64 : NetCDF Fortran openmpi libraries netcdf-fortran-openmpi-devel.x86_64 : NetCDF Fortran openmpi development files netcdf-fortran-openmpi-static.x86_64 : NetCDF Fortran openmpi static libraries netcdf-fortran-static.x86_64 : Static library for Fortran NetCDF API netcdf-mpich.x86_64 : NetCDF mpich libraries netcdf-mpich-devel.x86_64 : NetCDF mpich development files netcdf-mpich-static.x86_64 : NetCDF mpich static libraries netcdf-openmpi.x86_64 : NetCDF openmpi libraries netcdf-openmpi-devel.x86_64 : NetCDF openmpi development files netcdf-openmpi-static.x86_64 : NetCDF openmpi static libraries netcdf-static.x86_64 : Static libs for netcdf netcdf4-python.x86_64 : Python/numpy interface to netCDF octave-netcdf.x86_64 : A MATLAB compatible NetCDF interface for Octave python2-netcdf4.x86_64 : Python/numpy interface to netCDF python36-netcdf4.x86_64 : Python/numpy interface to netCDF etcd.x86_64 : A highly-available key value store for shared configuration flannel.x86_64 : Etcd address management agent for overlay networks nco.x86_64 : Suite of programs for manipulating NetCDF/HDF4 files ncview.x86_64 : A visual browser for netCDF format files netcdf.x86_64 : Libraries for the Unidata network Common Data Form Name and summary matches only, use "search all" for everything. [root@9a7df59dc26d ~]# yum install etcd Loaded plugins: fastestmirror, langpacks Loading mirror speeds from cached hostfile * base: mirror.softaculous.com * epel: mirror.23m.com * extras: mirror.softaculous.com * updates: ftp.rz.uni-frankfurt.de Resolving Dependencies --> Running transaction check ---> Package etcd.x86_64 0:3.3.11-2.el7.centos will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================================= Package Arch Version Repository Size ================================================================================================= Installing: etcd x86_64 3.3.11-2.el7.centos extras 10 M Transaction Summary ================================================================================================= Install 1 Package Total download size: 10 M Installed size: 45 M Is this ok [y/d/N]: y Downloading packages: etcd-3.3.11-2.el7.centos.x86_64.rpm | 10 MB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : etcd-3.3.11-2.el7.centos.x86_64 1/1 Verifying : etcd-3.3.11-2.el7.centos.x86_64 1/1 Installed: etcd.x86_64 0:3.3.11-2.el7.centos Complete!
インストールされたコマンド(etcd・etcdctl)の確認をします。
[root@9a7df59dc26d ~]# etcd --version etcd Version: 3.3.11 Git SHA: 2cf9e51 Go Version: go1.10.3 Go OS/Arch: linux/amd64 [root@9a7df59dc26d ~]# etcdctl --version etcdctl version: 3.3.11 API version: 2
一応、ヘルプでコマンドオプションなども確認。
| サブコマンド | 効果 |
|---|---|
| set | キーバリューの追加 |
| get | キーバリューの確認 |
| update | キーバリューの更新 |
| backup | etcdディレクトリのバックアップ |
とりあえず、こんなところがあればよさそうですね。
[root@9a7df59dc26d ~]# etcdctl --help
NAME:
etcdctl - A simple command line client for etcd.
WARNING:
Environment variable ETCDCTL_API is not set; defaults to etcdctl v2.
Set environment variable ETCDCTL_API=3 to use v3 API or ETCDCTL_API=2 to use v2 API.
USAGE:
etcdctl [global options] command [command options] [arguments...]
VERSION:
3.3.11
COMMANDS:
backup backup an etcd directory
cluster-health check the health of the etcd cluster
mk make a new key with a given value
mkdir make a new directory
rm remove a key or a directory
rmdir removes the key if it is an empty directory or a key-value pair
get retrieve the value of a key
ls retrieve a directory
set set the value of a key
setdir create a new directory or update an existing directory TTL
update update an existing key with a given value
updatedir update an existing directory
watch watch a key for changes
exec-watch watch a key for changes and exec an executable
member member add, remove and list subcommands
user user add, grant and revoke subcommands
role role add, grant and revoke subcommands
auth overall auth controls
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--debug output cURL commands which can be used to reproduce the request
--no-sync don't synchronize cluster information before sending request
--output simple, -o simple output response in the given format (simple, `extended` or `json`) (default: "simple")
--discovery-srv value, -D value domain name to query for SRV records describing cluster endpoints
--insecure-discovery accept insecure SRV records describing cluster endpoints
--peers value, -C value DEPRECATED - "--endpoints" should be used instead
--endpoint value DEPRECATED - "--endpoints" should be used instead
--endpoints value a comma-delimited list of machine addresses in the cluster (default: "http://127.0.0.1:2379,http://127.0.0.1:4001")
--cert-file value identify HTTPS client using this SSL certificate file
--key-file value identify HTTPS client using this SSL key file
--ca-file value verify certificates of HTTPS-enabled servers using this CA bundle
--username value, -u value provide username[:password] and prompt if password is not supplied.
--timeout value connection timeout per request (default: 2s)
--total-timeout value timeout for the command execution (except watch) (default: 5s)
--help, -h show help
--version, -v print the version
etcdの起動
では、etcdの起動から。
[root@9a7df59dc26d ~]# systemctl status etcd
● etcd.service - Etcd Server
Loaded: loaded (/usr/lib/systemd/system/etcd.service; disabled; vendor preset: disabled)
Active: inactive (dead)
[root@9a7df59dc26d ~]# systemctl start etcd
[root@9a7df59dc26d ~]# systemctl status etcd
● etcd.service - Etcd Server
Loaded: loaded (/usr/lib/systemd/system/etcd.service; disabled; vendor preset: disabled)
Active: active (running) since Wed 2021-10-20 15:58:47 UTC; 2s ago
Main PID: 2880 (etcd)
Tasks: 10
Memory: 14.2M
CGroup: /system.slice/etcd.service
└─2880 /usr/bin/etcd --name=default --data-dir=/var/lib/etcd/default.etcd --listen-...
Oct 20 15:58:47 9a7df59dc26d etcd[2880]: 8e9e05c52164694d received MsgVoteResp from 8e9e05c...m 2
Oct 20 15:58:47 9a7df59dc26d etcd[2880]: 8e9e05c52164694d became leader at term 2
Oct 20 15:58:47 9a7df59dc26d etcd[2880]: raft.node: 8e9e05c52164694d elected leader 8e9e05c...m 2
Oct 20 15:58:47 9a7df59dc26d etcd[2880]: published {Name:default ClientURLs:[http://localho...c32
Oct 20 15:58:47 9a7df59dc26d etcd[2880]: setting up the initial cluster version to 3.3
Oct 20 15:58:47 9a7df59dc26d etcd[2880]: ready to serve client requests
Oct 20 15:58:47 9a7df59dc26d etcd[2880]: serving insecure client requests on 127.0.0.1:2379...ed!
Oct 20 15:58:47 9a7df59dc26d etcd[2880]: set the initial cluster version to 3.3
Oct 20 15:58:47 9a7df59dc26d etcd[2880]: enabled capabilities for version 3.3
Oct 20 15:58:47 9a7df59dc26d systemd[1]: Started Etcd Server.
Hint: Some lines were ellipsized, use -l to show in full.
[root@9a7df59dc26d ~]# etcdctl ls
ちなみにetcdが起動していない場合、
以下のようなメッセージが出力されます。
[root@9a7df59dc26d ~]# etcdctl ls Error: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 127.0.0.1:4001: connect: connection refused ; error #1: dial tcp 127.0.0.1:2379: connect: connection refused error #0: dial tcp 127.0.0.1:4001: connect: connection refused error #1: dial tcp 127.0.0.1:2379: connect: connection refused
キーバリューデータの追加と確認
続いて、キーバリューデータの追加と確認をします。
[root@9a7df59dc26d ~]# etcdctl set Name Takahiro Takahiro [root@9a7df59dc26d ~]# etcdctl get Name Takahiro [root@9a7df59dc26d ~]# etcdctl set Age 39 39 [root@9a7df59dc26d ~]# etcdctl set Location Tokyo Tokyo [root@9a7df59dc26d ~]# etcdctl set Remarks " " [root@9a7df59dc26d ~]# etcdctl get Age 39 [root@9a7df59dc26d ~]# etcdctl get Location Tokyo [root@9a7df59dc26d ~]# etcdctl get Remarks [root@9a7df59dc26d ~]#
大丈夫そうですね！
少し深堀すると、各keyは内部的にはディレクトリとして管理されていることが分かります。
[root@9a7df59dc26d ~]# etcdctl ls /Location /Remarks /Name /Age [root@9a7df59dc26d ~]# etcdctl get /Location Tokyo [root@9a7df59dc26d ~]# etcdctl get /Age 39
ついでにデータの更新。
[root@9a7df59dc26d ~]# etcdctl get Age 39 [root@9a7df59dc26d ~]# etcdctl update Age 30 30 [root@9a7df59dc26d ~]# etcdctl get Age 30
そして、watchオプションも使ってみましょう。
watchでキーAgeを見張っておいて、
[root@9a7df59dc26d ~]# etcdctl watch Age
別のターミナルでキーAgeを更新すると、
[root@9a7df59dc26d ~]# etcdctl update Age 40 40
監視している方では、更新された内容が表示されました。
[root@9a7df59dc26d ~]# etcdctl watch Age 40
キーバリューデータの削除
こちらも大丈夫そうです。
[root@9a7df59dc26d ~]# etcdctl ls /Name /Age /Location /Remarks [root@9a7df59dc26d ~]# etcdctl rm Location PrevNode.Value: Tokyo [root@9a7df59dc26d ~]# etcdctl get Location Error: 100: Key not found (/Location) [10] [root@9a7df59dc26d ~]# etcdctl ls /Age /Remarks /Name
キーバリューデータのバックアップ
etcdプロセスが使っているデータディレクトリを確認してから、バックアップします。
具体的には、--data-dirというオプションの部分です。
[root@9a7df59dc26d ~]# ps aux | grep etcd etcd 2880 0.4 1.7 10939504 26604 ? Ssl 15:58 0:07 /usr/bin/etcd --name=default --data-dir=/var/lib/etcd/default.etcd --listen-client-urls=http://localhost:2379 root 3679 0.0 0.0 112712 960 pts/0 S+ 16:23 0:00 grep --color=auto etcd [root@9a7df59dc26d ~]# ls /tmp [root@9a7df59dc26d ~]# etcdctl backup --data-dir=/var/lib/etcd/default.etcd --backup-dir=/tmp 2021-10-20 16:24:15.439229 I | ignoring EntryConfChange raft entry 2021-10-20 16:24:15.439311 I | ignoring member attribute update on /0/members/8e9e05c52164694d/attributes [root@9a7df59dc26d ~]# ls /tmp member
なお、kubernetesのetcdのバックアップやリストアには、
"etcdctl snapshot save"・"etcdctl snapshot status"や、"etcdctl snapshot restore"を使うようです。
https://kubernetes.io/docs/tasks/administer-cluster/configure-upgrade-etcd/
以下のサイトだと、etcdのデータバージョンによってコマンドを使い分けるようです。
https://www.creationline.com/lab/36883
etcdがバージョン2/3のデータを保持している場合はetcdctl backup、
etcdのバージョン3のみ保持しているの場合はetcdctl snapshot save
このあたり、もう少し勉強が必要そうです。
