LinuCエヴァンジェリストの鯨井貴博@opensourcetechです。
はじめに
今回は、kubeadmを使ったKubernetes v1.34.9の構築を行っていきます。
構築する環境
・Clusterの構成
Master Node 1台 & Worker Node 1台
※予めUbuntu24.04LTSを2台用意済み
・使用するOS
Ubuntu 24.04 LTS Server
cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=24.04 DISTRIB_CODENAME=noble DISTRIB_DESCRIPTION="Ubuntu 24.04.4 LTS"
・Master NodeのPCスペック
4CPUs・Memory16GB・ストレージ50GB
・Worker NodeのPCスペック
4CPUs・Memory16GB・ストレージ50GB
・MasterNode、及びWorkerNodeのIPアドレス
10.30.128.84/24 ip-10-30-128-84 ※MasterNode
10.30.128.216/24 ip-10-30-128-216 ※WorkerNode
・Podネットワーク
IPv4：10.0.0.0/16
全Node共通設定① Swapの無効化
swapon -sで確認したところ無効となっていました。
全Node共通設定② /etc/hostsの編集
※ログは、Master Nodeだけ記載しています。
各Nodeの/etc/hostsに、クラスターを組む2台分のIPv4やホスト名をip addr showなどで確認し追記します。
sudo vi /etc/hosts sudo cat /etc/hosts 127.0.0.1 localhost 10.30.128.84 ip-10-30-128-84 10.30.128.216 ip-10-30-128-216 # The following lines are desirable for IPv6 capable hosts ::1 ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters ff02::3 ip6-allhosts
全Node共通設定③ iptablesがブリッジを通過するトラフィックを処理できるようにする
※ログは、Master Nodeだけ記載しています。
必要なモジュールのロードとLinuxカーネルのパラメータを有効化します。
lsmod | grep br_netfilter sudo modprobe br_netfilter lsmod | grep br_netfilter br_netfilter 32768 0 bridge 307200 1 br_netfilter
sudo vi /etc/sysctl.d/k8s.conf cat /etc/sysctl.d/k8s.conf net.bridge.bridge-nf-call-iptables = 1 sudo sysctl --system * Applying /usr/lib/sysctl.d/10-apparmor.conf ... * Applying /etc/sysctl.d/10-bufferbloat.conf ... * Applying /etc/sysctl.d/10-console-messages.conf ... * Applying /etc/sysctl.d/10-ipv6-privacy.conf ... * Applying /etc/sysctl.d/10-kernel-hardening.conf ... * Applying /etc/sysctl.d/10-magic-sysrq.conf ... * Applying /etc/sysctl.d/10-map-count.conf ... * Applying /etc/sysctl.d/10-network-security.conf ... * Applying /etc/sysctl.d/10-ptrace.conf ... * Applying /etc/sysctl.d/10-zeropage.conf ... * Applying /etc/sysctl.d/50-cloudimg-settings.conf ... * Applying /usr/lib/sysctl.d/50-pid-max.conf ... * Applying /etc/sysctl.d/99-cloudimg-ipv6.conf ... * Applying /usr/lib/sysctl.d/99-protect-links.conf ... * Applying /etc/sysctl.d/99-sysctl.conf ... * Applying /etc/sysctl.d/k8s.conf ... * Applying /etc/sysctl.conf ... kernel.apparmor_restrict_unprivileged_userns = 1 net.core.default_qdisc = fq_codel kernel.printk = 4 4 1 7 net.ipv6.conf.all.use_tempaddr = 2 net.ipv6.conf.default.use_tempaddr = 2 kernel.kptr_restrict = 1 kernel.sysrq = 176 vm.max_map_count = 1048576 net.ipv4.conf.default.rp_filter = 2 net.ipv4.conf.all.rp_filter = 2 kernel.yama.ptrace_scope = 1 vm.mmap_min_addr = 65536 net.ipv4.neigh.default.gc_thresh2 = 15360 net.ipv4.neigh.default.gc_thresh3 = 16384 net.netfilter.nf_conntrack_max = 1048576 kernel.pid_max = 4194304 net.ipv6.conf.all.use_tempaddr = 0 net.ipv6.conf.default.use_tempaddr = 0 fs.protected_fifos = 1 fs.protected_hardlinks = 1 fs.protected_regular = 2 fs.protected_symlinks = 1 net.bridge.bridge-nf-call-iptables = 1
全Node共通設定④ iptablesがnftablesバックエンドを使用しないようにする
※ログは、Master Nodeだけ記載しています。
nftablesバックエンドとkubeadmパッケージの互換性の調整のため、iptables-legacyが使用されるように変更します。
sudo apt update
sudo apt install -y iptables arptables ebtables
Reading state information... Done
iptables is already the newest version (1.8.7-1ubuntu5).
iptables set to manually installed.
The following NEW packages will be installed:
arptables ebtables
0 upgraded, 2 newly installed, 0 to remove and 16 not upgraded.
Need to get 123 kB of archives.
After this operation, 373 kB of additional disk space will be used.
[Working]
Get:1 http://jp.archive.ubuntu.com/ubuntu jammy/universe amd64 arptables amd64 0.0.5-3 [38.1 kB]
(Reading database ... 100%
(Reading database ... 73929 files and directories currently installed.)
Preparing to unpack .../arptables_0.0.5-3_amd64.deb ...
Scanning processes...
Scanning linux images...
Running kernel seems to be up-to-date.
No services need to be restarted.
No containers need to be restarted.
No user sessions are running outdated binaries.
No VM guests are running outdated hypervisor (qemu) binaries on this host.
sudo update-alternatives --set iptables /usr/sbin/iptables-legacy update-alternatives: using /usr/sbin/iptables-legacy to provide /usr/sbin/iptables (iptables) in manual mode sudo update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy update-alternatives: using /usr/sbin/arptables-legacy to provide /usr/sbin/arptables (arptables) in manual mode sudo update-alternatives --set arptables /usr/sbin/arptables-legacy update-alternatives: using /usr/sbin/ebtables-legacy to provide /usr/sbin/ebtables (ebtables) in manual mode sudo update-alternatives --set ebtables /usr/sbin/ebtables-legacy update-alternatives: using /usr/sbin/ebtables-legacy to provide /usr/sbin/ebtables (ebtables) in manual mode
全Node共通設定⑤ コンテナランタイムのインストール
コンテナランライム：ざっくりいうと、コンテナを起動してくれるプログラムのことです。
※ログは、Master Nodeだけ記載しています。
Docker・CRI-O・Containerdなどから選択できますが、
今回はContainerdを使いました。
参考：https://kubernetes.io/ja/docs/setup/production-environment/container-runtimes/
sudo vi /etc/modules-load.d/containerd.conf sudo cat /etc/modules-load.d/containerd.conf overlay br_netfilter sudo modprobe overlay sudo modprobe br_netfilter sudo vi /etc/sysctl.d/99-kubernetes-cri.conf net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward = 1 sudo sysctl --system * Applying /usr/lib/sysctl.d/10-apparmor.conf ... * Applying /etc/sysctl.d/10-bufferbloat.conf ... * Applying /etc/sysctl.d/10-console-messages.conf ... * Applying /etc/sysctl.d/10-ipv6-privacy.conf ... * Applying /etc/sysctl.d/10-kernel-hardening.conf ... * Applying /etc/sysctl.d/10-magic-sysrq.conf ... * Applying /etc/sysctl.d/10-map-count.conf ... * Applying /etc/sysctl.d/10-network-security.conf ... * Applying /etc/sysctl.d/10-ptrace.conf ... * Applying /etc/sysctl.d/10-zeropage.conf ... * Applying /etc/sysctl.d/50-cloudimg-settings.conf ... * Applying /usr/lib/sysctl.d/50-pid-max.conf ... * Applying /etc/sysctl.d/99-cloudimg-ipv6.conf ... * Applying /etc/sysctl.d/99-kubernetes-cri.conf ... * Applying /usr/lib/sysctl.d/99-protect-links.conf ... * Applying /etc/sysctl.d/99-sysctl.conf ... * Applying /etc/sysctl.d/k8s.conf ... * Applying /etc/sysctl.conf ... kernel.apparmor_restrict_unprivileged_userns = 1 net.core.default_qdisc = fq_codel kernel.printk = 4 4 1 7 net.ipv6.conf.all.use_tempaddr = 2 net.ipv6.conf.default.use_tempaddr = 2 kernel.kptr_restrict = 1 kernel.sysrq = 176 vm.max_map_count = 1048576 net.ipv4.conf.default.rp_filter = 2 net.ipv4.conf.all.rp_filter = 2 kernel.yama.ptrace_scope = 1 vm.mmap_min_addr = 65536 net.ipv4.neigh.default.gc_thresh2 = 15360 net.ipv4.neigh.default.gc_thresh3 = 16384 net.netfilter.nf_conntrack_max = 1048576 kernel.pid_max = 4194304 net.ipv6.conf.all.use_tempaddr = 0 net.ipv6.conf.default.use_tempaddr = 0 net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward = 1 fs.protected_fifos = 1 fs.protected_hardlinks = 1 fs.protected_regular = 2 fs.protected_symlinks = 1 net.bridge.bridge-nf-call-iptables = 1
sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl software-properties-common sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). gpg: conflicting commands gpg: no valid OpenPGP data found. sudo add-apt-repository \ "deb [arch=amd64] https://download.docker.com/linux/ubuntu \ $(lsb_release -cs) \ stable" Repository: 'deb [arch=amd64] https://download.docker.com/linux/ubuntu noble stable' Description: Archive for codename: noble components: stable More info: https://download.docker.com/linux/ubuntu Adding repository. Press [ENTER] to continue or Ctrl-c to cancel. Adding deb entry to /etc/apt/sources.list.d/archive_uri-https_download_docker_com_linux_ubuntu-noble.list Adding disabled deb-src entry to /etc/apt/sources.list.d/archive_uri-https_download_docker_com_linux_ubuntu-noble.list Hit:1 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu noble InRelease Hit:2 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu noble-updates InRelease Get:3 https://download.docker.com/linux/ubuntu noble InRelease [48.5 kB] Hit:4 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu noble-backports InRelease Hit:5 http://security.ubuntu.com/ubuntu noble-security InRelease Err:3 https://download.docker.com/linux/ubuntu noble InRelease The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 7EA0A9C3F273FCD8 Reading package lists... Done W: GPG error: https://download.docker.com/linux/ubuntu noble InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 7EA0A9C3F273FCD8 E: The repository 'https://download.docker.com/linux/ubuntu noble InRelease' is not signed. N: Updating from such a repository can't be done securely, and is therefore disabled by default. N: See apt-secure(8) manpage for repository creation and user configuration details.
sudo apt update && sudo apt install -y containerd.io
Hit:1 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu noble InRelease
Hit:2 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu noble-updates InRelease
Get:3 https://download.docker.com/linux/ubuntu noble InRelease [48.5 kB]
Hit:4 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu noble-backports InRelease
Err:3 https://download.docker.com/linux/ubuntu noble InRelease
The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 7EA0A9C3F273FCD8
Hit:5 http://security.ubuntu.com/ubuntu noble-security InRelease
Reading package lists... Done
W: GPG error: https://download.docker.com/linux/ubuntu noble InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 7EA0A9C3F273FCD8
E: The repository 'https://download.docker.com/linux/ubuntu noble InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml
sudo systemctl restart containerd
sudo vi /etc/containerd/config.toml
sudo cat -n /etc/containerd/config.toml
1 version = 3
2 root = '/var/lib/containerd'
3 state = '/run/containerd'
4 temp = ''
5 disabled_plugins = []
6 required_plugins = []
7 oom_score = 0
8 imports = ['/etc/containerd/conf.d/*.toml']
9
10 [grpc]
11 address = '/run/containerd/containerd.sock'
12 tcp_address = ''
13 tcp_tls_ca = ''
14 tcp_tls_cert = ''
15 tcp_tls_key = ''
16 uid = 0
17 gid = 0
18 max_recv_message_size = 16777216
19 max_send_message_size = 16777216
20 tcp_tls_common_name = ''
21
22 [ttrpc]
23 address = ''
24 uid = 0
25 gid = 0
26
27 [debug]
28 address = ''
29 uid = 0
30 gid = 0
31 level = ''
32 format = ''
33
34 [metrics]
35 address = ''
36 grpc_histogram = false
37
38 [plugins]
39 [plugins.'io.containerd.cri.v1.images']
40 snapshotter = 'overlayfs'
41 disable_snapshot_annotations = true
42 discard_unpacked_layers = false
43 max_concurrent_downloads = 3
44 concurrent_layer_fetch_buffer = 0
45 image_pull_progress_timeout = '5m0s'
46 image_pull_with_sync_fs = false
47 stats_collect_period = 10
48 use_local_image_pull = false
49
50 [plugins.'io.containerd.cri.v1.images'.pinned_images]
51 sandbox = 'registry.k8s.io/pause:3.10.1'
52
53 [plugins.'io.containerd.cri.v1.images'.registry]
54 config_path = ''
55
56 [plugins.'io.containerd.cri.v1.images'.image_decryption]
57 key_model = 'node'
58
59 [plugins.'io.containerd.cri.v1.runtime']
60 enable_selinux = false
61 selinux_category_range = 1024
62 max_container_log_line_size = 16384
63 disable_apparmor = false
64 restrict_oom_score_adj = false
65 disable_proc_mount = false
66 unset_seccomp_profile = ''
67 tolerate_missing_hugetlb_controller = true
68 disable_hugetlb_controller = true
69 device_ownership_from_security_context = false
70 ignore_image_defined_volumes = false
71 netns_mounts_under_state_dir = false
72 enable_unprivileged_ports = true
73 enable_unprivileged_icmp = true
74 enable_cdi = true
75 cdi_spec_dirs = ['/etc/cdi', '/var/run/cdi']
76 drain_exec_sync_io_timeout = '0s'
77 ignore_deprecation_warnings = []
78
79 [plugins.'io.containerd.cri.v1.runtime'.containerd]
80 default_runtime_name = 'runc'
81 ignore_blockio_not_enabled_errors = false
82 ignore_rdt_not_enabled_errors = false
83
84 [plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes]
85 [plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.runc]
86 runtime_type = 'io.containerd.runc.v2'
87 runtime_path = ''
88 pod_annotations = []
89 container_annotations = []
90 privileged_without_host_devices = false
91 privileged_without_host_devices_all_devices_allowed = false
92 cgroup_writable = false
93 base_runtime_spec = ''
94 cni_conf_dir = ''
95 cni_max_conf_num = 0
96 snapshotter = ''
97 sandboxer = 'podsandbox'
98 io_type = ''
99
100 [plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.runc.options]
101 BinaryName = ''
102 CriuImagePath = ''
103 CriuWorkPath = ''
104 IoGid = 0
105 IoUid = 0
106 NoNewKeyring = false
107 Root = ''
108 ShimCgroup = ''
109 SystemdCgroup = true
110
111 [plugins.'io.containerd.cri.v1.runtime'.cni]
112 bin_dir = ''
113 bin_dirs = ['/opt/cni/bin']
114 conf_dir = '/etc/cni/net.d'
115 max_conf_num = 1
116 setup_serially = false
117 conf_template = ''
118 ip_pref = ''
119 use_internal_loopback = false
120
121 [plugins.'io.containerd.differ.v1.erofs']
122 mkfs_options = []
123 enable_tar_index = false
124
125 [plugins.'io.containerd.gc.v1.scheduler']
126 pause_threshold = 0.02
127 deletion_threshold = 0
128 mutation_threshold = 100
129 schedule_delay = '0s'
130 startup_delay = '100ms'
131
132 [plugins.'io.containerd.grpc.v1.cri']
133 disable_tcp_service = true
134 stream_server_address = '127.0.0.1'
135 stream_server_port = '0'
136 stream_idle_timeout = '4h0m0s'
137 enable_tls_streaming = false
138
139 [plugins.'io.containerd.grpc.v1.cri'.x509_key_pair_streaming]
140 tls_cert_file = ''
141 tls_key_file = ''
142
143 [plugins.'io.containerd.image-verifier.v1.bindir']
144 bin_dir = '/opt/containerd/image-verifier/bin'
145 max_verifiers = 10
146 per_verifier_timeout = '10s'
147
148 [plugins.'io.containerd.internal.v1.opt']
149 path = '/opt/containerd'
150
151 [plugins.'io.containerd.internal.v1.tracing']
152
153 [plugins.'io.containerd.metadata.v1.bolt']
154 content_sharing_policy = 'shared'
155 no_sync = false
156
157 [plugins.'io.containerd.monitor.container.v1.restart']
158 interval = '10s'
159
160 [plugins.'io.containerd.monitor.task.v1.cgroups']
161 no_prometheus = false
162
163 [plugins.'io.containerd.mount-handler.v1.erofs']
164
165 [plugins.'io.containerd.nri.v1.nri']
166 disable = false
167 socket_path = '/var/run/nri/nri.sock'
168 plugin_path = '/opt/nri/plugins'
169 plugin_config_path = '/etc/nri/conf.d'
170 plugin_registration_timeout = '5s'
171 plugin_request_timeout = '2s'
172 disable_connections = false
173
174 [plugins.'io.containerd.nri.v1.nri'.default_validator]
175 enable = false
176 reject_oci_hook_adjustment = false
177 reject_runtime_default_seccomp_adjustment = false
178 reject_unconfined_seccomp_adjustment = false
179 reject_custom_seccomp_adjustment = false
180 reject_namespace_adjustment = false
181 reject_sysctl_adjustment = false
182 required_plugins = []
183 tolerate_missing_plugins_annotation = ''
184
185 [plugins.'io.containerd.runtime.v2.task']
186 platforms = ['linux/amd64']
187
188 [plugins.'io.containerd.service.v1.diff-service']
189 default = ['walking']
190 sync_fs = false
191
192 [plugins.'io.containerd.service.v1.tasks-service']
193 blockio_config_file = ''
194 rdt_config_file = ''
195
196 [plugins.'io.containerd.shim.v1.manager']
197 env = []
198
199 [plugins.'io.containerd.snapshotter.v1.blockfile']
200 root_path = ''
201 scratch_file = ''
202 fs_type = ''
203 mount_options = []
204 recreate_scratch = false
205
206 [plugins.'io.containerd.snapshotter.v1.btrfs']
207 root_path = ''
208
209 [plugins.'io.containerd.snapshotter.v1.devmapper']
210 root_path = ''
211 pool_name = ''
212 base_image_size = ''
213 async_remove = false
214 discard_blocks = false
215 fs_type = ''
216 fs_options = ''
217
218 [plugins.'io.containerd.snapshotter.v1.erofs']
219 root_path = ''
220 ovl_mount_options = []
221 enable_fsverity = false
222 set_immutable = false
223 default_size = ''
224
225 [plugins.'io.containerd.snapshotter.v1.native']
226 root_path = ''
227
228 [plugins.'io.containerd.snapshotter.v1.overlayfs']
229 root_path = ''
230 upperdir_label = false
231 sync_remove = false
232 slow_chown = false
233 mount_options = []
234
235 [plugins.'io.containerd.snapshotter.v1.zfs']
236 root_path = ''
237
238 [plugins.'io.containerd.tracing.processor.v1.otlp']
239
240 [plugins.'io.containerd.transfer.v1.local']
241 max_concurrent_downloads = 3
242 concurrent_layer_fetch_buffer = 0
243 max_concurrent_uploaded_layers = 3
244 check_platform_supported = false
245 config_path = ''
246 max_concurrent_unpacks = 1
247
248 [cgroup]
249 path = ''
250
251 [timeouts]
252 'io.containerd.timeout.bolt.open' = '0s'
253 'io.containerd.timeout.cri.defercleanup' = '1m0s'
254 'io.containerd.timeout.metrics.shimstats' = '2s'
255 'io.containerd.timeout.shim.cleanup' = '5s'
256 'io.containerd.timeout.shim.load' = '5s'
257 'io.containerd.timeout.shim.shutdown' = '3s'
258 'io.containerd.timeout.task.state' = '2s'
259
260 [stream_processors]
261 [stream_processors.'io.containerd.ocicrypt.decoder.v1.tar']
262 accepts = ['application/vnd.oci.image.layer.v1.tar+encrypted']
263 returns = 'application/vnd.oci.image.layer.v1.tar'
264 path = 'ctd-decoder'
265 args = ['--decryption-keys-path', '/etc/containerd/ocicrypt/keys']
266 env = ['OCICRYPT_KEYPROVIDER_CONFIG=/etc/containerd/ocicrypt/ocicrypt_keyprovider.conf']
267
268 [stream_processors.'io.containerd.ocicrypt.decoder.v1.tar.gzip']
269 accepts = ['application/vnd.oci.image.layer.v1.tar+gzip+encrypted']
270 returns = 'application/vnd.oci.image.layer.v1.tar+gzip'
271 path = 'ctd-decoder'
272 args = ['--decryption-keys-path', '/etc/containerd/ocicrypt/keys']
273 env = ['OCICRYPT_KEYPROVIDER_CONFIG=/etc/containerd/ocicrypt/ocicrypt_keyprovider.conf']
109行目の変更(false⇒true)に関しては、以下に記載あり。
https://kubernetes.io/docs/setup/production-environment/container-runtimes/#containerd-systemd
全Node共通設定⑥ kubeadm、kubelet、kubectlのインストール
※ログは、Master Nodeだけ記載しています。
sudo apt update sudo apt install -y apt-transport-https ca-certificates curl curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.34/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.34/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list sudo apt update sudo apt install -y kubelet kubeadm kubectl sudo apt-mark hold kubelet kubeadm kubectl
全Node共通設定⑦ コントロールプレーンノードのkubeletによって使用されるcgroupドライバーの設定
※ログは、Master Nodeだけ記載しています。
sudo systemctl daemon-reload sudo systemctl restart kubelet
Master Nodeのみ① CNI(Container Networking Interface)の設定
コンテナが稼働するPodネットワークに関する設定をします。
今回は、Calicoを使いました。
参考：https://kubernetes.io/docs/concepts/cluster-administration/addons/
以下にあるcalico.yamlを編集して使用します。
※4601行目・4602行目が変更対象
https://docs.projectcalico.org/manifests/calico.yaml
wget https://docs.projectcalico.org/manifests/calico.yaml sudo vi calico.yaml cat -n calico.yaml | grep 4601 4601 - name: CALICO_IPV4POOL_CIDR cat -n calico.yaml | grep 4602 4602 value: "10.0.0.0/16"
Master Nodeのみ② kubeadm-config.yamlの作成
sudo vi kubeadm-config.yaml sudo cat kubeadm-config.yaml apiVersion: kubeadm.k8s.io/v1beta4 kind: ClusterConfiguration kubernetesVersion: 1.34.9 controlPlaneEndpoint: "ip-10-30-128-84:6443" networking: podSubnet: 10.0.0.0/16
Master Nodeのみ③ kubeadm initの実行
kubeadm initのオプション --v= は出力ログのレベル(数字が大きくなればより多くの情報が出力される)なので、任意です。
sudo kubeadm init --v=5 --config=kubeadm-config.yaml --upload-certs | tee kubeadm-init.outubuntu@ip-10-30-128-84:~$ sudo kubeadm init --v=5 --config=kubeadm-config.yaml --upload-certs | tee kubeadm-init.out
I0704 01:12:03.760774 4252 initconfiguration.go:262] loading configuration from "kubeadm-config.yaml"
I0704 01:12:03.764897 4252 initconfiguration.go:124] detected and using CRI socket: unix:///var/run/containerd/containerd.sock
I0704 01:12:03.765973 4252 interface.go:432] Looking for default routes with IPv4 addresses
I0704 01:12:03.765991 4252 interface.go:437] Default route transits interface "enX0"
I0704 01:12:03.766098 4252 interface.go:209] Interface enX0 is up
I0704 01:12:03.766152 4252 interface.go:257] Interface "enX0" has 2 addresses :[10.30.128.84/24 fe80::40b:f9ff:fe5b:60c1/64].
I0704 01:12:03.766177 4252 interface.go:224] Checking addr 10.30.128.84/24.
I0704 01:12:03.766192 4252 interface.go:231] IP found 10.30.128.84
I0704 01:12:03.766218 4252 interface.go:263] Found valid IPv4 address 10.30.128.84 for interface "enX0".
I0704 01:12:03.766227 4252 interface.go:443] Found active IP 10.30.128.84
I0704 01:12:03.766258 4252 kubelet.go:195] the value of KubeletConfiguration.cgroupDriver is empty; setting it to "systemd"
[init] Using Kubernetes version: v1.34.9
[preflight] Running pre-flight checks
I0704 01:12:03.786788 4252 checks.go:542] validating Kubernetes and kubeadm version
I0704 01:12:03.786830 4252 checks.go:150] validating if the firewall is enabled and active
I0704 01:12:03.810048 4252 checks.go:185] validating availability of port 6443
I0704 01:12:03.810712 4252 checks.go:185] validating availability of port 10259
I0704 01:12:03.810771 4252 checks.go:185] validating availability of port 10257
I0704 01:12:03.810829 4252 checks.go:262] validating the existence of file /etc/kubernetes/manifests/kube-apiserver.yaml
I0704 01:12:03.811332 4252 checks.go:262] validating the existence of file /etc/kubernetes/manifests/kube-controller-manager.yaml
I0704 01:12:03.811424 4252 checks.go:262] validating the existence of file /etc/kubernetes/manifests/kube-scheduler.yaml
I0704 01:12:03.811445 4252 checks.go:262] validating the existence of file /etc/kubernetes/manifests/etcd.yaml
I0704 01:12:03.811471 4252 checks.go:412] validating if the connectivity type is via proxy or direct
I0704 01:12:03.812430 4252 checks.go:451] validating http connectivity to first IP address in the CIDR
I0704 01:12:03.812458 4252 checks.go:451] validating http connectivity to first IP address in the CIDR
I0704 01:12:03.812478 4252 checks.go:86] validating the container runtime
I0704 01:12:03.815226 4252 checks.go:618] validating whether swap is enabled or not
I0704 01:12:03.815336 4252 checks.go:352] validating the presence of executable losetup
I0704 01:12:03.815424 4252 checks.go:352] validating the presence of executable mount
I0704 01:12:03.815453 4252 checks.go:352] validating the presence of executable cp
I0704 01:12:03.815480 4252 checks.go:495] running all checks
I0704 01:12:03.829300 4252 checks.go:383] checking whether the given node name is valid and reachable using net.LookupHost
I0704 01:12:03.829509 4252 checks.go:584] validating kubelet version
I0704 01:12:03.875381 4252 checks.go:112] validating if the "kubelet" service is enabled and active
I0704 01:12:03.891945 4252 checks.go:185] validating availability of port 10250
I0704 01:12:03.892077 4252 checks.go:311] validating the contents of file /proc/sys/net/ipv4/ip_forward
I0704 01:12:03.892135 4252 checks.go:185] validating availability of port 2379
I0704 01:12:03.892191 4252 checks.go:185] validating availability of port 2380
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action beforehand using 'kubeadm config images pull'
I0704 01:12:03.892247 4252 checks.go:225] validating the existence and emptiness of directory /var/lib/etcd
I0704 01:12:03.895258 4252 checks.go:813] using image pull policy: IfNotPresent
I0704 01:12:03.896095 4252 checks.go:825] failed to detect the sandbox image for local container runtime, no 'sandboxImage' field in CRI info config
I0704 01:12:03.898455 4252 checks.go:849] pulling: registry.k8s.io/kube-apiserver:v1.34.9
I0704 01:12:06.115235 4252 checks.go:849] pulling: registry.k8s.io/kube-controller-manager:v1.34.9
I0704 01:12:07.926719 4252 checks.go:849] pulling: registry.k8s.io/kube-scheduler:v1.34.9
I0704 01:12:09.435822 4252 checks.go:849] pulling: registry.k8s.io/kube-proxy:v1.34.9
I0704 01:12:11.236672 4252 checks.go:849] pulling: registry.k8s.io/coredns/coredns:v1.12.1
I0704 01:12:13.063064 4252 checks.go:849] pulling: registry.k8s.io/pause:3.10.1
I0704 01:12:13.528601 4252 checks.go:849] pulling: registry.k8s.io/etcd:3.6.5-0
[certs] Using certificateDir folder "/etc/kubernetes/pki"
I0704 01:12:15.324460 4252 certs.go:111] creating a new certificate authority for ca
[certs] Generating "ca" certificate and key
I0704 01:12:15.499481 4252 certs.go:472] validating certificate period for ca certificate
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [ip-10-30-128-84 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 10.30.128.84]
[certs] Generating "apiserver-kubelet-client" certificate and key
I0704 01:12:15.773908 4252 certs.go:111] creating a new certificate authority for front-proxy-ca
[certs] Generating "front-proxy-ca" certificate and key
I0704 01:12:15.834471 4252 certs.go:472] validating certificate period for front-proxy-ca certificate
[certs] Generating "front-proxy-client" certificate and key
I0704 01:12:15.887187 4252 certs.go:111] creating a new certificate authority for etcd-ca
[certs] Generating "etcd/ca" certificate and key
I0704 01:12:16.061818 4252 certs.go:472] validating certificate period for etcd/ca certificate
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [ip-10-30-128-84 localhost] and IPs [10.30.128.84 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [ip-10-30-128-84 localhost] and IPs [10.30.128.84 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
I0704 01:12:16.469727 4252 certs.go:77] creating new public/private key files for signing service account users
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0704 01:12:16.679676 4252 kubeconfig.go:111] creating kubeconfig file for admin.conf
[kubeconfig] Writing "admin.conf" kubeconfig file
I0704 01:12:17.057762 4252 kubeconfig.go:111] creating kubeconfig file for super-admin.conf
[kubeconfig] Writing "super-admin.conf" kubeconfig file
I0704 01:12:17.267008 4252 kubeconfig.go:111] creating kubeconfig file for kubelet.conf
[kubeconfig] Writing "kubelet.conf" kubeconfig file
I0704 01:12:17.305029 4252 kubeconfig.go:111] creating kubeconfig file for controller-manager.conf
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0704 01:12:17.352931 4252 kubeconfig.go:111] creating kubeconfig file for scheduler.conf
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
I0704 01:12:17.470425 4252 local.go:67] [etcd] wrote Static Pod manifest for a local etcd member to "/etc/kubernetes/manifests/etcd.yaml"
I0704 01:12:17.470463 4252 manifests.go:125] [control-plane] getting StaticPodSpecs
I0704 01:12:17.470638 4252 certs.go:472] validating certificate period for CA certificate
I0704 01:12:17.470714 4252 manifests.go:151] [control-plane] adding volume "ca-certs" for component "kube-apiserver"
I0704 01:12:17.470733 4252 manifests.go:151] [control-plane] adding volume "etc-ca-certificates" for component "kube-apiserver"
I0704 01:12:17.470738 4252 manifests.go:151] [control-plane] adding volume "k8s-certs" for component "kube-apiserver"
I0704 01:12:17.470751 4252 manifests.go:151] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-apiserver"
I0704 01:12:17.470758 4252 manifests.go:151] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
I0704 01:12:17.471538 4252 manifests.go:180] [control-plane] wrote static Pod manifest for component "kube-apiserver" to "/etc/kubernetes/manifests/kube-apiserver.yaml"
I0704 01:12:17.471567 4252 manifests.go:125] [control-plane] getting StaticPodSpecs
I0704 01:12:17.471733 4252 manifests.go:151] [control-plane] adding volume "ca-certs" for component "kube-controller-manager"
I0704 01:12:17.471751 4252 manifests.go:151] [control-plane] adding volume "etc-ca-certificates" for component "kube-controller-manager"
I0704 01:12:17.471757 4252 manifests.go:151] [control-plane] adding volume "flexvolume-dir" for component "kube-controller-manager"
I0704 01:12:17.471770 4252 manifests.go:151] [control-plane] adding volume "k8s-certs" for component "kube-controller-manager"
I0704 01:12:17.471780 4252 manifests.go:151] [control-plane] adding volume "kubeconfig" for component "kube-controller-manager"
I0704 01:12:17.471792 4252 manifests.go:151] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-controller-manager"
I0704 01:12:17.471801 4252 manifests.go:151] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
I0704 01:12:17.472553 4252 manifests.go:180] [control-plane] wrote static Pod manifest for component "kube-controller-manager" to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
I0704 01:12:17.472583 4252 manifests.go:125] [control-plane] getting StaticPodSpecs
I0704 01:12:17.472763 4252 manifests.go:151] [control-plane] adding volume "kubeconfig" for component "kube-scheduler"
I0704 01:12:17.475723 4252 manifests.go:180] [control-plane] wrote static Pod manifest for component "kube-scheduler" to "/etc/kubernetes/manifests/kube-scheduler.yaml"
I0704 01:12:17.475751 4252 kubelet.go:69] Stopping the kubelet
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/instance-config.yaml"
[patches] Applied patch of type "application/strategic-merge-patch+json" to target "kubeletconfiguration"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
I0704 01:12:17.799941 4252 envvar.go:172] "Feature gate default state" feature="InOrderInformers" enabled=true
I0704 01:12:17.799977 4252 envvar.go:172] "Feature gate default state" feature="InformerResourceVersion" enabled=false
I0704 01:12:17.799997 4252 envvar.go:172] "Feature gate default state" feature="WatchListClient" enabled=false
I0704 01:12:17.800004 4252 envvar.go:172] "Feature gate default state" feature="ClientsAllowCBOR" enabled=false
I0704 01:12:17.800011 4252 envvar.go:172] "Feature gate default state" feature="ClientsPreferCBOR" enabled=false
I0704 01:12:17.828017 4252 kubeconfig.go:667] ensuring that the ClusterRoleBinding for the kubeadm:cluster-admins Group exists
I0704 01:12:20.501742 4252 kubeconfig.go:740] creating the ClusterRoleBinding for the kubeadm:cluster-admins Group by using super-admin.conf
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests"
[kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s
[kubelet-check] The kubelet is healthy after 1.402381ms
[control-plane-check] Waiting for healthy control plane components. This can take up to 4m0s
[control-plane-check] Checking kube-apiserver at https://10.30.128.84:6443/livez
[control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz
[control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez
[control-plane-check] kube-controller-manager is healthy after 6.388605ms
[control-plane-check] kube-scheduler is healthy after 7.868486ms
I0704 01:12:21.629234 4252 request.go:1500] "Body was not decodable (unable to check for Status)" err="couldn't get version/kind; json parse error: json: cannot unmarshal array into Go value of type struct { APIVersion string \"json:\\\"apiVersion,omitempty\\\"\"; Kind string \"json:\\\"kind,omitempty\\\"\" }"
[control-plane-check] kube-apiserver is healthy after 1.501163054s
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
I0704 01:12:22.126758 4252 uploadconfig.go:111] [upload-config] Uploading the kubeadm ClusterConfiguration to a ConfigMap
[kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
I0704 01:12:22.140104 4252 uploadconfig.go:125] [upload-config] Uploading the kubelet component config to a ConfigMap
[upload-certs] Storing the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace
[upload-certs] Using certificate key:
d78666a09bf6f7fc6182c815f3515c2024bcb0ef4374728dd76e786f4ea5eac0
[mark-control-plane] Marking the node ip-10-30-128-84 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
[mark-control-plane] Marking the node ip-10-30-128-84 as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]
[bootstrap-token] Using token: to1ywy.of2w7bq2cvdgekyp
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
I0704 01:12:22.334231 4252 request.go:683] "Waited before sending request" delay="120.246477ms" reason="client-side throttling, not priority and fairness" verb="POST" URL="https://10.30.128.84:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s"
[bootstrap-token] Configured RBAC rules to allow the API server kubelet client certificate to access the kubelet API
I0704 01:12:22.534577 4252 request.go:683] "Waited before sending request" delay="196.431683ms" reason="client-side throttling, not priority and fairness" verb="POST" URL="https://10.30.128.84:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s"
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
I0704 01:12:22.538326 4252 clusterinfo.go:58] [bootstrap-token] copying the cluster from admin.conf to the bootstrap kubeconfig
I0704 01:12:22.538622 4252 clusterinfo.go:70] [bootstrap-token] creating/updating ConfigMap in kube-public namespace
I0704 01:12:22.543761 4252 clusterinfo.go:84] creating the RBAC rules for exposing the cluster-info ConfigMap in the kube-public namespace
I0704 01:12:22.734179 4252 request.go:683] "Waited before sending request" delay="190.313118ms" reason="client-side throttling, not priority and fairness" verb="POST" URL="https://10.30.128.84:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-public/roles?timeout=10s"
I0704 01:12:22.934249 4252 request.go:683] "Waited before sending request" delay="196.299496ms" reason="client-side throttling, not priority and fairness" verb="POST" URL="https://10.30.128.84:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-public/rolebindings?timeout=10s"
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
I0704 01:12:22.937698 4252 kubeletfinalize.go:90] [kubelet-finalize] Assuming that kubelet client certificate rotation is enabled: found "/var/lib/kubelet/pki/kubelet-client-current.pem"
I0704 01:12:22.938593 4252 kubeletfinalize.go:144] [kubelet-finalize] Restarting the kubelet to enable client certificate rotation
[addons] Applied essential addon: CoreDNS
I0704 01:12:23.530169 4252 request.go:683] "Waited before sending request" delay="121.359827ms" reason="client-side throttling, not priority and fairness" verb="POST" URL="https://10.30.128.84:6443/api/v1/namespaces/kube-system/serviceaccounts?timeout=10s"
I0704 01:12:23.734764 4252 request.go:683] "Waited before sending request" delay="196.371299ms" reason="client-side throttling, not priority and fairness" verb="POST" URL="https://10.30.128.84:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s"
I0704 01:12:23.934976 4252 request.go:683] "Waited before sending request" delay="196.342482ms" reason="client-side throttling, not priority and fairness" verb="POST" URL="https://10.30.128.84:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s"
[addons] Applied essential addon: kube-proxy
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
You can now join any number of control-plane nodes running the following command on each as root:
kubeadm join ip-10-30-128-84:6443 --token to1ywy.of2w7bq2cvdgekyp \
--discovery-token-ca-cert-hash sha256:c2ca44d803df7f36bdebf70b08c49fe2d89ea681c5f02e44e82ff6e5b814a0f0 \
--control-plane --certificate-key d78666a09bf6f7fc6182c815f3515c2024bcb0ef4374728dd76e786f4ea5eac0
Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use
"kubeadm init phase upload-certs --upload-certs" to reload certs afterward.
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join ip-10-30-128-84:6443 --token to1ywy.of2w7bq2cvdgekyp \
--discovery-token-ca-cert-hash sha256:c2ca44d803df7f36bdebf70b08c49fe2d89ea681c5f02e44e82ff6e5b814a0f0
Master Nodeのみ③ kubernetesクラスター認証情報の作成
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
ls
calico.yaml kubeadm-config.yaml kubeadm-init.out
cat $HOME/.kube/configcat $HOME/.kube/config
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURCVENDQWUyZ0F3SUJBZ0lJWUxCaDB5eG9yQmd3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TmpBM01EUXdNVEEzTUROYUZ3MHpOakEzTURFd01URXlNRE5hTUJVeApFekFSQmdOVkJBTVRDbXQxWW1WeWJtVjBaWE13Z2dFaU1BMEdDU3FHU0liM0RRRUJBUVVBQTRJQkR3QXdnZ0VLCkFvSUJBUUMvbzA4RThhWVJiTzQvQnVqSE9VOUpwM1g3RnczVk13aGJ4Uy9DL21zQUltc2wzZWs2aUZnMUxudGMKdGhvaXFlVkp0dHBueHVmT3BiOG9vVG05ZG9XMlV6RmFxeUR0T0d2SWpJc1dBWnpPRkpjc3NMSlowT1NTcjRjQQplQVNzNUg0b3lJNHVwM1Q3ZXRFb1ZQQVlFb3FkaWJtNkhSMmlqSlg1MjVDUUZWaWQvMS9oZUJZb1J3VjVsM1V6CnV3Wjc5azZQSlRmc2FramV4NlZtUE8rZ2l4d2NtYkpJOU1jdS95S0QxOVQ0WVlKdVUxVjZZcmRhYnAzN2VMWUcKdW9qYkxMWGVsdENHbHNsZHpsRkFCWlNVOE1IR3gwdkdKNmNBcE5neFZRdzYyOWVFbWN1Z0wvYzFOZVhBZHJQLwpnTCtoblVTY0J0dk0rU1llNUVtMlR0OXgzSEZCQWdNQkFBR2pXVEJYTUE0R0ExVWREd0VCL3dRRUF3SUNwREFQCkJnTlZIUk1CQWY4RUJUQURBUUgvTUIwR0ExVWREZ1FXQkJRUTRxVUVuaXlhRXJEOUFrRE1ERUlmb0U5NFF6QVYKQmdOVkhSRUVEakFNZ2dwcmRXSmxjbTVsZEdWek1BMEdDU3FHU0liM0RRRUJDd1VBQTRJQkFRQWQ2T1V2UUs2YwpuMkRqN0ozQXA5L0IyTWFSZkJUYitaTk5OYjhoZDhTMGZmcUlVMDlESDgyYUdidVRCcjN0SkZ1c2Z4bWc4QXQ0Cks4eVJaUDA3dmpFT3Uxa0JsTnV1ZWpNZmV6djJwZnFmUHptMzZUQ0RXS3loVnlnQ3hLQjRzU2kwUDRuVDYycmEKeGF5Z0xwa1I0cVVPTm5SQVpRSEJWMTMwM3RRYjNKTUpEQ3YwYjJkdWhDejZ4NjZHdlZxNVV3SldjKzg2VWhhYgp6dEJxQ1dncElCMjE4OTc4d3JWRlFFV25hK3J1dkc1TUpxNXVWclE2eHk1dW92bjBjdEZocVBUdTl5YzY3a0dmCnlOWjQxb3RSanpTdThLZ0hWY0pYSVZ3NW5GK1JpVDlleDNGeG41TDIrSU1ySmtDZTFmU2syOXp3bUdKSnF3dEwKSXJrMVdtbkpvUVc3Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
server: https://ip-10-30-128-84:6443
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: kubernetes-admin
name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
users:
- name: kubernetes-admin
user:
client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURLVENDQWhHZ0F3SUJBZ0lJQnRkL2g0VzFnZXN3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TmpBM01EUXdNVEEzTUROYUZ3MHlOekEzTURRd01URXlNRE5hTUR3eApIekFkQmdOVkJBb1RGbXQxWW1WaFpHMDZZMngxYzNSbGNpMWhaRzFwYm5NeEdUQVhCZ05WQkFNVEVHdDFZbVZ5CmJtVjBaWE10WVdSdGFXNHdnZ0VpTUEwR0NTcUdTSWIzRFFFQkFRVUFBNElCRHdBd2dnRUtBb0lCQVFDd0dMSHcKWHh4eldYWjllWlhuS09BVjU5dnhGdUxCMFI1TmE5TjNRaVpRZFJ6K0RDeW5mUmJyYVBLQ2dWWVNxS2JEeTlWWQpXd1QzMkVhT1pxU1MvQlcvcktqYjVlK01VNVNrOFdFcWltNmlLNE1INFdOejMxTTRGcmV5d3VlaGtWRHJaaTd3ClFHMXhrVEhJREJQblF3QnpGWGEzTzc1WDJkMGVlQ2tHSmdvNUhsVjE1b2g5a2srTE9EQUZxM1RDanh3eG9EbEcKTlRPaERnNEZtOUM2d2NLMlg3TE9taEtSYUtRcWNUTHlPaG5nM1QybTJ4em1aY1o2NzNrNkpqOTM0VnlvbFhLWAplU1VlczN4amg2M1hGZWMwVmhaUjRDb21SOUNQUW1DbytKVjlmREdWSXYyMDl2SXhoOWk1Q1V6S1l2cUpPSjFQCkg3bVFSQU5VanFnWVpEK3BBZ01CQUFHalZqQlVNQTRHQTFVZER3RUIvd1FFQXdJRm9EQVRCZ05WSFNVRUREQUsKQmdnckJnRUZCUWNEQWpBTUJnTlZIUk1CQWY4RUFqQUFNQjhHQTFVZEl3UVlNQmFBRkJEaXBRU2VMSm9Tc1AwQwpRTXdNUWgrZ1QzaERNQTBHQ1NxR1NJYjNEUUVCQ3dVQUE0SUJBUUJlaHRtUTVHbGRvRVZNYUcvOGdRN0ZqTEZBCndYaXI1WEpFQnZmTW5vbzgxVENzWmlNN0RQRXdKakRuS3FKTWJYK2NoS0tVV09SZXl1QVN3aEVEZnpPVHN0bXgKVjBzTTV2eXZraWprTVNvZE5WMzYwelAycndXVnpoQStsdlRCUDlyaVJGYlJXKytlNE55bEduVktpYjIyeFh2cwpqTWZ2WG54UThYQm5CTk82anRyRzBmYjRWcGpOTitZQUtQMFE2bXc3dUZ0dHNreEdBWlV0bzV4RTM4YzFTNG9SCjdOSWUrSk5tem9OU01VWVluNWN4WXNwRWprU2FhWTJqZHJiamtxU0ZtL3p3bVZmQjQrdGlyMllVQUJuR1BXS08KZUpneW04UDloYXFIeWgvamZoWWRPaUJiRVhFaDZLS3NmOUtPSmNRVFFVU0ltMnFCR0ttaTB0NlRLSG9iCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBc0JpeDhGOGNjMWwyZlhtVjV5amdGZWZiOFJiaXdkRWVUV3ZUZDBJbVVIVWMvZ3dzCnAzMFc2Mmp5Z29GV0VxaW13OHZWV0ZzRTk5aEdqbWFra3Z3VnY2eW8yK1h2akZPVXBQRmhLb3B1b2l1REIrRmoKYzk5VE9CYTNzc0xub1pGUTYyWXU4RUJ0Y1pFeHlBd1Q1ME1BY3hWMnR6dStWOW5kSG5ncEJpWUtPUjVWZGVhSQpmWkpQaXpnd0JhdDB3bzhjTWFBNVJqVXpvUTRPQlp2UXVzSEN0bCt5enBvU2tXaWtLbkV5OGpvWjROMDlwdHNjCjVtWEdldTk1T2lZL2QrRmNxSlZ5bDNrbEhyTjhZNGV0MXhYbk5GWVdVZUFxSmtmUWowSmdxUGlWZlh3eGxTTDkKdFBieU1ZZll1UWxNeW1MNmlUaWRUeCs1a0VRRFZJNm9HR1EvcVFJREFRQUJBb0lCQUFSTHlKb3d5bnk2bG9HYQpJK3VSOE9HU21LZzBPRHBQNlRtQitMODU2R2hod0dBbTdqZEoweDRteWNwMW12Z2Y1WUwwL1QxSGFkZ2RZOG51CmtYT2FpSEgyMERSejBzUEN6WklMNTNuR1lIWStzZGJSTmdBVTF5NjduZENwSVBDQ256aEJubXlJWXkzTm1TclAKREo5Qml3aHVZaDZNbG1pd2w4eVBWRG8zckZ2WFZMSXlocS8xSjBGYWV3WWRkRUt5ZldXMHdlaGlIckplR3ZTNgp5L0JTWEFROWlDbjFOSC8wNnVLOS82V1FrakN0aVhZckdsVDBSOGxlVURKNDBCK25WWVVhVVRYVWhXaCtNTHAzCm80T1VsU2tDQmFiOGVjcHZ2L0d4a005YjJOd1NxZmsrbzZlVUdVc2MvWkovQ3BrU0dFRENydTlIVjVDTCtDZWgKVzYxNlY4Y0NnWUVBd0RwRFlpeXNxNU05YmkxUzljL0NJNE8zUEZpL1F6ck5zVW1iNzhoblM5RTh1Z2Fha1BVQQpxR0UvbFZ5VWhLbmZOYnRVZ3hQQzlQZHhHVWdsSHhpQURsbXprTWlEcC9zeGl1RzR2TitTNnRvQ1pRMXVkRjBsClhkZGdTVzZNMUgyWVFIb3dLekFlait5VzhibmQvY2RyMXJVbGgrS1JHZVBJRUhPQ0xFNlRqMU1DZ1lFQTZvUnQKbSt5aTFuVXVBNk5GUDVrWVh3cnB4VFlLNDJEclVLd0hCOEdNMGdNZndoT0tjT2YrbVR3ejdhalRjVE82RjZpZApCR1htRlE4Z1M3TEJ0SFdRSTVFcmlmekxtMWhiWmwzMnB6bUVoWTd6RUNLY3c0c0hpbnBldmJWV3Q0bnZGdFBQCjJsdEJIdWdBVEJYK2lub2RZZm9WOWhMdGNCdzJxbjJJdlB0cjRaTUNnWUFvSXdJMjJZVWtQaFl4angwblhqMzMKRkdXS0lDMnZaL1BTOUwrdWpYV2tDd2hXLzFIK3VtbHN2d0Y3dWx6NFprL0k5Rmh0Rmc4Ni92K0tlOW5rYUVVMAp1ZDRZZVFXa2FYcXF1UDdOMFJEb0lLVW5YR3FYdVdzMzhPU1Y4Sk5QT3ZvelhFLzZkSTJ0RjRWYzY2WkNld1UxClM1V09qZXdtTDcxeVVldjRQRi9yS1FLQmdRQzhjOTZBeGcrMHRrVmJ0d0FXc3pmYWFES2l2MkxRaFRtVGdpbkUKR2N6SGxxc1drZzFWM1pmQ0c1YS9ERU9kWEN2V2h6Qm1NRnhMMy9QYjRMRzRDaURoOCs1QTNGdmhZWWRSSTREcApzRmJGM0xHRjdjcDdlR1YrNmdibis2akxwSlAwakJFakx1R05ySEZRc3Z2dzIyQnhNejRpbCtNbGpBYUJ5bzd2CkRqc0s4UUtCZ0djUXM4bXdObUQ3TlNHWjFUczhxWXNHRDhiTmxkU1BNTHdqbkYxYUkzU3hKTVg5ZDltOG1iVjUKb1pZTWpkbE16UVVEdGc5bUppTnNlbUk0TFpLZTB5ak95d2dOZHFvTitUb1JqZ3VKNnFHTlhpK2diUUMxb0txOQp3dnRZajJMNnVSTlFMQytsQ1R6RldlYkFMOTIwbVNaQzF3aHlIZk9LRmt2VjZpQkY3T3QyCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==
kubectlコマンドが実行できるようになりました。
※MasterNodeは、まだSTATUS "NotReady"。
kubectl get nodes NAME STATUS ROLES AGE VERSION ip-10-30-128-84 NotReady control-plane 2m9s v1.34.9
Master Nodeのみ④ Calicoを起動
作成済みのcalico.yamlをapplyします。
※"Running"になるまで、数分かかります。
kubectl apply -f calico.yamlkubectl apply -f calico.yaml poddisruptionbudget.policy/calico-kube-controllers created serviceaccount/calico-kube-controllers created serviceaccount/calico-node created configmap/calico-config created customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created clusterrole.rbac.authorization.k8s.io/calico-kube-controllers created clusterrole.rbac.authorization.k8s.io/calico-node created clusterrolebinding.rbac.authorization.k8s.io/calico-kube-controllers created clusterrolebinding.rbac.authorization.k8s.io/calico-node created daemonset.apps/calico-node created deployment.apps/calico-kube-controllers created
MasterNodeが、"Ready"になりました。
kubectl get pods -n kube-system -w NAME READY STATUS RESTARTS AGE calico-kube-controllers-b45f49df6-gbxhl 1/1 Running 0 39s calico-node-wtxh8 1/1 Running 0 39s coredns-66bc5c9577-2d44f 1/1 Running 0 2m52s coredns-66bc5c9577-8hk4p 1/1 Running 0 2m52s etcd-ip-10-30-128-84 1/1 Running 0 3m kube-apiserver-ip-10-30-128-84 1/1 Running 0 3m kube-controller-manager-ip-10-30-128-84 1/1 Running 0 3m kube-proxy-r2v2c 1/1 Running 0 2m53s kube-scheduler-ip-10-30-128-84 1/1 Running 0 3m kubectl get nodes NAME STATUS ROLES AGE VERSION ip-10-30-128-84 Ready control-plane 3m11s v1.34.9
kubectl get nodes ip-10-30-128-84 -o yaml
apiVersion: v1
kind: Node
metadata:
annotations:
node.alpha.kubernetes.io/ttl: "0"
projectcalico.org/IPv4Address: 10.30.128.84/24
projectcalico.org/IPv4IPIPTunnelAddr: 10.0.62.128
volumes.kubernetes.io/controller-managed-attach-detach: "true"
creationTimestamp: "2026-07-04T01:12:20Z"
labels:
beta.kubernetes.io/arch: amd64
beta.kubernetes.io/os: linux
kubernetes.io/arch: amd64
kubernetes.io/hostname: ip-10-30-128-84
kubernetes.io/os: linux
node-role.kubernetes.io/control-plane: ""
node.kubernetes.io/exclude-from-external-load-balancers: ""
name: ip-10-30-128-84
resourceVersion: "806"
uid: 02782ed9-00ca-400a-8137-67c9430605e6
spec:
podCIDR: 10.0.0.0/24
podCIDRs:
- 10.0.0.0/24
taints:
- effect: NoSchedule
key: node-role.kubernetes.io/control-plane
status:
addresses:
- address: 10.30.128.84
type: InternalIP
- address: ip-10-30-128-84
type: Hostname
allocatable:
cpu: "4"
ephemeral-storage: "45795697384"
hugepages-1Gi: "0"
hugepages-2Mi: "0"
memory: 16268644Ki
pods: "110"
capacity:
cpu: "4"
ephemeral-storage: 49691512Ki
hugepages-1Gi: "0"
hugepages-2Mi: "0"
memory: 16371044Ki
pods: "110"
conditions:
- lastHeartbeatTime: "2026-07-04T01:15:02Z"
lastTransitionTime: "2026-07-04T01:15:02Z"
message: Calico is running on this node
reason: CalicoIsUp
status: "False"
type: NetworkUnavailable
- lastHeartbeatTime: "2026-07-04T01:15:27Z"
lastTransitionTime: "2026-07-04T01:12:19Z"
message: kubelet has sufficient memory available
reason: KubeletHasSufficientMemory
status: "False"
type: MemoryPressure
- lastHeartbeatTime: "2026-07-04T01:15:27Z"
lastTransitionTime: "2026-07-04T01:12:19Z"
message: kubelet has no disk pressure
reason: KubeletHasNoDiskPressure
status: "False"
type: DiskPressure
- lastHeartbeatTime: "2026-07-04T01:15:27Z"
lastTransitionTime: "2026-07-04T01:12:19Z"
message: kubelet has sufficient PID available
reason: KubeletHasSufficientPID
status: "False"
type: PIDPressure
- lastHeartbeatTime: "2026-07-04T01:15:27Z"
lastTransitionTime: "2026-07-04T01:15:07Z"
message: kubelet is posting ready status
reason: KubeletReady
status: "True"
type: Ready
daemonEndpoints:
kubeletEndpoint:
Port: 10250
features:
supplementalGroupsPolicy: true
images:
- names:
- docker.io/calico/cni@sha256:a38d53cb8688944eafede2f0eadc478b1b403cefeff7953da57fe9cd2d65e977
- docker.io/calico/cni:v3.25.0
sizeBytes: 87984941
- names:
- docker.io/calico/node@sha256:a85123d1882832af6c45b5e289c6bb99820646cb7d4f6006f98095168808b1e6
- docker.io/calico/node:v3.25.0
sizeBytes: 87185935
- names:
- docker.io/calico/kube-controllers@sha256:c45af3a9692d87a527451cf544557138fedf86f92b6e39bf2003e2fdb848dce3
- docker.io/calico/kube-controllers:v3.25.0
sizeBytes: 31271800
- names:
- registry.k8s.io/kube-apiserver@sha256:c63e3de35256e066a9462c8783667e85e18833d0098ebc1669d3315079a30c39
- registry.k8s.io/kube-apiserver:v1.34.9
sizeBytes: 27137304
- names:
- registry.k8s.io/kube-proxy@sha256:501b4d94d65ada1d491869d96c6f0ea4deb5a4766a2986a8ec3ff9859160c964
- registry.k8s.io/kube-proxy:v1.34.9
sizeBytes: 25998966
- names:
- registry.k8s.io/etcd@sha256:042ef9c02799eb9303abf1aa99b09f09d94b8ee3ba0c2dd3f42dc4e1d3dce534
- registry.k8s.io/etcd:3.6.5-0
sizeBytes: 22871747
- names:
- registry.k8s.io/kube-controller-manager@sha256:5166aca14a5c126b2920b9b05fe21a254cd89e0748c520bb9009ae5b5971a200
- registry.k8s.io/kube-controller-manager:v1.34.9
sizeBytes: 22848468
- names:
- registry.k8s.io/coredns/coredns@sha256:e8c262566636e6bc340ece6473b0eed193cad045384401529721ddbe6463d31c
- registry.k8s.io/coredns/coredns:v1.12.1
sizeBytes: 22384805
- names:
- registry.k8s.io/kube-scheduler@sha256:79a157c52816760d4868db0ce1a2287dc82f19ca3f9e35da17a74a76c0a7fb4a
- registry.k8s.io/kube-scheduler:v1.34.9
sizeBytes: 17398629
- names:
- registry.k8s.io/pause@sha256:278fb9dbcca9518083ad1e11276933a2e96f23de604a3a08cc3c80002767d24c
- registry.k8s.io/pause:3.10.1
sizeBytes: 320448
nodeInfo:
architecture: amd64
bootID: 41cfd061-ceeb-41d9-9735-727c7dacbd38
containerRuntimeVersion: containerd://2.2.5
kernelVersion: 6.17.0-1019-aws
kubeProxyVersion: ""
kubeletVersion: v1.34.9
machineID: ec21a8a5a69de725ed78d2b1f1f04ba1
operatingSystem: linux
osImage: Ubuntu 24.04.4 LTS
systemUUID: ec29f386-2cd4-dd5c-e420-4e1ef33b3de0
runtimeHandlers:
- features:
recursiveReadOnlyMounts: true
userNamespaces: true
name: ""
- features:
recursiveReadOnlyMounts: true
userNamespaces: true
name: runc
Worker Nodeのみ kubernetesクラスターへの参加(join)
Worker Nodeで実施します。
sudo kubeadm join ip-10-30-128-84:6443 --token to1ywy.of2w7bq2cvdgekyp --discovery-token-ca-cert-hash sha256:c2ca44d803df7f36bdebf70b08c49fe2d89ea681c5f02e44e82ff6e5b814a0f0ubuntu@ip-10-30-128-216:~$ kubeadm join ip-10-30-128-84:6443 --token to1ywy.of2w7bq2cvdgekyp --discovery-token-ca-cert-hash sha256:c2ca44d803df7f36bdebf70b08c49fe2d89ea681c5f02e44e82ff6e5b814a0f0 [preflight] Running pre-flight checks [preflight] Reading configuration from the "kubeadm-config" ConfigMap in namespace "kube-system"... [preflight] Use 'kubeadm init phase upload-config kubeadm --config your-config-file' to re-upload it. W0704 01:28:50.402280 9420 utils.go:69] The recommended value for "bindAddress" in "KubeProxyConfiguration" is: ::; the provided value is: 0.0.0.0 [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/instance-config.yaml" [patches] Applied patch of type "application/strategic-merge-patch+json" to target "kubeletconfiguration" [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Starting the kubelet [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s [kubelet-check] The kubelet is healthy after 1.000866237s [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap This node has joined the cluster: * Certificate signing request was sent to apiserver and a response was received. * The Kubelet was informed of the new secure connection details. Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
Master NodeのCLIで確認すると、kubernetesクラスラーに参加出来ていることが分かります。
kubectl get nodes -w NAME STATUS ROLES AGE VERSION ip-10-30-128-216 Ready <none> 17s v1.34.9 ip-10-30-128-84 Ready control-plane 16m v1.34.9
Master Nodeのみ Pods(コンテナ)の起動
正常にコンテナが起動出来るかを確認するため、
nginxのコンテナを使って確かめてみます。
kubectl run nginx --image=nginx --dry-run=client -o yaml > nginx.yaml
ls
calico.yaml kubeadm-config.yaml kubeadm-init.out nginx.yaml
cat nginx.yaml
apiVersion: v1
kind: Pod
metadata:
labels:
run: nginx
name: nginx
spec:
containers:
- image: nginx
name: nginx
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
kubectl apply -f nginx.yamlkubectl apply -f nginx.yaml
pod/nginx created
コンテナにPod用のIPv4が付与されているのが確認できます。
kubectl get pods -w-o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nginx 1/1 Running 0 12s 10.0.145.65 ip-10-30-128-216 <none> <none>
PodにアサインされているIPv4へのアドレスにアクセス成功。
curl 10.0.145.6510.0.145.65
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, nginx is successfully installed and working.
Further configuration is required for the web server, reverse proxy,
API gateway, load balancer, content cache, or other features.</p>
<p>For online documentation and support please refer to
<a href="https://nginx.org/">nginx.org</a>.<br/>
To engage with the community please visit
<a href="https://community.nginx.org/">community.nginx.org</a>.<br/>
For enterprise grade support, professional services, additional
security features and capabilities please refer to
<a href="https://f5.com/nginx">f5.com/nginx</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
今回の作業での気づきなど
以前にversion1.26/1.27のクラスターを構築して依頼の作業でしたが、APIバージョンやURLなど一部変更があることが発見出来ました。
また、新機能も追加されているので色々試してみたいと思います。
