LinuCエヴァンジェリストの鯨井貴博@opensourcetechです。
はじめに
Dual Stack環境のkubernetes(v1.26.0)におけるServiceの挙動(IPv4とIPv6のどっちが有効になるの？)に関するメモです。
なお、Dual Stack環境のkubernetesクラスターはこちらで構築したものを使っています。
やってみること
以下のようにkubernetes上にnginxのDeploymentsがあるので、
それを公開(Service)して挙動を確かめます。
※kubectl create deployments nginx --image=nginx:1.23.2
kubeuser@master01:~$ kubectl get nodes -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME master01 Ready control-plane 158m v1.26.0 192.168.1.41 <none> Ubuntu 22.04.2 LTS 5.15.0-67-generic containerd://1.6.18 worker01 Ready <none> 155m v1.26.0 192.168.1.45 <none> Ubuntu 22.04.2 LTS 5.15.0-67-generic containerd://1.6.18 worker02 Ready <none> 155m v1.26.0 192.168.1.46 <none> Ubuntu 22.04.2 LTS 5.15.0-67-generic containerd://1.6.18
kubeuser@master01:~$ kubectl get deployments.apps
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 1/1 1 1 147m
kubeuser@master01:~$ kubectl get deployments.apps nginx -o yaml
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "1"
creationTimestamp: "2023-03-14T13:50:35Z"
generation: 1
labels:
app: nginx
name: nginx
namespace: default
resourceVersion: "1156"
uid: 9083fb18-721d-4102-ae2d-100ebae18536
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app: nginx
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
creationTimestamp: null
labels:
app: nginx
spec:
containers:
- image: nginx:1.23.2
imagePullPolicy: IfNotPresent
name: nginx
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 30
status:
availableReplicas: 1
conditions:
- lastTransitionTime: "2023-03-14T13:50:59Z"
lastUpdateTime: "2023-03-14T13:50:59Z"
message: Deployment has minimum availability.
reason: MinimumReplicasAvailable
status: "True"
type: Available
- lastTransitionTime: "2023-03-14T13:50:35Z"
lastUpdateTime: "2023-03-14T13:50:59Z"
message: ReplicaSet "nginx-759fd59899" has successfully progressed.
reason: NewReplicaSetAvailable
status: "True"
type: Progressing
observedGeneration: 1
readyReplicas: 1
replicas: 1
updatedReplicas: 1
kubeuser@master01:~$ kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-759fd59899-59sgs 1/1 Running 0 147m 10.0.30.65 worker02 <none> <none>
kubeuser@master01:~$ kubectl get pods nginx-759fd59899-59sgs -o go-template --template='{{range .status.podIPs}}{{printf "%s\n" .ip}}{{end}}'
10.0.30.65
fd12:b5e0:383e:0:7bf:50a7:b256:1e40
kubeuser@master01:~$ curl http://[fd12:b5e0:383e:0:7bf:50a7:b256:1e40]
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
ipFamilies(IPv4/IPv6)指定なし
まずは、以下のService用のマニュフェストを使います。
特に、どのipFamilies(IPv4/IPv6)を指定していません。
kubeuser@master01:~$ cat svc_nginx_ipv4.yaml
apiVersion: v1
kind: Service
metadata:
labels:
app: nginx
name: nginx
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
結果：IPv4がアサインされる
kubeuser@master01:~$ kubectl apply -f svc_nginx_ipv4.yaml
service/nginx configured
kubeuser@master01:~$ kubectl get svc nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx ClusterIP 10.1.179.244 <none> 80/TCP 158m
kubeuser@master01:~$ kubectl get svc nginx -o yaml
apiVersion: v1
kind: Service
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app":"nginx"},"name":"nginx","namespace":"default"},"spec":{"ports":[{"port":80,"protocol":"TCP","targetPort":80}],"selector":{"app":"nginx"}}}
creationTimestamp: "2023-03-14T13:55:35Z"
labels:
app: nginx
name: nginx
namespace: default
resourceVersion: "15706"
uid: 97bbff5e-a1ad-4410-a9cb-267defd83fc3
spec:
clusterIP: 10.1.179.244
clusterIPs:
- 10.1.179.244
- fd12:b5e0:383f::2261
internalTrafficPolicy: Cluster
ipFamilies:
- IPv4
- IPv6
ipFamilyPolicy: PreferDualStack
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
sessionAffinity: None
type: ClusterIP
status:
loadBalancer: {}
ipFamilies(IPv6指定) & ipFamilyPolicy: SingleStack
まずは、以下のService用のマニュフェストを使います。
ipFamiliesでIPv6を指定します。
kubeuser@master01:~$ cat svc_nginx_ipv6.yaml
apiVersion: v1
kind: Service
metadata:
labels:
app: nginx
name: nginx
spec:
ipFamilies:
- IPv6
ipFamilyPolicy: SingleStack
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
結果：IPv6がアサインされる
kubeuser@master01:~$ kubectl apply -f svc_nginx_ipv6.yaml
service/nginx created
kubeuser@master01:~$ kubectl get svc nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx ClusterIP fd12:b5e0:383f::f75b <none> 80/TCP 5s
kubeuser@master01:~$ kubectl get svc nginx -o yaml
apiVersion: v1
kind: Service
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app":"nginx"},"name":"nginx","namespace":"default"},"spec":{"ipFamilies":["IPv6"],"ipFamilyPolicy":"SingleStack","ports":[{"port":80,"protocol":"TCP","targetPort":80}],"selector":{"app":"nginx"}}}
creationTimestamp: "2023-03-14T16:37:41Z"
labels:
app: nginx
name: nginx
namespace: default
resourceVersion: "16046"
uid: 9b2b69ec-fc71-4873-813b-9ab59fbdcc95
spec:
clusterIP: fd12:b5e0:383f::f75b
clusterIPs:
- fd12:b5e0:383f::f75b
internalTrafficPolicy: Cluster
ipFamilies:
- IPv6
ipFamilyPolicy: SingleStack
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
sessionAffinity: None
type: ClusterIP
status:
loadBalancer: {}
ipFamilies(IPv4/IPv6指定) & ipFamilyPolicy: PreferDualStack
まずは、以下のService用のマニュフェストを使います。
ipFamiliesでIPv4/IPv6を指定します。
また、ipFamilyPolicy: PreferDualStackとしています。
kubeuser@master01:~$ cat svc_nginx_dual_prefer.yaml
apiVersion: v1
kind: Service
metadata:
labels:
app: nginx
name: nginx
spec:
ipFamilies:
- IPv4
- IPv6
ipFamilyPolicy: PreferDualStack
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
結果：IPv4/IPv6(Dual Stack)となる
kubeuser@master01:~$ kubectl apply -f svc_nginx_dual_prefer.yaml
service/nginx created
kubeuser@master01:~$ kubectl get svc nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx ClusterIP 10.1.49.119 <none> 80/TCP 5s
kubeuser@master01:~$ kubectl get svc nginx -o yaml
apiVersion: v1
kind: Service
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app":"nginx"},"name":"nginx","namespace":"default"},"spec":{"ipFamilies":["IPv4","IPv6"],"ipFamilyPolicy":"PreferDualStack","ports":[{"port":80,"protocol":"TCP","targetPort":80}],"selector":{"app":"nginx"}}}
creationTimestamp: "2023-03-14T16:41:24Z"
labels:
app: nginx
name: nginx
namespace: default
resourceVersion: "16387"
uid: 164a6826-54ea-41f2-8820-75a6ca0fb1c1
spec:
clusterIP: 10.1.49.119
clusterIPs:
- 10.1.49.119
- fd12:b5e0:383f::f2e2
internalTrafficPolicy: Cluster
ipFamilies:
- IPv4
- IPv6
ipFamilyPolicy: PreferDualStack
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
sessionAffinity: None
type: ClusterIP
status:
loadBalancer: {}
ipFamilies(IPv4/IPv6指定) & ipFamilyPolicy: RequireDualStack
まずは、以下のService用のマニュフェストを使います。
ipFamiliesでIPv4/IPv6を指定します。
また、ipFamilyPolicy: RequireDualStackとしています。
kubeuser@master01:~$ cat svc_nginx_dual_reequire.yaml
apiVersion: v1
kind: Service
metadata:
labels:
app: nginx
name: nginx
spec:
ipFamilies:
- IPv4
- IPv6
ipFamilyPolicy: RequireDualStack
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
結果：IPv4/IPv6(Dual Stack)となる
kubeuser@master01:~$ kubectl apply -f svc_nginx_dual_reequire.yaml
service/nginx created
kubeuser@master01:~$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.1.0.1 <none> 443/TCP 3h12m
nginx ClusterIP 10.1.223.177 <none> 80/TCP 5s
kubeuser@master01:~$ kubectl get svc -o yaml
apiVersion: v1
items:
- apiVersion: v1
kind: Service
metadata:
creationTimestamp: "2023-03-14T13:45:53Z"
labels:
component: apiserver
provider: kubernetes
name: kubernetes
namespace: default
resourceVersion: "199"
uid: 55782f7e-062c-4cfb-acca-4e4a0d8177a3
spec:
clusterIP: 10.1.0.1
clusterIPs:
- 10.1.0.1
internalTrafficPolicy: Cluster
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
ports:
- name: https
port: 443
protocol: TCP
targetPort: 6443
sessionAffinity: None
type: ClusterIP
status:
loadBalancer: {}
- apiVersion: v1
kind: Service
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app":"nginx"},"name":"nginx","namespace":"default"},"spec":{"ipFamilies":["IPv4","IPv6"],"ipFamilyPolicy":"RequireDualStack","ports":[{"port":80,"protocol":"TCP","targetPort":80}],"selector":{"app":"nginx"}}}
creationTimestamp: "2023-03-14T16:58:24Z"
labels:
app: nginx
name: nginx
namespace: default
resourceVersion: "17914"
uid: 699a5e0f-bf6d-4941-85c9-ca0a1eade1b5
spec:
clusterIP: 10.1.223.177
clusterIPs:
- 10.1.223.177
- fd12:b5e0:383f::e44e
internalTrafficPolicy: Cluster
ipFamilies:
- IPv4
- IPv6
ipFamilyPolicy: RequireDualStack
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
sessionAffinity: None
type: ClusterIP
status:
loadBalancer: {}
kind: List
metadata:
resourceVersion: ""
ServiceのipFamilyPolicyとipFamiliesについて
ipFamilyPolicy
・SingleStack：IPv4 or IPv6のどちらか
・PreferDualStack：DualStack or SingleStackのどちらか
・RequireDualStack：DualStack(それ以外は失敗する)
ipFamilies
・IPv4/IPv6のどちらか、もしくは両方の指定が可能
・IPv4/IPv6のどれが適用されるかは、クラスターの構成とipFamilyPolicyで決定される
kubeuser@master01:~$ kubectl explain svc.spec.ipFamilyPolicy
KIND: Service
VERSION: v1
FIELD: ipFamilyPolicy <string>
DESCRIPTION:
IPFamilyPolicy represents the dual-stack-ness requested or required by this
Service. If there is no value provided, then this field will be set to
SingleStack. Services can be "SingleStack" (a single IP family),
"PreferDualStack" (two IP families on dual-stack configured clusters or a
single IP family on single-stack clusters), or "RequireDualStack" (two IP
families on dual-stack configured clusters, otherwise fail). The ipFamilies
and clusterIPs fields depend on the value of this field. This field will be
wiped when updating a service to type ExternalName.
kubeuser@master01:~$ kubectl explain svc.spec.ipFamilies
KIND: Service
VERSION: v1
FIELD: ipFamilies <[]string>
DESCRIPTION:
IPFamilies is a list of IP families (e.g. IPv4, IPv6) assigned to this
service. This field is usually assigned automatically based on cluster
configuration and the ipFamilyPolicy field. If this field is specified
manually, the requested family is available in the cluster, and
ipFamilyPolicy allows it, it will be used; otherwise creation of the
service will fail. This field is conditionally mutable: it allows for
adding or removing a secondary IP family, but it does not allow changing
the primary IP family of the Service. Valid values are "IPv4" and "IPv6".
This field only applies to Services of types ClusterIP, NodePort, and
LoadBalancer, and does apply to "headless" services. This field will be
wiped when updating a Service to type ExternalName.
This field may hold a maximum of two entries (dual-stack families, in
either order). These families must correspond to the values of the
clusterIPs field, if specified. Both clusterIPs and ipFamilies are governed
by the ipFamilyPolicy field.
