[7주차] Service Mesh : Traffic Management : TCP Traffic Shifting

CloudNet@ 가시다님이 진행하는 쿠버네티스 네트워크 스터디 3기

---

✅ TCP Traffic Shifting

특정 서비스를 여러 버전으로 나누고, 이들 버전 간에 트래픽을 분산시킬 수 있는 기능입니다. 예를 들어, 기존 서비스 버전(v1)과 새로운 서비스 버전(v2)을 운영하는 경우, 트래픽을 비율에 따라 각 버전으로 나누어 보낼 수 있습니다.

---

🧿 istio-injection 설정

Istio-injection 

• Istio는 각 애플리케이션 서비스의 네트워크 요청을 처리하기 위해 사이드카 프록시라는 별도의 컨테이너(일반적으로 Envoy 프록시)를 배포합니다.
• 이 프록시는 애플리케이션이 전송하는 모든 트래픽을 가로채서 관리하고, 필요한 경우 보안, 로깅, 트래픽 제어 등의 기능을 제공합니다.
• istio-injection은 이러한 사이드카 프록시를 자동으로 애플리케이션 Pod에 주입하는 기능입니다. 주입 방식에는 크게 두 가지가 있습니다.

자동 사이드카 주입 (Automatic Sidecar Injection):

• 사용자가 별도의 명령 없이도 Istio가 지정한 네임스페이스에 있는 Pod에 사이드카 프록시를 자동으로 추가합니다.
• 이를 위해, 네임스페이스에 istio-injection=enabled 라는 레이블을 설정합니다. 이렇게 하면 해당 네임스페이스에서 생성되는 모든 Pod에 자동으로 사이드카가 추가됩니다.

수동 사이드카 주입 (Manual Sidecar Injection):

• 자동 주입을 사용하지 않는 경우, 사용자가 직접 특정 Pod에 사이드카를 추가하는 방식입니다.
• istioctl kube-inject 명령을 사용해 Kubernetes 배포 yaml 파일에 Envoy 프록시 컨테이너 설정을 삽입합니다.

(⎈|default:N/A) root@k3s-s:~# kubectl create namespace istio-io-tcp-traffic-shifting
namespace/istio-io-tcp-traffic-shifting created
(⎈|default:N/A) root@k3s-s:~# kubectl label namespace istio-io-tcp-traffic-shifting istio-injection=enabled
namespace/istio-io-tcp-traffic-shifting labeled

---

🧿 요청 전송을 위한 테스트 소스로 사용할 sleep 샘플 앱을 배포합니다.

(⎈|default:N/A) root@k3s-s:~# cat ~/istio-$ISTIO_VERSION/samples/sleep/sleep.yaml
# Copyright Istio Authors
#
#   Licensed under the Apache License, Version 2.0 (the "License");
#   you may not use this file except in compliance with the License.
#   You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#   Unless required by applicable law or agreed to in writing, software
#   distributed under the License is distributed on an "AS IS" BASIS,
#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#   See the License for the specific language governing permissions and
#   limitations under the License.

##################################################################################################
# Sleep service
##################################################################################################
apiVersion: v1
kind: ServiceAccount
metadata:
  name: sleep
---
apiVersion: v1
kind: Service
metadata:
  name: sleep
  labels:
    app: sleep
    service: sleep
spec:
  ports:
  - port: 80
    name: http
  selector:
    app: sleep
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sleep
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sleep
  template:
    metadata:
      labels:
        app: sleep
    spec:
      terminationGracePeriodSeconds: 0
      serviceAccountName: sleep
      containers:
      - name: sleep
        image: curlimages/curl
        command: ["/bin/sleep", "infinity"]
        imagePullPolicy: IfNotPresent
        volumeMounts:
        - mountPath: /etc/sleep/tls
          name: secret-volume
      volumes:
      - name: secret-volume
        secret:
          secretName: sleep-secret
          optional: true
---

(⎈|default:N/A) root@k3s-s:~# kubectl apply -f ~/istio-$ISTIO_VERSION/samples/sleep/sleep.yaml -n istio-io-tcp-traffic-shifting
serviceaccount/sleep created
service/sleep created
deployment.apps/sleep created

---

🧿 tcp-echo 마이크로서비스의 v1 및 v2 버전을 배포합니다.

(⎈|default:N/A) root@k3s-s:~# cat ~/istio-$ISTIO_VERSION/samples/tcp-echo/tcp-echo-services.yaml
# Copyright 2018 Istio Authors
#
#   Licensed under the Apache License, Version 2.0 (the "License");
#   you may not use this file except in compliance with the License.
#   You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#   Unless required by applicable law or agreed to in writing, software
#   distributed under the License is distributed on an "AS IS" BASIS,
#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#   See the License for the specific language governing permissions and
#   limitations under the License.

apiVersion: v1
kind: Service
metadata:
  name: tcp-echo
  labels:
    app: tcp-echo
    service: tcp-echo
spec:
  ports:
  - name: tcp
    port: 9000
  - name: tcp-other
    port: 9001
  # Port 9002 is omitted intentionally for testing the pass through filter chain.
  selector:
    app: tcp-echo
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: tcp-echo-v1
  labels:
    app: tcp-echo
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: tcp-echo
      version: v1
  template:
    metadata:
      labels:
        app: tcp-echo
        version: v1
    spec:
      containers:
      - name: tcp-echo
        image: docker.io/istio/tcp-echo-server:1.3
        imagePullPolicy: IfNotPresent
        args: [ "9000,9001,9002", "one" ]
        ports:
        - containerPort: 9000
        - containerPort: 9001
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: tcp-echo-v2
  labels:
    app: tcp-echo
    version: v2
spec:
  replicas: 1
  selector:
    matchLabels:
      app: tcp-echo
      version: v2
  template:
    metadata:
      labels:
        app: tcp-echo
        version: v2
    spec:
      containers:
      - name: tcp-echo
        image: docker.io/istio/tcp-echo-server:1.3
        imagePullPolicy: IfNotPresent
        args: [ "9000,9001,9002", "two" ]
        ports:
        - containerPort: 9000
        - containerPort: 9001
        
(⎈|default:N/A) root@k3s-s:~# kubectl apply -f ~/istio-$ISTIO_VERSION/samples/tcp-echo/tcp-echo-services.yaml -n istio-io-tcp-traffic-shifting
service/tcp-echo created
deployment.apps/tcp-echo-v1 created
deployment.apps/tcp-echo-v2 created

---

🧿 모든 TCP 트래픽을 tcp-echo 마이크로서비스의 v1 버전으로 라우팅합니다.

(⎈|default:N/A) root@k3s-s:~# cat ~/istio-$ISTIO_VERSION/samples/tcp-echo/tcp-echo-all-v1.yaml
# Copyright 2018 Istio Authors
#
#   Licensed under the Apache License, Version 2.0 (the "License");
#   you may not use this file except in compliance with the License.
#   You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#   Unless required by applicable law or agreed to in writing, software
#   distributed under the License is distributed on an "AS IS" BASIS,
#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#   See the License for the specific language governing permissions and
#   limitations under the License.

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: tcp-echo-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 31400
      name: tcp
      protocol: TCP
    hosts:
    - "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: tcp-echo-destination
spec:
  host: tcp-echo
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: tcp-echo
spec:
  hosts:
  - "*"
  gateways:
  - tcp-echo-gateway
  tcp:
  - match:
    - port: 31400
    route:
    - destination:
        host: tcp-echo
        port:
          number: 9000
        subset: v1
 
(⎈|default:N/A) root@k3s-s:~# kubectl apply -f ~/istio-$ISTIO_VERSION/samples/tcp-echo/tcp-echo-all-v1.yaml -n istio-io-tcp-traffic-shifting
gateway.networking.istio.io/tcp-echo-gateway created
destinationrule.networking.istio.io/tcp-echo-destination created
virtualservice.networking.istio.io/tcp-echo created

---

🧿 ingress IP와 포트를 결정합니다:

• TCP_INGRESS_PORT 및 INGRESS_HOST 환경 변수를 설정합니다.

(⎈|default:N/A) root@k3s-s:~# export TCP_INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="tcp")].nodePort}')
(⎈|default:N/A) root@k3s-s:~# echo $INGRESS_HOST
192.168.10.10
(⎈|default:N/A) root@k3s-s:~# echo $TCP_INGRESS_PORT
30427

---

🧿 TCP 트래픽을 전송하여 tcp-echo 서비스가 실행 중인지 확인합니다.

• 모든 트래픽이 tcp-echo 서비스의 v1 버전으로 라우팅되었습니다.

(⎈|default:N/A) root@k3s-s:~# for i in {1..20}; do \
kubectl exec "$(kubectl get pod -l app=sleep -n istio-io-tcp-traffic-shifting -o jsonpath={.items..metadata.name})" \
-c sleep -n istio-io-tcp-traffic-shifting -- sh -c "(date; sleep 1) | nc $INGRESS_HOST $TCP_INGRESS_PORT"; \
done
one Tue Oct 15 15:35:47 UTC 2024
one Tue Oct 15 15:35:48 UTC 2024
one Tue Oct 15 15:35:50 UTC 2024
one Tue Oct 15 15:35:51 UTC 2024
one Tue Oct 15 15:35:53 UTC 2024
one Tue Oct 15 15:35:54 UTC 2024
one Tue Oct 15 15:35:56 UTC 2024
...

---

🧿 트래픽의 20%를 tcp-echo:v1에서 tcp-echo:v2로 전송합니다.

• TCP 트래픽의 80%가 v1 버전의 tcp-echo 서비스로 라우팅되고 20%가 v2로 라우팅되었습니다.

(⎈|default:N/A) root@k3s-s:~# cat  ~/istio-$ISTIO_VERSION/samples/tcp-echo/tcp-echo-20-v2.yaml
# Copyright 2018 Istio Authors
#
#   Licensed under the Apache License, Version 2.0 (the "License");
#   you may not use this file except in compliance with the License.
#   You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#   Unless required by applicable law or agreed to in writing, software
#   distributed under the License is distributed on an "AS IS" BASIS,
#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#   See the License for the specific language governing permissions and
#   limitations under the License.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: tcp-echo
spec:
  hosts:
  - "*"
  gateways:
  - tcp-echo-gateway
  tcp:
  - match:
    - port: 31400
    route:
    - destination:
        host: tcp-echo
        port:
          number: 9000
        subset: v1
      weight: 80
    - destination:
        host: tcp-echo
        port:
          number: 9000
        subset: v2
      weight: 20

(⎈|default:N/A) root@k3s-s:~# kubectl apply -f ~/istio-$ISTIO_VERSION/samples/tcp-echo/tcp-echo-20-v2.yaml -n istio-io-tcp-traffic-shifting
virtualservice.networking.istio.io/tcp-echo configured

---

🧿 라우팅 규칙 제거, sleep 샘플, tcp-echo 애플리케이션 및 테스트 네임스페이스를 제거합니다.

(⎈|default:N/A) root@k3s-s:~# kubectl delete -f ~/istio-$ISTIO_VERSION/samples/tcp-echo/tcp-echo-all-v1.yaml -n istio-io-tcp-traffic-shifting
gateway.networking.istio.io "tcp-echo-gateway" deleted
destinationrule.networking.istio.io "tcp-echo-destination" deleted
virtualservice.networking.istio.io "tcp-echo" deleted
(⎈|default:N/A) root@k3s-s:~# kubectl delete -f ~/istio-$ISTIO_VERSION/samples/tcp-echo/tcp-echo-services.yaml -n istio-io-tcp-traffic-shifting
service "tcp-echo" deleted
deployment.apps "tcp-echo-v1" deleted
deployment.apps "tcp-echo-v2" deleted
(⎈|default:N/A) root@k3s-s:~# kubectl delete -f ~/istio-$ISTIO_VERSION/samples/sleep/sleep.yaml -n istio-io-tcp-traffic-shifting
serviceaccount "sleep" deleted
service "sleep" deleted
deployment.apps "sleep" deleted
(⎈|default:N/A) root@k3s-s:~# kubectl delete namespace istio-io-tcp-traffic-shifting
namespace "istio-io-tcp-traffic-shifting" deleted