November 27, 2020
Citrix Virtual Summit Korea 2020
Hyunjong Lee
Senior Systems Engineer Citrix Systems Korea Ltd.
Netmarble에서는 클라우드 네이티브 환경을 어떻게 사용하고 있나요?
“클라우드 네이티브”에
대하여..
© 2020 Citrix | Confidential 3
가상화 기술의 변화
Virtual Mechine Container
어플리케이션 형태의 변화
Business
logic DataAccess UI
UI
Microservice Microservice Microservice
Monolithic Architecture
MicroServices
Architecture
© 2020 Citrix | Confidential 5
Release 주기의 변화
Monthly or yearly releases Every hour release, CI/CD
조직의 변화
Siloed teams Highly cross-functional teams
x
클라우드 네이티브 환경에서
Citrix ADC 구성의 특장점
Deployments
복잡성 혜택
Unified Ingress
2-Tier Ingress
Service Mesh Lite
Service Mesh
낮음 높음
높음
© 2020 Citrix | Confidential 9
Deployments for ingress
North-South App Traffic LB
Node Node
Node Node Citrix ADC
Kubeproxy Kubeproxy
Kubeproxy Kubeproxy
Citrix ADC를 이용하여, L4/L7 로드밸런싱
East-West App Traffic LB
Kubeproxy를 이용하여, L4 로드밸런싱
Node Node
Node Node Citrix ADC
Kubeproxy Kubeproxy
Kubeproxy Kubeproxy
North-South
App Traffic LB East-West
App Traffic LB
Citrix CPX
Automated ADC for developer agility Proxy per team, app,
namespace
Citrix ADC를 이용하여, L4/L7 로드밸런싱
Citrix CPX를 이용하여, L7 로드밸런싱
Kubeproxy를 이용하여, L4 로드밸런싱
Deployments for Servicemesh
Pod Pod
Pod Pod
Citrix ADC
Sidecar
Citrix CPX Citrix ADC
Pod
Sidecar
Pod
SidecarPod
Sidecar
Pod
North-South
App Traffic LB East-West
App Traffic LB North-South
App Traffic LB East-West
App Traffic LB
Citrix CPX
Best observability & security Requires new skillset
Service mesh-like benefits but simpler
Citrix ADC를 이용하여,
L4/L7 로드밸런싱 Citrix CPX를 이용하여,
L4/L7 로드밸런싱 Citrix ADC를 이용하여, L4/L7 로드밸런싱
Citrix CPX를 이용하여, L7 로드밸런싱
Citrix CPX를 이용하여, L7 로드밸런싱 및 sidecar로 동작
x
Use Case for Netmarble
Netmarble 소개
© 2020 Citrix | Confidential 13
Load-Balancer for bare metal kubernetes
Layer 2 mode / BGP mode
Layer 2 mode - leader node SPOF BGP mode – active connection reset when scaling worker node
- Netmarble Operation System NOS
Container Component
- Manage Kubernetes Cluster Lifecycle
Network Component
- Manage ADC List and VIP Pools - Mapping k8s cluster to ADC - Allocate VIP Pool to k8s cluster
OpenStack
Netmarble Container Service
Node Node
Node Node Citrix ADC
Cluster1
Kubeproxy Kubeproxy
Kubeproxy Kubeproxy
Citrix ADC Citrix ADC
Node Node
Node Node
Cluster2
Kubeproxy Kubeproxy
Kubeproxy Kubeproxy
Node Node
Node Node
Cluster3
Kubeproxy Kubeproxy
Kubeproxy Kubeproxy
Node Node
Node Node
ClusterN…
Kubeproxy Kubeproxy
Kubeproxy Kubeproxy
CIC CIC CIC
© 2020 Citrix | Confidential 15
Use case - Ingress
Load Balancer for nginx ingress
Citrix Ingress
Citrix ADC
Node Node
Node Node
Cluster
CIC
Path rule /apple /banana
Citrix ADC
Node Node
Node Node
Cluster
CIC
Load Balancer
Target – nginx ingress service
Path rule In nginx ingress
/apple
/banana
x
클라우드 네이티브 환경에서 효율적인 관측가능성에
관하여..
© 2020 Citrix | Confidential 17
Citrix Observability Solutions
Apps
Citrix ADC (any form factor)
Citrix ADM
Citrix Observability
Exporter Open Source Monitoring,
Logging, Tracing tools
2 1
New
Observability 을 확보하기 위한 기능별 특징
이벤트의 발생 시간을 기록하는 매우 세분화되어 있고, 제한된 검색 정보
일정 기간 동안 측정된 데이터 대시보드 형태로 과거 추세 확인
트래픽 플로우 및 지연 시간 추적 지연 시간 관련 문제 확인
서비스 상호 의존성, 서비스 상태 및 인사이트 확인
Logging
Metrics
Tracing
Service Graph
이점
© 2020 Citrix | Confidential 19
Citrix의 Observability 을 확보하기 위한 솔루션
Logging
Metrics
Tracing
Service Graph
Citrix LogProxy
Citrix Matrix Exporter
Citrix ADM
이점
Tools Chain Integration for Logging
Citrix LogProxy
Citrix ADC
Visualization Storage & Search UI
Filter, Buffering, Routing
& Translation of Logs Dashboards
© 2020 Citrix | Confidential 22
Logging Dashboards: Start with Base, then Build Custom
P90 Latency Distribution
Percentile 90 (worst 10%
experience)
1.3 Second average latency
Overall average latency is not a true measure to deliver great user experience
Time / Request
Average
2.25 Second / request
For all requests
Kibana 대시 보드 예제 : 사용자 지정 검색으로 숨겨진 문제 식별
Prometheus & Grafana integration for Metrics Dashboards
Citrix Metrics
Exporter
Scrape Data & Query
Dashboards
MPX/SDX
VPX
VPX
CPX
Scrape
Counters
© 2020 Citrix | Confidential 24
Metrics Dashboards: Start with Basics, then Build Custom
Grafana 대시 보드 예제
ADC Traffic
Health ADC Service Stats
System
Stats
Troubleshooting Slow Response with Kibana & Zipkin
Kibana 대시 보드 예제 : 사용자 지정 검색으로 숨겨진 문제 식별
P90 Latency Distribution
Percentile 90 (worst 10%
experience) =
1.3 Second average latency
Service G is the cause.
Fix it !
Time / Request
Average
2.25 Second / request
For all requests for
selected duration
© 2020 Citrix | Confidential 26
Citrix ADM
Citrix Application
Delivery Management
(ADM)
Single Pane of Glass
Across Traditional & Micro-services Apps
Automation | Orchestration | Management | Analytics
Hardware Appliances DC & Private Cloud,
MPX Virtual Appliances
DC & Private Cloud, VPX In Public Cloud VPX for AWS, Azure,
GCP
Containers Public & Private Cloud,
CPX
Bare Metal
Private & Public Cloud, BLX Multi-Tenant
MSP & Private Cloud,
SDX
Citrix ADM
시각화
Complete Micro-services map
통찰력 확보
Throughput, Saturation, Errors, Latency for Each Micro-service
Score 기반의 서비스 상태
Simple Colors, Composite Scores
이상 징후 감지
Identify Potential Problems
중요한 어플리케이션에는 마이크로서비스 트래픽 및 상태에 대한 가시성 확보가 중요합니다.
© 2020 Citrix | Confidential 28
Citrix ADM
Citrix ADM : Ingress
주요 메트릭의 가시성 – Hits, Processing Time and Data Volume
클러스터의 Microservices에 대한 연관성 및 들어오는 서비스에 대한 주요 메트릭의 가시성
© 2020 Citrix | Confidential 30
Citrix ADM : Ingress
TCP Metrics
• TCP connections – Total connections established between the services
• Data Volume – Total data processed by the service
• TCP Server / Client Reset – Total TCP resets initiated from the server/
client
• Hits – Indicates the total number of hits received by the service.
• Service Response Time – Indicates the average response time taken from the service to respond for Time To First Byte (TTFB).
• Errors – Indicates the total errors such as 4xx, 5xx, and so on.
• Data volume – Indicates the total volume of data processed by the service.
HTTP Metrics SSL Metrics
• SSL Server Errors – Indicates the total SSL errors from the server. (For example, SSL certificate unknown)
• SSL Protocol – Indicates the SSL protocol version used by the service
• SSL Client Errors - Indicate the total SSL errors from the client. (For example: Handshake Failure )
• SSL Server Errors - The total SSL backend errors
from the service. (For example: Client Auth Failure)
Citrix ADM : Service Details
© 2020 Citrix | Confidential 32
Citrix ADM : Service Graph – Distributed Tracing Insights
Kubernetes 서비스를 그래프 형태로 시각화
Citrix ADM : Service Graph Distributed Tracing
© 2020 Citrix | Confidential 34
Citrix ADM : Service Graph Distributed Tracing
Citrix ADM : Service Graph Distributed Tracing
© 2020 Citrix | Confidential 36
Microservices를 위한 보안성 확보
Ingress
N-S Security Intra-cluster E-W Security
WAF, Encryption, SSL Authentication, Authorization
API Gateway
Segmentation, mTLS, Encryption, SSL, Authentication, Authorization
Monolithic 및 Microservices 환경에서 일관된 애플리케이션 및 API 보안
Citrix 클라우트 네이티브 솔루션의 필요성
플랫폼 및 툴 통합
Get apps to production fast with vast K8s platform & opensource tools support
성능
Support large clusters &
very dynamic microservices
API 보안
Break the silos
of monoliths & microservices
유연한 아키텍처
Move at speed of your IT skill set Balance benefits & complexity
완벽한 관찰 가능성