Minimizing complexity with service mesh lite architecture

Priyanka Sharma

4 years ago

As more organizations adopt microservice architectures, it has become critical to manage, monitor, and secure service-to-service (east-west) communication. A service mesh is a dedicated infrastructure layer that tracks, secures, and relays all data flow between services and is usually implemented by deploying network proxies as sidecars in the data plane.

Popular service mesh solutions like Istio or LinkerD solve many of the challenges of service-to-service communication, but things can get complex quickly because of the presence of many data plane proxies that are used to connect disparate containers in the cluster. Also, service mesh solutions can create more resource overhead by having a control plane and one proxy for each microservice.

A service mesh lite architecture gives you the benefits of service mesh, all while minimizing complexity by using fewer proxies in the deployment. Learn more about the philosophy behind service mesh lite architecture and its benefits in this blog post. Here, we’ll take a look at how you can use Citrix ADCs to run your application in a service mesh lite architecture.

Benefits of Service Mesh Lite

An ingress solution (hardware, virtualized, or containerized) typically performs L7 proxy functions for north-south traffic. The service mesh lite architecture uses the same ingress solution to manage east-west traffic, as well.

In Kubernetes, east-west traffic traverses the built-in KubeProxy deployed in each node. As an L4 proxy, KubeProxy only does TCP/UDP-based load balancing without the benefits of L7 proxy. However, Citrix ADCs (MPX, VPX, or CPX) can provide other benefits for east-west traffic, including:

SSL offload
Content-based routing, allow/block traffic based on HTTP, and HTTPS header parameters
Advanced load balancing algorithms (least connections, least response time)
Observability of east-west traffic through measuring golden signals like errors, latencies, saturation, and traffic volume (Citrix ADM’s service graph is an observability solution to monitor and debug microservices.)

Understanding Service Mesh Lite

Citrix Ingress Controller is built around Kubernetes ingress and automatically configures one or more Citrix ADCs based on the ingress resource configuration present in a Kubernetes cluster. (We have resources available if you need a refresher on Citrix Ingress Controller and Citrix ADC form factors.)

Let’s look at service mesh lite and how you can run your application with this architecture using Citrix ADC CPX/VPX and Citrix Ingress Controller in your Kubernetes cluster.

East-West Communication in Kubernetes

Let’s consider an e-commerce application — Hipster — that has multiple microservices. Here is the manifest that can be used to deploy the Hipster application in Kubernetes. We’ll look at two microservices in the application — cartservice and checkoutservice — to understand how kube-proxy handles east-west traffic. When the cartservice microservice wants to access the checkoutservice microservice, the DNS returns the cluster IP of the checkoutservice microservice to the cartservice microservice. The cartservice microservice initiates a connection to the cluster IP, which the KubeProxy intercepts. The KubeProxy load balances the request between the set of checkoutservice pods.

E-W traffic to checkoutservice app via KubeProxy

With service mesh lite, we introduce a proxy in the east-west traffic path to leverage its features with the east-west application traffic in a Kubernetes cluster. Service mesh lite leverages the headless service concept instead of the cluster IP in Kubernetes. Kube-proxy does not handle headless services, and there’s no load balancing or proxying done for them by the platform.

In the next sections, we’ll look at two cases to help you understand how to use the headless service concept of Kubernetes to introduce Citrix ADC into the east-west traffic path so that it can control and load balance the traffic between various microservices running in a Kubernetes cluster.

Case I: East-West Communication with Citrix ADC CPX in Service Mesh Lite Architecture

Our goal here is to insert the Citrix ADC CPX in the east-west traffic path and use the Kubernetes Ingress rules to control the traffic between various microservices in the application.

Step 1: Modify the checkoutservice service definition to point to Citrix ADC CPX

We want all traffic destined for the checkoutservice microservice to land in the CPX. To achieve this, we want the checkoutservice DNS to be resolved to the IP of the CPX. You can do this by making service of checkoutservice microservice point to the CPX deployment. In the service manifest of checkoutservice, we specify the selector as a label given in the Citrix ADC CPX deployment manifest. In this case it’s ‘app:cpx-ingress1’.

E-W traffic to checkoutservice app via Citrix ADC CPX

After this modification, when a pod in the Kubernetes cluster resolves the FQDN for service of the checkoutservice microservice, the IP address of the Citrix ADC CPX is returned.

Step 2: Create the headless service checkoutservice-headless for checkoutservice microservice pods

Because we have modified the original service of checkoutservice microservice so that it points to the Citrix ADC CPX IP, we need to create one more service that represents the checkoutservice microservice deployment. The headless service will look like:

apiVersion: v1
kind: Service
metadata:
  name: checkoutservice-headless
spec:
  selector:
    name: checkoutservice
  ports:
  - name: grpc
    port: 5050
    targetPort: 5050
  #headless Service
  clusterIP: None

Step 3: Create an ingress resource with rules for checkoutservice-headless service

We need to have an ingress resource for the checkoutservice microservice that can be used by the Citrix Ingress Controller to configure the Citrix ADC CPX so it can handle all the east-west traffic destined to the checkoutservice microservice.

A sample ingress resource (for checkoutservice) is shown here:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: checkoutservice-ingress
  annotations:
    ingress.citrix.com/insecure-service-type: "tcp"
    ingress.citrix.com/insecure-port: "5050"
    kubernetes.io/ingress.class: "tier-2-cpx1"
spec:
  backend:
    serviceName: checkoutservice-headless
    servicePort: 5050

With the above changes for all the application’s microservices, all traffic coming to a microservice will land on the Citrix ADC CPX first. It can then load balance the east-west traffic in the cluster.

The following diagrams show the end-to-end flow of how the Citrix ADC CPX can be introduced in the east-west traffic path in the Kubernetes cluster so that it can provide L7 proxying communication between the cartservice and checkoutservice microservices using ingress rules.

The following diagram shows the Citrix ADC CPX configured as an east-west proxy using ingress rules.

This is a dual-tier topology. You can use this manifest to deploy the Hipster application in a service mesh lite architecture in the dual-tier topology. You will have make sure that you provide the following information in the manifest before you deploy it in the Kubernetes cluster:

<citrix-adc-vpx-nsip>: The management IP address of a standalone Citrix ADC appliance. For more information, see IP Addressing in Citrix ADC.
<citrix-adc-vpx-credential-secret>: Name of the Kubernetes Secret having ADC credential details.
<citrix-adc-vpx-vip>: Citrix Ingress Controller configures this virtual IP address in the Citrix ADC to receive ingress traffic.

Learn more about dual-tier topology and the information required to bring up the topology.

Once you can successfully deploy this manifest and everything is up and running in the tier-1 Citrix ADC VPX and the tier-2 Citrix ADC CPX, you will be able to access the application by following these steps:

Enter the application hostname in your host file against the Citrix ADC VPX VIP:x.x.x citrix.hipster.smlite.
You should be able to access the application via http://citrix.hipster.smlite in your browser.

In this deployment all the east-west traffic is handled and load-balanced by tier-2 Citrix ADC CPX.

Case II: East-West Communication with Citrix ADC MPX or VPX in Service Mesh Lite Architecture

Similarly (but with slight modifications), you can use Citrix ADC MPX or VPX, acting as an ingress, to load balance east-west microservice communication. Here’s how:

Step 1: Create an external service resolving the checkoutservice host name to the Citrix ADC MPX/VPX IP address

There are two ways to achieve this. You can add a service of type ExternalName by mapping a host name or using an IP address.

Mapping a host name (CNAME)

Create a domain name for the ingress endpoint IP (content switching virtual server IP address) in Citrix ADC MPX/VPX — myadc.mydomain.com, for example — and update it in your DNS server.
Create a Kubernetes service for checkoutservice with externalName as myadc.mydomain.com.
Now, when any pod looks up the checkoutservice microservice, a CNAME(myadc.mydomain.com) is returned.

apiVesion: v1
kind: Service
metadata:
  name: checkoutservice
spec:
  type: ExternalName
  externalName: myadc.mydomain.com

Mapping a host name to an IP address

When you want your application to use the host name checkoutservice, which will redirect to the virtual IP address hosted in Citrix ADC MPX or VPX, you can use the following.

apiVersion: v1
kind: Service
metadata:
  name: checkoutservice
spec:
  ports:
    - name: grpc
      port: 5050
      targetPort: 5050

---

apiVersion: v1
kind: Endpoints
metadata:
  name: checkoutservice
subsets:
  - addresses:
      - ip: "1.1.1.1" # Ingress IP in VPX/MPX
    ports:
      - port: 5050
        name: grpc

Step 2: Create a headless service checkoutservice-headless for the checkoutservice microservice pods

Because we have modified service of checkoutservice microservice to point to Citrix ADC MPX, we need to create one more service that represents the checkoutservice microservice deployment.

The headless service for checkoutservice microservice will look like:

apiVersion: v1
kind: Service
metadata:
  name: checkoutservice-headless
spec:
  selector:
    app: checkoutservice
  ports:
  - name: grpc
    port: 5050
    targetPort: 5050
  #headless Service
  clusterIP: None

Step 3: Create an ingress resource with rules for checkoutservice-headless service having the ingress.citrix.com/frontend-ip annotation

Create an ingress resource with the ingress.citrix.com/frontend-ip annotation, where the value matches the ingress endpoint IP address in Citrix ADC MPX or VPX. The Citrix Ingress Controller will use this ingress resource to configure the Citrix ADC VPX for handling traffic coming to the microservice checkoutservice.

Here’s a sample ingress resource:

apiVersion: extension/v1beta1
kind: Ingress
metadata:
  name: checkoutservice-ingress
  annotations:
    ingress.citrix.com/insecure-service-type: "tcp"
    ingress.citrix.com/insecure-port: "5050"
    kubernetes.io/ingress.class: "tier-2-cpx1"
    ingress.citrix.com/frontend-ip: "1.1.1.1"
spec:
  backend:
    serviceName: checkoutservice-headless
    servicePort: 5050

With the above changes in the microservices of an application, all traffic coming to a microservice will land on the Citrix ADC MPX/VPX first. Using the usual ingress load balancing methodology with the changes above, the Citrix ADC VPX/MPX can now load balance east-west traffic.

The following diagram shows Citrix ADC VPX or MPX configured as north-south and east-west proxies using ingress rules.

The above topology is a unified-ingress or single-tier topology. This manifest can be used to deploy the Hipster application in a service mesh lite architecture in the unified-ingress topology. You must make sure that you provide the following information in the manifest before you deploy it in the Kubernetes cluster:

<citrix-adc-vpx-nsip>: The management IP address of a standalone Citrix ADC appliance. For more information, see IP Addressing in Citrix ADC.
<citrix-adc-vpx-credential-secret>: Name of the Kubernetes secret having ADC credential details.
<citrix-adc-vpx-vip>: Citrix Ingress Controller configures this virtual IP address in the Citrix ADC to receive ingress traffic.

Learn more about unified ingress topology and the information required to bring up the topology.

Once you successfully deploy the above manifest and everything is up and running in tier-1 Citrix ADC VPX, you will be able to access the application by following these steps:

Enter the application hostname in your host file against Citrix ADC VPX VIP:x.x.x citrix.hipster.smlite
You should be able to access the application by accessing http:/citrix.hipster.smlite in your browser

In this deployment all the east-west traffic is handled and load-balanced by tier-1 Citrix ADC VPX.

Automated Deployment of Apps in Service Mesh Lite

To deploy an application in a service mesh lite architecture, you’ll need to perform multiple tasks, including:

Modifying the existing services to make them headless services
Creating a service to point to Citrix ADC
Creating ingress rules

Citrix provides an automated way to generate deployment-ready YAMLs to make deploying a service mesh lite solution easier. Learn more.

Conclusion

Service mesh lite architecture can give you the benefits of service mesh while minimizing complexity. You can run your app in a service mesh lite architecture using Citrix ADC CPX or VPX/MPX and Citrix Ingress Controller in your Kubernetes cluster. Citrix ADCs handle the east-west traffic of your application in the service mesh lite architecture and, as an L7 proxy, can provide features other proxies can’t.

To learn more about using Citrix Ingress Controller to enable different features on Citrix ADC CPX, VPX, and MPX, check out our GitHub repository.