NetScaler is a feature-rich application delivery controller that comes in various forms, including hardware (MPX), virtual (VPX), and containerized (CPX). The features NetScaler supports as well as its throughput capacity are usually unlocked by a license. The NetScaler Application Delivery Management (ADM) license server manages these NetScaler licenses and can be located in a separate network from the NetScalers.

You can configure licensing options using CLI/GUI/Nitro. NetScaler reaches out to the NetScaler ADM license server to check out the appropriate license, according to the configured capacity, and the NetScaler ADM license server periodically gathers the NetScalers’ health or connectivity status. The license server can reclaim licenses of unreachable NetScaler instances after a configured timeout (usually 10 minutes), assuming that NetScaler is either shut down or decommissioned. If it isn’t shut down, the NetScaler continues to work for a grace period of 30 days. If the NetScaler reconnects with the NetScaler ADM, it attempts to reconcile the license. If it can’t, the NetScaler loses the capacity when the grace period ends.

This works well for NetScaler MPX and VPX. But what about CPX?

In this blog post, we’ll go through the challenges that existed with the NetScaler CPX licensing mechanism in a Kubernetes environment and how that led us to develop NetScaler CPX License Aggregator — a Kubernetes cluster local proxy license service. We’ll also cover the topology, features, workflow, and a sample deployment of the NetScaler CPX License Aggregator.

Origins of the NetScaler CPX License Aggregator

We had observed licensing issues in the containerized NetScaler CPX with the existing modus operandi, which worked fine for the MPX and VPX forms of the NetScaler. Containers can be ephemeral in nature and have lifecycles of just a few minutes. It’s possible that a particular instance of NetScaler CPX could come up and go down and remain unlicensed. Why? Because the NetScaler ADM license server could not identify the instance coming up and going down due to its short lifespan.

Similarly, we also saw that NetScaler ADM took 10 minutes to reclaim the license of the terminated CPX instance. A new NetScaler CPX instantiated during this period might remain unlicensed due to the temporary unavailability of licenses in the NetScaler ADM license server because of the delayed reclamation.

Factors ranging from hardware problems to misconfigurations can cause connectivity issues between the NetScaler ADM license server and NetScaler CPX, resulting in stale information about CPX instances on the license server. It usually happens due to ungraceful termination of the NetScaler CPX. If the NetScaler ADM server is not reachable, you cannot license new NetScaler CPXs in the Kubernetes cluster. These issues can require manual debugging.

In the existing model of licensing, the connectivity between NetScaler CPX and NetScaler ADM is mandatory. NetScaler CPX can be deployed as a sidecar container in the service-mesh environment, but there might be restrictions placed on egress connectivity from the sidecars. To address these challenges and the need for a Kubernetes cluster local proxy license service, we created the NetScaler CPX License Aggregator!

NetScaler CPX License Aggregator: An Overview

The NetScaler CPX License Aggregator (CLA) is a Citrix-provided Kubernetes service that acts as a local license provider for NetScaler CPX instances deployed in a Kubernetes cluster. It’s an intermediary between NetScaler CPXs and the NetScaler ADM licensing server and keeps track of NetScaler CPXs and allotted licenses. It checks out licenses from the NetScaler ADM license server and reserves licenses to meet the needs of the clusterwide NetScaler CPXs. When NetScaler CPXs instances (pods) are created or deleted, licenses are either checked out from or checked back in to the NetScaler CPX License Aggregator.

With the NetScaler CPX License Aggregator, the NetScaler ADM server can reclaim licenses immediately when NetScaler CPXs instances are deleted. The NetScaler CPX License Aggregator is also responsible for relinquishing additional licenses that exceed immediate needs so no single instance will consume the maximum number of licenses from the NetScaler ADM license server.

In a Kubernetes cluster, the NetScaler CPX License Aggregator service supports both sidecar and standalone deployments of NetScaler CPXs.

Benefits of the NetScaler CPX License Aggregator

Let’s look at some key benefits of using the NetScaler CPX License Aggregator.

Scalability

A single NetScaler ADM licensing server can support up to 10,000 NetScaler CPX deployments. With NetScaler CPX License Aggregator service, each Kubernetes cluster can act as a single client to the NetScaler ADM licensing server, and each instance can handle the licensing needs of thousands of NetScaler CPXs. You can potentially handle millions of NetScaler CPXs with a single NetScaler ADM licensing server. That’s significant scalability!

Resource Optimization

The NetScaler CPX License Aggregator service can:

• Check out licenses from the NetScaler ADM server on an as-needed basis;
• Return additional licenses back to the NetScaler ADM server; and
• Handle ungraceful termination of NetScaler CPXs and reclaim licenses from these NetScaler CPXs after the configured waiting period.

These capabilities help to ensure optimal resource utilization of the NetScaler licenses.

Improved Performance

Because NetScaler CPX License Aggregator runs as a local service, NetScaler CPXs don’t need to establish connectivity to a more distant NetScaler ADM, avoiding a longer roundtrip time. The local NetScaler CPX License Aggregator service allots licenses with lower latency, and, thanks to the NetScaler CPX License Aggregator, we don’t need network connectivity between the cluster-local sidecar CPXs and the external, on-prem NetScaler ADM. Together, this improves overall cluster performance.

Workflow for Licensing NetScaler CPX

Here are the steps for licensing NetScaler CPX with NetScaler CPX License Aggregator.

1. Follow this guide to deploy NetScaler CPX license aggregator using Helm charts in a Kubernetes cluster. Ensure that the NetScaler ADM License Server details are in the Helm command.
2. CLA fetches licenses from the NetScaler ADM license server and reserves them for the local clusterwide CPX instances.
3. NetScaler CPX needs the address (FQDN or IP address) of the NetScaler CPX License Aggregator using an environment variable.
4. When the NetScaler CPX boots up, it sends a registration request to the NetScaler CPX License Aggregator on a secure communication channel.
5. NetScaler CPX License Aggregator validates the registration request and allots the required license, if available. If the license is unavailable, that is conveyed in the response to the NetScaler CPX.
6. The capacity on NetScaler CPX is unlocked after receiving a successful response from the NetScaler CPX License Aggregator.
7. Whenever the NetScaler CPX License Aggregator runs on a lower capacity of licenses (low-watermark), it fetches a quantum of licenses again from the license server.
8. NetScaler CPX License Aggregator monitors the CPX pod events, and whenever a CPX instance is deleted, the license of the deleted instance is retrieved by the NetScaler CPX License Aggregator .
9. If the NetScaler CPX License Aggregator determines it’s running with more than enough licenses, it returns additional licenses to the ADM license server.

Getting Insights on the NetScaler CPX License Aggregator Instance

NetScaler CPX License Aggregator runs as a Kubernetes service, and the user will likely want insights about the instance. There are two REST endpoints available to enable this. Let’s look at both.

Check the License Aggregator Status

The “/stats” URL path provides information about the type of license the NetScaler CPX License Aggregator can provide; how much capacity the NetScaler CPX License Aggregator holds for a particular license type; and how much capacity the active NetScaler CPXs have consumed. It also provides information about the configured values of variables low-watermark and quantum for a particular license type. Quantum represents the quantity of licenses to be checked out in bulk from the NetScaler ADM license server. If the NetScaler CPX License Aggregator’s available capacity falls below a certain level (i.e. low-watermark), additional “quantum” licenses will be checked out from the license server.

For example, if the NetScaler CPX License Aggregator provides instance-type licenses, it will provide information about how many CPXs can be licensed by the NetScaler CPX License Aggregator and how many are actually licensed at that time. In the case of bandwidth license, it will provide details about the total bandwidth capacity the NetScaler CPX License Aggregator holds and how much is distributed to the local NetScaler CPXs.

If NetScaler CPX License Aggregator is exposed to the client using NodePort, use the kubectl get service command to identify the node-port used for the CLA service. Use the following URL or a cURL utility.

Send an HTTP request to /stats path with https://<NodeIP:Nodeport-of-CLA>/stats.

A sample JSON output of stats is shown in the image below. From the image, we can see that the NetScaler CPX License Aggregator:

• Holds 5 Gbps of total enterprise bandwidth capacity, out of which 1 Gbps has been given to the active NetScaler CPXs, and it can further satisfy bandwidth license requests from CPX instances up to 4 Gbps.
• Owns the enterprise vCPU license and can allot a total of five vCPUs to CPXs. In this example, two vCPUs are allotted and three more vCPU requests can be successfully handled.
• Owns 18 licenses of type INSTANCE, and a total of 15 NetScaler CPX instances are licensed by the NetScaler CPX License Aggregator .

Check the Details of Licensed NetScaler CPXs

The “/cpxinfo” URL path generates a list of NetScaler CPXs licensed by the NetScaler CPX License Aggregator. Any HTTP request to this URL must contain an HTTP header named x-cla with value 1.0.0.

Make sure you send a HTTP request with a custom header to the /cpxinfo path. For example:

curl -H “x-cla: 1.0.0” https://<NodeIP:Nodeport-of-CLA>/cpxinfo

The image below shows the sample output of the cpxinfo request detailed above. Please note the presence of the sidecar field along with the license-Info details. This signifies whether the NetScaler CPX is deployed as a sidecar or a standalone instance.

Try a Sample Deployment

There’s nothing better than hands-on experience. Check out this sample deployment on GitHub to get your hands dirty. This example guide demonstrates the deployment of the NetScaler CPX License Aggregator in the Kubernetes cluster, and then how various types of NetScaler CPXs can be licensed using the NetScaler CPX License Aggregator.

Learn More

The NetScaler CPX License Aggregator streamlines the licensing process for NetScaler CPXs in Kubernetes. With the help of NetScaler CPX License Aggregator, you can get scalability with a single NetScaler ADM licensing server; optimal resource utilization of licenses; and lower latency and improved performance for licensing activities. The NetScaler CPX License Aggregator provides a seamless CPX licensing experience of the NetScaler user in a rapidly growing Kubernetes environment.

You’ll find a Helm chart to help with your NetScaler CPX License Aggregator deployment on the Citrix Helm Chart GitHub page. The page also has detailed steps (and quick steps, too) for deploying NetScaler CPX License Aggregator in a Kubernetes cluster. Give this deployment a try, and if you run into any issues, reach out to us on the GitHub issues page or send an email to AppModernization@citrix.com.