How to Cut Costs on AKS

How to Cut Costs on AKS

Image title

This article aims to showcase the ways in which one can cut costs by saving on resources like CPU’s, RAM and VM’s using both inbuilt objects of Kubernetes and third-party tools.

Here are some ways that can help you manage and optimize resources usage.

  1. LimitRanges
  2. ResourceQuotas
  3. Horizontal Pod Autoscaler
  4. ClusterAutoScaler
  5. Maestro

LimitRange

Assigning a memory request and a memory limit or a CPU request and a CPU limit to a container is called a LimitRange. A container can have as much memory as it requests but is not allowed to use more memory than its limit.

Make sure you have the metrics server installed in your cluster before applying LimitRanges. By default, AKS cluster comes with metrics server installed. See the metrics server pod in AKS cluster using  kubectl -n kube-system get pods .

Create LimitRange

Create a LimitRange to a namespace so that all the pods that exist in that namespace will get applied by that LimitRange.

  1. Create a namespace using kubectl create ns <namespace name>
  2. Create and apply the following limitrange to the namespace using kubectl create -f <file path>
apiVersion: v1
kind: LimitRange
metadata:
  name: cpu-memory
  namespace: <namespace name>
spec:
  limits:
  - default:
      cpu: 250m
      memory: 300Mi
    defaultRequest:
      cpu: 100m
      memory: 200Mi
    type: Container

Deploy a Simple Pod and Service

  1. Let’s deploy a pod and service that creates a single container to demonstrate how default values are applied to each pod. kubectl -n <namespace name> run php-apache --image=k8s.gcr.io/hpa-example --expose --port=80
  2. Get the pods and service using kubectl -n <namespace name> get pods and kubectl -n <namespace name> get services
NAME                          READY     STATUS    RESTARTS   AGE
php-apache-55c4bb8b88-bb7jp   1/1       Running   0          4m
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
php-apache   ClusterIP   10.96.125.210   <none>        80/TCP    4m

Get the Configuration of The Pod

Now get the configuration of the pod using kubectl -n <namespace name> get pod <podname> -o yaml

....................
....................
  containers:
  - image: k8s.gcr.io/hpa-example
    imagePullPolicy: Always
    name: php-apache
    ports:
    - containerPort: 80
      protocol: TCP
    resources:
      limits:
        cpu: 250m
        memory: 300Mi
      requests:
        cpu: 100m
        memory: 200Mi
...................
...................
  1. Get the metrics of the pods in your namespace using kubectl top pods -n <namespace name>.
NAME                          CPU(cores)   MEMORY(bytes)
php-apache-55c4bb8b88-bb7jp   1m           9Mi

Resource Quotas

In Kubernetes, ResourceQuota is used to limit the resources per namespace when multiple users sharing the cluster.

In this article, we’ll apply the ResourceQuota for the compute resources.

Create a Namespace

Create a namespace and apply the ResourceQuota to that namespace.

kubectl create ns <namespace name>

Create a ResourceQuota

  1. Create a ResourceQuota using the following YAML and apply it to the namespace by specifying the namespace name in the YAML file.
apiVersion: v1
kind: ResourceQuota
metadata:
  name: <resourcequota name>
  namespace: <namespace name>
spec:
  hard:
    requests.cpu: "1"
    requests.memory: 1Gi
    limits.cpu: "2"
    limits.memory: 2Gi

2. Save the above YAML file and create a ResourceQuota using kubectl create -f <file path>

3. Now use the kubectl -n <namespace name> get resourcequota <resourcequota name> -o yaml      command to get detailed information about the ResourceQuota.

Create a Pod

  1. Create a Pod in that namespace using the below YAML.
apiVersion: v1
kind: Pod
metadata:
  name: nginx-cpu-memory
  namespace: <namespace name>
spec:
  containers:
  - name: nginx-cpu-memory-quota
    image: nginx
    resources:
      limits:
        memory: "700Mi"
        cpu: "700m" 
      requests:
        memory: "500Mi"
        cpu: "300m"

2. Deploy the pod using kubectl create -f <file path>

3. See the container status kubectl -n <namespace name> get pods

4. Get detailed information about the ResourceQuota.

kubectl -n <namespace name> get resourcequota <resuorcequota name> -o yaml

Example:

spec:
  hard:
    limits.cpu: "2"
    limits.memory: 2Gi
    requests.cpu: "1"
    requests.memory: 1Gi
status:
  hard:
    limits.cpu: "2"
    limits.memory: 2Gi
    requests.cpu: "1"
    requests.memory: 1Gi
  used:
    limits.cpu: 700m
    limits.memory: 700Mi
    requests.cpu: 300m
    requests.memory: 500Mi

5. The output shows how much of the quota has been used along with the quota.

Create Another Pod

  1. Create another pod by specifying the memory request more than its ResuorceQuota request.
apiVersion: v1
kind: Pod
metadata:
  name: redis-cpu-memory
  namespace: resourcequota
spec:
  containers:
  - name: redis-cpu-memory-quota
    image: redis
    resources:
      limits:
        memory: "1Gi"
        cpu: "900m"      
      requests:
        memory: "600Mi"
        cpu: "500m"

2. Create the pod using kubectl create -f <file path>

3. Now the second pod doesn’t get created and gives an error because of the exceeding memory request.

You can also restrict the totals for memory limit, CPU request, and CPU limit.

Horizontal Pod Autoscaler

Horizontal Pod Autoscaler increases the number of pods in a deployment, replica controller or replica set automatically based on the CPU utilization.

Apply Horizontal Pod Autoscaling

  1. Apply the HPA configuration to the existing deployment.

kubectl -n <namespace name> autoscale deployment php-apache --cpu-percent=50 --min=1 --max=10

  1. Get the HPA configuration using kubectl -n <namespace name> get hpa
  NAME         REFERENCE               TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
  php-apache   Deployment/php-apache   1%/50%    1         10        1          10m

Generate Load

  1. Now we will use a load generator to generate some load on Apache.
  2. Open an additional terminal window and run the below command.
kubectl -n <namespace name> run -i --tty load-generator --image=busybox /bin/sh
  1. Hit enter and run below command to generate load on Apache.
while true; do wget -q -O- http://php-apache.<namespace name>.svc.cluster.local; done

Output should be:

OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!
  1. Within few minutes, we should see the higher CPU load by executing kubectl -n <namespace name> get hpa
NAME         REFERENCE               TARGETS    MINPODS   MAXPODS   REPLICAS   AGE
php-apache   Deployment/php-apache   158%/50%   1         10        4          16m
  1. You can see the number of apache pods increased due to load on apache using kubectl -n <namespace name> get pods
NAME                              READY     STATUS    RESTARTS   AGE
load-generator-5ff6784f85-7wgnm   1/1       Running   0          9m
php-apache-55c4bb8b88-2j2nl       1/1       Running   0          7m
php-apache-55c4bb8b88-bp5mf       1/1       Running   0          5m
php-apache-55c4bb8b88-jx5qr       1/1       Running   0          7m
php-apache-55c4bb8b88-kc68r       1/1       Running   0          52m
php-apache-55c4bb8b88-m4hzb       1/1       Running   0          5m
php-apache-55c4bb8b88-trvrm       1/1       Running   0          7m

Stop Load

  1. You can stop the load on apache by typing “Ctrl + C” on the new terminal.
  2. You can verify the result within a minute using kubectl -n <namespace name> get hpa.
NAME         REFERENCE               TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
php-apache   Deployment/php-apache   44%/50%   1         10        6          24m

Cluster Autoscaler on Azure

Cluster Autoscaler:

1. Cluster Autoscaler is an element that automatically by itself ( no direct human control) adjusts the size of the Kubernetes cluster so that all pods have a place to run when there are no redundant nodes. Cluster Autoscaler works with GCP, Azure, and AWS.

Cluster Autoscaler on Azure dynamically scales Kubernetes worker nodes.

Pre-Requirements:

az ad sp create-for-rbac --role="Contributor" --scopes="/subscriptions/<subscription-id>" --output json

from DZone Cloud Zone

Sharing is caring!

Comments are closed.