Kube-Green is a Kubernetes operator to reduce the carbon footprints and CO2 emission on your Kubernetes clusters. It has been added as the CNCF landscape project in the Scheduling and orchestrating subcategory. It is a simple Kubernetes addon that automatically shuts down your resources when you don't need them.
In the previous KubeCon, environmental sustainability and lesser Carbon footprints have been a rising issue in the domain due to the large amount of carbon footprints generated in the workloads due to the unrequired running of clusters and pods within the clusters.
There has been quite a few times when the issue has been raised by many projects and the tag environment sustainability, which is a part of CNCF. These mentions were brought multiple times through keynotes and speeches.
Thus the rise in the environment sustainability efforts gave rise to the increment in the projects related to this concern of solving the issue of reducing carbon emission through sustainable methods and how can we enable those, tweak them, autoscale based on certain footprints of severs where there are workloads still running.
Functioning
kube-green is a CNCF landscape project in the Scheduling and orchestrating subcategory.
Kube-Green is an operator to reduce the CO2 footprint on your Kubernetes clusters. It helps by making your pods go to sleep or suspends them when no one is using them, thus scale down your clusters to save energy and reduce CO2 emissions.
To showcase this, they have created a Dashboard to show the amount of CO2 emission that have been reduced by our efforts after adopting kube-green on our clusters and workloads.
We are bring using a lot of different environments like we are reviewing Dev Stage production and a lot more resources for our development and staging/ preview, even regularly on weekends and holidays which is an obvious waste of resources, capital and money as we don't want to run them on such time when no one is being using it.
How many CO2 is produced by a Pod?
Emission for cloud server using 100% green electricity : 160 kg CO2 eq. /year per pods.
Cluster node of 2 CPU approx, 1 node is 1 server.
15 pods per node.
Mean consumption of CO2 per pod in a year = 160 /5 = 11 Kg of CO2 /year per pod.
CO2 Calculator
CO2 per pod per year (kg CO2 eq.) = 11 Kg
Let's take total number of pods = 100
Total pods when kube-green active = 100
Result
Total (Kg CO2 eq. /week):
without kube-green \= 21 Kg CO2
with kube-green \= 5 Kg CO2
Difference produced as an impact of adopting kube-green in our cluster**:** 16Kg CO2.
that's how it can be impactful, we can be with one server with one load of 2 CPU and you can calculator the number of huge cluster that we will be running for your Dev Stage kind of environment and how much CO2 can we serve.
How it Works
The use of kube-green is very simple, once installed on cluster, configure the desired CRD to make it work. The CRD used by kube-green is called Sleeping. In configuration, it's possible to configure the sleep and wake up time for namespace.
So, we can define a specific time for a particular namespace and the wake-up time for a particular namespace so you can define a specific time where these pods and deployments will go to sleep. We can also specify the woke-up-time. The CRD is simple with SleepInfo ,weekdays, sleepTime, timeZone, suspended CronJobs, exclude ref. to specify which ones to exclude.
SleepInfo: It is the Schema for the sleepinfos API.ExcludeRef: This is used to exclude the pods and resources where aren't to be included in the measures and rules we set for the rest of our cluster while using kube-green.SleepInfoSpec: It defines the desired state of SleepInfo.SleepInfoStatus: It defines the observed state of SleepInfo
What Resources Handles?
Kube-Green handles:
Deployments
CronJobs
By default, on sleep, deployment resources are stopped (if not excluded). If you want to suspend also cronjobs, set suspendCronJobs to true.
What means sleep and wake up a resource?
The sleep of the resource is the stop of the resource, and it depends on the type of resources.
The wake up of the resource is the restore of the status of the resource before the sleep.
Deployment
To sleep the Deployment resources, replicas are set to 0.
To wake up, the number of replicas is set to the number of replicas before the sleep.
CronJobs
To sleep the CronJob resources, are set as suspended.
To wake up, the suspend field is restored.
How state is saved across sleep and wake up?
The state of the resources before the sleep is saved in a secret with the name of the SleepInfo in the namespace.
Configuration
In the namespace where you want to enable kube-green, apply the SleepInfo CRD. An example of CRD is accessible at this link.
The SleepInfo spec contains:
weekdays: day of the week.
*is every day,1is monday,1-5is from monday to fridaysleepAt: time in hours and minutes (HH:mm) when namespace will go to sleep. Valid values are, for example, 19:00or
*:*for every minute and every hour. Resources sleep will be deployments (setting replicas value to 0) and, ifsuspendCronjobsoption is set to true, cron jobs will be suspended.wakeUpAt (optional): time in hours and minutes (HH:mm) when namespace should be restored to the initial state (before sleep). Valid values are, for example, 19:00or
*:*for every minute and every hour. If wake up value is not set, pod in namespace will not be restored. So, you will need to deploy the initial namespace configuration to restore it.timeZone (optional, default to UTC): time zone in IANA specification. For example for italian hour, set
Europe/Rome.suspendDeployments (optional, default to true): if set to false, deployments will not be suspended.
suspendCronJobs (optional, default to false): if set to true, cronjobs will be suspended.
excludeRef (optional): an array of object containing the resource to exclude from sleep. It can specify exactly the name of the specified resource or match based from the labels. The possible formats are:
apiVersion: version of the resource. Now it is supported "apps/v1", "batch/v1beta1" and "batch/v1"
kind: the kind of resource. Now it is supported "Deployment" and "CronJob"
name: the name of the resource or
matchLabels: an object of strings with the labels to identify a resources click here to see an example.
Examples
Simplest SleepInfo resource
apiVersion: kube-green.com/v1alpha1
kind: SleepInfo
metadata:
name: working-hours
spec:
weekdays: "1-5"
sleepAt: "20:00"
wakeUpAt: "08:00"
timeZone: "Europe/Rome"
With this CRD, it's configured a sleep to 20:00 and wake up to 08:00 on weekdays only for Deployments. The sleepAt and the wakeUpAt parts of the the spec can be updates to the various different formats as wlel like for throughout the time we can set em out to "*" while "*:*/5" can be concluded as doing the task at every 5 minutes throughout the day.
Installing to our Clusters
Prerequisite
kube-green support all Kubernetes cluster versions >= 1.19 <= 1.28 and OpenShift Container Platform v4.
Supported architectures are: amd64, arm64.
To successfully install kube-green, in the cluster must be installed a cert-manager. If it is not already installed installed, click here.
kubectl apply
With this method, you can not change the default configuration parameters.
The default static configuration for the latest release can be installed with:
kubectl apply -f https://github.com/kube-green/kube-green/releases/latest/download/kube-green.yaml
If you want to install a specific release version, run:
kubectl apply -f https://github.com/kube-green/kube-green/releases/download/${RELEASE_TAG}/kube-green.yaml
with ${RELEASE_TAG} correctly filled.
kustomize
Change default configuration
You can change default configuration editing the config files.
For example, to deploy the controller in another namespace, change the file kustomization.yaml with the desired namespace name.
Deploy
To install kube-green in the cluster, clone the repository and run
make deploy
This will create a new namespace, kube-green, which contains the pod of the operator.
Operator Lifecycle Manager (OLM)
Install from OperatorHub.io
You can find kube-green in the OperatorHub.io website. Click the Install button, and follow the instructions.
Install on OpenShift
kube-green is in the Red Hat-provided Operator catalogs called community operators. On OpenShift 4 you can install kube-green from OperatorHub. To install it, follow this guide.
Cloud compatibility
GKE
When Google configure the control plane for private cluster, they automatically configure VPC peering between your Kubernetes cluster network and a separate Google managed project.
To restrict what Google is able to access in your cluster, the firewall rules configured restrict access to your Kubernetes pods. This means that the webhook won't work, and you see an error like Internal error occurred: failed calling webhook ...:
So, to use the webhook component with a GKE private cluster, you need to configure an additional firewall rule to allow the GKE control plane to access to your webhook pod.
kube-green uses webhook (exposed on port 9443) to check that SleepInfo CRD is valid. In order to make it works, you must open the 9443 port (or change the exposed port by configuration) otherwise it would not possible to add SleepInfo CRD.
Here you can read more information about how to add firewall rules to GKE.
AWS EKS
When using a custom CNI on EKS (such as cilium), the webhook cannot be reached by kube-green. This happens because the control plane cannot be configured to run on a custom CNI, so the CNIs differ between control plane and worker nodes.
To address this, set hostNetwork: true in the deployment of the webhook to run it in the host network.
To set this, uncomment the # - host_network_patch.yaml line (in this file) to apply the patch with the hostNetwork: true value.
Metrices
kube-green exposes Prometheus metrics available on port 8080 by default, exposed with /metrics path.
current_sleepinfo can be used for getting info about SleepInfo resource, with information about the name and the position where the SleepInfo is installed.
if your want to knwo abotu the project in real depth then please prefer goign through their Official:
Documentation: https://kube-green.dev/docs/getting-started/
Blog: https://kube-green.dev/blog/
Hope, you get to know some interesting facts and useful data which you might include in your cluster and organization to contribute 8in saving this Planet with reducing Carbon footprints. Soon, I will be releasing some more projects for Event Driven Automating like Keda, and other CNCF tools like Prometheus and Thanos very soon.
If you like my Article then please react to it and connect with me on Twitter if you are also a tech enthusiast. I would love to collaborate with people and share the experience of tech😄😄.
My Twitter Profile: