Security | Jul 16, 2021

Kubernetes Security: Protect Internal Traffic with Policy as Code (CVE-2021-25737 and CVE-2021-25740)

Kubernetes security: Terrascan as a Validating Admission Controller Banner (1920x1080)

Kubernetes security can be frustrated by its built-in customizability and the fact that no default settings will be secure for every situation.  Users must configure certain functionality in order to secure their cluster. This means that the engineers responsible for deploying the Kubernetes platform need to understand how poor and incomplete configuration can lead to potential attack vectors and vulnerabilities.

Given the rapid pace of innovation in the Kubernetes ecosystem, it is not feasible to keep track of all potential attack vectors across different versions. The best practice is to ensure you are running the latest available version of Kubernetes and its components, including control plane and node components.

However, vulnerabilities are occasionally discovered even in the latest versions, as we’ve discussed before (for example in this earlier blog).  The blog you are currently reading describes two additional vulnerabilities, one of which exists in the latest version of Kubernetes.

So, how can we ensure our Kubernetes clusters remain secure, and we mitigate such vulnerabilities? That’s a complicated topic that I don’t have space to cover in this post; instead, I refer you to an earlier blog which goes into more detail. 

Securing internal Kubernetes traffic with Policy as Code

In this blog post, I will be breaking down two recent vulnerabilities disclosed by the Kubernetes team. The first is CVE-2021-25740, which enables a potential attacker to create or edit Endpoints or EndpointSlices in the Kubernetes API, potentially allowing a LoadBalancer or Ingress implementation to expose backend IPs the attacker should not have access to. The second is CVE-2021-25737, which allows EndpointSlice validation to enable an attacker to hijack host network traffic.

I will discuss how these vulnerabilities can be mitigated using a Policy as Code tool such as Terrascan in your GitOps pipelines as well as how imperative changes to your cluster can be mitigated using Terrascan as an admission controller.

CVE-2021-25740

Versions AffectedAll versions of Kubernetes
Impact Attacker can gain access to backend services to which they are not entitled.
CVSS ScoreCVSS:3.0/AV:N/AC:H/PR:L/UI:N/S:U/C:L/I:N/A:N
Attack VectorDesign flaw where a service is created or already present without a selector.
MitigationThere is no patch for this issue, and it can currently only be mitigated by restricting access to the vulnerable features. To mitigate the exposure, we recommend restricting write access to Endpoints and EndpointSlices by updating the system:aggregate-to-edit role

Vulnerable Example

apiVersion: v1
kind: Service
metadata:
 name: cve-example
spec:
 ports:
   - name: http
     protocol: TCP
     port: 80
     targetPort: 8080
 type: LoadBalancer
 

CVE-2021-25737

Versions Affected

This vulnerability is for Kube API Server and the versions affected are: 

  • kube-apiserver v1.21
  • kube-apiserver v1.20.0 to v1.20.6
  • kube-apiserver v1.19.0 to v1.19.10
  • kube-apiserver v1.16.0 – v1.18.18
Impact Attacker can hijack your cluster’s network traffic; leak sensitive data; exfiltrate credentials that could lead to lateral movement and further compromise.
CVSS ScoreCVSS:3.1/AV:N/AC:L/PR:H/UI:N/S:U/C:L/I:N/A:N
Attack VectorPermission to create an Endpoint for a Backend service such as one accessible on loopback addresses.
Mitigation

This issue has been fixed in the following versions of kube-apiserver:

  • v1.21.1
  • v1.20.7
  • v1.19.11
  • V1.18.19

Vulnerable Example

apiVersion: discovery.k8s.io/v1
  kind: EndpointSlice
  metadata:
   name: example-abc
   labels:
     kubernetes.io/service-name: example
  addressType: IPv4
  ports:
   - name: http
     protocol: TCP
     port: 80
  endpoints:
   - addresses:
       - "169.254.169.254"
     conditions:
       ready: true
     hostname: pod-1
     nodeName: node-1
     zone: us-west2-a
 

Securing your GitOps workflow with Terrascan

CVE-2021-25737 does not have any patch and for CVE-2021-25737 let’s assume your organisation does not want to upgrade to the latest version of kube-apiserver. Provided your organisation is using Argo CD as a declarative GitOps framework to manage Kubernetes, then you can use Terrascan’s integration with Argo CD announced during KubeCon + CloudNativeCon Europe 2021.

I won’t go through the steps to integrate Terrascan, you can find it in Terrascan’s documentation. Let’s look at the policies that Terrascan will use to detect misconfigured K8s configurations that will trigger the vulnerabilities mentioned above.  If we detect that such vulnerabilities will be triggered, we will break the Argo CD build.

Policy AC_K8S_0114 for CVE-2021-25740

package accurics
 
{{.prefix}}{{.name}}{{.suffix}}[service.id] {
   service := input.kubernetes_service[_]
   service_config := service.config
   service_config.spec.type == ["LoadBalancer", "Ingress"][_]
   object.get(service_config.spec, "selector", "undefined") == "undefined"
} 

Terrascan policies are written in Rego.  Policy AC_K8S_0114 above will look at all K8s service definitions, and the build will fail if any services have no selector defined. 

Policy AC_K8S_0113 for CVE-2021-25737

package accurics
{{.prefix}}{{.name}}{{.suffix}}[endpoint_slice.id] {
  endpoint_slice = input.kubernetes_endpoint_slice[_]
  address := endpoint_slice.config.endpoints[_].addresses[_]
 
  not_allowed_addresses := ["127.0.0.0/8", "169.254.0.0/16"]
  net.cidr_contains(not_allowed_addresses[_], address)
}
 

Policy AC_K8S_0113 will be triggered and the build will fail whenever the manifest includes endpoint addresses in the 127.0.0.0/8 and 169.254.0.0/16 internal range.

If you configure ArgoCD with Terrascan as a PreSync hook and scan the remote repository, the flow diagram will be like this:

Terrascan integration enables policy as code in Argo CD workflows
Terrascan integration enables policy as code in Argo CD workflows

We can see the job terrascan-hook has failed and if we look at the logs we can see the violations generated against the manifests:

Argo CD logs show Terrascan policy violations when present
Argo CD logs show Terrascan policy violations when present.

Securing your Kubernetes runtime with Terrascan

Now, to proceed further, let’s pass our Argo CD build by adding a selector field and changing the endpoint address from 169.254.169.254 to 10.10.10.1.

There is a high likelihood that changes to the cluster can be made by tools other than your CD tool, either by mistake or by malice. To protect against such imperative changes, Terrascan can be configured as a validating admission controller. One of the great things about Terrascan is that the same policy discussed above can also be used by Terrascan running as an admission controller: same tool, same policies, one Terrascan

If you want to learn how to configure Terrascan as an admission controller, my colleague Yusuf has written a blog post about it.

Terrascan can be deployed as a server along with validating webhook using the Helm chart available here.

Before installing the Helm chart ensure you have created a config.toml and placed it in the path deploy/helm/data/config.toml.

A configuration to identify the violations described in this post might use a config.toml like this:

[severity]
  level = "low"
[rules]
  scan-rules = [
    "AC_K8S_0113",
    “AC_K8s_0114”
  ]
[k8s-admission-control]
  denied-categories = [
    "Infrastructure Security"
  ]
  denied-severity = "low" 

We can test by using kubectl to create a service without a selector, similar to the vulnerable example described above for CVE-2021-25740.  With the admission controller in place, we can see that the problem was identified and the change was not accepted by the server:

Terrascan admission controller for Kubernetes rejects change that violates policy
Terrascan admission controller for Kubernetes rejects change that violates policy

To conclude, integrating policy as code with your GitOps workflow and then implementing an admission controller to reject imperative changes in runtime will not only help you achieve your Kubernetes security posture at the early stages of development but also help you maintain the security posture against imperative changes directly made to the cluster.

Accurics + Tenable: Building the Future of Cloud Security

Extend–don’t shift–left cloud security posture management (CSPM)

Terraform Security: Six Habits for Secure Terraform Workflows

We use cookies to ensure you get the best experience on our website. By continuing to browse this site, you acknowledge the use of cookies.