Building a Resilient Kubernetes Cluster: Enforcing Policies with OPA

Policy controllers, such as Open Policy Agent (OPA), are used in Kubernetes to enforce policy-based governance and to ensure that the cluster and its resources adhere to specific rules and requirements.

Here are some reasons why policy controllers like OPA are essential in Kubernetes:

Policy Enforcement: Policy controllers help enforce security, compliance, and operational policies within a Kubernetes cluster. They allow administrators to define and enforce policies that regulate various aspects of the cluster, such as pod security, resource quotas, network policies, access controls, and more. By applying policies, organizations can ensure that their cluster configurations align with best practices and compliance standards.

Flexible Policy Language: OPA provides a flexible policy language, known as Rego, which allows users to express complex policies in a declarative and concise manner. This language enables policy authors to define fine-grained rules and conditions for enforcing policies. With OPA, policies can be written and maintained independently of the underlying applications, providing a separation of concerns, and making it easier to update or modify policies as needed.

Dynamic Policy Evaluation: Policy controllers continuously evaluate policies against the current state of the cluster and its resources. This dynamic evaluation enables real-time policy enforcement and ensures that any changes or updates to the cluster are immediately evaluated against the defined policies. This capability is particularly useful in dynamic environments where the cluster and its workloads are constantly evolving.

Auditing and Compliance: Policy controllers like OPA can generate detailed audit logs and reports to track policy violations and ensure compliance with regulatory requirements. These logs can be used for monitoring, troubleshooting, and demonstrating adherence to security and compliance standards during audits or assessments. By having a policy controller in place, organizations can have greater visibility into policy violations and take necessary actions to rectify them.

Integration with Kubernetes API: Policy controllers integrate directly with the Kubernetes API server, allowing them to intercept and evaluate requests for cluster resources before they are allowed or denied. This integration provides a seamless and centralized approach to policy enforcement across the cluster, ensuring consistent application of policies regardless of the access points used by users or applications.

What is Gatekeeper?

Gatekeeper is an open-source project and a policy controller for Kubernetes that enforces custom policies for resource admission. It is built on top of the Open Policy Agent (OPA) framework and provides a declarative way to define and enforce policies during the admission process of Kubernetes resources.

https://github.com/open-policy-agent/gatekeeper

The main purpose of Gatekeeper is to validate and ensure that Kubernetes resources created or modified within a cluster comply with specific rules and requirements defined by the policies. It acts as a gatekeeper at the time of resource admission, preventing the creation or modification of resources that violate the defined policies.

OPA Gatekeeper in Action:

In this example, we will use a sample policy that creates a constraint in creating any Kubernetes objects without labels.

Install OPA Gatekeeper:

Installation Guide: https://open-policy-agent.github.io/gatekeeper/website/docs/install

Once the OPA Gatekeeper is deployed in your kubernetes cluster, we need to create a ConstraintTemplate with our rules, here in this example below, our rule specifies that every kubernetes object we create needs to have a resource label.

opa_template.yaml

apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
  name: k8srequiredlabels
spec:
  crd:
    spec:
      names:
        kind: K8sRequiredLabels
      validation:
        # Schema for the `parameters` field
        openAPIV3Schema:
          properties:
            labels:
              type: array
              items: string
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8srequiredlabels
        violation[{"msg": msg, "details": {"missing_labels": missing}}] {
          provided := {label | input.review.object.metadata.labels[label]}
          required := {label | label := input.parameters.labels[_]}
          missing := required - provided
          count(missing) > 0
          msg := sprintf("you must provide labels: %v", [missing])
        }

controlplane $ k -f /root/opa_template.yaml apply
constrainttemplate.templates.gatekeeper.sh/k8srequiredlabels created

controlplane $ k get constrainttemplates
NAME               AGE
k8srequiredlabels  108s

opa_constraint.yaml

We are more specific with the constraint rule now, we are saying every namespace that we create should have "cks" in the label.

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabels
metadata:
  name: ns-must-have-cks
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Namespace"]
  parameters:
    labels: ["cks"]

controlplane $ k -f /root/opa_constraint.yaml apply
k8srequiredlabels.constraints.gatekeeper.sh/ns-must-have-cks created

controlplane $ k get k8srequiredlabels
NAME               AGE
ns-must-have-cks   1s

controlplane $ k create ns test
Error from server ([ns-must-have-cks] you must provide labels: {"cks"}): admission webhook "validation.gatekeeper.sh" denied the request: [ns-must-have-cks] you must provide labels: {"cks"}
controlplane $

In Summary, Gatekeeper enhances the security, compliance, and governance aspects of Kubernetes clusters by allowing administrators to define and enforce custom policies during the resource admission process. It provides a powerful tool to ensure that only resources that meet the defined policies are allowed into the cluster, reducing the risk of misconfigurations and unauthorized access.