Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

🏠 Back to Blog

Cluster Hardening (15%)

Securing Access to the KubeAPI Server

  • A request to the KubeAPI server goes through 4 stages before it is processed by KubeAPI:
    • Authentication
      • Validates the identity of the caller by inspecting client certificates or tokens
    • Authorization
      • The authorization stage verifies that the identity found in the first stage can access the verb and resource in the request
    • Admission Controllers
      • Admission Control verifies that the requst is well-formed and/or potentially needs to be modified before proceeding
    • Validation
      • This stage ensures that the request is valid.
  • You can determine the endpoint for the kubeapi server by running: kubectl cluster-info
  • KubeAPI is also exposed via a service named ‘kubernetes’ in the default namespace
    kubeadmin@kube-controlplane:~$ k get svc kubernetes -n default -o yaml
      apiVersion: v1
      kind: Service
      metadata:
        creationTimestamp: "2024-11-11T10:57:42Z"
        labels:
          component: apiserver
          provider: kubernetes
        name: kubernetes
        namespace: default
        resourceVersion: "234"
        uid: 768d1a22-91ff-4ab3-8cd7-b86340fc319a
      spec:
        clusterIP: 10.96.0.1
        clusterIPs:
        - 10.96.0.1
        internalTrafficPolicy: Cluster
        ipFamilies:
        - IPv4
        ipFamilyPolicy: SingleStack
        ports:
        - name: https
          port: 443
          protocol: TCP
          targetPort: 6443
        sessionAffinity: None
        type: ClusterIP
      status:
        loadBalancer: {}
    
    • The endpoint of the kube-api server is also exposed to pods via environment variables:
    kubeadmin@kube-controlplane:~$ k exec -it other -- /bin/sh -c 'env | grep -i kube'
     KUBERNETES_SERVICE_PORT=443
     KUBERNETES_PORT=tcp://10.96.0.1:443
     KUBERNETES_PORT_443_TCP_ADDR=10.96.0.1
     KUBERNETES_PORT_443_TCP_PORT=443
     KUBERNETES_PORT_443_TCP_PROTO=tcp
     KUBERNETES_SERVICE_PORT_HTTPS=443
     KUBERNETES_PORT_443_TCP=tcp://10.96.0.1:443
     KUBERNETES_SERVICE_HOST=10.96.0.1
        ``` 
    
    

Authentication

  • There are two types of accounts that would need access to a cluster: Humans and Machines. There is no such thing as a ‘user account’ primitive in Kubernetes.

User accounts

  • Developers, cluster admins, etc.

Service Accounts

  • Service Accounts are created and managed by the Kubernetes API and can be used for machine authentication
  • To create a service account: kubectl create serviceaccount <account name>
  • Service accounts are namespaced
  • When a service account is created, it has a token created automatically. The token is stored as a secret object.
  • You can also use the base64 encoded token to communicate with the Kube API Server: curl https://172.16.0.1:6443/api -insecure --header "Authorization: Bearer <token value>"
  • You can grant service accounts permission to the cluster itself by binding it to a role with a rolebinding. If a pod needs access to the cluster where it is hosted, you you configure the automountServiceAccountToken boolean parameter on the pod and assign it a service account that has the appropriate permissions to the cluster. The token will be mounted to the pods file system, where the value can then be accessed by the pod. The secret is mounted at /var/run/secrets/kubernetes.io/serviceaccount/token.
  • A service account named ‘default’ is automatically created in every namespace
  • As of kubernetes 1.22, tokens are automatically mounted to pods by an admission controller as a projected volume.
    • https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/1205-bound-service-account-tokens/README.md
  • As of Kubernetes 1.24, when you create a service account, a secret is no longer created automatically for the token. Now you must run kubectl create token <service account name> to create the token.
    • https://github.com/kubernetes/enhancements/issues/2799
  • One can also manually create a token for a service account:
kubectl create token <service-account-name> --duration=100h

TLS Certificates

  • Server certificates are used to communicate with clients
  • Client certificates are used to communicate with servers
  • Server components used in Kubernetes and their certificates:
    • kube-api server: apiserver.crt, apiserver.key
    • etcd-server: etcdserver.crt, etcdserver.key
    • kubelet: kubelet.crt, kubelet.key
  • Client components used in kubernetes and their certificates:
    • user certificates
    • kube-scheduler: scheduler.crt, scheduler.key
    • kube-controller-manager: controller-manager.crt, controller-manager.key
    • kube-proxy: kubeproxy.crt, kubeproxy.key
  • To generate a self-signed certificate: openssl req -nodes -x509 -keyout my.key -out my.crt --subj="/CN=mysite.com"
  • To generate certificates, you can use openssl:
    • Create a new private key: openssl genrsa -out my.key 2048
    • Create a new certificate signing request: openssl req -new -key my.key -out my.csr -subj "/CN=ryan"
    • Sign the csr and generate the certificate or create a signing request with kube-api:
      • Sign and generate: openssl x509 -req -in my.csr -out my.crt
      • Create a CertificateSigningRequest with kube-api:
        # extract the base64 encoded values of the CSR:
        cat my.csr | base64 | tr -d '\n'
        
        # create a CertificateSigningRequest object with kube-api, provide the base64 encoded value
        .... see the docs
        
  • kubeadm will automatically generate certificates for clusters that it creates
    • kubeadm generates certificates in the /etc/kubernetes/pki/ directory
  • To view the details of a certificate, use openssl: openssl x509 -in <path to crt> -text -noout
  • Once you have a private key, you can sign it using the CertificateSigningRequest object. The controller manager is responsible for signing these requests. You can then use the signed certificate values to authenticate to the Kube API server by placing the signed key, certificate, and ca in a kube config file (~/.kube/config)

kubelet Security

  • By default, requests to the kubelet API are not authenticated. These requests are bound to an ‘unauthenticated users’ group. This behavior can be changed by setting the --anonymous-auth flag to false in the kubelet config
  • kubelet ports
    • port 10250 on the machine running a kubelet process serves an API that allows full access
    • port 10255 on the machine running a kubelet process serves an unauthenticated, read-only API
  • kubelet supports 2 authentication mechanisms: bearer token and certificated-based authentication
  • You can find the location of the kubelet config file by looking at the process: ps aux |grep -i kubelet

Authorization

Roles and ClusterRoles

  • Roles and clusteroles define what a user or service account can do within a cluster
  • The kubernetes primitive role is namespaced, clusterrole is not

Role Bindings and Cluster Role Bindings

  • rolebinding and clusterrolebinding link a user or service account to a role