In our cloud-agnostic digital world, security is a crucial aspect, be it a Fortune 500 company or a small service provider gaining traction.
The importance of security elevates when deployment relies on cloud-based environments. And all companies rely extensively on microservices when deploying an application or integrating a feature onto an existing application. Due to its unparalleled reputation, Kubernetes is the go-to-market leader.
Kubernetes is the #1 open source orchestration platform for managing containerized applications. When it first came onto the scene, it drastically altered how DevOps teams’ approaches to creating, managing, and operating container-based applications.
One exclusive feature of Kubernetes is that Kubernetes is an evolving project, with improvements being made all the time. The Kubernetes team is already progressing towards cooking their latest solutions and approaches to address security vulnerabilities that have come to light and will offer them to us in their next release.
To fully take advantage of the versatility of this tool, it is important to be smart in your build process, knowing it so you can best utilize it. This is what we will be discussing in this article.
When building a CI/CD pipeline to deploy containers and images, Kubernetes security best practices come into the picture at three different levels in Kubernetes: cluster level, node level, and underlying host OS level. Scanning and patching container images to detect vulnerabilities and report them is crucial:
Note: Keep K8s up to date to improve security.
Let’s unwrap the levels one by one and explore the best practices and how to handle the situation with K8s.
Application Level Security
This level comes under developer consideration and is the topmost layer of security. Resource and data isolation from one another is achieved by containerization of the application, but this alone is not enough for securing the services.
Some of the best security practices at the application level are as follows:-
Setting API server access right and using secrets
Kubernetes API server offers a powerful REST API for controlling Kubernetes. The write permissions on this API has the equivalent of root access on every machine in the cluster.
Kubectl is a CLI client for this API. Resources and workloads can be managed by making requests of kubectl. By default, this API server will listen to the insecure port: port 8080. Requests to this port will bypass authentication and authorization checks.
If you leave this port open, anyone who gains access to the host on which your master is running will have complete control over your entire cluster.
Solution: Ensure that the — insecure-bind-address is not set and that all sensitive data, such as passwords or tokens to connect to an API of an application, are stored within Kubernetes Secrets.
Access control through RBAC
Role-based access control (RBAC) is a method of governing access to resources based on the roles that help you define who has access to the Kubernetes API with what permissions.
With the insecure port 8080 closed, the API can be accessed only over a secure encrypted TLS connection. You may want to further restrict API access to known, authenticated users.
Solution: set — anonymous-auth=false. Opt for namespace-specific permissions instead of cluster-wide permissions.
As long as we are leveraging the functionality of RBAC, we can skip the above step and allow access to anonymous access. RBAC default settings will take care of it.
Cluster hardening
By default, Docker containers run as ‘root user’ which results in security issues sometimes. Restricting user access can be the best way to secure applications deployed within a K8s cluster.
Solution: Specify the Run as User/Run as Groupin the deployment YAML file for a pod definition.
Cluster Level Security
This is the middle level and central layer where K8s clusters and their components interact with each other.
Configuring and restricting etcd
Kubernetes stores configuration and state information in a distributed key-value store called etcd. Someone gaining read/write permission to etcd gives the person potential control over the K8s cluster.
Solution: Establish HTTPS connections etcd by setting — cert-file and — key-file.
The traffic to etcd should be encrypted. Ensuring access to etcd requires applying the authentication setting to — client-cert-auth=true .
Make sure that the API server will identify itself to etcd by specifying — etcd-certfile and — etcd-keyfile and by setting many other configurations.
Scanning and limiting container images
Software is imported into K8s clusters via container images. Security breaches can occur when there is a misconfiguration or an outdated software is used. Leveraging a container image scanner to inspect the packages included in an image can be helpful for reporting known vulnerabilities from the packages.
Solution: Use the AlwaysPullImages admission controller to ensure that the most recent version is used.
Tag images with semantic versioning.
Use container scanners.
Configuring Admission Controllers
Kubernetes Admission Controller is an advanced plugin for gating and governing the usage of clusters. Making use of an Admission Controller can provide a great intent of control over security.
Solution: Defining access restrictions to API server nodes and designing custom network policies.
Implement dynamic admission control for ensuring only successfully scanned images are deployed.
Host Level Security
This is the level where the actual Kubernetes is installed.
Restricting privilege access
Remote execution through application pods should be restricted and should not expose host-level OS to ssh.
Solution: Set up IAM policies to restrict access to K8s resources.
Limit modification access to admins
Installing new software at the host level and updating them should be limited because this could act as a backdoor to hackers installing malicious software.
Solution: Restrict execution of sudo/yum installs only to sysadmins.
Conclusion
Security should be at the heart when designing and developing complex and advanced systems targeted at end-users or giant firms. The most efficient way for build a complex application or system is by using microservices and Kubernetes as a containerized deployment engine.
As developers and architects, it is our responsibility to make sure we have the best and most secure products and applications to deliver by avoiding any weak points that can be a security threat or a vulnerability to the whole ecosystem.
This post provides the best practices to keep in mind when building containerized applications in a CI/CD pipeline.
