Follow these exercises to learn about native controls you can use to lock down your Kubernetes applications.
They’re meant to be done in order, but you should be able to complete each one on its own, too.
Exercise
Setup
To complete the exercises, you’ll need:
a Kubernetes cluster, and a command-line terminal with the kubectl client. Get a cluster If you’re completing these exercises on your own, you’ll need to provide your own cluster. Here are a few ways to get a cluster running:
Install Docker for Mac or Docker for Windows and enable Kubernetes. On Windows, keep the default setting so you can run Linux containers.
Exercise
Streamlined images
Introduction In this exercise, we cover:
Removing unnecessary tools from an image Changing the base image to a slimmer one Setup We’ll use an image with application-level vulnerabilities. This is a common case for both in-house applications and those that use common components. For example, Apache Struts is a framework that has had well-known exploitable vulnerabilities in the past.
For this example, we won’t have applied other controls already; many people tackle image security first, so this lets us see the impact of minimal images without other controls keeping adversaries isolated or at bay.
Exercise
Read-only root file system
Introduction In this exercise, we cover:
How to mark the root of your container read-only How to make certain paths writable if you need to Setup We’ll use the same image as before, because app vulnerabilities are useful to demonstrate attacks!
We’ll start with a writable file system and show how useful that is for an adversary.
This is the same as the previous exercise. Skip ahead to the Countermeasure if you just did that one.
Exercise
Read-only host mounts
Introduction In this exercise, we cover:
The risk of host mounts How to use read-only mounts if they suffice Setup In this example, we’ll just directly use a shell inside the app. Our example app will be a simulated monitoring agent mounting the same paths as the Datadog DaemonSet:
volumeMounts: - {name: dockersocket, mountPath: /var/run/docker.sock} - {name: procdir, mountPath: /host/proc, readOnly: true} - {name: cgroups, mountPath: /host/sys/fs/cgroup, readOnly: true} - {name: s6-run, mountPath: /var/run/s6} - {name: logpodpath, mountPath: /var/log/pods} ## Docker runtime directory, replace this path with your container runtime ## logs directory, or remove this configuration if `/var/log/pods` ## is not a symlink to any other directory.
Exercise
Network policies
Introduction In this exercise, we cover:
Interesting types of access that pods have How you can effectively limit network access Setup We’ll use an application that simulates a Server Side Request Forgery (SSRF), like the one involved in the Shopify bug bounty report, which ultimately allowed an adversary to steal cloud credentials from the metadata server.
We’ll use the simulated SSRF to see what a real problem like this could expose.
Exercise
Tune Kubernetes RBAC and account provisioning
Introduction In this exercise, we cover:
How service account identities are given to pods What access they have How you can remove them if you don’t need them Setup In this example, we’ll just directly use a shell inside the app. We’ll start with a default configuration and see what we can do.
We’ll also deploy an example where a service account has more access than required–say, it was intended to be used in one app and is not needed in another.
Exercise
Use separate namespaces
Introduction In this exercise, we cover:
What namespaces are How various types of Kubernetes objects treat namespace boundaries Setup In this example, we’ll use shells inside of apps to see how they can talk to one another. We’ll also intentionally make some mistakes and see if Kubernetes tells us we did something wrong.
Let’s start with everything in the default namespace:
kubectl apply -f https://securek8s.dev/namespaces/default.yaml “Attack” Contact other services in the same namespace, and accidentally mount a secret.
Exercise
Use a non-root user
Introduction In this exercise, we cover:
How user identities work in Kubernetes How to use a non-root user ID and enforce this in the future We will show how running as root:
Is the default behavior Lets you modify host files if mounted Allows other host modifications Still blocks other host modifications due to other controls (more on this in the following exercise) Note that almost all clusters run without username remapping, so generally the root user in the container is the same as the root user on the host.
Exercise
Avoid privileged mode
Introduction In this exercise, we cover:
What --privileged mode does The crazy things it allows How you might be able to replace it Setup We’ll exec directly into a container in this example.
This example is based on Ian Coldwater and Duffie Cooley’s example from Black Hat USA 2019.
As a preliminary step, we’ll deploy nginx and then spy on it later. (Note: If needed, adjust the replica count to match the number of nodes in your cluster.
Exercise
Set resource limits
Introduction In this exercise, we cover:
How to set resource limits What can happen if you don’t Note: This exercise can be a little finicky in cloud environments. And be careful that you don’t request too much memory if your cluster is running on your machine and doesn’t enforce overall memory limits.
Setup In this example, we’ll use an app with an exploitable memory exhaustion denial of service. This will be fun to watch…