Configuring K8s security boundaries
Security boundaries
- Hypervisor is security boundary for VM
- container is security boundary
Hypervisor is stronger. With container, attacker can bypass the boundary using kernel layer.
- firewall is security boundary in nw, traditionally.
- netpol is security boundary in nw in k8s
- RW access is security boundary in data layer.
- chroot is security boundary in data layer in k8s
Definition
a security domain is a set of entities that are within the same access level.
a security boundary is a point of demarcation between different security domains
Trusted Zones Components
1. Internet: kubectl, application clients
2. API Server: kube-apiserver
3. Master Node components: kube-controller-manager, cloud-controller-manager
4. Master Node database: etcd
5. Wokrer Node: kubelet, kubeproxy
6. Container: Container Runtime.
Security Domain
1. K8s Master Components: kube-apiserver, etcd, kube-controller-manager, DNS server, kube-scheduler
2. K8s Worker Components: kublet, kubeproxy
3. K8s objects:
K8s objects + components and their boundary
1. Container: cgroups, namespace, AppArmor profile, seccomp profile
2. Pod: above + nw, IPC, SecurityContext, netpol, psp
3. Node: nodeSelectors, kernel, optinally hypervisor, host - hardening by AppArmor and SELinux
4. Cluster:
5. NS: LimitRanger admission controller, netpol
6. The K8s API Server
Threat actor and their boundary
1. End user: nodes, pods, kube-apiserver, firewall.
2. Internal attacker: Netpol and RBAC
3. Privileged attacker: Only kube-apiserver
Security boundary in system layer. Use PSP
1. NS
- HostNetwork
- HostIPC
- HostPID
- shareProcessNamespace
2. Linux Capabilities https://man7.org/linux/man-pages/man7/capabilities.7.html
Default capabilities
• CAP_SETPCAP
• CAP_MKNOD
• CAP_AUDIT_WRITE
• CAP_CHOWN
• CAP_NET_RAW
• CAP_DAC_OVERRIDE
• CAP_FOWNER
• CAP_FSETID
• CAP_KILL
• CAP_SETGID
• CAP_SETUID
• CAP_NET_BIND_SERVICE
• CAP_SYS_CHROOT
• CAP_SETFCAP
Security boundary in nw layer : Netpol
Least privilege for K8s workloads
Definition: Any subject, user, program, process, and so on should only have the minimum required privileges to perform its function.
Authorization Models
1. ACL: Subject, Object, Operations: Linux and network firewalls
2. RBAC: Subject has role. Role, Object, Operations. DB
3. ABAC: Subject has attributes (e.g. label, property). Attribute, Object, Operations.
RBAC
- Subjects
-- Regular Users
-- SA (If a Microservice does not interact with kube-apiserver or any other K8s object, then it does not need any privileges)
-- Annonymus users
-- Groups
- Role / ClusterRole
-- Collection of persmissions
-- Role creatoin / modification is restricted.
- RoleBinding / ClusterRoleBinding
-- Binds role and subject.
NS: In K8s, NS allows a single cluster to be logically shared among teams/projects
- NS have resource quota
- RBAC per NS
- NetPol per NS
- Nodes and PV do not belong to any NS.
One can specify NS with "k apply -f" command also.
Security Context
- at pod level and at container level
- It about accessing worker node system resources.
1. Discretionary Access Control (DAC)
-- User ID
-- Group ID
-- container's root file system is RO or not.
2. Security Enhanced Linux (SELinux)
- Label for level, role, type, user
- volumes on the node
3. Privileged Mode
4. Linux Capabilities : https://man7.org/linux/man-pages/man7/capabilities.7.html
5. AppArmor profile
- Linux capabilities
- file access
- nw resources access
6. Secure Computing Mode (seccomp): witelist and blacklist for system calls.
7. AllowPrivilegeEscalation: Process can give more privileges than its parents or not.
if (CAP_SYS_ADMIN || Privileged mode) then AllowPrivilegeEscalation = true
PodSecurityPolicy: Pod's scrutiny attributes. Defined at cluster level.
- privileged container (Privileged Mode)
- host-level ns (network, PID, IPC)
- host ports
- different types of volumes
- host's filesystem
- RO root filesystem for containers (DAC)
- user IDs and group IDs (DAC)
- containers' privilege escalation (AllowPrivilegeEscalation)
- containers' Linux capabilities (Linux Capabilities)
- SELinux
- seccomp and AppArmor profiles
- sysctls that a pod can run
- a proc mount type to be used
- an FSGroup to volumes
Resource Limit Control
- request
- limit
Least privilege for nw resources using netpol
Least privilege for app resources with proper design of app.
K8s Threat Modelling
Threat Modelling
1. Asset
2. Security control
3. Threat Actor
- End user
- Internal attacker: Malicious containers/pods
- Privileged attacker: Infra admin, compromised API server, malicious node.
- script kiddies,
- hacktivists,
- nation-state actors
4. Attack Surface
- End user: LB, Ingress, NodePort service
5. Threat
6. Mitigation
Threat Scenarios
● An External Attacker without access to the client application
● An External Attacker with valid access to the client application
● An Internal Attacker with access to cluster
● A Malicious Internal User
Threat Modelling Approach
1. STRIDE: Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, and Escalation of Privilege.
What can go wrong with System
2. PASTA: Process for Attack Simulation and Threat Analysis
attacker-centric approach to develop asset-centric migration strategies
3. VAST: Visual, Agile, and Simple Threat
Trusted Zones
Zone: Components
1. Internet: kubectl, application clients
2. API Server: kube-apiserver
3. Master Node components: kube-controller-manager, cloud-controller-manager
4. Master Node database: etcd
5. Wokrer Node: kubelet, kubeproxy
6. Container: Container Runtime.
Trust Zone connections
1. Internet -> API Server
2. Internet -> Container
3, API Server -> Master node database
4. Master Node components -> API Server
5. ApPI Server -> Worker Node
6. Worker node -> API Server
7. Worker Node -> Container
krew
Krew is plugin manager for kubectl command line tool
Krew helps you:
- discover kubectl plugins,
- install them on your machine,
- and keep the installed plugins up-to-date.
