Express YourSelf !

Please execute this command at master node, to see the values of configuration parameters / flags for K8s Controller Manager

ps -ef | grep kube-controller-manager

As per section 1.3 of CIS Benchmark below table describe (1) parameter (2) its default value and (3) its desired value

 Please execute this command at master node, to see the values of configuration parameters / flags for K8s Scheduler

ps -ef | grep kube-scheduler

As per section 1.4 of CIS Benchmark

* --profiling should be change from default true to false

* --bind-address should be change from 0.0.0.0 to 127.0.0.1 

Additional tips not in CIS Benchmark

* Ensure that AppArmor is not disabled for kube-scheduler.

* AllowExtTrafficLocalEndpoints feature is disabled using --feature-gates

Please execute this command at master node, to see the values of configuration parameters / flags for etcd

ps -ef | grep etcd

As per section 2 of CIS Benchmark below table describe (1) parameter (2) its default value and (3) its desired value

Please execute this command at master node, to see the values of configuration parameters / flags for kubelet. This is for first column. 

ps -ef | grep kubelet | grep config

The default configuration path can be found from /etc/systemd/system/kubelet.service.d/10-kubeadm.conf This is for second column. 

As per section 4.2 of CIS Benchmark below table describe (1) parameter with desired value (2) its default value

Please execute this command at master node, to see the values of configuration parameters / flags for K8s API server

ps -ef | grep kube-apiserver

As per section 1.2 of CIS Benchmark below table describe (1) parameter (2) its default value and (3) its desired value

 Security boundaries

- Hypervisor is security boundary for VM

- container is security boundary

Hypervisor is stronger. With container, attacker can bypass the boundary using kernel layer. 

- firewall is security boundary in nw, traditionally. 

- netpol is security boundary in nw in k8s

- RW access is security boundary in data layer. 

- chroot is security boundary in data layer in k8s

Definition

a security domain is a set of entities that are within the same access level.

a security boundary is a point of demarcation between different security domains

Trusted Zones Components

1. Internet: kubectl, application clients

2. API Server: kube-apiserver

3. Master Node components: kube-controller-manager, cloud-controller-manager

4. Master Node database: etcd

5. Wokrer Node: kubelet, kubeproxy

6. Container: Container Runtime.

Security Domain

1. K8s Master Components: kube-apiserver, etcd, kube-controller-manager, DNS server, kube-scheduler 

2. K8s Worker Components: kublet, kubeproxy

3. K8s objects: 

K8s objects + components and their boundary

1. Container: cgroups, namespace, AppArmor profile, seccomp profile

2. Pod: above + nw, IPC, SecurityContext, netpol, psp

3. Node: nodeSelectors, kernel, optinally hypervisor, host - hardening by AppArmor and SELinux

4. Cluster: 

5. NS: LimitRanger admission controller, netpol

6. The K8s API Server

Threat actor and their boundary

1. End user: nodes, pods, kube-apiserver, firewall.

2. Internal attacker: Netpol and RBAC

3. Privileged attacker: Only kube-apiserver

Security boundary in system layer. Use PSP

1. NS

- HostNetwork

- HostIPC

- HostPID

- shareProcessNamespace

2. Linux Capabilities https://man7.org/linux/man-pages/man7/capabilities.7.html

Default capabilities

• CAP_SETPCAP

• CAP_MKNOD

• CAP_AUDIT_WRITE

• CAP_CHOWN

• CAP_NET_RAW

• CAP_DAC_OVERRIDE

• CAP_FOWNER

• CAP_FSETID

• CAP_KILL

• CAP_SETGID

• CAP_SETUID

• CAP_NET_BIND_SERVICE

• CAP_SYS_CHROOT

• CAP_SETFCAP

Security boundary in nw layer : Netpol

Definition: Any subject, user, program, process, and so on should only have the minimum required privileges to perform its function.

Authorization Models

1. ACL: Subject, Object, Operations: Linux and network firewalls

2. RBAC: Subject has role. Role, Object, Operations. DB

3. ABAC: Subject has attributes (e.g. label, property). Attribute, Object, Operations. 

RBAC

- Subjects

-- Regular Users

-- SA (If a Microservice does not interact with kube-apiserver or any other K8s object, then it does not need any privileges)

-- Annonymus users

-- Groups

- Role / ClusterRole

-- Collection of persmissions

-- Role creatoin / modification is restricted. 

- RoleBinding / ClusterRoleBinding

-- Binds role and subject. 

NS: In K8s, NS allows a single cluster to be logically shared among teams/projects 

- NS have resource quota

- RBAC per NS

- NetPol per NS

- Nodes and PV do not belong to any NS. 

One can specify NS with "k apply -f" command also. 

Security Context

- at pod level and at container level

- It about accessing worker node system resources. 

1. Discretionary Access Control (DAC)

-- User ID

-- Group ID

-- container's root file system is RO or not. 

2. Security Enhanced Linux (SELinux)

- Label for level, role, type, user 

- volumes on the node

3. Privileged Mode

4. Linux Capabilities : https://man7.org/linux/man-pages/man7/capabilities.7.html 

5. AppArmor profile

- Linux capabilities

- file access

- nw resources access

6. Secure Computing Mode (seccomp): witelist and blacklist for system calls. 

7. AllowPrivilegeEscalation: Process can give more privileges than its parents or not.

if (CAP_SYS_ADMIN || Privileged mode) then AllowPrivilegeEscalation = true

PodSecurityPolicy: Pod's scrutiny attributes. Defined at cluster level. 

- privileged container (Privileged Mode)

- host-level ns (network, PID, IPC)

- host ports

- different types of volumes

- host's filesystem

- RO root filesystem for containers (DAC)

- user IDs and group IDs (DAC)

- containers' privilege escalation (AllowPrivilegeEscalation)

- containers' Linux capabilities (Linux Capabilities)

- SELinux

- seccomp and AppArmor profiles

- sysctls that a pod can run

- a proc mount type to be used

- an FSGroup to volumes

Resource Limit Control

- request

- limit

Least privilege for nw resources using netpol

Least privilege for app resources with proper design of app. 

Threat Modelling

1. Asset

2. Security control

3. Threat Actor

- End user

- Internal attacker: Malicious containers/pods 

- Privileged attacker: Infra admin, compromised API server, malicious node.

- script kiddies, 

- hacktivists, 

- nation-state actors

4. Attack Surface

- End user: LB, Ingress, NodePort service

5. Threat

6. Mitigation

Threat Scenarios

● An External Attacker without access to the client application

● An External Attacker with valid access to the client application

● An Internal Attacker with access to cluster

● A Malicious Internal User

Threat Modelling Approach

1. STRIDE: Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, and Escalation of Privilege.

What can go wrong with System

2. PASTA: Process for Attack Simulation and Threat Analysis

attacker-centric approach to develop asset-centric migration strategies

3. VAST: Visual, Agile, and Simple Threat

Trusted Zones

Zone: Components

1. Internet: kubectl, application clients

2. API Server: kube-apiserver

3. Master Node components: kube-controller-manager, cloud-controller-manager

4. Master Node database: etcd

5. Wokrer Node: kubelet, kubeproxy

6. Container: Container Runtime. 

Trust Zone connections

1. Internet -> API Server

2. Internet -> Container 

3, API Server -> Master node database

4. Master Node components -> API Server

5. ApPI Server -> Worker Node

6. Worker node -> API Server

7. Worker Node -> Container