5. Securing Kube-APIServer: PSP, IAM, CIS

Posted by Manish Panchmatia on Wednesday, November 3, 2021 Labels: , , ,

Pod Security Policy (PSP)

- A set of rules

- provide/modify default values for fields

- change pod

- PSP ordered by name before applied. 

- Deprecated in K8s 1.21

- will be removed in K8s 1.25

Even if you are only planning on changing a single value, the policy file must contain several entries. Sample PSP, where pod can do anything

apiVersion: policy/v1beta1

kind: PodSecurityPolicy
metadata:
   name: basicpolicy
spec:
privileged: true
runAsUser:
   rule: RunAsAny
seLinux:
  rule: RunAsAny
fsGroup:
  rule: RunAsAny
supplementalGroups:
  rule: RunAsAny
allowedCapabilities:
  - '*'
volumes:
  -'*'

Most commonly changed parameters

1. privileged

2. runAsUser

Reference: https://kubernetes.io/docs/concepts/policy/pod-security-policy/

For allowedUnsafeSysctls  and forbiddenSysctls 

  • kernel (common prefix: kernel.)

    • kernel.shm*,
    • kernel.msg*,
    • kernel.sem,
  • networking (common prefix: net.)
  • virtual memory (common prefix: vm.)
  • MDADM (common prefix: dev.)
  seLinux:
    rule: RunAsAny

Means AppArmor is used instead of SELinux

If we have PodSecurityPolicy admission plugin enable, but no PSP defined, then by default, any new pod creation will fail. 

In order to use PSP, the requesting user or target pod's service account must be authorized to use the policy, by allowing the use verb on the policy.

With the plugin enable and appropriate policy, only pod create is allowed. Not deployment, not replicaset creation. To allow pod creation using deployment->replicaset use following ClusterRole and ClusterRoleBinding. Thee methods. Difference between them are highlighted with bold face

1. 

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: use-restricted-psp
rules:
- apiGroups: ['policy']
  resources: ['podsecuritypolicies']
  verbs:     ['use']
  resourceNames:
  - restricted-psp
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: restricted-role-bind
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: use-restricted-psp
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:serviceaccounts

2. 

31apiVersion: rbac.authorization.k8s.io/v1
32kind: ClusterRole
33metadata:
34  name: psp:restricted
35rules:
36- apiGroups:
37  - policy
38  resourceNames: <- resourceName is optional. It give control for individual PSP. 
39  - psp.restricted
40  resources:
41  - podsecuritypolicies
42  verbs:
43  - use
44---
45apiVersion: rbac.authorization.k8s.io/v1
46kind: ClusterRoleBinding
47metadata:
48  name: psp:restricted:binding
49roleRef:
50  apiGroup: rbac.authorization.k8s.io
51  kind: ClusterRole
52  name: psp:restricted
53subjects:
54  - kind: ServiceAccount
55    name: replicaset-controller
56    namespace: kube-system

If we use RoleBinding instead of ClusterRoleBinding then it is for same namespace

3. kubectl -n "namespace" create role "anyNameForRole" \
    --verb=use \
    --resource=podsecuritypolicy \
    --resource-name=" # This Is optional"
kubectl -n "namespace" create rolebinding "anyNameForRoleBinding" \
    --role="anyNameForRole" \
    --serviceaccount=namespace:default

The replicaset controller use default SA. So we should able to create deployment with about 2 commands also. 

If controller manager connects to API server using trusted/insecure port then all PSS allowed, as authorization (and authentication) is bypass. 

After enabling PodSecurityPolicy admission control plugin, we should have

1. This policy

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: default-allow-all
spec:
  allowPrivilegeEscalation: true
  allowedCapabilities:
  - '*'
  fsGroup:
    rule: RunAsAny
  hostIPC: true
  hostNetwork: true
  hostPID: true
  hostPorts:
  - max: 65535
    min: 0
  privileged: true
  runAsUser:
    rule: RunAsAny
  seLinux:
    rule: RunAsAny
  supplementalGroups:
    rule: RunAsAny
  volumes:
  - '*'

2. We need clusterrole in target namespace

k -n team-red create clusterrole cr --verb=use --resource=psp

3. To add any new PSP, it should have min these fields

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: example
spec:
  privileged: false  # Don't allow privileged pods!
  # The rest fills in some required fields.
  seLinux:
    rule: RunAsAny
  supplementalGroups:
    rule: RunAsAny
  runAsUser:
    rule: RunAsAny
  fsGroup:
    rule: RunAsAny
  volumes:
  - '*'

4. At each NS, we should have rolebidning.

k -n team-red create rolebinding rb --clusterrole=cr --user=system:serviceaccounts

OR

We can have clusterrolebinding

k -n team-red create clusterrolebinding crb --clusterrole=cr --user=system:serviceaccounts

References

https://banzaicloud.com/blog/pod-security-policy/

https://www.suse.com/c/rancher_blog/enhancing-kubernetes-security-with-pod-security-policies-part-2/

IAM using tools: keycloak , Active Directory, Amazon IAM

CIS It provides huge amount of free and paid resources to improve IT It provides security. tools, including benchmarks, scanning tools, threat tools, and hardened images. The CIS-CAT®Pro tool evaluates a target system against known issues and performance configurations. CIS also offers dashboards to view the ongoing state of compliance and security considerations.

For minikube setup we need to install kube-bench tool on individual node and run test. The test result recommend steps, for failure and warning cases. We can also run job.yaml at K8s cluster. 

Have a look to summary of CIS for K8s in this Excel file

0 comments:

Post a Comment