Express YourSelf !

Security Context

Only for container

allowPrivilegeEscalation: false . It is for setuid and setgid

allowPrivilegeEscalation:

if (CAP_SYS_ADMIN || Privileged mode) then AllowPrivilegeEscalation = SSeue

=============

in PSP we have 

  allowedCapabilities:

  - '*'

  requiredDropCapabilities:

    - ALL

=============

privileged: true avoid it.

=============

procMount: true is for nested container useful for in-cluster build process

=============

readOnlyRootFilesystem: true for immutable container

=============

For container and pod both

container settings will get precedence. 

=============

runAsUser and runAsGroup are runtime configuration. If not defined then UID as per data in image

If runAsNonRoot = True then image should have UID. Same UID should be in host. 

=============

securityContext:

we can specify level, role, type, user. They are labels for file, process and ports. They are collectively called called "context" in SELinux terms. 

=============

type = Localhost | Unconfined  | RuntimeDefault 

where kuelet-root-dir = /var/lib/kubelet . 

It is configured with seccomp-profile-root flag for kubelet. This flag is deprecited since 1.19 . If seccomp-profile-root flag is not defined, the default path will be used, which is <root-dir>/seccomp where <root-dir> is specified by the --root-dir flag.

* type: RuntimeDefault means same seccomp profile as container runtime default is applied

* type: Localhost, then only, we need to set localhostProfile

We need to use annotation (optional) 

seccomp.security.alpha.kubernetes.io/pod: localhost/profiles/audit.json

Path for seccomp profiles: https://k8s.io/examples/pods/security/seccomp/profiles/

=============

Only for Pod

fsGroup

the permission bit will be | with rw-rw----

ownership and permission change recursively for all content in mounted volume as per fsGroup

=============

fsGroupChangePolicy = OnRootMismatch | Always

no impact on emptyDir, secret and configMap

OnRootMismatch: It will save time. The permission and ownership only change if root level folder has mismatch with expected fsGroup. 

If DelegateFSGroupToCSIDriver feature gate is enabled then this is done by CSI driver. CSI driver will not respect fsGroupChangePolicy 

=============

supplementalGroups

A list of groups applied to the first process run in each container, in addition to the container's primary GID.

=============

sysctls

====================================================

Here Discretionary Access Control (DAC) is related fields are: runAsUser, runAsGroup, runAsNonRoot, readOnlyRootFilesystem

Here volume related fields are: fsGroup and seLinuxOptions. We set only level at seLinuxOptions and as per level, labels applies to all containers and volumes. 

PodSecurityPolicy

Pod's scrutiny attributes. Defined at cluster level. It controls security sensitive aspects of the pod specification. It define a set of conditions that a pod must run with in order to be accepted into the system, as well as defaults for the related fields. 

It (1) restrict pod creation (2) restrict pod update (3) provide default value

=============

- "privileged" container (Privileged Mode). It is part of PSP. But it control container level "security context" 

=============

- host-level ns (network, PID, IPC) "hostPID", "hostIPC", "hostNetwork". If hostPID allowed then container can escalate privilege using ptrace system call.

============= 

- host ports "hostPorts"

- different types of volumes. E.g. "allowedFlexVolumes" "volumes"

=============

- host's filesystem E.g. "fsGroup"

fsGroup = MustRunAs | MayRunAs | RunAsAny

We shall specify 1+ range if fsGroup = MustRunAs | MayRunAs. In case of MustRunAs the fsGroup at Pod Security Context is set as min value. in case of MayRunAs, the default value for fsGroup at Pod Security Context, is unset

=============

- RO root filesystem for containers (DAC) "readOnlyRootFilesystem"

=============

- user IDs and group IDs (DAC) "runAsUser" "runAsGroup" "supplementalGroups"

runAsUser = MustRunAs | MustRunAsNonRoot | RunAsAny

runAsGroup = MustRunAs | MayRunAs | RunAsAny

supplementalGroups = MustRunAs | MayRunAs | RunAsAny

=============

- containers' privilege escalation "allowPrivilegeEscalation" "defaultAllowPrivilegeEscalation". Here defaultAllowPrivilegeEscalation, set values for allowPrivilegeEscalation, if not set.

=============

- containers' Linux capabilities (Linux Capabilities): "defaultAddCapabilities", "requiredDropCapabilities", "allowedCapabilities"

=============

- SELinux "seLinux"

RunAsAny means: Apparmor is used instead of SELinux. 

=============

- seccomp and AppArmor profiles : using annotations. 

seccomp.security.alpha.kubernetes.io/allowedProfileNames

apparmor.security.beta.kubernetes.io/defaultProfileName

If this annotation is not specified then default seccomp cannot be changed

=============

- sysctls that a pod can run: "forbiddenSysctls", "allowedUnsafeSysctls"

=============

- a proc mount type to be used. "allowedProcMountTypes" and "DefaultProcMount"

=============

PSP Policy Order

1. non-mutating policy in any order

2. mutating policy in alphabetical order of name

Good Reference: 

A Good article: https://www.linux.com/training-tutorials/overview-linux-kernel-security-features/

Abbreviation

ASLR Address Space Layout Randomization

DAC Discretionary Access Control 

IMA Integrity Measurement Architecture

MAC Mandatory Access Control

PAM Pluggable Authentication Modules

Falco by Sysdig: multiple components (user space program, configuration, driver) working together in order to evaluate system calls against rules, and generate alerts when a rule is broken:

rule has lists. rule can have reference to list. List can be part of macro and other list, in addition to part of rule.

rule has 5 k-v pairs. (1) name, (2) description , (3) condition : Filtering expression for events. (4) output, (5) priority. (emergency, alert, critical, error, warning, notice, informational, debug) 

rule has 4 optional K-v pairs.(1) enabled. default is true (2) tags (filesystem, software_mgmt, process, database, host, shell, container, cis, users, network) . -T option to disable rules with given tag. -t option to enable. (3) warn_evttypes default is true. (4) skip-if-unknown-filter default is false. 5th one added (5) exceptions : a set of conditions that cause the rule to not generate an alert. 

- Falco comes with many rules in /etc/falco/falco_rules.yaml file. They can be overwritten by /etc/falco/falco_rules.local.yaml file. E.g. to disable rule : We can add rule with same name and "append: true" + "enabled: false" 

- evt.dir = < indicates end of system call and evt.dir = > indicates beginning of system call. dir = direction

- We have K8s related context: k8s.[pod | rc | svc | rs | deployment].[name | id \ label | labels] + many fields from K8s audit logs.

with macro part of rule can be re-used. There are many default macros.  

Falco runs with K8s-audit on. So we need to specify audit policy file at API server argument --audit-policy-file

We can configure webhook in API server with this arguement

--audit-webhook-config-file=/etc/kubernetes/audit-webhook-kubeconfig

This YAML file shall define Config kind. 

We can see Falco output with journalctl command.

Reference :

https://github.com/falcosecurity/falco/blob/master/rules/k8s_audit_rules.yaml

https://v1-17.docs.kubernetes.io/docs/tasks/debug-application-cluster/falco/

 Tracee (uses eBPF) :monitors system call and kernel events. 

- It captures :  (1) precise time stamp, (2) uts_name, (3) UID, (4) Command COMM (5) PID, (6) TID/host (7) return code, RET (8) event, and (9) arguments.

- At least 3 volume locations needed. (1) /lib/modules , (2) /usr/src , (3) /tmp/tracee Tracee provides in-depth tracing of container or pod. 

list: list of system calls and other events. 

trace: events. specific pid, uid, uts, mntns, pidns, command (comm), system call etc. We can use comparison operator to filer. 

Examples:

  --trace pid=new                                              | only trace events from new processes

  --trace pid=510,1709                                         | only trace events from pid 510 or pid 1709

  --trace p=510 --trace p=1709                                 | only trace events from pid 510 or pid 1709 (same as above)

  --trace container=new                                        | only trace events from newly created containers

  --trace container                                            | only trace events from containers

  --trace c                                                    | only trace events from containers (same as above)

  --trace '!container'                                         | only trace events from the host

  --trace uid=0                                                | only trace events from uid 0

  --trace mntns=4026531840                                     | only trace events from mntns id 4026531840

  --trace pidns!=4026531836                                    | only trace events from pidns id not equal to 4026531840

  --trace 'uid>0'                                              | only trace events from uids greater than 0

  --trace 'pid>0' --trace 'pid<1000'                           | only trace events from pids between 0 and 1000

  --trace 'u>0' --trace u!=1000                                | only trace events from uids greater than 0 but not 1000

  --trace event=execve,open                                    | only trace execve and open events

  --trace set=fs                                               | trace all file-system related events

  --trace s=fs --trace e!=open,openat                          | trace all file-system related events, but not open(at)

  --trace uts!=ab356bc4dd554                                   | don't trace events from uts name ab356bc4dd554

  --trace comm=ls                                              | only trace events from ls command

  --trace close.fd=5                                           | only trace 'close' events that have 'fd' equals 5

  --trace openat.pathname=/tmp*                                | only trace 'openat' events that have 'pathname' prefixed by "/tmp"

  --trace openat.pathname!=/tmp/1,/bin/ls                      | don't trace 'openat' events that have 'pathname' equals /tmp/1 or /bin/ls

  --trace comm=bash --trace follow                             | trace all events that originated from bash or from one of the processes spawned by bash

  --trace container=new  | all the events from container created after issuing this command

 We can use

 kubectl auth reconcile -f  "filename.yaml"

to create missing objects and ns. It does not create sa

We can also run with 

 kubectl auth reconcile -f  "filename.yaml" --dry-run=client

--remove-extra-permissions will remove extra permission in role

--remove-extra-subjects will remove extra subjects in binding

The kubectl auth reconcile command will ignore any resources that are not Role, RoleBinding, ClusterRole, and ClusterRoleBinding objects, so you can safely run reconcile on the full set of manifests. Next we can run kubectl apply command. 

With kubectl apply command for rolebinding, we cannot update roleRef. it is immutable. However with this command kubectl auth reconcile, we can do it. 

All the above points are applicable to ClusterRole and ClusterRoleBinding also. 

Reference: https://www.mankier.com/1/kubectl-auth-reconcile

====================================

Regardless of namespace, by default, SA with name "default" is added to pod, in all namespace. 

====================================

In rolebinding if kind = User then only name is sufficinet.