Securing kube-controller-manager


Please execute this command at master node, to see the values of configuration parameters / flags for K8s Controller Manager

ps -ef | grep kube-controller-manager

As per section 1.3 of CIS Benchmark below table describe (1) parameter (2) its default value and (3) its desired value

Flags Default Value
--terminated-pod-gc-threshold 12500 (pods) set appropriate
--profiling TRUE FALSE
--use-service-account-credentials FALSE TRUE
--service-account-private-key-file Not Set Appropriate
--root-ca-file Not Set Appropriate
RotateKubeletServerCertificate TRUE TRUE
--bind-address 0.0.0.0 127.0.0.1

when --use-service-account-credentials=true with RBAC, it ensures, control loop runs with minimum privilege. 

Securing kube-scheduler


 Please execute this command at master node, to see the values of configuration parameters / flags for K8s Scheduler

ps -ef | grep kube-scheduler

As per section 1.4 of CIS Benchmark

* --profiling should be change from default true to false

* --bind-address should be change from 0.0.0.0 to 127.0.0.1 

Additional tips not in CIS Benchmark

* Ensure that AppArmor is not disabled for kube-scheduler.

* AllowExtTrafficLocalEndpoints feature is disabled using --feature-gates

Securing etcd


Please execute this command at master node, to see the values of configuration parameters / flags for etcd

ps -ef | grep etcd

As per section 2 of CIS Benchmark below table describe (1) parameter (2) its default value and (3) its desired value

Flags Default Value
--cert-file not set Appropriate
--key-file not set Appropriate
--client-cert-auth FALSE TRUE
--auto-tls FALSE FALSE
--peer-client-file Not Set Appropriate
--peer-key-file Not Set Appropriate
--peer-client-cert-auth FALSE TRUE
--peer-auto-tls FALSE FALSE

* --trusted-ca-file: this value should be different from --client-ca-file for API Server

Additional tips not in CIS Benchmark

* Restrict node access: Use Linux firewalls to ensure that only nodes that need access to etcd are allowed access.

* Encrypt data at rest: --encryption-provider-config is passed to the API server to ensure that data is encrypted at rest in etcd.

Securing kubelet


Please execute this command at master node, to see the values of configuration parameters / flags for kubelet. This is for first column. 

ps -ef | grep kubelet | grep config

The default configuration path can be found from /etc/systemd/system/kubelet.service.d/10-kubeadm.conf This is for second column. 

As per section 4.2 of CIS Benchmark below table describe (1) parameter with desired value (2) its default value


Command line arguments Config File Default
--anonymous-auth=false authentication: anonymous: disabled true
KUBELET_AUTHZ_ARGS --authorization-mode!=AlwaysAllow authorization: mode!=AlwaysAllow AlwaysAllow
KUBELET_AUTHZ_ARGS --authorization-mode=Webhook authorization: mode=Webhook
KUBELET_AUTHZ_ARGS --client-ca-file authentication: x509: clientCAFile
KUBELET_SYSTEM_PODS_ARGS --read-only-port=0 readOnlyPort:0 0 or 10255
KUBELET_SYSTEM_PODS_ARGS --streaming-connection-idle-timeout=NON-ZERO streamingConnectionIdleTimeout=NON-ZERO 4 hrs
KUBELET_SYSTEM_PODS_ARGS --protect-kernel-defaults=true protectKernelDefaults=true fasle
KUBELET_SYSTEM_PODS_ARGS --make-iptables-util-chains=true makeIPTablesUtilChains=true true
KUBELET_SYSTEM_PODS_ARGS --hostname-override=ABSENT ABSENT
KUBELET_SYSTEM_PODS_ARGS --event-qps=0 OR APPROPRIATE eventRecordQPS:0 OR APPROPRIATE 5
--rotate-certificates=true rotateCertificates=true true
KUBELET_CERTIFICATE_ARGS --rotateserver-certificates=true RotateKubeletServerCertificate=true false
--tls-cipher-suites TLSCipherSuites
KUBELET_CERTIFICATE_ARGS --tls-cert-file
KUBELET_CERTIFICATE_ARGS --tls-private-key-file

* --tls-cipher-suites : kubelet supports a wide range of cipher suits : TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
One should select a stonger cipher suite

Additional tips not in CIS Benchmark

* the APIs exposed by kubelet shall be accessible by K8s API Server only, using RBAC

* Enable the NodeRestriction admission controller: The NodeRestriction admission controller only allows kubelet to modify the node and pod objects on the node it is bound to.

K8s API Server security settings


Please execute this command at master node, to see the values of configuration parameters / flags for K8s API server

ps -ef | grep kube-apiserver

As per section 1.2 of CIS Benchmark below table describe (1) parameter (2) its default value and (3) its desired value

Flags Default Value
--anonymous-auth TRUE FALSE
--basic-auth-file ABSENT ABSENT
--token-auth-file ABSENT ABSENT
--kubelet-https TRUE TRUE
--kubelet-client-certificate Not Set <path/to/client-certificate-file>
--kubelet-client-key Not Set <path/to/client-key-file>
--kubelet-certificate-authority Not Set <ca string>
--authorization-mode NOT 'AlwaysAllow' NOT 'AlwaysAllow'
--authorization-mode Not enabled ...,RBAC,Node,...
--insecure-bind-address ABSENT ABSENT
--insecure-port 8080 0 //Must be 0 in k8s v1.20. Also used for bootstrap access. It will be removed in v1.24
--secure-port 6443 1 to 65535
--profiling TRUE FALSE
--audit-log-maxage Disable Minimum 30 (days)
--audit-log-maxbackup Disable Minimum 10 (files)
--audit-log-maxsize Disable 100 (MB) or appropriate
--request-timeout 60 s Set appropriate. 300s
--service-account-lookup TRUE TRUE
--service-account-key-file ABSENT Public Key file for SA. Private key to controller manager
--etcd-certfile Not Set Appropriate
--etcd-keyfile Not Set Appropriate
--tls-cert-file Not Set Appropriate. Do not use self-signed certificate
--tls-private-key-file Not Set Appropriate
--client-ca-file Not Set Appropriate
--etcd-cafile Not Set Appropriate
--encryption-provider-config Not Set EncryptionConfig file

* --tls-cipher-suites : K8s API server support a wide range of cipher suits : TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
One should select a stonger cipher suite

* --enable-admission-plugins: One should enable following admission plugins

EventRateLimit,AlwaysPullImages, SecurityContextDeny || PodSecurityPolicy,NodeRestriction

* --disable-admission-plugins: One should disable following admission plugins

ServiceAccount, NamespaceLifecycle, AlwaysAdmit

* --audit-log-path: By default it is absent. One should specify a path to secure storage with maxage, maxsize, and maxbackup Parameters. E.g. /var/log/apiserver/audit.log

* EncryptionConfig file: aesgcb, aescbc, kms OR secretbox. Specify provider with EncryptionConfiguration k8s object. 

Additional tips not in CIS Benchmark

* --tls-min-version: Its value should be >= 1.2

* One should enable following feature gate: --feature-gates AdvancedAuditing=TRUE