3. Install OPA
TUF & Notary
Gatekeeper project uses Admission Controller and "OPA constraint framework" (CRD). It enables users to customize admission control by configuration not by code. It enforces policies executed by OPA
Gatekeeper is extensible, parameterized policy library. constraint CRD instantiate the policy library. constraint template CRD extend the policy library.
- Each Constraint is written with Rego, a declarative query language used by OPA. All Constraints are evaluated as a logical AND. If one Constraint is not satisfied, then the whole request is rejected.
- Each constraint template describes both (1) the Rego logic that enforces the Constraint and (2) the schema for the Constraint, which includes (2.1) the schema of the CRD and (2.2) the parameters that can be passed into a Constraint, much like arguments to a function.
- No namespace for constraint template. No namespace for constraint.
The audit functionality enables periodic evaluations of replicated resources. It evaluates against the Constraints. So it detects pre-existing misconfigurations. Gatekeeper stores audit results as violations listed in the status field of the relevant Constraint.
k get constraints "name of constraint" -o yaml
check status field here
- In k8s, OPA replaces PSP. OPA is also used for Envoy Proxy, SSH and sudo
- Gatekeeper is validation webhook + audit + CRD based policy and OPA is policy engine.
- failurePolicy: Ignore means if webhook is down, then constraint will not be apply. Later on, audit shall determine such events.
Use cases
- Validating
- Require specific labels on all resources. E.g. All namespaces must have a label that lists a point-of-contact
- Require container images come from the corporate image registry. (here, no error shown for deployment creation, not at at pod, not at rs. number of pod=0. when you describe deployment and rs, you will see error in rs creation. No, you can also see error at status of constraint. It is only for existing pods)
- Require all Pods specify resource requests and limits.
- Prevent conflicting Ingress objects from being created. E.g. All ingress hostnames must be globally unique
- No run as root
- No privileged container
- Mutating
- Inject sidecar containers into Pods.
- Set specific annotations on all resources.
- Rewrite container images to point at the corporate image registry.
- Include node and pod (anti-)affinity selectors on Deployments.
- input.request.kind specifies the type of the object (e.g., Pod, Service, etc.)
- input.request.operation specifies the type of the operation, i.e., CREATE, UPDATE, DELETE, CONNECT.
- input.request.userInfo specifies the identity of the caller.
- input.request.object contains the entire Kubernetes object.
- input.request.oldObject specifies the previous version of the Kubernetes object on UPDATE and DELETE
- We can use "review" instead of "request" in above path.
When we define constraint, we mention, to whom it applies using "match". The match field supports following matches : kind, namespaces, excludedNamespaces, labelSelector, namespaceSelector and scope.
We can set enforcementAction: dryrun at constraint
Related Tools
1. conftest It can be used in CICD pipelineKyverno is alternative for gatekeeper.
Reference:
https://kubernetes.io/blog/2019/08/06/opa-gatekeeper-policy-and-governance-for-kubernetes/
Sample Constraints https://github.com/open-policy-agent/gatekeeper/tree/master/demo/agilebank/templates and https://github.com/open-policy-agent/gatekeeper/tree/master/demo/basic/templates
3. Install: Runtime sandbox for container isolation
Traditional container shares host kernel
gVisor has application kernel Sentry to execute system calls, securely.
kata has hardware virtualization to provide kernel per container.
gVisor https://github.com/google/gvisor
- Application kernel (Sentry). It includes Netstack (user space network stack)
- implements OCI runtime runsc
- isolation between app and host kernel (Gofer for access to filesystem) with seccomp
- Each pod is gVisor instance and container inside pod is process.
- Often used with SELinux or seccomp
- gVisor uses runsc binary instead of runc
- Sentry and Gofer communicates using 9P protocol (common n/w protocol by Bell Labs)
- more overhead to execute system calls using gVisor
Follow steps as per https://sbulav.github.io/certifications/cks-gvisor/ at all worker node.
We can see on worker node runsc-gopher and runsc-sandbox (santry) processes are running.
Kata https://katacontainers.io/
Speed of containers and security of VMs
Lightweight VM, plug into container eco system
hypervisors: QEMU, NEMU, Firecracker
Works with Docker and cri-o architecture. cri-o and containerd engine.
Intel's clear container is now part of kata container.
Kata containers are part of OSF Open Stack Foundation.
PouchContainer http://pouchcontainer.io/
Open source container engine by Alibaba.
Strong isolation
less runtime overhead
incremental image distribution
cross platform and cross OS
compatible with OCI spec
* Rich container: Can have multiple process + init process. Package fat application easily.
* Strong isolation: Can create VM
* Compatible with many kernel versions
* P2P image distribution
Use with K8s: https://github.com/alibaba/pouch/blob/master/docs/kubernetes/pouch_with_kubernetes_deploying.md
Firecracker https://github.com/firecracker-microvm/firecracker-containerd
to manage Firecracker microVM using containerd
Unik https://github.com/solo-io/unik
- Tool to compile app source into unikernels and microVM. Not binaries.
Unikernels are lightweight, immutable operating systems compiled specifically to run a single application. No ssh/telnet. Mostly no File System. single binary | process | address space | mode, no forked. no shell, so no "shell injection". Less code, so less bug so less likely buffer overflow.
- Simple docker-like CLI
- Good for IoT smart devices
1. Go code to Firecracker microVM
2. Python, Node.js, Go to rump (rumprun) unikernel
3. Java, Node.js, C, C++ to OSv unikernel
4. C++ to IncludeOS unikernel
5. OCaml to MirageOS unikernel
Supported Providers: Firecracker, Virtualbox, AWS, Google Cloud, vSphere, QEMU, UKVM, Xen, OpenStack, Photon Controller
=========
RuntimeClass
Enable at (1) Kube-apiserver and (2) kubelet
High level container runtime containerd support it. (1) cri-o and (2) docker are yet to support custom runtime configuration.
apiVersion: node.k8s.io/v1beta1
kind: RuntimeClass
metadata:
name: gvisor
handler: runsc //CRI Configuration
=======
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
runtimeClassName: gvisor
CRI Configuration
* dockershim : Default. Does not support runtime handler
* containerd: [plugins.cri.containerd.runtimes.${HANDLER_NAME}] section at file /etc/containerd/config.toml
Reference: https://github.com/containerd/cri/blob/master/docs/config.md
* cri-o:
[crio.runtime.runtimes.${HANDLER_NAME}]
runtime_path = "${PATH_TO_BINARY}"
section at file /etc/crio/crio.conf
Reference: https://github.com/cri-o/cri-o/blob/master/docs/crio.conf.5.md#crioruntime-table
3. Install : Image Supply Chain
Trow: Registry and image management
Prisma Cloud: Cloud Security Platform
NeuVector: scan container images for vulnerabilities. DPI, Firewall
Clair: static analysis for container image vulnerabilities. Open Source Project
Aqua: Complete enterprise security solution for container LCM
Trivy: scan container images for vulnerabilities.
use command
trivy image "image name" | more
trivy i "image name" | more
They provide all CVEs. Summaries number of CVE: high, medium, low, critical, unknown.
Note, it is "trivy i" it is not "trivy -i"
Trivy can find misconfiguration and vulnerabilities in (1) file system and (2) git repository.
We can check all YAML file with command
trivy conf "folder path of YAML files"
Notary: TUF based CNCF project. Image signing
Harbor: secure registry and manage artifacts
Kritis: Validation webhook of K8s admission controller and CRDs. It uses Grafeas and Voucher.
1. GenericAttestationPolicy : kritis.grafeas.io/v1beata1
spec: attestationAuthoriy
2. AttestationAuthority: kritis.grafeas.io/v1beata1
spec: privateKeySecretName, publicData, noteReference.
3. ImageSecurityPolicy: kritis.grafeas.io/v1beata1
spec: imageWhitelist / imageAllowlist
spec: packageVulnerabilityRequirement
maximumSeverity:
whitelistCVEs: / allowlistCVEs
4. Default admittance fallback policy
Grafeas: API for image vulnerabilities meta data, attestation meta data all other structured metadata for artifacts.
Terminology
Note: High level description of type of metadata. E.g. CVE as Vulnerability Note. Note ID format: /projects/"project_id"/notes/"note_id"Note ID is unique within project. It can be CVE ID
Occurrence: instance of note in artifact E.g. CVE presence in an image. Occurrence ID format: /projects/<project_id>/occurrences/<occurrence_id> Occurrence ID is random number
Providers: E.g. Vulnerability scanning. add notes and occurrence in DB
Consumer: E.g. Kritis
Resource URL: Part of occurrence. It is identifier for artifact. Immutable. It ties occurrence to image
Kind Specific Shema: Strict and structured. ATTESTATION, BUILD, DEPLOYMENT, DISCOVERY, IMAGE, PACKAGE, VULNERABILITY, INTOTO. Total 8. Easy to add new. It helps to support multi-vendor.
The metadata can be part of binary authorization tool.
Architecture
Auth system for strict access control.
Client: not part of Grafeas
Grafeas API: core component
DB: Pluggable storage backends
Voucher: Make sure image after build and image at deployment, both are same.
