Express YourSelf !

 Kyverno Policy

- 2 types: Policy and ClusterPolicy

- Multiple Rules

- Match / Exclude

-- Match resources

kind is mandatory. names, namespaces, operations, selector are optional. 

Wildcards * supported in kinds, names, namespaces

All are AND condition. 

Any means OR condition

We can mention based on who created

exclude:

any:
- clusterRoles:
- cluster-admin
- subjects:
- kind: User
name: John

match AND exclude

exclude must be a subset of match

- Action

1. Validate ( allow / deny )

2. Mutate

3. Generate new K8s object

4. Verify image (Cosign/Sigstore)

- Enforce / Audit

* It uses JAMESPath

- Filter JSON

# Returns a list of container objects that match the condition
{{ request.object.spec.containers[?starts_with(image, 'nginx')] }}

validate:

message: "Nginx images are not allowed!"
deny:
conditions:
all:
# Filter the list. Use length() to count.
# If count > 0, it means a violation exists -> Block.
- key: "{{ request.object.spec.containers[?starts_with(image, 'nginx')] | length(@) }}"
operator: GreaterThan
value: 0

- pipe

Used to extract a specific field from a complex object into a flat list.

# Input: List of container objects
# Output: ["nginx:latest", "redis:alpine", "busybox"]
key: "{{ request.object.spec.containers[].image }}"

kyverno jp query -i object.json 'spec.containers[].name'

Combo (Filter + Pipe): “Get the containerPort of the container named ‘app’”

 request.object.spec.containers[?name == 'app'].ports[].containerPort

You want to find the names of all volumes that are of type emptyDir.

spec.volumes[?emptyDir != null].name

kyverno jp query -i object.json "spec.volumes[?emptyDir != null]"

- null handling

Risk: {{ request.object.metadata.labels.team }} (If null -> Error).

Safe: {{ request.object.metadata.labels.team || '' }} (If null -> treat as empty string).

- The length function

Example: “A Pod must not have more than 3 containers.”

validate:
deny:
conditions:
all:
- key: "{{ request.object.spec.containers | length(@) }}"
operator: GreaterThan
value: 3

- if / else 

- sum

- contain means exist : contains(request.object.metadata.labels, 'production')

advance mutate foreach

mutate:

foreach:
- list: "request.object.spec.containers"
patchStrategicMerge:
spec:
containers:
- name: "{{ element.name }}"
securityContext:
readOnlyRootFilesystem: true

* Aut-Gen: If rule for pod then automatically generate rules for Deployment, StatefulSet, DaemonSet, etc.

https://release-1-8-0.kyverno.io/docs/writing-policies/autogen/

Generate rule has synchronize flag

synchronize: true: Kyverno complete managed lifecycle

synchronize: false: Kyverno created for the first time then user can edit it manually without getting revert back like synchronize: true

* request.object is the incoming resource configuration (the new state) that is being submitted to Kubernetes API server

Documentation: https://kyverno.io/docs/policy-types/cluster-policy/variables/ 

  # -------------------------------------------------------------

  # ACTION: If Kyverno is dead, just let the request go through.

  # -------------------------------------------------------------

  failurePolicy: Ignore # or Fail

PolicyReport

PolicyReport stores the results of those rules.

The PolicyReport is essentially a “Health Check Report Card” for your Kubernetes resources.

Its main goal is Observability & Auditing.

It provides summary and then detail about all failures. For Example

apiVersion: wgpolicyk8s.io/v1alpha2
kind: PolicyReport
metadata:
name: polr-ns-default
namespace: default # It lives next to the Pod, not at the cluster level
labels:
app.kubernetes.io/managed-by: kyverno
summary:
pass: 0
fail: 1
warn: 0
error: 0
skip: 0
results:
- policy: require-labels # The name of the ClusterPolicy responsible
rule: check-for-team-label # The specific rule name
category: Best Practices
severity: medium
result: fail # The outcome (fail, pass, warn, error, skip)
message: "Validation error: label 'team' is required"
source: kyverno
resources: # The specific object that failed
- apiVersion: v1
kind: Pod
name: nginx
namespace: default
uid: a1b2c3d4-e5f6...

* Background scan never delete resource even with enforce mode

* For background scan following variable are not relevant

request.userInfo.*

request.operation

request.dryRun

serviceAccountName in admission context

Document: https://kyverno.io/docs/policy-reports/background/

* Cleanup policy delete pods

https://kyverno.io/docs/policy-types/cleanup-policy/

Kyverno CLI

* for given resource, policy is pass or fail

kyverno apply policy.yaml --resource pod.yaml

* test JAMESPath expression against JSON

kyverno jp query -i object.json 'metadata.labels'

* Check policy YAML is written correctly or not

kyverno validate policy.yaml

External Data Source

Purpose: It allows you to load data from outside into a variable before the rule logic (validate/mutate) runs.

In order to consume data from a ConfigMap in a rule, a context is required... The context data can then be referenced in the policy rule using JMESPath notation.

Kyverno supports 3 main data sources in context:

1. Kubernetes Resources (via API Call): Look up existing data in the cluster (e.g., ConfigMaps, Secrets, Services).

2. External APIs: Make an HTTP call to a service outside the cluster.

3. Image Registry: Fetch metadata about a container image (e.g., image size, architecture).

verifyImages:
- imageReferences:
- "ghcr.io/myorg/*"
attestors:
- entries:
- keys:
publicKeys: |-
-----BEGIN PUBLIC KEY-----
...
-----END PUBLIC KEY-----

https://main.kyverno.io/docs/policy-types/cluster-policy/external-data-sources/

Kyverno Mutate — JSON Patch

the standard patchStrategicMerge merges YAMLs together.

patchesJson6902 is a mutation method specific operations (RFC 6902 standard) to tell Kyverno exactly how to change the data.

Usage: 

* Removing a field (impossible with standard merge).

* Adding an item to a specific position in a list (arrays).

* Replacing a value entirely without merging.

It follows the JSON Patch format:

* op: The action (add, remove, replace).

* path: The location of the field (e.g., /metadata/labels/mytag).

* value: The data to put there.

Example: “Add a sidecar container”

https://main.kyverno.io/docs/policy-types/cluster-policy/mutate/#rfc-6902-jsonpatch

TTL Default of Certification in Kyverno:

https://pkg.go.dev/github.com/kyverno/kyverno/pkg/tls

const (

    CAValidityDuration = 365 * 24 * time.Hour      // 365 days

    TLSValidityDuration = 150 * 24 * time.Hour     // 150 days

    CertRenewalInterval = 12 * time.Hour           // 12 hours

)

https://www.udemy.com/course/complete-certified-kyverno-associate-kca-exam-prep/

https://medium.com/@kienlt.qn/prepare-for-the-kyverno-certified-associate-kca-exam-c144906f9bc2

Concurrent Policies Generation Number Default!

PolicyException CRD

CEL

For AI application, we need automation of 

1. Data preparation
2. model tuning
3. Deployment
4. Maintenance and 
5. Monitoring

• Managing Dependency adds complexity. 

E2E workflow for LLM based application. 

MLOps framework

1. data ingestion

2. data validation

3. data transformation

4. model

5. model analysis

6. serving model

7. logging. 

LLM System Design

boarder design of E2E app including front end, back end, data engineering etc. 

Chain multiple LLMs together

* Grounding : provides additional information/fact with prompt to LLM. 

* Track History. how it works past. 

LLM App

User input->Preprocessing->grounding->prompt goes to LLM model->LLM Response->Grounding->Post processing + Responsible AI->Final output to user.

Model Customization

1. Data Prep

2. Model Tuning

3. Evaluate

It is iterative process

LLMOps Pipeline (Simplified)

1. Data Preparation and versioning (for training data)

2. Supervised tuning (pipeline) 

3. Artifact = config and workflow : are generated. 

- config = config for workflow

E.g. 

Which data set to use

- Workflow = steps 

4. Pipeline execution

5. deploy LLM 

6. Prompting and predictions

7. Responsible AI

Orchestration = 1 + 2 . Orchestration : What is first, then next step and further next step. sequence of step assurance. 

Automation = 4 + 5

Fine Tuning Data Model using Instructions (Hint)

1. rules

2. step by step

3. procedure

4. example

File formats

1. JSONL: JSON Line. Human readable. For small and medium size dataset. 

2. TFRecord 

3. Parquet for large and complex dataset. 

MLOps Workflow for LLM

1. Apache Airflow

2. KubeFlow

DSL = Domain Specific Language

Decorator 

@dls.component

@dls.pipeline

Next compiler will generate YAML file for pipeline

YAML file has

- components

- deploymentSpec

Pipeline can be run on

- K8s

- Vertex AI pipeline execute pipeline in serverless enviornment

PipelineJob takes inputs

1. Template Path: pipline.yaml

2. Display name

3. Parameters

4. Location: Data center

5. pipeline root: temp file location

Open Source Pipeline

https://us-kfp.pkg.dev/ml-pipeline/large-language-model-pipelines/tune-large-model/v2.0.0

Deployment

Batch and REST

1. Batch. E.g. customer review. Not real time. 

2. REST API e.g. chat. More like teal time library. 

* pprint is library to format 

LLM provides output and 'safetyAttributes'

- blocked

* We can find citation also from output of LLM

===========

vertexAI SDK

https://cloud.google.com/vertex-ai

BigQuery 

https://cloud.google.com/bigquery

sklearn

To decide data 80-20% for training and evaluation. 

Building AI/ML apps in Python with BigQuery DataFrames | Google Cloud Blog

===========

Examples: 

Istio, Kong, Envoy , Gluee , Trafeik, kgateway, Contour, NGINX, Kong Gateway and many more as per https://gateway-api.sigs.k8s.io/implementations/#gateway-controller-implementation-status

Protocols: gRPC, HTTP/2, and WebSockets

The structure of a Kubernetes Custom Resource Definition (CRD) or manifest file is referred to as an API. This is because it refers to the structure of the API in the Kubernetes control plane

Migration from ingress https://gateway-api.sigs.k8s.io/guides/migrating-from-ingress/#migrating-from-ingress

extension points

ingress has 2 extension points

1. annotations

2. resource end point

primary extension points in GW API:

1. External references

1.1 HTTP Route Filter

1.2 Backend Object Reference

1.3 Secret Object Reference

Here GW API reference 'external reference'

2. Custom implementations

e.g. The RegularExpression type of the 'HTTP Path Match'

3. Policies

A "Policy Attachment" is a specific type of metaresource

Here, Policy reference 'GW API'

• GW API is not API GW
• GAMMA (Gateway API for Mesh Management and Administration) initiative
• A 'waypoint proxy' is a proxy server deployed inside the mesh for E-W traffic. It can be (1) per namespace level destination services (2) few service(s) within a namespace as destination (3) destination services from multiple namespaces. 

- It is at cluster level. so no namespace

- Annotations at GatewayClassfor vendor specific

- It defines controller capabilities

2. Gateway

- Each Gateway defines one or more listeners, which are the ingress points to the cluster

- You can control which services can be connected to this listener (allowedRoutes) by way of their namespace — this defaults to the same namespace as the Gateway 

- Advanced featues like 

-- request mirroring, 

-- direct response injection, 

-- and fine-grained traffic metrics

-- Traffic spilt

- In Istio APIs, a Gateway configures an existing gateway Deployment/Service that has been deployed. In the Gateway APIs, the Gateway resource both configures and deploys a gateway

- one can attach HPA and PodDisuptionBudget to gateway deployment. 

3. HTTP Route: 

- any combinations of hostname, path, header values, HTTP method and query parameters.

• Paths (e.g., /headers, /status/*)
• Headers (e.g., User-Agent: Mobile)
• Query Parameters (e.g., ?version=beta)
• Methods (e.g., GET, POST)

- hostname (optional) at HTTP route shall match with hostname at Gateway->Listener->hostname

- A definition of the Gateway to use (in ParentRefs), is referenced by name and namespace

- The backendRefs that defines the service to route the request to for this match

- advanced pattern matching and filtering on arbitrary headers as well as paths.

1. RequestRedirect : E.g. Redirect HTTP traffic to HTTPS

2. URLRewrite

3. <Request|Response>HeaderModifier

4. RequestMirror

5. CORS

6. ExtensionRef for custom filter. E.g. DirectResponse

    filters:

    - type: ExtensionRef

      extensionRef:

       name: direct-response

       group: gateway.kgateway.dev

       kind: DirectResponse

- In the Istio VirtualService, all protocols are configured within a single resource. In the Gateway APIs, each protocol type has its own resource, such as HTTPRoute and TCPRoute.

- Traffic splitting is done by specifying multiple backendRef, with weight

- timeout, retry, sessionPersistence Session persistence, (= sticky sessions or strong session affinity), ensures that a client's requests are consistently routed to the same backend instance for the duration of a session. based on cookie or a header

- Route and Gateway can be in different namespace. If Gateway is defined with

    allowedRoutes:

      namespaces:

        from: Same

then Route and Gateway shall be in same namespace. We can have group of namespaces with label selector and specify those namespace using label at Gatway resource. 

    allowedRoutes:

      namespaces:

        from: Selector

        selector:

          matchLabels:

            self-serve-ingress: "true"

* 4. TLS Route

5. GRPCRoute

* 6. TCPRoute

* not v1, GA

Details: https://gateway-api.sigs.k8s.io/reference/spec/

If you are using a service mesh, it would be highly desirable to use the same API resources to configure both ingress traffic routing and internal traffic, similar to the way Istio uses VirtualService to configure route rules for both. Fortunately, the Kubernetes Gateway API is working to add this support. Although not as mature as the Gateway API for ingress traffic, an effort known as the Gateway API for Mesh Management and Administration (GAMMA) initiative is underway to make this a reality and Istio intends to make Gateway API the default API for all of its traffic management in the future.

https://gateway-api.sigs.k8s.io/mesh/

Gateway controller is for North South traffic. mesh controller is for East West traffic

7. ReferenceGrant: for cross-namespace reference. 

8. Inference Extension

K8s offers following mechanisms to optimize GPU usage, 

- time slicing, 

- Multi-Instance GPU (MIG) partitioning, 

- virtual GPUs, and 

- NVIDIA MPS 

for concurrent processing.

Effective GPU utilization means

- hardware allocation; 

- how inference requests are routed across model-serving instances. 

- how inference requests are load-balanced across model-serving instances. 

Simple load-balancing strategies often fall short in handling AI workloads effectively, leading to suboptimal GPU usage and increased latency.

Inference requests V/s traditional web traffic

- It often takes much longer to process, sometimes several seconds (or even minutes!) rather than milliseconds, 

- It has significantly larger payloads (ie, with RAG, multi-turn chats, etc). So a single request can consume an entire GPU, So making scheduling decisions far more impactful than those for standard API workloads. So, these requests need to queue up while others are being processed.

AI Models are stateful

- They maintain in-memory caches, such as KV storage for prompt tokens,

- They load fine tuned adapters like LoRA to customize response for specific user/organisation. 

So routing decisions are based on

- current state (in-memory caches, adapters)

- available memory, and 

- request queue depth.

So Inference aware routing through

8.1 Inference Model

- maps user facing model name to backend model

- traffic splitting between fine-tuned adapaters

- Priority based on real time interaction OR best-effort batch job

8.2 Inference Pool

- It is for platform operators managing model-serving infrastructure.

- a group of model-serving instances 

- specialized backend service for AI workloads.

- It manages

-- inference-aware endpoint selection, 

-- intelligent routing decisions based on real-time metrics such as 

--- request queue depth and 

--- GPU memory availability.

* Inference Pool is mapped with HTTP Route->backendRefs

* Inference Model has poolRef to link with Inference Pool

* Inference Pool has extensionRef (EPP = Endpoint picker) . If name for Inference Pool is xyz then extensionRef is "xyz-endpoint-picker" It is similar to K8s service, as it also has selector and target port

9. DirectResponse

10. Backend

For external endpoint

Ref: https://tetrate.io/blog/kubernetes-envoy-gateway-extensions