Express YourSelf !

Shopify has very large scale deployment with AI use cases algorithm : 

- Vector relations of products. 

- Credit Card frauds 

- Many GPUs

* GPU utilization v/s developer productivity is trade off. 

Challenges

1. Build v/s buy 

2. Dev experience : skypilot and rocket ML

==========================

Shadow role in K8s release team is best place to start contributing at K8s

Cato is for AI. This is another good place to start with. 

He showed many Indian architectures like Taj Mahal (Agra), Jantar Mantar  (Jaipur) and inspire Indian to have largest contributors in the world

==========================

TajMahal also built with diversity. 

Conscious and continuous effort for diversity is invisible, important. 

Now many meetings started and will start in APAC friendly timezone

Very hard to justify open source contribution to employer.

Contributors shall be move to maintainers.

==========================

2014 Stateless

2017 Stateful

2019 Serverless

2023 AI

Cloud Native AI (CNAI) working group : Streamline the integration of AI with cloud native ecosystem. 

Whitepaper CNAI

CN is ideal platform for AI

- Cost efficiency

- Scalability

- Containerization

- Harmony among dev, test, staging and production

- High Availability

- Microservice Architecture

CNAI from 3 perspective

1. K8s: 

- DRA Dynamic Resource Allocation. inspired by PV/PVC (1.26, 1.32 beta)

2. ML engineers

- Kubeflow has many projects for different use cases

- Queue for ML batch processing

3. App Developer

- OPEA - Open Platform for Enterprise AI

website: opea.dev

1. Data Prep

2. Embedding *

3. LLM/SLM *

4. Vector DB *

6. Receiver

7. Reranking

* OPEA provides recipes for all options. 20+ GenAI recipes 

They are validated at Intel, ARM, AMD architecture

MongoDB / Neo4J Graph Database. no need of Vector DB.

Minio is common data layer

OPEA is available on Azure, AWS

CNAI has its own landscape on CNCF website

WG

- Scheduling

- Security

- Sustainability

AI Playground validate OPEA samples on ARM with free Oracle Credit. CNAI needs people. 

==========================

1980 data Spreadsheet

1990 Information DataBase

2000 Knowledge Data Warehouse

2010 Insight Analytics (Hadoop, Spark)

2020 Intelligence AIML

2025+ Action Agents

Analogy

- Agents Apps

- GenAI OS

- LLM Kernel

Characteristics

1. Decision Making

2. Memory

3. Reasoning

4. Action

Analogy

Container - Agent

OCI runtime - LLM

Deterministic Logic - Adaptive Logic

stateless by default - stateful by nature

static resource limit - dynamic resource

Identical replicas - Unique instance

Docker run -> compose -> K8s

Agent -> Multiple agents that needs orchestration. Here K8s fits

K8s is universal control plane for VM, DB, Iot edge, docker, WA. Agent will be yet another workload type. 

Arch : Agent Operator

1. Agent controller

2. Schedular

3. CR

LLM will tell Agent Controller what agent to create. 

Agent CR YAML will have Task, model, memory, tools, person 

AI : Crewai, metaflow, airflow, 

CN: Argo, dapar, Numaflow, KServe (came out of Kubeflow)

They are not just feature proposal. It is about

- Feature Design

- Different Alternatives

- Testing plan

Feature LCM = Feature, Deprecation, update. Captured in KEP

What is KEP? KEP-0000 K8s Enhancements Proposals

A unit of work: combined features or single feature or major changes 

Capture decisions in well crafted artifacts 

KEP has YAML based metadata. 

/enhancements/keps

Stages: 

1. Alpha: Disable by default. Enable with Feature Gate

2. Beta: API groups disable by default. Feature is enable.

3. GA: All enables

Demo

Git Issue : 114465 Add Sleep for pod life cycle handler 

Let sleep is first class citizen, so no need to add its binary.

KEP is not hard but time taking. We need to take sign in from different people of community.

Feature = Enhancement = KEP

Non-Goals are very important, so distraction is avoided

Proposal section of KEP has technical part. Then read the code

Then "Design Details" may have pseudo code

"Graduation Criteria" for all 3 stages. 

"Upgrade / Downgrade strategy" 

K8s repo has staging repository. 

SCHEO

/hack/update-codegen.sh will generate auto code

We can have KIND image  with our own k8s code using config.yaml for KIND Also enable Alpha feature flag

KubeAPI Server YAML

Look for post-release blogs for user-firendly docs. Mid-release blogs also. 

Beta to GA then remove feature enable check

KEP also captures all PR

NIM is off the self interfacing framwork

NIM is about decide which model to use, based on number of GPU? Which GPU? Performance criteria (throughput v/s latency)? floating point library. NIM can autodetect hardware

GenAI with RAG has many services.

NIM operator to deploy RAG application CR:  1. NIM Cache (PVC) 2. NIM service 3. NIM pipeline all service can increase together. 

NIM monitoring and autoscaling: Prometheus 

1. Utilization of hardware

2. inter token latency

3. first token time generation. 

4. request per second

Monitoring of NIM

2 seconds, 15 chat user etc are input for SLA. 

NIM monitoring operator choose metrics from many metrics exposed by NIM

Autoscaling

In the sample chat application : milvus DB is needed. RAG is frontend service

Canary has mode deployment time, because more iteration. 10% increase means 10 iteration. 

Blue Green has less. 

Fine tune Max Surge, Max unavailable etc. 

Traffic Mirroring is for ML load also. 

image compression, local registry etc needed because model size is big. 

There is no inter cluster redundancy by K8s. We need to use proprietary solution OR cloud. 

In telecom a component is connected with multiple. E.g vCU with vDU, EMS, 5GC

Sync Driver

GRAF framework with AI, Management Data analytics function(MDAF), policy driven 

A1 interface is better than MDAF. As MDAF is at core network. it will add latency. 

Nephio : open source project : LF + Google

Automate LCM of cloud infra and NF. Intent based declarative approach

LinkedIN : saurabhswaraj

GitOps based approach

1. N/w driver is GR aware

2. DB drive sync 2 MySQL

YAML PV and DB table synch

Each cluster has GRAPH controller

We can have several other use cases also

GRAPH is framework. We can develop our own driver. 

GRAPH f/w = Redundancy Manager at Orchestrator + GRAPH controller at each cluster

Based on CRD, different driver will be deployed at each K8s cluster.

Still GRAPH is not open source. It is in process for open source. At present it is in R&D stage.

GRAPH can work with many orchestrator including Nephio 

All DB has Replication Manager. Why do we need DB drive? DB drive is not novelty. Our novelty is framework. 

Policy based, when failed comes up again, what will happen. 

Kept has Two Operators: App LCM and for Metrics

Day1 and Day2 operation

Integrated with Argo

Metrics based scaling: HPA and Scala

works for outside K8s application

It works with DORA metrics

Multistage application delivery. 

Each pod will have local bucket and then implement leaky bucket algorithm. 

Start bucket size with big number

Learning from rate limiting journey

Service with smaller threshold has higher precision

higher threshold has upto 10% error rate

pod has its own swimlane 

-------------------------------------------------------------------------

For inter+intra rate limit, need to synch API-GW and serviceMesh. 

Unified config management 

Naavik knows, where service resides (Kubecon Paris) 

Canary release pattern : First canary call, the service registry will read from DB and put it in cache. For remaining, no need to access DB. Response from cache. Canary pod can put value in S3 bucket, for further optimization, and notify all pods to read form s3 bucket

* rust based data plane is  pretty light

* We can use any Ingress Controllers (IC) with Linkerd. IC are based on (1) NGINX (2) Envoy (2) HA Proxy. NGINX IC has Mister-Minion model

* security : authz policy, mTLS 

* intra cluster URL. So no need to go to LB for intra cluster. 

* Gateway API (GW-API) integration

* 1.17 Linkerd has many new features.

* canary deployment

github.com/thotz/python-vectordbapp-ceph

It can search image and text both

it uses RGW bucket and in-memory channel

CR has embedded function

CR is per bucket

CR is matched with milvus collection like, vector dimensions, index type, vector metric, staticjk schema 

LB has many features

These features are supported by data plane technologies

data plane has many technologies, including open source. 

1. eBPF : 

2. VPP

- L2-L4 n/w

- graph based network

- fast lookup

DPDK : 

- latency reduce by bypassing kernel space

- large page and pool.

1. eBPF - XDP model

2. VPP-DPDK model

• Upcoming KCDs

1. April, 2025 at  Chennai

2. June, 2025 at  Bangalore

• Kubestronaut : Takeoffs begin in 2025. In this announcement, India's map was incorrect. 

• Upcoming KubeCon and CloudNativeCon

Hydrabad August 6 to 7 , 2025

LF Networking will be launched in phase 1 under LF India

=============================================================

Takeaway points : Flipkart

Ambient proxy is better. Sidecar cannot scale as per pod's TPS, Fan out etc. 

For PaaS Controller is better than Helm to deploy

=============================================================

Takeaway points : 

• K8s is everywhere
• Nothing is complete with AI
• Starching the limits of K8s

1.  eBPF programme in C, Rust or other programming languages. 
2. Compiler converts it to bytecode (obj file)
3. Application pass this OBJ to kernel using  eBPF library
4. Kernel verifies it
5. When event happens the program runs

1.  eBPF programme in C, Rust or other programming languages. 
2. Compiler converts it to bytecode (obj file)
3. Application pass this OBJ to kernel using  eBPF library
4. Kernel verifies it
5. JIT compiler converts byte code to machine code
6. When event happens the program runs

Sanskrit Learning. 

Alphabet 

42 = 9 vowels + 33 consonants 

9 vowels = 5 short + 8 long + 9 PLUT (3 MAATRA)

33 consonants = 5 * 5 + 8

= [AGHOSH 2 * 5 + 3 S = 13 ] + [3 * 5 + 5 = 20 GHOSH]

=================================================================

Similar words

सलिल (n) = waves, surge, rain, सलील = with play

चिर = long, चीर (m,n) = cloth

कूल (n) = slop, pond, bank , कुल (n) = family

वराङ्गना = beautiful / excellent woman , वाराङ्गना = brave woman

दिन = day , दीन = poor

गिरीश = mountain lord , गिरिश = inhabiting mountains

सुत = son , सूत = driver

=================================================================

Vocabulary

किङ्करः = servant 

कर्दमः = mud

मरालः = swan

अङ्गणम् = courtyard

पुरीषम् = output stool 

ऊर्जः = energy

उपोषणम् = fast 

दशनम् = tooth

चत्वरम् = CHORO

हर्म्यम् = HAVELI

उपवीतम् = JANOI

कलत्रम् = wife

नीराजनम् = AARATI

बलिवर्द (m) = ox

एजति = vibrate 

कन्था = GODADI, mattress

रथ्या = street

शिबिका = palanquin

प्रभवः = origin, source, birth

निदाध = summer 

अद्भुत = wonderful

सितम् = white, असितम् = black

शारम् , कर्बुरम् = zibra color

कपिशम् = KHAKHI

हृष्टम् =  happy

कृत्स्नम् = complete 

पीवरम् , पीनम् = fat

वदान्यम् = generous 

सव्यम् = left

स्थविरम् = immobile, big

करालम् = terrible 

वायस (m) = crow

=================================================================

SANDHI 1

र् ऋ ॠ ष् + न = र् ऋ ॠ ष् + ण 

र् ऋ ॠ ष् + ह य व कु पु [१+] + न  = र् ऋ ॠ ष् + ह य व कु पु [१+] + ण 

=================================================================

SANDHI 2

1st char becomes 3rd with vowel and GHOSH consonant  

=================================================================

Verb 1

उपधा = second last character 

गुण 

इ -> ए 

उ -> ओ 

ऋ -> अर्

लृ -> अल्

In verb, we shall change as per उपधा and गुण 

इदानीम् now तदानीम् at that time

आपरेद्युः = on the second day

इह = here

क्व = where (comparative) 

किन्तु = but

इत्थम् = like this way

अजस्त्रम् = continuous 

दिष्ट्या = fortunately 

सुष्ठु = nicely

Sanskrit Learning. Key take away points

from Man Ki Bat (मनोगतम्) Episode 1

स्नानार्थम् उपयुज्यमानः प्रोच्छः Towel 

आस्तरणम् Bedsheet

उपधानम् Pillow

अपकर्षः Discount

अभिज्ञान​म् Recognition, Remembrance, Knowledge

सिंही Lioness 

शावक Yong child of an animal

मदीय My, Mine, My own (Adjective) 

चितः चिता चितम्  Selected, Chosen 

उत Or / Else

उत न  Or not

8th VIBHAKTI भ्रातः पितः कर्तः 

विधि लिंग 

https://mycoaching.in/kri-ke-dhatu-roop

3rd October 2014

Here is list of few drama in simple Sanskrit language from various events

• Drama "Eklavya" https://www.youtube.com/watch?v=x65XcpzWor8&list=PL0faGNhyuhciydl4p3ftVb-mx8nHcQ6e-&index=2 and https://www.youtube.com/watch?v=O-ueNy4RkL4&list=PL0faGNhyuhciydl4p3ftVb-mx8nHcQ6e-&index=6
• Which Finger is the best? https://www.youtube.com/watch?v=KdTo9-L4OEA&list=PL0faGNhyuhciEIbpsPGFQSxAQOT_xlLTk&index=6
• Kavi Chakravarti Kalidas https://www.youtube.com/watch?v=2tqJdqyuNeg&list=PL0faGNhyuhciydl4p3ftVb-mx8nHcQ6e-&index=4
• NarSinha Sanskrit Drama https://www.youtube.com/watch?v=EfZ-fruR6jI&list=PL38D181E632683E50&index=11
• Who is the best devotee? https://www.youtube.com/watch?v=bYixNeZ_wbo&list=PL0faGNhyuhcivnD_xcfwIqooojUBto9l6&index=12
• Foolish King https://www.youtube.com/watch?v=mvc2MKwwbHQ&list=PL38D181E632683E50&index=13
• Cricket commentary in Sanskrit https://www.youtube.com/watch?v=VZ17VAqj1nA&list=PL0faGNhyuhcivnD_xcfwIqooojUBto9l6&index=4
• Small comedy drama https://www.youtube.com/watch?v=kbjS8DRln1Q&list=PL0faGNhyuhcgGDlwwyKmiL40XtK3VN8mR&index=9
• Mind and Intelligence https://www.youtube.com/watch?v=Vr_J2zJppbY&list=PL0faGNhyuhcivnD_xcfwIqooojUBto9l6&index=11
• Causal Samskrit Talk https://www.youtube.com/watch?v=cHhGfDTo5Ps&list=PL0faGNhyuhciEIbpsPGFQSxAQOT_xlLTk&index=8
• Dialogue https://www.youtube.com/watch?v=_DgS1QTps7o&list=PL0faGNhyuhcgGDlwwyKmiL40XtK3VN8mR&index=10
• Sanskrit Dialogue https://www.youtube.com/watch?v=GkOmv3WDcTU&list=PL0faGNhyuhcivnD_xcfwIqooojUBto9l6&index=6
• Small drama about the special hand-fan https://www.youtube.com/watch?v=rTKMtLNBbdQ&list=PL0faGNhyuhcgGDlwwyKmiL40XtK3VN8mR&index=14
• "A Friend in need, is friend indeed" https://www.youtube.com/watch?v=AQgfhDUCDT4&list=PL0faGNhyuhcgle-nkBDCGVBWTxMjQlZpl&index=4

My Key Take Away Points

vCluster allows us to create multiple virtual cluster using vCluster CLI commands on a single physical k8s cluster in matter of seconds and reduce cost. (1)syncer and (2) core dns are two building blocks to realize vCluster. We can try it with Minikube. Here entire cluster runs inside a pod (KIND) with Rancher's k3s distribution. The syncer component is greatly extensible with various plugins. Excellent tool vcluster, insightful talk and engaging questions answers

DevPod is another userful tool, that connects local IDE to remote development machine. 

Keploy is developer-centric API testing tool that creates tests along with built-in-mocks, faster than unit tests. It uses eBPF

Good talk on observability that covers all relevant tools Prometheus, OTel , Grafana etc. and nice QA

Kids Day was another event part of recent Kubecon. I did not find any YouTube video about the event. https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/program/kids-day/

There are many security tools XRay, Trivy, Synk, Grype, Syft etc. XRay is part of JFrog. 

We had quiz and social media contest to engage the audience. I met few regular friends. We had photo session with polarized camera. Here I am. 

Overall it was very good event. 

Meetup event link: https://www.meetup.com/loft-meetup-india/events/301365666/

 SREs and platform engineers use Hashicorp Vault to store Kubernetes secret. 

Hi all, sharing my speech on મારા મનગમતા સંત ( My Favorite Saint ). Written by my class teacher Fatamaben. It was one of my best speeches. Later on I read few books about "Pujya Mota" from Kamdaar club at my native place Mithapur. I planned to join engineering college at Nadiad, because Nadiad has Hari OM Ashram established by Pujya Mota. Finally I did Maun Ekant Sadhana at Hari Om Ashram at Nadiad and Surat. School days memories that gave direction to my life.

Key Takeaway points

O11y = Observability = logs (e.g. FluentD) + Metrics (Prometheus) + Traces (Jaeger) 

OTel = OpenTelemetry. OTel is CNCF specs for generation, collection and export of O11y. There are OTel API, OTel SDK, OTel auto instrumentation and Tools. 

OTel auto instrumentation has different maturity for different programming language. With OTel auto instrumentation, no need to modify existing code and yet we can integrate with OTel collector. 

Even we can pass B3 headers (for trace, span etc) header from incoming message to outgoing message, without modifying code. Nowadays, log message also contains trace ID and span ID. 

OTel Collector can collect data from K8s, L7Proxy, Cloud and shared infrastructure. OTel Collector emits the data towards Prometheus (for metrics), Grafana Loki (for log) and Grafana Tempo (for distributed tracing). OTel collector has three components (1) Receiver (2) Processor and (3) exporter. Yes the exporter can export to multiple endpoints. 

OTel collector can be deployed as sidecar container. We can have custom OTel collector also. 

=========================================

Kapacitor is an open source data processing framework that makes it easy to create alerts, run ETL jobs and detect anomalies. Kapacitor is a native data processing engine for InfluxDB

Ansible can be used to install minikube

https://github.com/robertdebock/ansible-playbook-minikube

https://github.com/gantsign/ansible_role_minikube

=========================================

Zabbix is enterprise class open source monitoring solution for network monitoring and application monitoring https://www.zabbix.com/index

=========================================

Meetup event https://www.meetup.com/geekyants-events-meetup-group/events/301078538/