Programming Kubernetes - I

Meetup Event :

on 24th August 2019

* Only API server interact with etcd

* Extend K8s

1. Extend life cycle of API server by adding new controller
2. add more APIs : by adding new API resource types and by adding CRDs
3. cluster daemons for automation. E.g. collecting metrics
4. cluster assist to extend functionality. E.g. SSL certificate manager. 

* Action or commands on K8s resources

1 create
2 read
- get
- list
- watch
3 update
- patch
- replace
4 delete

Here are all such actions/command on Deployment resource



Short form : GVR = group, version, resource 

Possible versions: 
Alpha (v1alpha1)
Beta (v1beata1)
Stable (v1)

* Few useful commands

kubectl api version
kubectl api-resource

we can get short names
kubectl get --raw 

* KIND used at YAML file is corrosponding to golang type. For any built-in K8s object:
1. has multiple struct in Go langugae corrpospond to each K8s object at YAML file.
2. folder has multiple files. Each file is corrsoponding to one K8s object. 
* K8s support for client library implemented in many languages: 
Official support for : Go, Python, Java, DotNet, JavaScript, Haskell
Many more language supported by community implementaiton
Please refer: The source code is available at
Out of all of them: client-go has cache and many more features. Kubectl and prometheus-operator also internally use client-go library. Important packages / utilities of client-go library are described at blog E.g. Kubernetes, Discovery, Dynamic, Transport, Plugin, Scale, Util, tool/cache etc. tools/clientcmd package is useful to setup client from kubeconfig file. 

* Flatcar Linux: 

- an immutable Linux distribution for containers. 
- a friendly fork of CoreOS’s Container Linux and as such, compatible with it. 
- a self-updating operating system 
- It is designed for containers, making it ideal for running Kubernetes and other container platforms. 
- It aims to be an independently built, distributed, and supported Linux distribution designed for container workloads.
Reference :

* apimachienry
- Scheme, typing, encoding, decoding, and conversion packages for Kubernetes and Kubernetes-like API objects.
- This library is a shared dependency for servers and clients to work with Kubernetes API infrastructure without direct type dependencies. 
- Its first consumers are,, and

* "~/.kube/config" path has all the security related file to interface with K8s master node (or minikybe) kubeconfig has context

* controller uses service account. secreat is mounted inside pod

* jq is a lightweight and flexible command-line JSON processor. It is like sed for JSON. Reference :

* YAML file consists 

1. api version
2. kind
3. matadata
4. spec
5. status
everything under metadata is in type of go language

Read Annotations

Unit testing of controller with fake client

* Join K8s slack channel "k8sBLR"


Go Client to communicate with K8s cluster:

Schema of the external API types that are served by the Kubernetes API server: 

All relevant blogs: 

PPT slide deck:

Github accounts

Relevant blogs


Alternatives of tcpdump

There are many tools similar to tcpdump, as per

Here, I choose only Free and Open Source tools, whose docker image is available and tool is lightweight.

  1. Ngrep is best, for capture only those packets, whose payload has certain pattern.

  1. Packetbeat is lightweight open source packet analyzer. It sends data to Elastic Search OR Logstash. It is not inline to datapath. So no impact on latency. It consumes high CPU. Packetbeat can run as sidecar Docker container:
It can capture all HTTP headers from request and response

  1. Tranalyzer is Lightweight open-source flow generator and packet analyzer for practitioners and researchers

  1. Justniffer is like tcpdump. Tcpdump is for TCP, while Justniffer for HTTP. Useful to debug webserver.

Digital Certificate and SSL

1. cryptographic algorithms, 
1.1 Conventional cryptography (symmetric key)
1.2 Public Key cryptography 
2. message digest functions, = one-way hash
3. digital signatures
Encrypt 1. digest 2. seq number 3. etc. using private key

DER is based on BER. Digital certificate is converted to binary format using DER. Then Base64 convert + add prefix BEGIN + add suffix END = PEM format.

Cipher suite
1. Key Exchange Method : RSA, DH. with / without signature
2. Cipher for data transfer
2.1 No encryption
2.2 Steam cipher
2.3 Block cipher
3. Message Digest for creating MAC.
3.1 no digest
3.2 MD5
3.3 SHA

SSL Record Protocol between TCP and HTTP layers
1. Input from HTTP goes to RPU (Record Protocol Unit)
2. Compress input
3. add MAC
4. encrypt
5. output as TCP payload

TCPdump inside docker

For TCPDump 1

k get pod productpage-v1-8554d58bff-rz25r -o json | grep containerID

docker exec ff8e6d1a169bc225ad1e357b54445e9055423189b900176bcc6cdd393f9cd83d /bin/bash -c 'cat /sys/class/net/eth0/iflink'

ip link | grep ^47

tcpdump -i

For TCPDump 2


- name: tcpdump
   image: corfr/tcpdump
     - /bin/sleep
     - infinity
k get pod productpage-v1-8554d58bff-rz25r -o json | grep containerID

docker exec 867662a10a0324059b71d3be9765069b900eca4f2f5f29fdb2e7b7792fcfc726 tcpdump -s 0 -n -w /tmp/container.pcap

docker cp 867662a10a0324059b71d3be9765069b900eca4f2f5f29fdb2e7b7792fcfc726:/tmp/container.pcap .

Istio Practical - 1


Istio version istio-1.3.0-rc.1 at path Downloads/istio-1.3.0-rc.1
Helm version helm-v2.14.3

sudo apt-get install socat

kubectl create serviceaccount tiller --namespace kube-system

kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller

helm init --wait --service-account tiller

kubectl create namespace istio-system

helm repo add

helm repo update

helm template Downloads/istio-1.3.0-rc.1/install/kubernetes/helm/istio-init --name istio-init --namespace istio-system | kubectl apply -f -

helm install --wait --name istio --namespace istio-system Downloads/istio-1.3.0-rc.1/install/kubernetes/helm/istio \
  --set gateways.istio-ingressgateway.type=NodePort \
  --set gateways.istio-egressgateway.type=NodePort \
  --set grafana.enabled=true \
  --set kiali.enabled=true \
  --set kiali.dashboard.grafanaURL=http://localhost:3000 \
  --set kiali.dashboard.jaegerURL=http://localhost:16686 \
  --set servicegraph.enabled=true \
  --set telemetry-gateway.grafanaEnabled=true \
  --set telemetry-gateway.prometheusEnabled=true \
  --set tracing.enabled=true \
  --set sidecarInjectorWebhook.enabled=true \
  --set global.mtls.enabled=false