Programming Kubernetes - I
Meetup Event : https://www.meetup.com/
* Only API server interact with etcd
* Extend K8s
1. Extend life cycle of API server by adding new controller
2. add more APIs : by adding new API resource types and by adding CRDs
3. cluster daemons for automation. E.g. collecting metrics
4. cluster assist to extend functionality. E.g. SSL certificate manager.
* Action or commands on K8s resources
1 create
2 read
- get
- list
- watch
3 update
- patch
- replace
4 delete
Here are all such actions/command on Deployment resource
https://github.com/kubernetes/client-go/blob/master/kubernetes/typed/apps/v1/deployment.go
* API
apis/// /resource
Short form : GVR = group, version, resource
Possible versions:
Alpha (v1alpha1)
Beta (v1beata1)
Stable (v1)
* Few useful commands
kubectl api version
kubectl api-resource
we can get short names
kubectl get --raw
* KIND used at YAML file is corrosponding to golang type. For any built-in K8s object:
1. https://github.com/kubernetes/api/blob/master///types.go has multiple struct in Go langugae corrpospond to each K8s object at YAML file.
2. https://github.com/kubernetes/client-go/tree/master/kubernetes/typed// folder has multiple files. Each file is corrsoponding to one K8s object.
* K8s support for client library implemented in many languages:
Official support for : Go, Python, Java, DotNet, JavaScript, Haskell
Many more language supported by community implementaiton
Please refer: https://kubernetes.io/docs/reference/using-api/client-libraries/ The source code is available at https://github.com/kubernetes-client
Out of all of them: client-go has cache and many more features. Kubectl and prometheus-operator also internally use client-go library. Important packages / utilities of client-go library are described at blog https://itnext.io/how-to-
* Flatcar Linux:
- an immutable Linux distribution for containers.
- a friendly fork of CoreOS’s Container Linux and as such, compatible with it.
- a self-updating operating system
- It is designed for containers, making it ideal for running Kubernetes and other container platforms.
- It aims to be an independently built, distributed, and supported Linux distribution designed for container workloads.
Reference : https://www.flatcar-linux.org/
* apimachienry
- Scheme, typing, encoding, decoding, and conversion packages for Kubernetes and Kubernetes-like API objects.
- This library is a shared dependency for servers and clients to work with Kubernetes API infrastructure without direct type dependencies.
- Its first consumers are k8s.io/kubernetes, k8s.io/client-go, and k8s.io/apiserver.
* "~/.kube/config" path has all the security related file to interface with K8s master node (or minikybe) kubeconfig has context
* controller uses service account. secreat is mounted inside pod
* jq is a lightweight and flexible command-line JSON processor. It is like sed for JSON. Reference : https://stedolan.github.io/jq/
* YAML file consists
1. api version
2. kind
3. matadata
4. spec
5. status
everything under metadata is in type of go language
Read Annotations
https://github.com/kubernetes/api/tree/master/core/v1
https://github.com/kubernetes/apimachinery/tree/master/pkg
Unit testing of controller with fake client
https://github.com/kubernetes-sigs/controller-runtime/blob/master/pkg/client/fake/client_test.go
* Join K8s slack channel "k8sBLR"
Reference
Go Client to communicate with K8s cluster: https://github.com/kubernetes/client-go
Schema of the external API types that are served by the Kubernetes API server: https://github.com/kubernetes/api
All relevant blogs: https://kubernauts.dev/
PPT slide deck: https://docs.google.com/
Github accounts
https://github.com/surajssd
https://github.com/surajssd/lspods
Relevant blogs
https://kubernetes.io/blog/2018/01/introducing-client-go-version-6/
https://itnext.io/how-to-
https://medium.com/
https://jvns.ca/blog/2017/06/04/learning-about-kubernetes/
Book
https://go.heptio.com/rs/383-
https://www.oreilly.com/library/view/programming-kubernetes/9781492047094/
https://stackoverflow.com/questions/49953980/watch-customresourcedefinitions-crd-with-client-go
on 24th August 2019
* Extend K8s
1. Extend life cycle of API server by adding new controller
2. add more APIs : by adding new API resource types and by adding CRDs
3. cluster daemons for automation. E.g. collecting metrics
4. cluster assist to extend functionality. E.g. SSL certificate manager.
* Action or commands on K8s resources
1 create
2 read
- get
- list
- watch
3 update
- patch
- replace
4 delete
Here are all such actions/command on Deployment resource
https://github.com/kubernetes/client-go/blob/master/kubernetes/typed/apps/v1/deployment.go
* API
apis/
Short form : GVR = group, version, resource
Possible versions:
Alpha (v1alpha1)
Beta (v1beata1)
Stable (v1)
* Few useful commands
kubectl api version
kubectl api-resource
we can get short names
kubectl get --raw
* KIND used at YAML file is corrosponding to golang type. For any built-in K8s object:
1. https://github.com/kubernetes/api/blob/master/
2. https://github.com/kubernetes/client-go/tree/master/kubernetes/typed/
* K8s support for client library implemented in many languages:
Official support for : Go, Python, Java, DotNet, JavaScript, Haskell
Many more language supported by community implementaiton
Please refer: https://kubernetes.io/docs/reference/using-api/client-libraries/ The source code is available at https://github.com/kubernetes-client
Out of all of them: client-go has cache and many more features. Kubectl and prometheus-operator also internally use client-go library. Important packages / utilities of client-go library are described at blog https://itnext.io/how-to-
* Flatcar Linux:
- an immutable Linux distribution for containers.
- a friendly fork of CoreOS’s Container Linux and as such, compatible with it.
- a self-updating operating system
- It is designed for containers, making it ideal for running Kubernetes and other container platforms.
- It aims to be an independently built, distributed, and supported Linux distribution designed for container workloads.
Reference : https://www.flatcar-linux.org/
* apimachienry
- Scheme, typing, encoding, decoding, and conversion packages for Kubernetes and Kubernetes-like API objects.
- This library is a shared dependency for servers and clients to work with Kubernetes API infrastructure without direct type dependencies.
- Its first consumers are k8s.io/kubernetes, k8s.io/client-go, and k8s.io/apiserver.
* "~/.kube/config" path has all the security related file to interface with K8s master node (or minikybe) kubeconfig has context
* controller uses service account. secreat is mounted inside pod
* jq is a lightweight and flexible command-line JSON processor. It is like sed for JSON. Reference : https://stedolan.github.io/jq/
* YAML file consists
1. api version
2. kind
3. matadata
4. spec
5. status
everything under metadata is in type of go language
Read Annotations
https://github.com/kubernetes/api/tree/master/core/v1
https://github.com/kubernetes/apimachinery/tree/master/pkg
Unit testing of controller with fake client
https://github.com/kubernetes-sigs/controller-runtime/blob/master/pkg/client/fake/client_test.go
* Join K8s slack channel "k8sBLR"
Reference
Go Client to communicate with K8s cluster: https://github.com/kubernetes/client-go
Schema of the external API types that are served by the Kubernetes API server: https://github.com/kubernetes/api
All relevant blogs: https://kubernauts.dev/
PPT slide deck: https://docs.google.com/
Github accounts
https://github.com/surajssd
https://github.com/surajssd/lspods
Relevant blogs
https://kubernetes.io/blog/2018/01/introducing-client-go-version-6/
https://itnext.io/how-to-
https://medium.com/
https://jvns.ca/blog/2017/06/04/learning-about-kubernetes/
Book
https://go.heptio.com/rs/383-
https://www.oreilly.com/library/view/programming-kubernetes/9781492047094/
https://stackoverflow.com/questions/49953980/watch-customresourcedefinitions-crd-with-client-go
Alternatives of tcpdump
There
are many tools similar to tcpdump, as per https://en.wikipedia.org/wiki/Comparison_of_packet_analyzers
Here, I choose only Free and Open Source tools, whose docker
image is available and tool is lightweight.
- Ngrep
is best, for capture only those packets, whose payload has certain
pattern.
- Packetbeat is
lightweight open source packet analyzer. It sends data to Elastic Search
OR Logstash. It is not inline to datapath. So no impact on latency. It
consumes high CPU. Packetbeat can run as sidecar Docker container: https://www.elastic.co/guide/en/beats/packetbeat/current/running-on-docker.html
It supports various protocols https://www.elastic.co/guide/en/beats/packetbeat/current/exported-fields.html
It can capture all HTTP headers from request and
response https://www.elastic.co/guide/en/beats/packetbeat/current/exported-fields-http.html
Some more reference: https://www.elastic.co/products/beats/packetbeat
- Tranalyzer
is Lightweight open-source flow generator and packet analyzer for
practitioners and researchers
- Justniffer
is like tcpdump. Tcpdump is for TCP, while Justniffer for HTTP. Useful to
debug webserver.
https://github.com/reneluria/justniffer
5. Moloch is a large scale, open source, indexed packet capture and search system.
5. Moloch is a large scale, open source, indexed packet capture and search system.
Digital Certificate and SSL
1. cryptographic algorithms,
1.1 Conventional cryptography (symmetric key)
1.2 Public Key cryptography
2. message digest functions, = one-way hash
3. digital signatures
Encrypt 1. digest 2. seq number 3. etc. using private key
Certificate
DER is based on BER. Digital certificate is converted to binary format using DER. Then Base64 convert + add prefix BEGIN + add suffix END = PEM format.
Cipher suite
1. Key Exchange Method : RSA, DH. with / without signature
2. Cipher for data transfer
2.1 No encryption
2.2 Steam cipher
2.3 Block cipher
3. Message Digest for creating MAC.
3.1 no digest
3.2 MD5
3.3 SHA
SSL Record Protocol between TCP and HTTP layers
1. Input from HTTP goes to RPU (Record Protocol Unit)
2. Compress input
3. add MAC
4. encrypt
5. output as TCP payload
1.1 Conventional cryptography (symmetric key)
1.2 Public Key cryptography
2. message digest functions, = one-way hash
3. digital signatures
Encrypt 1. digest 2. seq number 3. etc. using private key
Certificate
DER is based on BER. Digital certificate is converted to binary format using DER. Then Base64 convert + add prefix BEGIN + add suffix END = PEM format.
Cipher suite
1. Key Exchange Method : RSA, DH. with / without signature
2. Cipher for data transfer
2.1 No encryption
2.2 Steam cipher
2.3 Block cipher
3. Message Digest for creating MAC.
3.1 no digest
3.2 MD5
3.3 SHA
SSL Record Protocol between TCP and HTTP layers
1. Input from HTTP goes to RPU (Record Protocol Unit)
2. Compress input
3. add MAC
4. encrypt
5. output as TCP payload
TCPdump inside docker
For TCPDump 1
=============
k get pod productpage-v1-8554d58bff-rz25r -o json | grep containerID
docker exec ff8e6d1a169bc225ad1e357b54445e9055423189b900176bcc6cdd393f9cd83d /bin/bash -c 'cat /sys/class/net/eth0/iflink'
ip link | grep ^47
tcpdump -i
For TCPDump 2
=============
add
- name: tcpdump
image: corfr/tcpdump
command:
- /bin/sleep
- infinity
k get pod productpage-v1-8554d58bff-rz25r -o json | grep containerID
docker exec 867662a10a0324059b71d3be9765069b900eca4f2f5f29fdb2e7b7792fcfc726 tcpdump -s 0 -n -w /tmp/container.pcap
docker cp 867662a10a0324059b71d3be9765069b900eca4f2f5f29fdb2e7b7792fcfc726:/tmp/container.pcap .
=============
k get pod productpage-v1-8554d58bff-rz25r -o json | grep containerID
docker exec ff8e6d1a169bc225ad1e357b54445e9055423189b900176bcc6cdd393f9cd83d /bin/bash -c 'cat /sys/class/net/eth0/iflink'
ip link | grep ^47
tcpdump -i
For TCPDump 2
=============
add
- name: tcpdump
image: corfr/tcpdump
command:
- /bin/sleep
- infinity
k get pod productpage-v1-8554d58bff-rz25r -o json | grep containerID
docker exec 867662a10a0324059b71d3be9765069b900eca4f2f5f29fdb2e7b7792fcfc726 tcpdump -s 0 -n -w /tmp/container.pcap
docker cp 867662a10a0324059b71d3be9765069b900eca4f2f5f29fdb2e7b7792fcfc726:/tmp/container.pcap .
For TCPDump 3
=============
=============
1. run 'docker ps' at worker node.
get the container Id, for target container
2. get pid of that container
docker inspect --format '{{.State.pid}}' "Container ID from previous command"
3. nsenter -t "PID value from previous command" -n tcpdump
Istio Practical - 1
Installation
Istio version istio-1.3.0-rc.1 at path Downloads/istio-1.3.0-rc.1
Helm version helm-v2.14.3
sudo apt-get install socat
kubectl create serviceaccount tiller --namespace kube-system
kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller
helm init --wait --service-account tiller
kubectl create namespace istio-system
helm repo add istio.io https://storage.googleapis.com/istio-release/releases/1.2.5/charts/
helm repo update
helm template Downloads/istio-1.3.0-rc.1/install/kubernetes/helm/istio-init --name istio-init --namespace istio-system | kubectl apply -f -
helm install --wait --name istio --namespace istio-system Downloads/istio-1.3.0-rc.1/install/kubernetes/helm/istio \
--set gateways.istio-ingressgateway.type=NodePort \
--set gateways.istio-egressgateway.type=NodePort \
--set grafana.enabled=true \
--set kiali.enabled=true \
--set kiali.dashboard.grafanaURL=http://localhost:3000 \
--set kiali.dashboard.jaegerURL=http://localhost:16686 \
--set servicegraph.enabled=true \
--set telemetry-gateway.grafanaEnabled=true \
--set telemetry-gateway.prometheusEnabled=true \
--set tracing.enabled=true \
--set sidecarInjectorWebhook.enabled=true \
--set global.mtls.enabled=false
Istio version istio-1.3.0-rc.1 at path Downloads/istio-1.3.0-rc.1
Helm version helm-v2.14.3
sudo apt-get install socat
kubectl create serviceaccount tiller --namespace kube-system
kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller
helm init --wait --service-account tiller
kubectl create namespace istio-system
helm repo add istio.io https://storage.googleapis.com/istio-release/releases/1.2.5/charts/
helm repo update
helm template Downloads/istio-1.3.0-rc.1/install/kubernetes/helm/istio-init --name istio-init --namespace istio-system | kubectl apply -f -
helm install --wait --name istio --namespace istio-system Downloads/istio-1.3.0-rc.1/install/kubernetes/helm/istio \
--set gateways.istio-ingressgateway.type=NodePort \
--set gateways.istio-egressgateway.type=NodePort \
--set grafana.enabled=true \
--set kiali.enabled=true \
--set kiali.dashboard.grafanaURL=http://localhost:3000 \
--set kiali.dashboard.jaegerURL=http://localhost:16686 \
--set servicegraph.enabled=true \
--set telemetry-gateway.grafanaEnabled=true \
--set telemetry-gateway.prometheusEnabled=true \
--set tracing.enabled=true \
--set sidecarInjectorWebhook.enabled=true \
--set global.mtls.enabled=false
