Container Orchestration


What is Container Orchestration ?

- Fault tolerant
- Scaling
- Optimally use resources
- Discovery
- Access from outside world
- Update/rollback with 0 downtime

What Container Orchestrators do?

- cluster = multiple host together
- schedule container(pod) to run on host/node
- network among pod on different nodes
- bind container with storage
- services = group of containers
- keep resource usage in check and optimize if needed
- secure access to app running inside continer

Deployment options

on-premise v/s cloud
bare-matel v/s VM

Container Orchestrators 

- Kubernetes
- Docker Swarm
- Mesos Marathon
- Docker Compose : Single machine. Not for large scale. With one command, "docker compose up" it will bring up : containers, volumes, networks
- Hashicorp Nomad

- Amazon ECS (Amazon EC2 container service)
1. Task == Pod
2. It has its own repository. 
3. Task can be part of CloudFormation stack. Task, Queue, EC2 Volume all together in 
4. CloudFormation to start and to cleanup
5. To get started https://aws.amazon.com/ecs/
- AWS Fargate https://aws.amazon.com/fargate
- Google Kubernetes Engine (^L = clear = cls at Google Cloud Shell)
- Microsoft Azure Kubernetes Services (AKS)
- Cloud Foundry
- Rackspace
- Oracle Cloud Infrastructure 
- Rancher

K8s Features

schedule pod based on resource usage and constrain with HA
self-healing
scaling
- service discovery + load balancing. 
- auto rollout and rollback
- secrets and config mgt
- storage orchstration with SDS
- batch execution
- RBAC

Developers With Possibilities - CallForCode2019



Can technology can really solve the issues concerning to Natural Disasters or Health? Can it really become a powerful medium to answer some of biggest problem of the mother earth? Definitely Yes, It can certainly solve or prove a powerful medium to reach the last person affected due to Natural Calamities. In simple terms, Technology can help to reduce the impact of Natural Disasters.

A study conducted by World Bank in year 2008 about Philippines stated that, “50% of its total land and more than 80% of its total population is venerable to Natural Disasters”. In fact, Philippines is the 3rd most venerable country in the world as far as Natural Disasters and its aftermath is considered.

Heavy and sudden floods; intense monsoons play a vital role to destroy the land and its population the most. Can something be done about it? As a developer myself, I can say Yes.

With proper use of technology, Analytical Data can be easily used to provide essential information as to how a particular structure will respond in case of any unfortunate situation. Cities or Countries which are more affected by such Disaster can easily integrate and make use of geospatial data and come up proper maintenance plan of Entire Township. Geospatial Data provide precise geographical component of a particular areas and thus can play pivotal role in Township Planning and Maintain acne.



Let us take another example and understand How Innovation and Technology Together Can Help us in staying ahead in case of Natural Disaster. In last one decade or so total loss due to natural disaster is around $200 Billion which was at year on year average of 50 billion USD. Trends suggest that rising population and urbanization are driving losses in vulnerable regions. More than million people are moving towards cities every week, most of which is happening in region of Africa and Asia. On the other hand, climate change threatens push 100 million more people into poverty by 2030. Such information is enough to make it clear that disaster risk isn’t static, but rapidly evolving.

Global Facility for Disaster Reduction and Recovery (GFDRR) – A World Bank Initiative along with 34 countries and 9 international organizations across globe are motivating governments of various countries to come forward and invest in smarter technologies such as AI, Mobile Applications, and Navigations etc. and make citizen aware of risk much before it is going to happen. Today, Mobile Phone has a wider reach and even a small SMS in regional language can save millions of life even before a disaster take place.

Do you know that an air balloon can deliver internet in the most rural and unconnected places in the world, Kites can be used to generate electricity in unexpected region. Surprised!

These are some of the issues which are actually solved with the effective use of Technology. So why can’t we come ahead in great numbers and pledge to build solutions to protect the interest of last man living on earth and protect his/her from the aftermath of Natural Disaster!

More than a 1,00,000 developers from 156 countries participated in year 2018 call For Code Challenge organized by IBM and David Clark Cause. More than 2500 applications were designed, developed with use of IBM products and technologies by sharing of knowledge, resource and proper mentorship. Each of the solutions focused on some real life challenges. The aim was simple, Give back To the Society, To Your People something that can save their life and help them to stay healthy.

The mission remains the same this time too. Come forward and join #Callforcode2019 challenge and build solutions that can solve or reduce the impact of Natural Disaster on Human Life. 

The Process to Participate is simple:-
  • Register Yourself with IBM ID on the website https://developer.ibm.com/callforcode
  • Make use of Available Resources and Technologies and Build Your Solutions
  • Be the part of Developer’s Community and share your idea , take necessary mentorship , build team and complete the project.
  • Submit your code with proper demo with participant agreement.

Don’t miss the opportunity this time. Last date is July 29, 2019.
Be the Best. The Winner will get a loud shout on Award Night with a cash reward of $ 2,00,000 apart from Investors and Technology Support.
It’s time to rock!! Go ahead and mark your presence with a solution that matters the most.

Bangalore Kubernetes May 2019


Hrishikesh Shinde talked about "Intro to sample controller"

He introduced to following github repositories

https://github.com/kubernetes/client-go
Go clients for talking to a kubernetes cluster. It implements watch. 

https://github.com/kubernetes/apimachinery
Scheme, typing, encoding, decoding, and conversion packages for Kubernetes and Kubernetes-like API objects. It is used for filter. 

https://github.com/kubernetes/api
Schema of the external API types that are served by the Kubernetes API server.

https://github.com/kubernetes/code-generator 
Golang code-generators used to implement Kubernetes-style API types.

Hrishikesh further explained his own code repository. 

https://github.com/hrishin/podset-operator
Build replicaset kind of k8s object. Build controller from scratch to demonstrate  KuberBuilder or Operator SDK frameworks

We need to develop boiler plate code at path /pkg/apis/demo/v1alpha1/
1. doc.go
2. register.go
3. types.go

The controller takes 3 actions

1. Check for desire state
2. Strive for desire state
3. Update about resourse status using AvailableReplicaStatus variable. 

Controller uses various data structures (modules) like informer, listers, reflect etc. 

Hrishikesh suggested books like
https://www.amazon.com/Programming-Kubernetes-Developing-Native-Applications/dp/1492047104 and others. 

Slide deck

Nikhil Tomas talked about "Intro to operator SDK"

I have updated my blog "K8s Operator" with useful details. 

- quick start: https://github.com/operator-framework/operator-sdk#quick-start

- user guide: https://github.com/operator-framework/operator-sdk/blob/master/doc/user-guide.md

Slide deck

Suraj Narwade gave valuable tips about CKA examNeependra Khare also added some more points. 

One need to focus on concepts and tasks, by studying them four to five times. 

Also study 4 to 5 times the course
Kubernetes - the hard way
https://github.com/kelseyhightower/kubernetes-the-hard-way
https://github.com/kinvolk/kubernetes-the-hard-way-vagrant

Practise, Practise and Practise
Use GCP, AWS, minkube on Linux box or online like Katacoda and CloudYuga
Also refer my blog : Kubernetes Practicals 

There will be only one termianl. So better get familar with tmux

They suggested book : Kubernetes up and runningdive into the future of infrastructure

At begining of exam set alias for various frequently used kubectl commands. 
Also set auto completion of commands
  
Few useful blogs/URLs
https://kubernauts.dev/
https://suraj.pro/
https://suraj.io/

Suraj's slide deck

Neependra mentioned about upcomign webinar from CloudYuga

Meetup Reference
https://www.meetup.com/kubernetes-openshift-India-Meetup/events/261130273/

Python : collections module


Python has general purpose data types like dict, list, set, tuple. Collections module has following additional useful datatype

1. namedtuple

Useful to contruct objects

2. Counter

It works with string, list, and sentence. Sentence should be split with ' ' to convert into list of words. 

suppose

c = Counter(list)

then 
c.values() gives only count. So sum(c.values()) givestotal of all counts 
c.most_common() sort based on frequency and return list of tuples
c.most_common()[0][0] gives the item with maximum occurance
c.most_common()[:-2:-1] gives the item with maximum occurance
c.most_common()[:-n-1:-1]  gives the item with n least common elements 
c.substract(d) Here is d is another Counter. The result will be frequency for each element will be substraced as per its frequency in d. 

3. defaultdict(object)
It gives default empty dictionary. 

4. OrderedDict

OrderedDict(sorted(d.items(), key=lambda t: t[0])) to sort with key
OrderedDict(sorted(d.items(), key=lambda t: t[1])) to sort with value

5. deque

To add : append(), appendleft()
To remove : pop() , popleft()
to count: count()
To insert as specific index i : insert(i, x)


6. ChainMap
This is to join 2 dict as a list with 2 elements as dict

Open Container Initiative


1. Image specification

An OCI Image is an ordered collection of root filesystem changes and the corresponding execution parameters for use within a container runtime

https://github.com/opencontainers/image-spec/blob/master/spec.md

1.1 manifest
It contains metadata (annotation = key, value pair) about the contents and dependencies of the image 
JSON file
https://github.com/opencontainers/image-spec/blob/master/manifest.md
https://docs.docker.com/engine/reference/commandline/manifest/

1.2 image index (optional)
For platform specific version of image

1.3 image layout

1.4 file system layers

1.5 configuration
JSON file https://github.com/opencontainers/image-spec/blob/master/config.md

1.6 conversion

1.7 descriptor 

Tools

2. Distribution specification

It describes the API that might be used by a registry to distribute images, to easily 
- share, 
- search, 
- obtain and 
- verify 
container image

https://github.com/opencontainers/distribution-spec/blob/master/spec.md

It is implemented by Docker Registry and https://github.com/atlaskerr/stori

3. Runtime Specification

It aims to specify the configuration, execution environment, and lifecycle of a container.
aims to specify the configuration, execution environment, and lifecycle of a container.

runc is Runtime specification OCI implementation 

LifeCycle


3.1 create command
3.2 create runtime environment using config.json
3.3 start command
3.4 pre-start hook
3.5 user program specified by process
3.6 post-start hook
3.7 exit due to exit, error, kill, crash
3.8 delete
3.9 destroy container. undo create
3.10 post-stop hook

https://github.com/opencontainers/runtime-spec/blob/master/glossary.md

CRI = Container Runtime Interface = protocol buffers + gRPC APIs + Libraries