1. Design

API -> Primitives (Building Blocks) for 
1. deploy 
2. maintain 
3. scale 

1.1 Pod

* Scheduling unit
* Pod = 1+ co-located containers. 
* Pod has unique IP within cluster. 
* Can be managed by Kubernetes API or controller. 

1.2 Labels & Selectors

* Key-Value pair
* attached to pod and node
* grouping mechanism 

1.3 Controllers

* Manage a set of podes as per "Labels and Selector"
* reconciliation loop drive cluster state from actual to desirable 
E.g Replication controller, Daemonset controller to run 1 pod on 1 machine, Job controller 

1.4 Services

* set of pods works together, E.g. tier in multi-tier
* set defined by labels & selector.
* service discovery by Kubernetes

2. Architecture

* Master-slave

2.1 C-plane

2.1.1 etcd

* key value data store
* configuration data of cluster
* represent overall state of cluster
* other componenets monitors changed at etcd

2.1.2 API server

* JSON over HTTP
* Validate REST request and update API objects's state at etcd
* so client can configure workloads, containers across the worker nodes

2.1.3 Schedular

* plugable 
* match resource "supploy" to workload "demands"
* select nod to run pod
* inputs
- resource availability
- resource untilization
- resource requirement
- QoS
- afinity requirements
- anti-afinity requirements
- data locality 

2.1.4 contoller manager
* process to run (1) Daemonset controller (2) Replication controller 
* communicate with API server to create, update, delete (1) pod, (2) service end points (3) etc.

2.2 Kubernetes Node

= Worker = Minion 
* run container runtime. e.g Docker and below componenets

2.2.1 Kubelet

* hearbeat for health of node.

2.2.2 Kube-proxy

* n/w proxy + load balancer
* route to container based on IP + port

2.2.3 cAdvisor

Agent to collect resource usage. 

Artificial Intelligence / Machine Learning : Videos

Machine Learning online courses

1. Machine Learning

2. Deep Learning

3. Convolutional Neural Network : CS231N

6. Intro to Machine Learning

7. Distributed Machine Learning with Apache Spark

Machine Learning

Today, Let me share a list of website related to Machine Learning.This list is created by referring to dhilipsiva@gmail.com 's e-mails to BangML meetup group. Acknowledged. 


Today Let's revise / refresh few aspects of C++ object oriented programming language

  • A class has implicit members
  1. Default constructor 
  2. Copy constructor. It cannot take object as argument. It always takes object reference as argument. 
  3. Assignment operator
  4. Default destructor 
  • Now some specific points about constructor
  1. A class can have default constructor and explicit constructor. Explicit constructor force compiler for not to do implicit conversion
  2. However a constructor cannot be virtual
  • The copy constructor is called in following cases
  1. If function return a object by value
  2. Function argument is object by value
  3. When we construct a object using another object
  4. Many times compiler generates temporary object
  • Storage class
  1. auto
  2. register. Here '&' operator cannot be used
  3. static. The variable is initialize with value 0
  4. extern The variable is initialize with value 0
  • Storage qualifier OR type qualifier
  1. volatile 
  2. mutable
  3. restrict C99 standard. compiler optimization that avoid multiple loading of same variable. 
  4. _Atomic . C99 standard 
  5. const . In the context of pointer, const can be used 3 different ways
    1. int const * const x
    2. int const * x
    3. int * const x
    • As we know for member function of class "this" pointer is passed by default. It is always passed as constant pointer. 
  • The following operator cannot be overloaded 
  1. sizeof
  2. "."
  3. ".*"
  4. typeid
  5. "::"
  6. "?:"
  7. alignof
  • Now about type casting of variables 
  1. dynamic_cast . Runtime type check. It is used to case 
    1. base class pointer to derived class pointer
    2. derive class pointer to another derive class pointer of same base class as parent. 
  2. reinterpret_cast : cast to unrelated object
  3. static cast. compile time type check. for related object
  4. const_cast
  5. duration cast from chrono library
  • C++ has specific features like
  1. Class
  2. Inline function
  3. Default argument 
  4. Exception
  5. Namespace
  6. Boolean type
  • Object Oriented Programming features
  1. Abstraction
  2. Encapsulation (information hiding) 
  3. Inheritance 
  4. Polymorphism 
  • Something about Inheritance 
    • Virtual inheritance is used to avoid diamond problem
    • Inheritance applications are (1) code reuse and (2) overriding base class implementaiton
    • 5 types of inheritance
      • Single
      • Multiple
      • Multilevel
      • Hierarchical 
      • Hybrid
  • Something about polymorphism 
    • Static polymorphism
      • Function overloading is also called ad-hoc polymorphism
      • Templates is also called parametric polymorphism
    • Dynamic polymorphism
      • Subtyping is also called inclusion polymorphism
      • C++ implements virtual table with single dispatch only. No multiple dispatch
We will some more topics about C++ and C programming language. Stay tunes

Service-Oriented Architecture

Service-oriented architecture is a buzzword. It is all about 1. Loose coupling 2. Modular programme 3. connected components using various protocols over IP. SOA is the philosophy of encapsulating application logic in services with a uniformly defined interface and making these publicly available via discovery mechanism. Let's look at some more aspects about SOA

Core Values
  1. More focus on 'business value' compare to 'technical strategy'
  2. So more focus on 'strategic goal' compare to 'project benifits'
  3. The components should have 'intrinsic inter-operability'
  4. More importance to 'shared services' compare to 'implementation for specific purpose'
  5. Flexibility is more important compare to optimisation
  6. Here, 'Evolution refinement' is more important compare to 'initial perfection'
SOA is similar to 1. Modular programming 2. Distributed computing 3. mashups 4. SaaS and 5. Cloud computing. Still it is different. SOA has 11 principals. 

1. The service contract should be standardized. That includes functional expressions may be using WSDL. Data model may be using XML schema and 3. Policy document. 

2. Reference autonomy standardization : The other components are merely aware about presence of other service component. It fosters the loose coupling

3. Service Location Transparency : Service can be executed, deployed anywhere. 

4. Service Longevity : Service should run for ever. It also means backward compatible implementation. 

5. Service Abstraction means the service is jut a black box. (1) The functional aspects of service (2) Technical information (3) Logic / Algorithm and (2) Quality of service, all these are hidden. 

6. Service autonomy : It means, service should be more predictable, reliable and less depends on external environment.  Run time service autonomy achieved by dedicated CPU and memory so external environment has lesser impact. Design time service autonomy means, service can be evolve and re-design without impacting consumers. More applicable to entity service and utility service. 

7. Service statelessness: The service is implemented as stateless. However it can take support for other system to maintain state. Service may need to maintain state about (1) Context data (2) Business data and (3) Session data. 

8. Service granularity define the scope of service. It impacts performance and message size for the service. 

9. Service normalization is trade off between redundancy and performance 

10. Service composability  A service can play two role here (1) Service controller and (2) Service member 

11. Service discovery : Using service discovery protocol 

12. Service re-usability : The service, which is part of multiple business processes is more reusable. The service with extra functionality will be reusable in future. 

13. Service encapsulation: The service not part of SOA, initially. 

Each SOA building block plays any of the three roles

1. Service provider

2. Service broker = service registry = service repository

3. Service consumer = service requester


SOA can be implemented in multiple ways

1. SOA can be implemented using any service based technology. It can be web services using SOAP+WSDL or RESTful protocol. It can be using (1) Jini and JBI (2) CORBA (3) REST (4) WCF

2. Messaging technology can be used (1) ActiveMQ (2) JMS (3) RabbitMQ

3. OPC-UA is a M2M protocol. It can be web service oriented. It can be binary protocol over TCP/IP. It can by hybrid approach of binary protocol over SOAP. 

4. Apache Thrift is a RPC based framework by FaceBook. It provides IDL and binary protocol. 

5.  SORCER is Java based distributed computing platform, for service oriented computing environment. 

6. The programming language like C# .Net, Java, BPEL also has support for SOA

7. WS-CDL and WS-Coordination Specifications extend SOA further for workflows and business process.

Service Oriented Modelling

In SOA framework, Service Oriented Modelling identifies various disciplines to conceptualize, analyze, design, and architect service-oriented assets. Here three approaches are identified

1. SDDM: Service-oriented design and development methodology

2. SOMA: Service-oriented modeling and architecture

3. SOMF: Service-oriented modeling framework. It is about what to do. 

Extension and variants

SOA can be further extended with various variants

1. SOA 2.0 is combination of SOA and EDA. Event Driven Architecture. It has three components. (1) Event emitters (agents) (2) Event consumers (sinks OR event processing engine) (3) Event channel. It can be (3.1) message oriented middleware or (2) TCP/IP link (3) File: XML, flat file, or even e-mail. 

2. Web 2.0 has same philosophy as SOA, but it serves different user needs. 
3. Microservices : (1) Fine grained services may be a process of OSGi Bundle (2) Light weight protocol may be HTTP or shared memory. Results in continuous software development process. Jolie programming language allows to think and program in terms of microservices. The complexity moves to network as (1) network latency (2) message format (3) load balancing and (4) fault tolerance

I hope, the readers like this overview of SOA. 

Database Key

Today one new label is added. "Database". There are set of articles planned in this category. Today is the first one. 

Most of us know, in RDBMS, database table has primary key and foreign key. Any idea about other types of key? Below tables describes all database keys and relations among them. 

May be
May be
May be
May be
May be
May be
May be
May be
May be
May be
May be
May be
May be
May be
May be
May be
May be
May be
May be
May be
May be
May be
Minimal super
May be
May be
May be
Tirival Super
May be
May be
May be
May be
May be
May be
May be
May be
May be
May be
May be
May be

Super Minaml super Tirival Super Candidate Alternate Natural
1 Primary May be May be 0 1 0 May be
2 Unique 1 May be May be 1 May be May be
3 Surrogate 0 0 0 May be May be 0
4 Foreign May be May be 0 0 0 0
5 Composite May be May be 1 1 May be 0
6 Simple 0 May be 0 May be May be May be
7 Super 1 May be 1 0 0
8 Minimal super 1 1 0 1 May be May be
9 Tirival Super 1 0 1 1 May be 0
10 Candidate 1 1 1 1 May be May be
11 Alternate 1 1 0 1 1 May be
12 Natural 0 1 May be 1

The above table should be read, row wise

For example 1st row: 

Is primary key is unique? 1 = Yes
Is it surrogate? The surrogate key can become primary key. However the primary key is not always surrogate. So answer "May be"
and so on. Here 1 = Yes and 0 = No

Here are some key points about database keys
  • The primary key cannot be NULL
  • The unique key can be NULL
  • The primary key is always unique
  • All unique keys are candidate key to become primary key
  • Primary key can be one of the candidate key or it can be surrogate key
  • The candidate key, who is not primary key is alternate key. 
  • If surrogate key is primary key, then all candidate keys are alternate keys. 
  • Simple key and composite key are mutually exclusive. 
  • The compound key should have at least two simple keys
  • The composite key is extension of compound key
  • Natural key and domain key are synonyms.
  • Composite key, compound key, aggregate key and concatenated key. All are in a way same. 
  • Minimal super key and candidate key are in a way same. 
  • Trivial super key means all columns.   
Stay tune to this blog "Express YourSelf !" for more interesting articles.