Express YourSelf !

Levels of DevOps Practice

1. Values:
======

CAMS Culture Automation Measurement and Sharing

1.1 culture
1.2 automation,
1.3 measurement,

* MTTR
* cycle Time
* Costs
* Revenue
* Employee Satisfaction

1.4 sharing: Kizen

Later on, L for Lean is also added.

CALMS Culture Automation Lean Measurement and Sharing

2. Principles
========

2.1 System thinking : Concept to Cash
2.2 Amplifying feedback loop
2.3 Continuous experiments and learning

3. Methods
=======

3.1 People (Who is responsible) , Process, Tools
3.2 Continuous Delivery
3.3 Lean Management
3.4 Change Control
3.5 Infrastructure as code

4. Practices
========

4.1 Chaos Monkey
4.2 Blue / Green Deployment
4.3 Dependency Injection (Inversion of control) using service discovery
4.4 Andon Cord
4.5 Cloud : API driven way
4.6 Embedded Teams
4.7 Blameless Postmortem (industrial safety) should happen within 24 or 48 hours
4.8 Public Status Page is also about Transparent uptime
4.9 Developers on call
4.10 Incident Command System (Like fire department)

5. Tools
=====

Criteria for choosing tools

1. Tools should be programmable. Can invoke by API and CLI, not only GUI.
2. verifiable : events and matrix
3. well behave: deploy automatically, acceptance test case. it can be in form for .deb OR chef cookbook

5.1 Code – code development and review, source code management tools, code merging
5.2 Build – continuous integration tools, build status
5.3 Test – continuous testing tools that provide feedback on business risks
5.4 Package – artifact repository, application pre-deployment staging
5.5 Release – change management, release approvals, release automation
5.6 Configure – infrastructure configuration and management, infrastructure as code tools (e.g. Ansible, Puppet, Chef)
5.7 Monitor – applications performance monitoring, end-user experience (e.g. NewRelic, Kibana, Datadog)

Six Phases of tool chain

5.1 Version Control : Git (Github OR Bigbucket) as SAS or enterprise version. Hands-on http://try.github.io/

5.2 CI systems : Jenkins, commercial Jenkins from CloudBees, GoCD, Bamboo, Shippable, TeamCity. CI as Service : Travis CI, Circle CI

5.3 Build : Make, Rake, Maven (Workflow approach), Gulp for front end, HashiCorp's Packer for infrastructure

5.4 Test : Testing for developers before check in

5.4.1 Unit Test and code hygiene : JUnit, golint/gofmt, RuboCop,
Integration Testing : Robot, Protractor, Cucumber, Selenium (for UI testing), Sauce Labs from SAS for front end acceptance testing

5.4.2 Infrastructure Testing : Kitchen Chef

5.4.3 Performance Testing : Apache Bench, Meter, Apache JMeter

5.4.4 Security Testing : Brakeman, VeraCode, gauntlet
Vagrant, Otto, Docker Compose

5.5 Artifact Repository

Artifactory
Nexus
DockerHub
AWS S3

5.6 Deployment

RunDeck
UrbenCode
ThoughtWorks
Deployinator
Elastic Beanstalk
Octopus
Vamp
Saltstack
rerun

Most important tools are : Continuous Integration (e.g. Jenkins) and Infrastructure as a code (e.g. Puppet)

Containerization — Docker, Rocket, Unik
Orchestration — Kubernetes, Swarm, Mesos

Organisational culture

1. Pathological (Power oriented)
2. Bureaucratic (rule oriented)
3. Generative (performance oriented) cooperation and inquiry

Processes are like training wheels: So go for minimal viable process

How to build Trust, Respect and eliminate blame and hostility ?

* Some people may introvert
* People think others job is easy to do
* bring transparency "chat ops" : Hip Chat, slack, ICR, Hubot, Lita, Cog, Err

Organisational boundary is also communication boundary.

Progress Charts

1. Epic burn down charts
2. Release burn down charts
3. Sprint burn down charts
4. Velocity charts (committed v/s completed)

Kanban based matrix and charts

1. WIP : Cumulative Flow Diagram
2. throughput
3. lead time = work is requested till delivered
4. cycle time = amount of time we spend working on it.

"The Phoenix Project" Modeled on "The Goal" but DevOps specific

Reference
https://theagileadmin.com/

Building Blocks of DevOps

DevOps includes both

1. New project management principles : agile and lean
2. old-school principles : ITIL (Information Technology Infrastructure Library) , ITSM (IT Service Management), and SDLC

Let's see three building blocks of DevOps.

1. DevOps should be extensions of Agile

Iterative and incremental process

Agile Manifesto

1.1 Individual and interactions > processes and tools
1.2 working s/w > comprehensive documentation
1.3 customer collaboration > contract negotiona
1.4 Responding to change > following plan

12 principles

1.1 highest priority is satisfy customer with early + continuous valuable s/w delivery
1.2 welcome requirement change
1.3 frequent delivery of working s/w
1.4 business people and developer works together daily
1.5 motivated individual
1.6 face to face conversation
1.7 working s/w is primary measure of progress
1.8 constant pace and sustainable development
1.9 technical excellence and good design
1.10 simplicity
1.11 self organising team
1.12 retrospection

Methodologies

Scrum
Kanabn
DSDM : Dynamic System Development Method
FDD : Feature Driven Development
ASD
eXtreme Programming
Lean
Crystal
Rational Unified Process

Widely used are scrum, Kanban and lean.

1.1 scrum

from rugby game
meaning: tight player formation that is trying to move the ball towards goal
Scrum Life Cycle : Product backlog -> Sprint Backlog -> Sprint -> Valuable product
Sprint = timebox of a month or less

1.2 kanban

5 principles

1.2.1 Visualize workflow
1.2.2 Limit WIP
1.2.3 Manage the flow
1.2.4 Make process policies explicit
1.2.5 Improve collaboratively

SAFe (Scaled Agile Framework) to reduce complexity for large scale implementation.
2. Lean :

Lean inspired by SPC, Kizen, DoE, JIT, ToC, quality tools

Value Stream Mapping

From Supplier to Customer : What Information and Materials are flowing

Seven principles

2.1 Eliminating waste

waste = The effort spend on anything other than creating value

* Muda : Absorbs resources but no value. 7 wastes
Type 1: Technical waste for compliance
Type 2: Plain waste
* Mura : unevenness , No regular flow of work. Overburden
* Muri : overburden process, person or system. unevenness

2.1.1 Partially done work
2.1.2 Extra Features
2.1.3 Re-Learning
2.1.4 Handoffs
2.1.5 Delays
2.1.6 Task Switching
2.1.7 Defects

2.2 Amplify Learning
2.3 Decide as late as possible
2.4 Deliver as fast as possible
2.5 Empower the team
2.6 Build an integrity
2.7 See the whole

Plan Do Check Act of Kaizen
is
Build Measure Learn Repeat in Lean

Four Principles

2.1. Pull scheduling
2.2. Kanban
2.3. Visual Management
2.4. Employee empowerment

For software industry

2.1 Limiting WIP
2.2 Kanban Visibility to flow
2.3 Customer f/b
2.4 Empowered team

Upcoming : Lean UX

3. Six principles of Kaizen

3.1. Good process brings good results
3.2. Gemba
3.3. Speak with data
3.4. Take action to correct root cause

* Five Whys
* Focus on causes not symptoms
* Fish Bone : Equipment, Process, People, Materials, Environment, Management
* Do not accept (1) No time (2) human error . People does not fail. Process fails.

3.5. Team work
3.6. Kaizen is for everyone

4. ITIL (Information Technology Infrastructure Library), ITSM (IT Service Management) and SDLC

PyCharm : My Notes

PyCharm is an IDE for Python development. Here are few key take away points:

$ pip freeze > requirements.txt

This will create a requirements.txt file, which contains a simple list of all the packages in the current environment, and their respective versions. Later, when a different developer (or you, if you need to re- create the environment) can install the same packages, with the same versions by running

$ pip install -r requirements.txt

================================================================================

Code Completion

1. Basic : Ctrl + Space

2. Smart : Ctrl + Shift + Space

3. Within context : Atl + / Atl + Shift + /

4. Post Fix completion : Space OR Enter OR Tab

Ctrl + Shift + Enter to make all auto corrections

================================================================================

Pipenv is a tool that provides all necessary means to create a virtual environment for your Python project. It automatically manages project packages through the Pipfile file as you install or uninstall packages.

================================================================================

Two settings

1. Project level settings are applied to current project and stored at :

C:\Users\\PycharmProjects\schoop_app\.idea

2. Global settings are like template, it applies to new projects. Files-> Settings for new projects

================================================================================

Keyboard short cut

Double Shift : search

Ctrl+N

Ctrl+Shift+N

Ctrl+Shift+Alt+N : Find and jump to the desired class, file, or symbol.

Ctrl+E : View Recent Files-

================================================================================

AI-ML Algorithm

Choosing the right estimator (scikit-learn) : http://scikit-learn.org/stable/tutorial/machine_learning_map/
How to choose algorithms for Azure Machine Learning Studio : https://docs.microsoft.com/en-us/azure/machine-learning/studio/algorithm-choice
A Tour of Machine Learning Algorithms : https://machinelearningmastery.com/a-tour-of-machine-learning-algorithms/
Guidelines to Select Machine Learning Scheme for Classification of Biomedical Datasets http://nexginrc.org/Publications/pub_files/evobio09-ajay.pdf
A Tour of The Top 10 Algorithms for Machine Learning Newbies : https://towardsdatascience.com/a-tour-of-the-top-10-algorithms-for-machine-learning-newbies-dde4edffae11
Machine Learning Algorithms: Which One to Choose for Your Problem : https://blog.statsbot.co/machine-learning-algorithms-183cc73197c

Yantra

Yantra (यन्त्र) (Sanskrit) (literally "machine, contraption" is a mystical diagram, mainly from the Tantric traditions of the Indian religions.

They are used for the worship of deities in temples or at home; as an aid in meditation; used for the benefits given by their supposed occult powers based on Hindu astrology and tantric texts. They are also used for adornment of temple floors, due mainly to their aesthetic and symmetric qualities. Specific yantras are traditionally associated with specific deities.

Yantras are usually associated with a particular deity and are used for specific benefits, such as: for meditation; protection from harmful influences; development of particular powers; attraction of wealth or success, etc.They are often used in daily ritual worship at home or in temples, and sometimes worn as a talisman.

As an aid to meditation, yantras represent the deity that is the object of meditation. These yantras emanate from the central point, the bindu. The yantra typically has several geometric shapes radiating concentrically from the center, including triangles, circles, hexagons, octagons, and symbolic lotus petals. The outside often includes a square representing the four cardinal directions, with doors to each of them

Yantras can be on a flat surface or three dimensional. Yantras can be drawn or painted on paper, engraved on metal, or any flat surface. They tend to be smaller in size than the similar mandala, and traditionally use less color than mandalas.

Occult yantras are used as good luck charms, to ward off evil, as preventative medicine, in exorcism, etc., by using their magical power. When used as a talisman, the yantra is seen to represent a deity who can be called on at will by the user. They are traditionally consecrated and energized by a priest, including the use of mantras which are closely associated to the specific deity and yantra. Practitioners believe that a yantra that is not energized with mantra is lifeless