OpenStack Services and OpenStack Distributions


Compute

* NOVA Compute Service
- Main part of IaaS
ZUN Containers Service
QINLING Functions Service

Bare Metal

IRONIC Bare Metal Provisioning Service
- Preboot eXecution Environment PXE
- Intelligent Platform Management Interface IPMI
- can extend with vendor specific plugin
CYBORG Accelerators resource management

Storage

SWIFT Object store
-scalable redundant storage system 
* CINDER Block Storage
MANILA Shared filesystems

Networking

NEUTRON Networking
- old name Quantum
OCTAVIA Load balancer
DESIGNATE DNS service
- old name Moniker

Shared Services

* KEYSTONE Identity service
- common authentication system
- Can integrate with LDAP
* GLANCE Image service
- It can use SWIFT
- Heat and Nova interface with Glance
BARBICAN Key management
KARBOR Application Data Protection as a Service
SEARCHLIGHT Indexing and Search
- Integrated with Horizon and CLI

Orchestration

* HEAT Orchestration
- OpenStack-native REST API 
- CloudFormation-compatible Query API
SENLIN Clustering service
MISTRAL Workflow service
ZAQAR Messaging Service
- Messages between various components of SaaS and mobile Apps. 
- old name Marconi
BLAZAR Resource reservation service
AODH Alarming Service
- rule based
- defined rules against metric
defined rules against event data 
- metric and event data collected by Ceilometer or Gnocchi

Workload Provisioning

MAGNUM Container Orchestration Engine Provisioning
- K8s
- Apache Mesos
- Docker Swarm
SAHARA Big Data Processing Framework Provisioning
- Elastic MapReduce
- To provision Hadoop cluster
- old name Savanna
TROVE Database as a Service
- Rational DB
- Non-rational DB
- old name RedDwarf

Application Lifecycle

MASAKARI Instances High Availability Service
MURANO Application Catalog
SOLUM Software Development Lifecycle Automation
FREEZER Backup, Restore, and Disaster Recovery

API Proxies

EC2API EC2 API proxy

Web Frontend

* HORIZON Dashboard
- native OpenStack API 
- EC2 compatibility API.

Telemetry
Ceilometer 
- Single Point of Contact for billing system
Gnocchi

RCA
Vitrage 
- for organizing, analyzing and expanding OpenStack alarms & events, yielding insights regarding the root cause of problems and deducing their existence before they are directly detected

Services marked with * are main services

Distributions

  • Bright Computing
  • Canonical (Ubuntu)
  • HPE (which was spin-merged to Micro Focus/Suse)
  • IBM
  • Mirantis
  • Oracle OpenStack for Oracle Linux, or O3L
  • Oracle OpenStack for Oracle Solaris
  • Red Hat
  • Sardina Systems
  • Stratoscale
  • SUSE
  • VMware Integrated OpenStack (VIO)

Service Function Chaining


Network monitoring/measurement

  • sFlow RFC 3176
  • Cisco's NetFlow 
  • IPFIX Protocol RFC 7011

Cloud native technologies include 

  • containers, 
  • service meshes, 
  • microservices, 
  • immutable infrastructure and 
  • declarative APIs 

that allow deployment in public, private and hybrid cloud environments through loosely coupled and automated systems

Various planes

  • infrastructure plane, 
  • virtual infrastructure plane, 
  • service plane,
  • user plane,
SFC Path identification

* NSH Network Service Header
* VLAN SFC
* Ethernet MAC chaining
* SFC using MPLS - SPRING

NSH is new tunneling protocol. RFC 8300

Then service function forwarders (SFFs) will create the service function paths (SFPs) in the form of an overlay by forwarding packets based on their NSH header.

The NSH header is composed of 

  • service path identification, 
  • transport independent per-packet service metadata and 
  • optional variable type-length-value (TLV) metadata.

physical probe or virtual probe functionality deployed as 

  • switches,
  • classifiers, 
  • SFs, or 
  • SFFs.

The term probe to designate any network node capable of reading and writing to a NSH header


Middleboxes are also interchangeably called 

  • services, 
  • inline services, 
  • appliances, 
  • network functions (NFs), 
  • virtual NFs
  • (vNFs), or 
  • service functions (SFs)

Example SFs includes 

  • firewalls, 
  • content filters, 
  • virus scanners (VS), 
  • intrusion detection systems (IDS), 
  • deep packet inspection (DPI), 
  • network address translation (NAT), 
  • content caches, 
  • load-balancers, 
  • wide area network (WAN) accelerators,
  • multimedia transcoders, 
  • multiservice proxies, 
  • application acceleration,
  • Lawful Intercept (LI),
  • HTTP header enrichment functions
  • TCP Optimizer
  • logging/metering/charging/advanced charging applications,  
  • or any other function that requires processing of packets 
SFC

ETSI NFV uses the term "network function forwarding graph" (NF-FG) 
IETF uses the term "service function chaining" (SFC) 

Fundamentally SFC is the ability to cause network packet flows to route through a network via a path other than the one that would be chosen by routing table lookups on the packet’s destination IP address.

VNF Forwarding Graph (VNFFG)
The combination of 

  • VNFs, 
  • SFC, and 
  • the classification of traffic to flow through them 

is described as the VNF Forwarding Graph (VNFFG). 


It is described as YAML file as per TOSCA VNF Forwarding Graph Descriptor (VNFFGD). VNFFGD = Forwarding Path + VNFGG

NSD = VNFFGD + VNFD

Each node is really a logical port, which is defined in the path as a Connection Point (CP) belonging to a specific VNFD. 

Tacker = OpenStack service addressing uses cases of 

  • NFV Orchestration and 
  • VNF Infrastructure Manager VIM ( Nova, Neutron, Cinder)

using standards based architecture

NFVO Renders VNF Forwarding Graphs using SDN Controller or a SFC API

Tacker allows for managing VNFs

Example CLI calls:

To create VNFFG

openstack vnf descriptor create --vnfd-file tosca-vnffg-vnfd1.yaml VNFD1
openstack vnf create --vnfd-name VNFD1 VNF1

openstack vnf descriptor create --vnfd-file tosca-vnffg-vnfd2.yaml VNFD2

openstack vnf create --vnfd-name VNFD2 VNF2

To create VNFFG SFC (where testVNF1, and testVNF2 are VNF instances):

tacker vnffg-create –name mychain –chain testVNF2,testVNF1 –symmetrical True

To create VNFFG SFC by abstract VNF types (ex. “firewall”, “nat”): 

tacker vnffg-create –name mychain –chain firewall,nat –abstract-types

To create SFC Classifier for a VNFFG:

tacker vnffg-classifier-create –name myclass –chain mychain –match tcp_dest=80,ip_proto=6

vnffg, vnffg_classifier are schema. Can be represented as dictionary. 

For classifier, one can use tenant_id attribute to implement 


Reference

K8S Tools


K8S Native Tools
===========

Minikube

It has many addons

minikube addons list

--insecure-registry flag for private docker registry.
OR
registry-creads addon to use GCR ECR and private docker registry.  

Advanced topics: https://github.com/kubernetes/minikube/tree/master/docs

Kops

To manage production-grade k8s clusters using CLI on AWS etc. It creates configuration file, that can be used to create actual clusters. It is like kubectl for AWS. 

https://github.com/kubernetes/kops/tree/master/docs

kubeadm

Master Node needs : docker, kubeadm, kubelet, kubectl. Worker Node needs : Kubeadm
Master : kubeadm init It gives "joint token" to be used at worker node. with command kubeadm join

https://www.ianlewis.org/en/how-kubeadm-initializes-your-kubernetes-master

Dashboard

1. Manage k8s apps
2. troubleshoot issue with k8s apps
3. manage entire k8s cluster. 

It is add-on for Minikube and application for real K8s cluster. It needs kubectl proxy. 

https://github.com/kubernetes/dashboard

kubefed

1. sync resources across clusters
2. cross-cluster discovery (DNS and load balancer) 

With federated clustered we can have hybrid cloud and multi-vendor cloud. 

https://kubernetes.io/docs/tasks/federation/set-up-cluster-federation-kubefed/

Kompose

converts Docker compose to K8s objects like deployments and services

Docker -> Compose
K8s -> Replication Controller = deployments + replica sets
Rancher -> Cattle
Stack Engine -> applications and deployments


Github link: https://github.com/kubernetes/kompose
Architecture: http://kompose.io/architecture/

Helm

Installations and management of K8s apps. it is like package manager. 
chart = pre-configured k8s resources
Helm (client at local host) -> Tiller (server at K8s cluster) 

chart = 
1. chart.yml
2. Templates
3. values.yml

Kubernetes/Charts at github has list of important projects https://github.com/helm/charts

Helm charts: https://github.com/kubernetes/helm/blob/master/docs/charts.md
Stable charts: https://github.com/kubernetes/charts/tree/master/stable

Draft, Gitkube, Helm, Ksonnet, Metaparticle and Skaffold are some of the tools around that help developers build and deploy their apps on Kubernetes

kubectl

3 namespace always exists
1. default
2. kube-public
3. kube-system

Auto Complete

https://blog.hasura.io/kubectl-bash-completion-on-coreos-b147ae94ff10/
https://www.cyberciti.biz/faq/add-bash-auto-completion-in-ubuntu-linux/

knative

Knative helps developers build, deploy, and manage modern serverless workloads on Kubernetes. Kubeless is one more such solution for serverless. 


CNCF Tools
=======

gRPC will replace SOAP and REST. Payload is protobuf. 

Consul and etcd are for service discovery. CoreDNS is from CNCF that can replace kube-dns

Service-mesh handles communication among micro services and network intricacies. Linkerd  transparent network proxy. Envoy small server with small footprint. Both support gRPC and http2 

CNI is plugin-based networking solutions for containers. Calico and flannel are most popular networking provider. 

GlusterFS and Ceph are for storage. Rook file, object and block storage system. Rook runs as an operator and creates Rook cluster using PV.

rkt and containerd are for container runtime

Prometheus is CNCF project for monitoring and many vendor specific such similar projects. Add metrics to application and to add exporter to use at Prometheus. PromoQL is its query language. Its alert manager has many good features and can integrate with PagerDuty. Prometheus for backend. Front end can be Grafana. 

Logging : Beats / Elastic Stack, Grayling, Fluentd. Fluentd

Tracing : Jaeger, OpenTracing, Zipkin. Application instrumentation is exposed using OpenTracing API to Jaeger agent. Jaeger has Client, agent, collector and UT

Security : (1) Image security and (2) Key management. Notary and TUF for secure image by CNCF. Valut and Confident stores sensitive data of image in secure manner and encrypt in REST. TUF is framework for software update system. Notary is implementation of TUF specification. Acquasec Product Suite for complete security platform. 

Kubeless and Fission providing equivalents to functions-as-a-service but running within Kubernetes

Reference 

Kelsey Hightower: https://twitter.com/kelseyhightower
Kubernetes Docs: https://kubernetes.io/docs/home/
Kubernetes Slack: http://slack.k8s.io/
The CNCF: https://www.cncf.io/
CNCF Meetups: https://www.meetup.com/pro/cncf/
Kubeconf: http://events.linuxfoundation.org/events/kubecon
The agile admin: https://theagileadmin.com/

https://ramitsurana.github.io/awesome-kubernetes/

Identity and Access Management


Directory

1. Active Directory : Windows solution
2. LDAP Directory

Safeguard personal information Legal

1. Safe Harbor (US)
2. TRUSTe
3. GDPR (Europe) 

Programms

1. penetration tests
2. network scans
3. bug bounty 

Vulnerabilities

1. Open Web Application Security Project (OWASP) for Web Application Security
2. SANS Institute

Other initiatives

1. Health Insurance Portability and Accountability Act HIPAA to protect patient data
2. Gramm-Leach-Bliley Act GLBA for consumer financial information. Federal Financial institutions Examination Council FFIEC provides guidelines for it
3. National Institute of Standards and Technology (NIST) Framework for Improving Critical Infrastructure Cybersecurity 
4. Family Educational Rights and Privacy Act (FERPA) to protect the privacy of student education records.
5. G-Cloud by UK government for cloud services. 
6. Federal Information Security Management Act (FISMA) defines a comprehensive framework to protect government information

Open Standards

1. Security Assertion Markup Language (SAML) for web browser Single Sign-On (SSO) using secure tokens. XML based protocol. No password needed. 
2. OpenID : Decentralized authentication protocol by 3rd party
3. OAuth. OpenID is built on OAuth. REST API using JSON
4.  System for Cross-Domain Identity Management SCIM to exchange user identity information. REST API using JSON or XML

KubeCon Seattle 2018 - Announcements


KubeCon Seattle 2018 - Announcements
(via CNCF)

Kubecon seattle 2018 recap


https://blog.openshift.com/openshift-commons-gathering-at-seattle-kubecon-2018-recap-with-video-and-slides/

https://www.cncf.io/blog/2018/12/14/closing-out-2018-with-a-top-notch-cloud-native-community-event/

https://www.forbes.com/sites/jasonbloomberg/2018/12/15/top-nine-vendor-highlights-from-kubecon/

https://aws.amazon.com/blogs/opensource/kubecon-seattle-2018-recap/

https://blogs.oracle.com/cloudnative/kubecon-2018-cloud-native-recaps-and-highlights

https://blog.openshift.com/podcast-podctl-reviewing-kubecon-seattle-2018/

https://www.storagereview.com/kubecon_2018_bits

https://www.ibm.com/blogs/bluemix/2018/12/highlights-ibm-cloud-kubecon-2018/

https://blog.openshift.com/podcast-podctl-reviewing-kubecon-seattle-2018/

https://medium.com/awesome-tech-confs/all-things-kubecon-and-cloudnativecon-seattle-2018-db84eb121217

https://chrisshort.net/my-kubecon-cloudnativecon-na-2018-recap/

https://thenewstack.io/this-week-on-the-new-stack-kubecon-highlights/

https://vexxhost.com/blog/recap-kubecon-2018-seattle/

https://diamanti.com/main-blog/kubecon-2018-recap/

UDS


DevOps for Practitioners


  • ·         It’s never done that before.
  • ·         It worked yesterday.
  • ·         How is that possible?
  • ·         It must be a hardware problem.
  • ·         What did you type in wrong to get it to crash?
  • ·         There is funky in your data.
  • ·         I haven’t touch that module in weeks!
  • ·         You must have the wrong version. Are you sure, you picked up correct binary?
  • ·         It’s just some unlucky coincidence.
  • ·         I can’t test everything!
  • ·         THIS can’t be the source of THAT.
  • ·         It works, but it hasn’t been tested.
  • ·         Somebody must have changed my code.
  • ·         Did you check for a virus on your system?
  • ·     Even though it doesn’t work, how does it matter? Is customer using this feature?
  • ·         You can’t use that version on your system.
  • ·         Why do you want to do that way?
  • ·         Where were you when the program blew up? Where are the log files?
  • ·         It works on my machine.

Sounds familiar? These are day to day replies by development team to operation team/testing team. Recently, Aricent had organization wide Learn DevOps initiative. “DevOps for Practitioners” training curriculum was assigned to employees with E2 to E6 grade. It is a self-paced e-learning courses at online learning platform of Lynda now Linkedin Learning.



DevOps should be extension of Agile. It includes new project management techniques like Agile and Lean, as well as, old school of principals SDLC etc. All the principals of Agile, Lean, Kanban, Kaizen etc are Building Blocks of DevOps
DevOps is about organisation wide cultural change and new set of matrices to measure progress. Read more about Levels of DevOps Practice. DevOps also means a set of values (Culture Automation Measurement and Sharing), principals, methods, practices and tools for all phases of software lifecycle.

Here are some of the tools, worth to explore. Now almost all development teams are using
Git as code repository. Ansible is software provisioning, configuration management, and application deployment tool with many useful Ansible Modules. Docker performs operating system level virtualization on top of Container Runtime. Kubernetes is container orchestration system for automating application deployment, scaling, and management. For hands-on with Kubernetes, one can refer online browser based solution and kubectl command cheat sheet.

Infrastructure automation, Continuous delivery and Reliability Engineering are three pillars of DevOps. DevSecOps is another emerging area with focus on security. One can find more DevOps Resources on Internet

Some more interesting posts : 

KubeCon Seattle 2018 - Announcements
Kubecon Seattle 2018 recap
K8S meetup
DevOps & Digital Transformation
Bangalore Kubernetes May 2019

Sixer

Is Ekta Kapoor active contributor for Kubernetes related tools?
No. Why?
There are many tools like kops, kubeadm, kubefed, kompose, kubectl, knative, kubeless etc, whose name starts with K…….

:-)