Go / Golang


https://github.com/alco/gostart

https://www.digitalocean.com/community/tutorials/how-to-install-go-and-set-up-a-local-programming-environment-on-ubuntu-18-04

krew


Krew is plugin manager for kubectl command line tool

Krew helps you:

  • discover kubectl plugins,
  • install them on your machine,
  • and keep the installed plugins up-to-date.
Reference: 

Krew: 

Extend kubectl with plugins

plugins

Some useful plugins



modify-secret It allows user to modify the secret without having to worry about doing base64 encoding/decoding.

Image Policy Admission Controller & Admission Controllers


Image Policy Admission Controller

* ImagePolicyWebhook admission controller

* adds new field to Pod, e.g. pod.spec.container[i].imageID (or an annotation). and kubelet will enforce that both the imageID and image match the image pulled.


* It does not cover Post Pod creation audit (out of scope)

** images allowed due to backend not reachable

** images that kept running after policy change (e.g. CVE discovered)

** images started via local files or http option of kubelet

** checking SHA of images allowed by a tag which was remapped


* Alternatives

1 Admission Control on Controller Objects

2 Enforcement at Docker level:  twistlock/authz Docker plugin

3 Policy Stored in API (PSP) depends on

*** Dockerfile, 

*** the source code, 

*** the source repo, 

*** the source review tools, 

*** vulnerability databases

4 Registry whitelist/blacklist by prefix match on image strings

5 Send every Request to a Generic Admission Control Backend

How to implement?

Three relevant files

1 . Configuration of admission control. Let's say it A.yaml. It is specified using --admission-control-config-file flag at API-server pod's YAML

--admission-control-config-file=/etc/kubernetes/A.yaml

Content of /etc/kubernetes/A.yaml

kind: AdmissionConfiguration
plugins:
- name: ImagePolicyWebhook
  path: /etc/kuberenets/B.yaml

2. Configuration of Image Policy. Let's say it is B.yaml OR B.json. It is specified in A.yaml as above

Here is content of /etc/kuberenets/B.yaml

      imagePolicy:
        kubeConfigFile: /etc/kubernetes/admission/C
        allowTTL: 50
        denyTTL: 50
        retryBackoff: 500
        defaultAllow: false # DENY ALL PODS IF SERVICE NOT AVAILABLE

3. kubeconfig file. Ler's tell it as file C. It is specified in file B, as above

Here is partial C file

# clusters refers to the remote service.
clusters:
- name: name-of-remote-imagepolicy-service
  cluster:
    certificate-authority: /path/to/ca.pem    # CA for verifying the remote service.
    server: https://images.example.com/policy # URL of remote service to query. Must use 'https'.
# users refers to the API server's webhook configuration.
users:
- name: name-of-api-server
  user:
    client-certificate: /path/to/cert.pem # cert for the webhook admission controller to use
    client-key: /path/to/key.pem          # key matching the cert

Note: We can remove B.yaml 2 and put its content to A.yaml. Now A.yaml will point to kubeConfigFile C, as below

apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
- name: ImagePolicyWebhook
  configuration:
    imagePolicy:
      kubeConfigFile: /etc/kubernetes/admission/C
      allowTTL: 50
      denyTTL: 50
      retryBackoff: 500
      defaultAllow: true

Admission Controllers

List of admission controllers are enabled by default

  • NamespaceLifecycle
  • LimitRanger, 
  • ServiceAccount
  • TaintNodesByCondition, 
  • Priority, 
  • DefaultTolerationSeconds, 
  • DefaultStorageClass, 
  • StorageObjectInUseProtection, 
  • PersistentVolumeClaimResize, 
  • RuntimeClass, 
  • CertificateApproval, 
  • CertificateSigning, 
  • CertificateSubjectRestriction, 
  • DefaultIngressClass, 
  • MutatingAdmissionWebhook, 
  • ValidatingAdmissionWebhook, 
  • ResourceQuota

As per CSI following plugins shall be enabled

  • EventRateLimit,
  • AlwaysPullImages,
  • PodSecurityPolicy OR SecurityContextDeny
  • NodeRestriction

As per CSI following plugins shall be disabled

  • ServiceAccount
  • NamespaceLifecycle
  • AlwaysAdmit

Container Images and Practical


Let me share few useful container images to play around with K8s

1. katacoda/docker-http-server
2. k8s.gcr.io/echoserver:1.10
3. nginx
4. busybox
- command: 
  - tail
  - -f
  - /dev/null
5. mendhak/http-https-echo
6.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: echoserver
spec:
  replicas: 3
  selector:
    matchLabels:
      app: echoserver
  template:
    metadata:
      labels:
        app: echoserver
    spec:
      containers:
      - image: gcr.io/google_containers/echoserver:1.0
        imagePullPolicy: Always
        name: echoserver
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: echoserver
spec:
  ports:
  - port: 80
    targetPort: 8080
    protocol: TCP
  selector:
    app: echoserver
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: echoserver
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: echo.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: echoserver
          servicePort: 80
---
7. curl
apiVersion: apps/v1 
kind: Deployment
metadata:
  name: client
  namespace: cli
spec:
  selector:
    matchLabels:
      app: cli
  replicas: 1
  template:
    metadata:
      labels:
        app: cli
    spec:
      containers:
      - name: client
        image: curlimages/curl:latest
        env:
        - name: ADDRESS
          value: "service_name.svc.cluster.local"
        command: [ "/bin/sh""-c""while true; do date; curl $ADDRESS 2>/dev/null | grep '<title>'; sleep 0.5; done;" ]

8. kubectl run -it --rm test --image=curlimages/curl --restart=Never -- /bin/sh

9. Docker Image: us-docker.pkg.dev/google-samples/containers/gke/hello-app:2.0

K8s microservice: kubectl expose deploy hw11 --port=80 --target-port=8080 --dry-run=client -o yaml

Script : please run in pod as per 8 above. 

while true; 
do curl n2iwf;
sleep 1; 
done

10. Echo server: hashicorp/http-echo:0.2.3 Port 5678

Here are example Dockerfiles

Here are few posts about hands on with K8s


CKA and CKAD: My Notes and Website Links


Let me post all links, for CKA and CKAD exam


CKA


To know more about K8s from my blog

Apart form there are many links material on Internet. Here is the list of website, that I am aware about

https://github.com/StenlyTU/K8s-training-official

https://www.shrlrn.com/

https://www.manning.com/books/kubernetes-in-action#toc

https://amartus.com/amartus-kubernetes-exam-tips/


CKAD
https://itnext.io/the-kubernetes-workloads-3bcf99168c05
https://itnext.io/introduction-to-stateful-services-kubernetes-6018fd99338d
https://itnext.io/learn-how-to-configure-your-kubernetes-apps-using-the-configmap-object-d8f30f99abeb

https://github.com/kelseyhightower/kubernetes-the-hard-way around 10 hrs
https://azure.microsoft.com/en-in/resources/kubernetes-up-and-running/
https://kubernetes.io/
https://discuss.kubernetes.io/
https://kubernetes.io/training/
https://training.linuxfoundation.org/training/kubernetes-for-developers/#outline
https://www.edx.org/course/introduction-to-kubernetes
https://www.edx.org/course/introduction-to-linux
https://killer.sh/course/preview/e84d0e31-4fff-4c42-8afd-be1bdbc0d994
https://github.com/dgkanatsios/CKAD-exercises/blob/master/a.core_concepts.md
https://kubernetes.io/docs/reference/kubectl/cheatsheet/
https://learnk8s.io/troubleshooting-deployments
https://learnk8s.io/academy
https://kodekloud.com/
https://kubernauts.de/en/training/kubernetes-training-course.html
https://docs.google.com/presentation/d/13EQKZSQDounPC1I6EC4PmqaRmdCrpT3qswQJz9KRCyE/edit#slide=id.g33599df588_13_58  It has many excerises
https://collabnix.github.io/kubelabs/
https://www.katacoda.com/courses/kubernetes/playground
https://www.youtube.com/watch?v=c63qr8EFVWE
https://docs.google.com/spreadsheets/d/10NltoF_6y3mBwUzQ4bcQLQfCE1BWSgUDcJXy-Qp2JEU/edit#gid=0
https://killer.sh/
https://github.com/ShubhamTatvamasi/kubernetes-practice-tests
https://github.com/suchakra012/kubernetes-practice-tests

CKA Tips


 * Network Policy: 

- If we put podSelector: {} then it means all pods

- if we do not mention about podSelector at all, then it means none of the pod. 

* We can set env, sa, image, selector, and resources (Requests and Limit)  with kubectl set command

* We can set subject (user | sa | group) for rolebinding | clusterrolebinding

* now (k8s 1.18) kubectl run is only for pod. Not for deployment and job

* We cannot create pv, pvc , .... cannot be with kubectl create

* With 'kubectl expose' command we cannot specify nodePort value. With kubectl create service we can. 

* with 'kubectl create deploy' we can specify replica. Not containerPort

kubectl auth can-i VERB_ACTION RESOURCE_OBJECT --as USER --namespace NS

kubectl explain K8s_OBJECT is useful. kubectl explain K8s_OBJECT --recursive is more useful

kubectl api-resources is useful to know short form

* grep -A N , grep -B N, grep -C N: Here N is number of line. A means after, B is before 

* to run on specific node : nodeName

* mount secret : secretName ; mount configMap : name at Volumes section

* securityContext.capabilities is only for container, not for pod

* useful command: kubectl config set-context --current --namespace=default

* verify taint with kubectl describe node | grep -i Taint

* If we edit replicaset, it impacts when new pod is created. 

* If replicaset is part of deployment and if we delete replicaset, it will be recreate with same name. 

* If pod is part of replicaset and if we delete pod, it will be created with different name. 

* Replicaset has name. Pod has genearateName



Developing Secure Software


Terms

  • Vulnerability: Any exploitable weakness to breach confidentiality, integrity, availability 
  • Threat: harm
  • Defect: any error introducing software security vulnerability
  • threat vector: means through which cyber security criminal gain unauthorized access to protected resources. 
  • TPM Trusted Platform Module: secure encryption keys
  • Risk Manifest: Probability and Consequences
  • Requirement level threats: 
  • Architectural patterns: E.g. Single point of access 
  • Flaws: design error causing software security vulnerability E.g.: Improper input validation-> SQL injection attacks
  • Bugs: coding error leading to software security vulnerability. They can be avoided by: 
- static and dynamic code analysis
- peer code review
  • Threat Modelling Process TMP
1. identification
2. analysis
3. categorization for priorities
4. mitigation
STRIDE threat model
S: Spoofing identity
T: Tampering with data
R: Repudiation
I: Information disclosure
D: Denial of Service
E: Elevation of privilege

Hardware level threats
1. eavesdropping: hardware key logger
2. man-made disruption: power outage, sabotage
3. natural disaster 
Countermeasures
1. geo-graphically dispersed redundancy
2. UPS
3. Physical security

Secure design 
1. Security Tactics.
- Detect Attacks
- Resist Attacks
- React to Attacks
- Recover from Attacks
2. Security Patterns. Two Types 
- Design Patterns
- Architectural Patterns
Design patterns become Architectural patterns when applied consistently
3. Security Vulnerability
- CVE DB maintained by MITRE: Product specific details
- CWE: Category of CVE: Useful insights about what can go wrong. 
4. Architectural Analysis for Security AAFS Three phases
- ToAA Tactic-Oriented Architectural Analysis
- PoAA Pattern-Oriented Architectural Analysis
- VoAA Vulnerability-Oriented Architectural Analysis: Code inspection
5. Software Security Anti-Patterns
- unrestricted upload of files
- unrestricted path traversal
- hardcoded password

Secure Coding
1. Buffer overflow attacks due to Lack of user input validation
- automatic bounds checking
- built in language specific library model for safe buffer handling
- code scanner
- advanced compiler
- OS support
2. broken authentication and session management
- Software Framework 
- Standard Application Security Verification Standard ASVS V2(Authentication) and V3(Session Management) by OWSAP
3. Insecure Direct Object Reference
4. Exposing sensitive data
5. Access Control
- Identification
- Authentication
- Authorization
6. Input Validation
- Buffer overflow
- SQL injection
- Cross site scripting XSS
Intercepting validation security pattern

Testing for security
1. Static Analysis
2. Dynamic Analysis
- HCL AppScan
- Nikto2
- Qualys
3. Penetration Testing (Ethical Hacking)
- Kali Linux
4. Vulnerability Management Tools
- Nessus uses CVSS Common Vulnerability Scoring System

Recent Development and Future Directions
1. DevOps
2. DevSecOps https://www.redhat.com/en/topics/devops/what-is-devsecops
3. Cloud Security
- Hypervisor vulnerability
- Cloud Service Provider access physical machines
4. Developer friendly security tools and training
5. IoT and software security
6. Rules and Regulations
- GDPR General Data Protection Regulations. Compliance expectations 
-- Data controller
-- Data Producer
- HIPAA Health Insurance Portability and Accountability Act 
- PCI DSS Payment Card Industry Data Security Standard
7. Certification
- Global Information Assurance Certification GIAC
- The International Council of Electronic Commerce Consultants (EC-Council): Certified Application Security Engineer (CASE)
- International Information System Security Certification Consortium (ISC Square)
(1) Certified Information System Security Professional CISSP 
(2) Certified Secure Software Lifecycle Professional CSSLP 

The evolution of Ingress through the Gateway API


We have three types components. 

1. Transparent Proxies: sidecar, kube-proxy
2. Cloud LB: GCP, Azure, AWS
3. Middle Proxies: nginx, HAProxy, Envoy

Now their functionalities are overlapping 

In K8s, Ingress is simple L7 description 

Roles: 
1. Infrastructure Provider 
2. Cluster Operator / NetOps / SRE 
3. Applicatoin DEveloper 

In case of Ingress, Infrastructure Provider provides IngressClass. No role for SRE. Application Developer defines Ingress and Service

In case of Gateway API, Infrastructure Provider provides GatewayClass. It is associated with type of LB. SRE provides Gateway implementation. Application Developer defines Route and Service

We have K8s objects like

1. GatewayClass
2. Gateway provides: Exposure and access, LB
3. Route: HTTP, TCP, UDP, SNI and TLS: HTTPS, TLS
4. Service about grouping and selection 

API groups
1. Core: Must be supported
2. Extended: Feature specific. May be supported, must be portable
3. Implementation specific: Not K8s API schema. No guarantee for portability

Gateway contains route. Route can be CRD. 
Earlier we had single ingress resource. Now it splits in Gateway and Route. So there may be conflict. E.g. same host, same path in multiple routes. 

Extension Points in Alpha
1. GatewayClass parameters for LB configuration
2. Gateway.Listener have ExtensionRef to customize listener properties
3. Route: Custom filter via ExtenstionRef. Backend is more than Services, like ingress

ExtenstionRef is for CRD. 

Reference: