CKA 9 : Networking
Command
ip link //to list out interfaces
ip addr add IPV4/24 dev eth0
route
ip route add IPV4_NW/24 via GW
ip route add default via GW
is same as
ip route add 0.0.0.0 via GW
/proc/sys/net/ipv4/ip_forward
Files
/etc/sysctl.conf file. Modify Boolean value for /proc/sys/net/ipv4/ip_forward
For persistence across reboot, modify
/etc/network/interfaces
DNS
/etc/hosts
/etc/resolv.conf
nameserver IPV4
nameserver 8.8.8.8
serach mycompany.com
Whatever mentioned in search is appended to all entries.
Preference to /etc/hosts and then to DNS server
as per /etc/nsswtich.conf
check for line starts with "hosts:" as key
see the value part
files for /etc/hosts
dns for nameserver
CNAME records map, one name to another name
DNS tools: nslookup, dig, hostlookup
nslookup and dig do not consider entries at /etc/hosts
coreDNS
Corefile is configuration file for coreDNS. it is at /etc/coredns/Corefile inside pod. coreDNS has many plugins https://coredns.io/plugins/
Here only we define name of K8s cluster. Default is cluster.local
This file is created as configmap.
kubelet adds file /etc/resolv.conf to each pod. The content of file is
nameserver "IP_ADDRESS"
the value for IP_ADDRESS is as per clusterIP of coreDNS service
coreDNS service generally runs with replica count = 2 for redundancy.
kubelet has config file: /var/lib/kubelet/config.yaml
it has clusterDNS, clusterIP for coreDNS service is specified. it has clusterDomain where K8s cluster name "cluster.local" is specified.
PTR Record
if clusterIP is a.b.c.d for SERVICE
then PTR record is
d.c.b.a.in-addr.arpa. "ttl" IN PTR SERVICE.NS.svc.ZONE.
similar PTR record for IPv6. Note: PTR record with IPv6 shall have same sequence.
If we have headless service HL having a pod with hostname POD then 2 A records for each pods (2 AAAA records, if IPv6)
HL.NS.svc.cluster.local. 4 IN A a.b.c.d
and
POD.HL.NS.avc.cluster.local. 4 IN A a.b.c.d
If HL listens on port 80 then SRV record for each pod
_http._tcp.HL.NS.svc.cluster.local 4 IN SRV "weight" "priority" 80 POD.HL.NS.avc.cluster.local.
PTR record for HL service
d.c.b.a.in-addr.arpa. 4 IN PTR POD.HL.NS.svc.cluster.local
similar PTR record for IPv6. Note: PTR record with IPv6 shall have same sequence.
CNAME record
SERVICE.NS.cluster.local. 10 IN CNAME www.example.com
Then we can have A and/or AAAA record for www.example.com
References
https://github.com/kubernetes/dns/blob/master/docs/specification.md
Before K8s 1.12 DNS server was kube-dns. After K8s 1.12 onwards it is CoreDNS. However the K8s microservice name is still kube-dns
Namespace
Commands
1. Create network namespace
ip netns add "NW_NS_NAME"
ip netns // to list NW NS
ip netns exec "NW_NS_NAME" ip link
is same as
ip -n "NW_NS_NAME" link
To connect two NS
ip link add VETH-1 type veth peer name VETH-2
ip link set VETH-1 netns NW_NS1
ip link set VETH-2 netns NW_NS2
ip -n NW_NS1 addr add IP1 dev VETH-1
ip -n NW_NS2 addr add IP2 dev VETH-2
Then set the link up
ip -n NW_NS1 link set dev VETH-1 up
ip -n NW_NS2 link set dev VETH-2 up
Cleanup
If we delete any one of the link and other will be deleted automatically.
ip -n NW_NS1 link del VETH-1
To connect many NS and build a virtual network within host, we need virtual switch. (1)Linux Bridge (2) Open vSwitch OVS
We will see Linux Bridge
2. Create bridge network/interface
ip link add v-net-0 type bridge
this is added in host. it is listed in response to ip link command
ip link set dev v-net-0 up
3. Create VETH pairs (pipe, virtual cable)
Now create virtual cable to connect to bridge
ip link add VETH-1 type veth peer name VETH-1-br
ip link add VETH-2 type veth peer name VETH-2-br
ip link add VETH-3 type veth peer name VETH-3-br
ip link add VETH-4 type veth peer name VETH-4-br
4. Attach veth to NS
ip -n NW_NS1 link set dev VETH-1 up
ip -n NW_NS2 link set dev VETH-2 up
ip -n NW_NS3 link set dev VETH-3 up
ip -n NW_NS4 link set dev VETH-4 up
5. Attach other veth to bridge
ip link set VETH-1-br master v-net-0
ip link set VETH-2-br master v-net-0
ip link set VETH-3-br master v-net-0
ip link set VETH-4-br master v-net-0
6. Assign IP address
ip -n NW_NS1 addr add IP1 dev VETH-1
ip -n NW_NS2 addr add IP2 dev VETH-2
ip -n NW_NS3 addr add IP3 dev VETH-3
ip -n NW_NS4 addr add IP4 dev VETH-4
ip addr add IP5/24 dev v-net-0
7. Bring the interface up
8. Then each namespace add host IP as default GW
ip netns exec NS1 ip route add IP0/24 via IP5
ip netns exec NS2 ip route add IP0/24 via IP5
ip netns exec NS3 ip route add IP0/24 via IP5
ip netns exec NS4 ip route add IP0/24 via IP5
9. Then add firewall rule for NAT
iptables -t nat -A POSTROUTING -s IP0/24 -j MASQUERADE
Also add default GW
ip netns exec NS1 ip route add default via IP5
ip netns exec NS2 ip route add default via IP5
ip netns exec NS3 ip route add default via IP5
ip netns exec NS4 ip route add default via IP5
* We can open a port at any of the namespace using firewall
iptables -t nat -A PREROUTING --dport 80 --to-destination IP2:80 -j DNAT
Always set NETMASK, while setting IP address using ip addr add command by adding /24
Docker Networking
- None: --network none Docker container is not attached to any network outside. All docker containers cannot talk with each other too.
- Host: --network host
- Bridge
docker network ls
ip link command list docker0
Docker internally run:
ip link add docker0 type bridge
But this docker0 is down
ip netns
docker inspect NS // NS from previous command. it is for one running container.
For each container there is a device VETH and
ip link set VETH master docker0
If we run
ip -n NS link
We can see VETH. It will have IP address also.
ip -n NS addr
docker run -p "HOST_PORT":"CONTAINER_PORT"
It will add iptables rule
iptables -t nat -A PREROUTING -j DNAT --dport HOST_PORT --to-destination CONTAINER_PORT
iptables -t nat -A DOCKER -j DNAT --dport HOST_PORT --to-destination CONTAINER_IP:CONTAINER_PORT
This can be verified with
iptables -nvL -t nat
CNI
Docker follows same steps as above for bridge network, except naming convention is different. Same steps by : Docker, rocket (rkt), Mesos, K8s. So there is a script name "bridge". Here bridge script is CNI plugin
bridge add "container id" "network namespace"
As per CNI step 1
* CRI must create n/w ns
* identify network, that container must attach to
* CRI invoke CNI plugin when container is added
* CRI invoke CNI plugin when container is deleted
* JSON format for n/w configuration
at CNI plugin steps 2 onwards
* must support command line arguments ADD/DEL/CHECK
* must support parameter container id, network namespace
* must manage IP address assignment
* must return result in specific format
Default CNI plugins : bridge, VLAN, IPVLAN, MACVLAN, Windows
Default IPAM CNI plugin: DHCP, host-local
Docker is not CNI. It has its own standard called CNM
K8s create docker container with --network none
Then invoke CNI plugins
Cluster Networking
ip link //to list out interfaces
ip addr add IPV4/24 dev eth0
route
ip route add IPV4_NW/24 via GW
ip route add default via GW
is same as
ip route add 0.0.0.0 via GW
/proc/sys/net/ipv4/ip_forward
Files
/etc/sysctl.conf file. Modify Boolean value for /proc/sys/net/ipv4/ip_forward
For persistence across reboot, modify
/etc/network/interfaces
DNS
/etc/hosts
/etc/resolv.conf
nameserver IPV4
nameserver 8.8.8.8
serach mycompany.com
Whatever mentioned in search is appended to all entries.
Preference to /etc/hosts and then to DNS server
as per /etc/nsswtich.conf
check for line starts with "hosts:" as key
see the value part
files for /etc/hosts
dns for nameserver
CNAME records map, one name to another name
DNS tools: nslookup, dig, hostlookup
nslookup and dig do not consider entries at /etc/hosts
coreDNS
Corefile is configuration file for coreDNS. it is at /etc/coredns/Corefile inside pod. coreDNS has many plugins https://coredns.io/plugins/
Here only we define name of K8s cluster. Default is cluster.local
This file is created as configmap.
kubelet adds file /etc/resolv.conf to each pod. The content of file is
nameserver "IP_ADDRESS"
the value for IP_ADDRESS is as per clusterIP of coreDNS service
coreDNS service generally runs with replica count = 2 for redundancy.
kubelet has config file: /var/lib/kubelet/config.yaml
it has clusterDNS, clusterIP for coreDNS service is specified. it has clusterDomain where K8s cluster name "cluster.local" is specified.
PTR Record
if clusterIP is a.b.c.d for SERVICE
then PTR record is
d.c.b.a.in-addr.arpa. "ttl" IN PTR SERVICE.NS.svc.ZONE.
similar PTR record for IPv6. Note: PTR record with IPv6 shall have same sequence.
If we have headless service HL having a pod with hostname POD then 2 A records for each pods (2 AAAA records, if IPv6)
HL.NS.svc.cluster.local. 4 IN A a.b.c.d
and
POD.HL.NS.avc.cluster.local. 4 IN A a.b.c.d
If HL listens on port 80 then SRV record for each pod
_http._tcp.HL.NS.svc.cluster.local 4 IN SRV "weight" "priority" 80 POD.HL.NS.avc.cluster.local.
PTR record for HL service
d.c.b.a.in-addr.arpa. 4 IN PTR POD.HL.NS.svc.cluster.local
similar PTR record for IPv6. Note: PTR record with IPv6 shall have same sequence.
CNAME record
SERVICE.NS.cluster.local. 10 IN CNAME www.example.com
Then we can have A and/or AAAA record for www.example.com
References
https://github.com/kubernetes/dns/blob/master/docs/specification.md
Before K8s 1.12 DNS server was kube-dns. After K8s 1.12 onwards it is CoreDNS. However the K8s microservice name is still kube-dns
Namespace
Commands
1. Create network namespace
ip netns add "NW_NS_NAME"
ip netns // to list NW NS
ip netns exec "NW_NS_NAME" ip link
is same as
ip -n "NW_NS_NAME" link
To connect two NS
ip link add VETH-1 type veth peer name VETH-2
ip link set VETH-1 netns NW_NS1
ip link set VETH-2 netns NW_NS2
ip -n NW_NS1 addr add IP1 dev VETH-1
ip -n NW_NS2 addr add IP2 dev VETH-2
Then set the link up
ip -n NW_NS1 link set dev VETH-1 up
ip -n NW_NS2 link set dev VETH-2 up
Cleanup
If we delete any one of the link and other will be deleted automatically.
ip -n NW_NS1 link del VETH-1
To connect many NS and build a virtual network within host, we need virtual switch. (1)Linux Bridge (2) Open vSwitch OVS
We will see Linux Bridge
2. Create bridge network/interface
ip link add v-net-0 type bridge
this is added in host. it is listed in response to ip link command
ip link set dev v-net-0 up
3. Create VETH pairs (pipe, virtual cable)
Now create virtual cable to connect to bridge
ip link add VETH-1 type veth peer name VETH-1-br
ip link add VETH-2 type veth peer name VETH-2-br
ip link add VETH-3 type veth peer name VETH-3-br
ip link add VETH-4 type veth peer name VETH-4-br
4. Attach veth to NS
ip -n NW_NS1 link set dev VETH-1 up
ip -n NW_NS2 link set dev VETH-2 up
ip -n NW_NS3 link set dev VETH-3 up
ip -n NW_NS4 link set dev VETH-4 up
5. Attach other veth to bridge
ip link set VETH-1-br master v-net-0
ip link set VETH-2-br master v-net-0
ip link set VETH-3-br master v-net-0
ip link set VETH-4-br master v-net-0
6. Assign IP address
ip -n NW_NS1 addr add IP1 dev VETH-1
ip -n NW_NS2 addr add IP2 dev VETH-2
ip -n NW_NS3 addr add IP3 dev VETH-3
ip -n NW_NS4 addr add IP4 dev VETH-4
ip addr add IP5/24 dev v-net-0
7. Bring the interface up
8. Then each namespace add host IP as default GW
ip netns exec NS1 ip route add IP0/24 via IP5
ip netns exec NS2 ip route add IP0/24 via IP5
ip netns exec NS3 ip route add IP0/24 via IP5
ip netns exec NS4 ip route add IP0/24 via IP5
9. Then add firewall rule for NAT
iptables -t nat -A POSTROUTING -s IP0/24 -j MASQUERADE
Also add default GW
ip netns exec NS1 ip route add default via IP5
ip netns exec NS2 ip route add default via IP5
ip netns exec NS3 ip route add default via IP5
ip netns exec NS4 ip route add default via IP5
* We can open a port at any of the namespace using firewall
iptables -t nat -A PREROUTING --dport 80 --to-destination IP2:80 -j DNAT
Always set NETMASK, while setting IP address using ip addr add command by adding /24
Docker Networking
- None: --network none Docker container is not attached to any network outside. All docker containers cannot talk with each other too.
- Host: --network host
- Bridge
docker network ls
ip link command list docker0
Docker internally run:
ip link add docker0 type bridge
But this docker0 is down
ip netns
docker inspect NS // NS from previous command. it is for one running container.
For each container there is a device VETH and
ip link set VETH master docker0
If we run
ip -n NS link
We can see VETH. It will have IP address also.
ip -n NS addr
docker run -p "HOST_PORT":"CONTAINER_PORT"
It will add iptables rule
iptables -t nat -A DOCKER -j DNAT --dport HOST_PORT --to-destination CONTAINER_IP:CONTAINER_PORT
This can be verified with
iptables -nvL -t nat
CNI
Docker follows same steps as above for bridge network, except naming convention is different. Same steps by : Docker, rocket (rkt), Mesos, K8s. So there is a script name "bridge". Here bridge script is CNI plugin
bridge add "container id" "network namespace"
As per CNI step 1
* CRI must create n/w ns
* identify network, that container must attach to
* CRI invoke CNI plugin when container is added
* CRI invoke CNI plugin when container is deleted
* JSON format for n/w configuration
at CNI plugin steps 2 onwards
* must support command line arguments ADD/DEL/CHECK
* must support parameter container id, network namespace
* must manage IP address assignment
* must return result in specific format
Default CNI plugins : bridge, VLAN, IPVLAN, MACVLAN, Windows
Default IPAM CNI plugin: DHCP, host-local
Docker is not CNI. It has its own standard called CNM
K8s create docker container with --network none
Then invoke CNI plugins
Cluster Networking
- kube-apiserver Master 6443
- kube-scheduler Master 10251
- kube-controller-manager Master 10252
- etcd Master 2379
- etcd Multiple Master 2380
- kublet both 10250
- NodePort Worker 30000-32767
kubelet is invoked with
--cni-conf-dif=/etc/cni/net.d
--cni-bin-dir=/etc/cni/bin
--network-plugin=cni
the CNI binary is located at /etc/cni/bin and it is invoked with "add | del | check" , "container id" and "network namespace"
At /etc/cni/net.d path, we have JSON file for each CNI plugin. It contains
- cniVersion
- name
- type // the name of binary at /etc/cni/bin
- bridge
- isGateway // The network within node (bridge), shall have its IP address? So it can act as a gateway
- isMasq // If NAT rule should be added for IP MASQUERADE or not?
- ipam
IPAM
weave CNI plugin IPAM range is 10.32.0.0/12 = 10.32.0.1 to 10.47.255.254
total about 1e06 IP addresses. They are divided equally as per number of worker nodes.
For ClusterIP range
kube-api-server --service-cluster-ip-range ipNet
Default is 10.0.0.0 /24
We can check command line arguments for k8s master node by checking file at /etc/kubernetes/manifests path.
Ingress Controller
it consists of
- deployment or deamonset
- configmap
- service
- service account
- role
- clusterrole
- rolebinding
ingress.yaml has multiple rules for multiple host. For single host, and multiple path, we can have single rule.
backend:
serviceName:
servicePort:
rules:
- host: //optional
http:
paths:
- path:
backend:
serviceName:
servicePort:
Ingress can have
- tls at same level as rule
tls:
- hosts:
- HOST_FQDN
secretName: SECRET
Ingress can be in different namespace and ingress controller can be in different namespace. Generally ingress shall be in same namespace, where application resides. we can have multiple K8s ingress objects
We need to add
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
Reference:
Also refer
http://layers7.blogspot.com/2020/04/ckad-7exposing-applications.html
https://www.objectif-libre.com/en/blog/2018/07/05/k8s-network-solutions-comparison/
Ingress Controller
it consists of
- deployment or deamonset
- configmap
- service
- service account
- role
- clusterrole
- rolebinding
ingress.yaml has multiple rules for multiple host. For single host, and multiple path, we can have single rule.
backend:
serviceName:
servicePort:
rules:
- host: //optional
http:
paths:
- path:
backend:
serviceName:
servicePort:
Ingress can have
- tls at same level as rule
tls:
- hosts:
- HOST_FQDN
secretName: SECRET
Ingress can be in different namespace and ingress controller can be in different namespace. Generally ingress shall be in same namespace, where application resides. we can have multiple K8s ingress objects
We need to add
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
Reference:
Also refer
http://layers7.blogspot.com/2020/04/ckad-7exposing-applications.html
https://www.objectif-libre.com/en/blog/2018/07/05/k8s-network-solutions-comparison/
0 comments:
Post a Comment