Services & Networking (20%)

Notes on cluster networking, Services, Ingress, and NetworkPolicies.

Topics

• Understand host networking configuration on the cluster nodes
• Understand connectivity between Pods
• Understand ClusterIP, NodePort, LoadBalancer service types and endpoints
• Know how to use Ingress controllers and Ingress resources
• Know how to configure and use CoreDNS
• Choose an appropriate container network interface plugin

The Kubernetes network model

Rules every conformant cluster follows:

1. Every Pod gets its own IP address.
2. Any Pod can talk to any other Pod directly without NAT, regardless of node.
3. Agents on a node (kubelet, system daemons) can talk to all Pods on that node.

Three separate address ranges must not overlap: the node network, the Pod CIDR, and the Service CIDR. Service IPs are virtual; no interface owns them. kube-proxy (or the CNI in kube-proxy-free setups) translates them to Pod IPs.

Host networking on the nodes

What to check on a node:

ip addr                    # interfaces and IPs (node IP, CNI bridge, veth pairs)
ip route                   # routes; Pod CIDR routes are installed by the CNI
ip link
ss -tulpn                  # listening ports (kubelet 10250, etc.)
iptables -t nat -L -n | grep <svc-name>     # kube-proxy rules in iptables mode
ipvsadm -Ln                                 # IPVS mode
cat /etc/cni/net.d/*                        # CNI config
ls /opt/cni/bin                             # CNI binaries
sysctl net.ipv4.ip_forward                  # must be 1
cat /etc/resolv.conf                        # node DNS

Key points:

• net.ipv4.ip_forward=1 and br_netfilter are required so bridged Pod traffic hits iptables.
• Pods with hostNetwork: true share the node’s network namespace and IP and may bind node ports.
• The node’s kubelet reads the CNI config from /etc/cni/net.d and binaries from /opt/cni/bin (path set by the container runtime).
• Find a cluster’s CIDRs: kubectl cluster-info dump | grep -m1 cluster-cidr, or look at --cluster-cidr and --service-cluster-ip-range in the controller-manager and apiserver manifests.

Pod to Pod connectivity

• Same node: Pods connect through a virtual ethernet pair (veth) into a bridge or directly routed by the CNI.
• Different nodes: the CNI either routes the Pod CIDR (BGP, host routes) or encapsulates traffic (VXLAN, IP-in-IP, Geneve), or uses eBPF.
• A Pod’s network namespace is shared by all containers in it; they reach each other on localhost.

Quick tests:

kubectl run tmp --rm -it --image=busybox:1.36 --restart=Never -- sh
# inside: wget -qO- <pod-ip>:<port>   or   nslookup <svc>
kubectl exec <pod> -- curl -s <other-pod-ip>:8080
kubectl get pods -o wide                   # shows Pod IPs and nodes

Services

A Service gives a stable virtual IP and DNS name to a changing set of Pods selected by labels. The EndpointSlice controller maintains the list of ready Pod IPs behind it.

apiVersion: v1
kind: Service
metadata:
  name: web
spec:
  type: NodePort
  selector:
    app: web
  ports:
  - port: 80          # Service port
    targetPort: 8080  # container port (number or named port)
    nodePort: 30080   # optional, NodePort only

kubectl expose deployment web --port=80 --target-port=8080 --type=ClusterIP
kubectl expose pod web --port=80 --name=web-svc
kubectl get svc,endpoints,endpointslices
kubectl describe svc web

Troubleshooting a Service with no traffic:

1. kubectl get endpoints web: empty means the selector matches no Ready Pods (label mismatch or failing readiness probe).
2. targetPort must match the port the app actually listens on.
3. Check NetworkPolicies and kube-proxy health (kubectl -n kube-system get pods -l k8s-app=kube-proxy).

Other details:

• externalTrafficPolicy: Local preserves the client source IP and avoids an extra hop, but only routes to Pods on the receiving node.
• sessionAffinity: ClientIP gives sticky sessions.
• Environment variables <SVC>_SERVICE_HOST are injected for Services that existed before the Pod started; DNS is the preferred method.

Ingress

Ingress is an L7 (HTTP/HTTPS) routing resource: host and path rules mapped to Services. It does nothing alone; an Ingress controller (ingress-nginx, Traefik, HAProxy, Contour, and others) must be installed and watches the Ingress objects. The controller itself is usually exposed via a LoadBalancer or NodePort Service.

Use ingressClassName to pick the controller.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx
  tls:
  - hosts: [app.example.com]
    secretName: app-tls
  rules:
  - host: app.example.com
    http:
      paths:
      - path: /api
        pathType: Prefix
        backend:
          service:
            name: api
            port: { number: 80 }
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web
            port: { number: 80 }

kubectl create ingress web --class=nginx --rule="app.example.com/=web:80" --rule="app.example.com/api*=api:80"
kubectl get ingress
kubectl describe ingress web
kubectl get ingressclass

pathType: Exact, Prefix, or ImplementationSpecific.

The Gateway API (GatewayClass, Gateway, HTTPRoute) is the successor to Ingress, with role-oriented resources and richer routing. Expect it to appear in the current curriculum.

NetworkPolicy

By default all Pods accept all traffic. A NetworkPolicy selects Pods and, once any policy selects a Pod for a direction, only the traffic explicitly allowed for that direction is permitted. Policies are additive. They require a CNI that enforces them (Calico, Cilium, Weave; not plain Flannel).

# Default deny all ingress in the namespace
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-ingress
  namespace: prod
spec:
  podSelector: {}
  policyTypes: [Ingress]
---
# Allow only frontend Pods and the monitoring namespace to reach db on 5432
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: db-allow
  namespace: prod
spec:
  podSelector:
    matchLabels: { app: db }
  policyTypes: [Ingress, Egress]
  ingress:
  - from:
    - podSelector:
        matchLabels: { app: frontend }
    - namespaceSelector:
        matchLabels: { kubernetes.io/metadata.name: monitoring }
    ports:
    - protocol: TCP
      port: 5432
  egress:
  - to:
    - namespaceSelector: {}
      podSelector:
        matchLabels: { k8s-app: kube-dns }
    ports:
    - { protocol: UDP, port: 53 }
    - { protocol: TCP, port: 53 }

Gotchas:

• Within a single from list, separate items are OR’d; podSelector and namespaceSelector in the same item are AND’d. This indentation difference is a classic trap.
• When restricting egress, remember to allow DNS (port 53) or name resolution breaks.
• Policies are namespaced and select by labels, so label the namespaces you reference.

CoreDNS

CoreDNS is the cluster DNS, deployed as a Deployment (coredns) in kube-system behind the Service kube-dns (the IP the kubelet writes into each Pod’s resolv.conf via clusterDNS).

Record formats:

• Service: <svc>.<ns>.svc.cluster.local resolves to the ClusterIP (headless: Pod IPs).
• Pod: <ip-with-dashes>.<ns>.pod.cluster.local.
• StatefulSet Pod via headless Service: <pod>.<svc>.<ns>.svc.cluster.local.
• Pods’ /etc/resolv.conf has search <ns>.svc.cluster.local svc.cluster.local cluster.local and ndots:5, so short names such as web or web.other-ns work.

Configuration lives in the coredns ConfigMap (a Corefile):

.:53 {
    errors
    health { lameduck 5s }
    ready
    kubernetes cluster.local in-addr.arpa ip6.arpa {
       pods insecure
       fallthrough in-addr.arpa ip6.arpa
       ttl 30
    }
    prometheus :9153
    forward . /etc/resolv.conf     # upstream for non-cluster names
    cache 30
    loop
    reload
    loadbalance
}

Customize by editing the ConfigMap, for example a stub domain:

example.internal:53 {
    forward . 10.10.0.53
}

reload picks up changes automatically; otherwise kubectl -n kube-system rollout restart deployment coredns.

Pod DNS policy: ClusterFirst (default), Default (inherit node resolv.conf), None with dnsConfig, ClusterFirstWithHostNet for hostNetwork Pods.

kubectl -n kube-system get pods -l k8s-app=kube-dns
kubectl -n kube-system logs -l k8s-app=kube-dns
kubectl -n kube-system get cm coredns -o yaml
kubectl run dns --rm -it --image=busybox:1.36 --restart=Never -- nslookup web.default.svc.cluster.local
kubectl exec <pod> -- cat /etc/resolv.conf

Choosing a CNI plugin

The CNI plugin allocates Pod IPs and wires Pod networking. Kubeadm does not install one, so the cluster has no working Pod network until you do.

Selection criteria: NetworkPolicy requirement, performance and overlay overhead, cloud integration, observability, encryption (WireGuard/IPsec), operational familiarity, and scale. On the exam the CNI is normally given or specified by the task; remember to apply its manifest after kubeadm init, and that the Pod CIDR must match.

Cheat sheet

kubectl get svc -A
kubectl get endpoints <svc>
kubectl get ingress -A
kubectl get netpol -A
kubectl describe netpol <name>
kubectl port-forward svc/web 8080:80
kubectl port-forward pod/web-abc 8080:80
kubectl run curl --rm -it --image=curlimages/curl --restart=Never -- curl -s http://web.default
kubectl get pods -A -o wide | grep <ip>
kubectl -n kube-system get cm kube-proxy -o yaml | grep mode
kubectl cluster-info dump | grep -E "cluster-cidr|service-cluster-ip-range"