Routing Modes

• Cilium routing modes answer the question of “how do external (to the cluster) network devices route traffic to pods running on nodes, if the external devices have no concept of internal cluster networking (including the IP address scope assigned to the node.)”
• Cilium has two routing modes: Encapsulation or Native

Encapsulation Routing Mode (default)

• In the configuration, this is “tunnel” routing mode

  routingMode: tunnel
  

• When a pod wants to send a packet to a pod on another node, it constructs a packet with a source address of its own interface and a destionation address of the other pod. It will then encapsulate the packet in another packet with a source address of the node and a destination address of the node where the destination pod is scheduled. The packet is then routed from the source node to the other. Once the destination node receives the packet, it decapsulates it, retrieving the packet the source pod created, discovers the destination pod IP address, resolves ARP, and sends the packet to the destination pod.
• Encapsulation mode is simple to configure, you simply need to ensure that the nodes can route traffic to eachother and that any network devices in between the nodes allow VXLAN traffic.
• There are two different encapsluation mode protocols:
  • VXLAN: 8472/udp (default)
  • Geneve: 6081/udp
• Disadvantages of Encapsulation Routing Mode:
  • encapsulation mode requires roughly 50 bytes of the header (so ‘MTU - 50’) for the additional data
  • additional latency
  • debugging is more complex
  • potential for packet fragmentation
• No additional configuration required

Native Routing Mode

• Cilium can run routing protocols (BGP)
• The cilium agents run BGP and advertise their routes on the network
• No packet encapsulation! Woo hoo! But, BGP is complicated and cumbersome.
• You could always configure static routing as well, nothing wrong with that :)
• to configure:

  routingMode: "native"
  ipv4NativeRoutingCIDR: "10.244.0.0/16"
  ipv6NativeRoutingCIDR: ""