Applies to SUSE Edge 3.6

23 Learning External Routes with MetalLB BGP #

MetalLB is a load-balancer implementation for bare-metal Kubernetes clusters, using standard routing protocols.

In this guide, we demonstrate how to use MetalLB to learn routes from external routers via BGP.

23.1 Learning External Routes with MetalLB BGP #

Note

FRR-K8s is now the default backend for BGP. Community documentation of FRR-K8s is here.

23.2 Prerequisites #

All prerequisites here are the same as for Chapter 22, MetalLB on K3s (using Layer 3 Mode) apart from the need for a free IP address.
Note
The example here does not include setting up a service so there is no need for an IP address.
A K3s cluster with MetalLB already deployed.
Router(s) on the network that support the BGP protocol.

23.3 Configuration to Accept Incoming Routes #

Out of the box MetalLB BGP advertises a Service IP address to all the BGP peers that are configured. These peers, which are usually routers, will receive a route for each Service IP address with a 32 bit network mask. When using FRR-K8s with the FRRConfiguration CR, it is also possible to receive routes from external routers. These external routes will show up in each node’s routing table. There are multiple benefits from this, but the main one is that it eliminates the need to manually update Linux routing tables on nodes when the external network changes, specifically in cases where it is desirable to avoid sending traffic through the default gateway.

23.4 Deployment #

MetalLB is deployed by default for L2 VIP functionality, so there is no need to install it separately.

Verify that you have 4 pods in the metallb-system namespace and that they are all running without a problem:

k get pods -n metallb-system
NAME                                                      READY   STATUS    RESTARTS     AGE
metallb-controller-7fbfd8977d-m2q9t                       1/1     Running   0            46s
metallb-metallb-frr-k8s-9w7wl                             6/6     Running   0            46s
metallb-metallb-frr-k8s-webhook-server-5d9d67ffd6-8jqnc   1/1     Running   1 (6s ago)   46s
metallb-speaker-qx8bl                                     1/1     Running   0            46s

23.5 Configuration #

Create an FRRConfiguration for the FRR-K8s based BGP backend:

cat <<-EOF | kubectl apply -f -
apiVersion: frrk8s.metallb.io/v1beta1
kind: FRRConfiguration
metadata:
  name: frrdemo
  namespace: metallb-system
spec:
  bgp:
    routers:
    - asn: 64513
      neighbors:
        - address: 192.168.20.154
          asn: 64512
          port: 179
          toAdvertise:
            allowed:
              mode: all
          toReceive:
            allowed:
              mode: all
EOF

The external BGP router has ASN 64512 while our BGPPeer will have 64513. We can also see that the external BGP Router has an IP address that is 192.168.20.154.

Verify that the FRRConfiguration has been deployed:
```
k get FRRConfiguration -A
NAMESPACE        NAME      AGE
metallb-system   frrdemo   4s
```
Note
The toReceive settings above will make your cluster accept all incoming routes. This might not be advisable in a production environment as, depending on your external routers, it can cause your nodes' routing tables to fill up. See documentation on how to filter toReceive further.

With FRR-K8s these are all the settings needed. Any route that your external BGP router is receiving will be shared with your cluster and all the nodes will have their routing tables updated.

The setup can be tested with what is described in Chapter 22, MetalLB on K3s (using Layer 3 Mode). In order to combine these setups there are some requirements:

The FRR-K8s configuration is done on a separate cluster. This is required to avoid cluster internal routing.
Changes to ASN IDs and to the Router IP address are required to match both setups.

With both these setups in place the following can be observed:

Once the Deployment and Service have been applied on the cluster described in Chapter 22, MetalLB on K3s (using Layer 3 Mode), the route to the service will be visible on the external FRR Router.
The route will be added to all nodes on the FRR-K8s cluster.

Note

In standard configurations of FRR Routing routes are shared with the next-hop set to the IP Address of the FRR Router. To achieve a next-hop without the FRR Router IP Address, one can add a line "neighbor BGPPG next-hop-unchanged" and a line "neighbor BGPPG as-override" to the /etc/frr/frr.conf file on the FRR Router. With this in place, the nodes on the FRR-K8s cluster will get a direct route to the service.