Home

Awesome

Linen CNI plugin

A CNI plugin designed for overlay networks with Open vSwitch.

About Linen CNI plugin

Linen provides a convenient way to easily setup networking between pods across nodes. To support multi-host overlay networking and large scale isolatio, VxLAN tunnel end point (VTEP) is used instead of GRE. Linen creates an OVS bridge and added as a port to the linux bridge.

This CNI plugin implementation was inspired by the document from Kubernetes OVS networking and designed to meet the requirements of SDN environment.

Please read CNI for more detail on container networking.

Prerequisite

$ sudo apt-get install openvswitch-switch

Should I use this or ovn-kubernetes?

ovn-kubernetes provides more advanced features and use vRouter (Layer 3 approach) to achieve multi-host networking. If you're going to create vRouters and vSwitches to build any network topologies you desire, ovn-kubernetes is a complete solution.

This CNI plugin creates only vSwitches in each node and uses VxLAN for achieving network overlay. For the PODs in cluster are managed by linux bridges and the IP allocation is configured through IPAM plugin.

Kubernetes

Kubernetes 1.7+ and CNI 0.6.0 are required.

Linen CNI is not only a plugin which support for network namespace (e.g., docker, ip-netns), but also a option for Kubernetes cluster networking.

Usage

  1. Create a Linen CNI configuration list file in the /etc/cni/net.d/linen.conflist directories.
  2. Make sure that the linen, bridge and host-local binaries are in the /opt/cni/bin directories directories.
  3. (Optional) Create a daemon set to manager ovsdb kubectl create -f flaxd.yaml.
  4. Test to create a POD/Deployment.

Architecture

Management Workflow

<p align="center"> <img src="/images/mgmt-workflow.png" width="541" /> </p>

Packet Processing

To provide overlay network, Linen utilize Open vSwitch to create VxLAN tunneling in the backend.

<p align="center"> <img src="/images/ovs-networking.png" width="586" /> </p>

Example network configuration

Please check example network configuration in the examples folder

Network configuration reference

Linen is a chained plugin. It always comes after bridge plugin, so configure Linux Bridge is needed.

For the Linux Bridge plugin options

For the Open vSwitch Bridge plugin options

Build

You may need to build the binary from source. The "build-essential" package is required.

$ sudo apt-get install build-essential

Execute build.sh

$ ./build.sh

When build succeed, binary will be in the bin folder.

Example

Linen-CNI also provides a vagrantfile to help you setup a demo environment to try Linen-CNI.

Environment

You should install vagrant in your system and make sure everything goes well.

Setup Linen-CNI

cd linen-cni
sh build.sh
cp bin/linen ../cni/ 
sudo cp examples/master.linen.conflist  /root/linen.conflist

Create NS

In this vagrant environment, we don't install docker related services but you can use namespace(ns) to test Linen-CNI. Type following command to create a namespace named ns1

sudo ip netns add ns1

Start CNI

We have setup Linen-CNI environement and testing namespace(ns1), we can use the following commands to inform CNI to add a network for the namespace.

cd ~/cni
sudo CNI_PATH=`pwd` NETCONFPATH=/root ./cnitool \ add linen-network /var/run/netns/ns1

and the result looks like below

{
    "cniVersion": "0.3.1",
    "interfaces": [
        {
            "name": "veth7df4d2c0",
            "mac": "56:b1:e8:32:e4:b7"
        },
        {
            "name": "eth0",
            "mac": "0a:58:0a:f4:01:0a",
            "sandbox": "/var/run/netns/ns1"
        }
    ],
    "ips": [
        {
            "version": "4",
            "interface": 2,
            "address": "10.244.1.10/16",
            "gateway": "10.244.1.1"
        }
    ],
    "routes": [
        {
            "dst": "0.0.0.0/0"
        },
        {
            "dst": "0.0.0.0/0",
            "gw": "10.244.1.1"
        }
    ],
    "dns": {}
}

Now, we can use some tools to help us check the current network setting, for example.
You can use ovs-vsctl show to show current OVS setting and it looks like:

e6289dc2-a181-4316-b902-a50fc6d854b6
    Bridge "br0"
        Controller "tcp:192.168.2.100:6653"
        fail_mode: standalone
        Port "vxif10_245_2_2"
            Interface "vxif10_245_2_2"
                type: vxlan
                options: {key=flow, remote_ip="10.245.2.2"}
        Port "br0"
            Interface "br0"
                type: internal
        Port "vxif10_245_2_3"
            Interface "vxif10_245_2_3"
                type: vxlan
                options: {key=flow, remote_ip="10.245.2.3"}
    ovs_version: "2.5.2"

In this setting, the OVS will try to connect to Openflow controller (it not exist, change to L2 bridge mode) and it also contains three ports, including two vxlan ports.

Besides, you can use brctl show to see that the OVS bridge (br0) has been attached to Linux bridge(kbr).

bridge name     bridge id               STP enabled     interfaces
kbr0            8000.0a580af40101       no              br0
                                                        veth7df4d2c0

If you want to check the namepsace's networking setting, you can use sudo ip netns exec ns1 ifconfig to see it's IP config.

ubuntu@dev:~$ sudo ip netns exec ns1 ifconfig
eth0      Link encap:Ethernet  HWaddr 0a:58:0a:f4:01:0a
          inet addr:10.244.1.10  Bcast:0.0.0.0  Mask:255.255.0.0
          inet6 addr: fe80::bc15:faff:fe6b:b414/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1400  Metric:1
          RX packets:18 errors:0 dropped:0 overruns:0 frame:0
          TX packets:10 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1476 (1.4 KB)  TX bytes:828 (828.0 B)