Awesome

Install eUPF(eBPF/XDP UPF) on Host

This briefly describes the steps and configuration to build and install eUPF. There are installation instructions in the eUPF repository, but I would like to write down the steps for actually installing it. It is intended to be prepared for use with Open5GS and free5GC.

Sample Configurations and Miscellaneous for Mobile Network

<a id="toc"></a>

Table of Contents

• Simple Overview of eUPF and Data Network Gateway
• Build eUPF on VM-UP
  • Install required packages
  • Install Golang and Setting
  • Install the Swag command line tool for Golang
  • Clone eUPF
  • Run the code generators
  • Build eUPF
• Setup eUPF on VM-UP
  • Create configuration file
• Run eUPF on VM-UP
• Setup Data Network Gateway on VM-DN
• Sample Configurations
  • For 5G
  • For 4G
• Changelog (summary)

<a id="overview"></a>

Simple Overview of eUPF and Data Network Gateway

This describes a simple configuration of eUPF and Data Network Gateway, focusing on U-Plane. Note that this configuration is implemented with Proxmox VE VMs.

The following minimum configuration was set as a condition.

• One UPF and Data Network Gateway

The built simulation environment is as follows.

<img src="./images/network-overview.png" title="./images/network-overview.png" width=800px></img>

The eBPF/XDP UPF used is as follows.

• eBPF/XDP UPF - eUPF v0.6.4 (2024.05.01) - https://github.com/edgecomllc/eupf

Each VMs are as follows.

The network interfaces of each VM are as follows.

Linux Bridges of Proxmox VE are as follows.

<a id="build"></a>

Build eUPF on VM-UP

Please refer to the following for building eUPF.

• eUPF v0.6.4 (2024.05.01) - https://github.com/edgecomllc/eupf

<a id="install_packages"></a>

Install required packages

# apt install git clang llvm gcc-multilib libbpf-dev

<a id="install_golang"></a>

Install Golang and Setting

# wget https://go.dev/dl/go1.22.8.linux-amd64.tar.gz
# tar -C /usr/local -zxvf go1.22.8.linux-amd64.tar.gz
# mkdir -p ~/go/{bin,pkg,src}
# echo 'export GOPATH=$HOME/go' >> ~/.bashrc
# echo 'export GOROOT=/usr/local/go' >> ~/.bashrc
# echo 'export PATH=$PATH:$GOPATH/bin:$GOROOT/bin' >> ~/.bashrc
# echo 'export GO111MODULE=auto' >> ~/.bashrc
# source ~/.bashrc

<a id="install_swag"></a>

Install the Swag command line tool for Golang

# go install github.com/swaggo/swag/cmd/swag@v1.8.12

<a id="clone"></a>

Clone eUPF

# git clone https://github.com/edgecomllc/eupf.git
# cd eupf

<a id="generate_codes"></a>

Run the code generators

# go generate -v ./cmd/...

If you want to output kernel logs for debugging, add the following option.

# BPF_CFLAGS="-DENABLE_LOG" go generate -v ./cmd/...

In that case, to see debug log from eBPF programs:

# cat /sys/kernel/debug/tracing/trace_pipe

<a id="build_1"></a>

Build eUPF

# go build -v -o bin/eupf ./cmd/

<a id="setup_up"></a>

Setup eUPF on VM-UP

Please refer to the following for setup eUPF.

• eUPF v0.6.4 (2024.05.01) - https://github.com/edgecomllc/eupf/blob/main/docs/Configuration.md

First, uncomment the next line in the /etc/sysctl.conf file and reflect it in the OS.

net.ipv4.ip_forward=1

# sysctl -p

Next, down the default interfaceens18 of the VM-UP and set the VM-DN IP address to default GW on the N6 interfaceens22.

# ip link set dev ens18 down
# ip route add default via 192.168.16.152 dev ens22

<a id="conf"></a>

Create configuration file

Create /root/eupf directory and put the configuration file there.

• /root/eupf/config.yml

interface_name: [ens20, ens22]
xdp_attach_mode: native
api_address: :8080
pfcp_address: 192.168.14.151:8805
pfcp_node_id: 192.168.14.151
metrics_address: :9090
n3_address: 192.168.13.151
gtp_peer:
echo_interval: 10
qer_map_size: 1024
far_map_size: 1024
pdr_map_size: 1024
resize_ebpf_maps: false
heartbeat_retries: 3
heartbeat_interval: 5
heartbeat_timeout: 5
logging_level: info
feature_ueip: true
feature_ftup: true
ueip_pool: 10.45.0.0/16
teid_pool: 65536

If xdp_attach_mode is generic(Kernel-level implementation), the performance will not be improved. native(Driver-level implementation) or offload(NIC-level implementation) will be better. For reference, a list of drivers that support XDP can be found here.

<a id="run"></a>

Run eUPF on VM-UP

# cd /root/eupf
# bin/eupf --config config.yml
2024/10/16 01:02:45 Startup config: map[api_address::8080 echo_interval:10 far_map_size:1024 feature_ftup:true feature_ueip:true gtp_peer:[] heartbeat_interval:5 heartbeat_retries:3 heartbeat_timeout:5 interface_name:[ens20 ens22] logging_level:info metrics_address::9090 n3_address:192.168.13.151 pdr_map_size:1024 pfcp_address:192.168.14.151:8805 pfcp_node_id:192.168.14.151 qer_map_size:1024 resize_ebpf_maps:false teid_pool:65536 ueip_pool:10.45.0.0/16 xdp_attach_mode:native]
2024/10/16 01:02:45 Apply eUPF config: {InterfaceName:[ens20 ens22] XDPAttachMode:native ApiAddress::8080 PfcpAddress:192.168.14.151:8805 PfcpNodeId:192.168.14.151 MetricsAddress::9090 N3Address:192.168.13.151 GtpPeer:[] EchoInterval:10 QerMapSize:1024 FarMapSize:1024 PdrMapSize:1024 EbpfMapResize:false HeartbeatRetries:3 HeartbeatInterval:5 HeartbeatTimeout:5 LoggingLevel:info UEIPPool:10.45.0.0/16 FTEIDPool:65536 FeatureUEIP:true FeatureFTUP:true}
2024/10/16 01:02:45 INF Attached XDP program to iface "ens20" (index 4)
2024/10/16 01:02:45 INF Attached XDP program to iface "ens22" (index 6)
2024/10/16 01:02:45 INF Initialize resources: UEIP pool (CIDR: "10.45.0.0/16"), TEID pool (size: 65536)
2024/10/16 01:02:45 INF Starting PFCP connection: 192.168.14.151:8805 with Node ID: 192.168.14.151 and N3 address: 192.168.13.151
[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.

[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:   export GIN_MODE=release
 - using code:  gin.SetMode(gin.ReleaseMode)

[GIN-debug] GET    /api/v1/health            --> github.com/edgecomllc/eupf/cmd/api/rest.(*ApiHandler).InitRoutes.func1 (4 handlers)
[GIN-debug] GET    /api/v1/xdp_stats         --> github.com/edgecomllc/eupf/cmd/api/rest.(*ApiHandler).displayXdpStatistics-fm (4 handlers)
[GIN-debug] GET    /api/v1/packet_stats      --> github.com/edgecomllc/eupf/cmd/api/rest.(*ApiHandler).displayPacketStats-fm (4 handlers)
[GIN-debug] GET    /api/v1/config            --> github.com/edgecomllc/eupf/cmd/api/rest.(*ApiHandler).displayConfig-fm (4 handlers)
[GIN-debug] POST   /api/v1/config            --> github.com/edgecomllc/eupf/cmd/api/rest.(*ApiHandler).editConfig-fm (4 handlers)
[GIN-debug] GET    /api/v1/uplink_pdr_map/:id --> github.com/edgecomllc/eupf/cmd/api/rest.(*ApiHandler).getUplinkPdrValue-fm (4 handlers)
[GIN-debug] PUT    /api/v1/uplink_pdr_map/:id --> github.com/edgecomllc/eupf/cmd/api/rest.(*ApiHandler).setUplinkPdrValue-fm (4 handlers)
[GIN-debug] GET    /api/v1/qer_map           --> github.com/edgecomllc/eupf/cmd/api/rest.(*ApiHandler).listQerMapContent-fm (4 handlers)
[GIN-debug] GET    /api/v1/qer_map/:id       --> github.com/edgecomllc/eupf/cmd/api/rest.(*ApiHandler).getQerValue-fm (4 handlers)
[GIN-debug] PUT    /api/v1/qer_map/:id       --> github.com/edgecomllc/eupf/cmd/api/rest.(*ApiHandler).setQerValue-fm (4 handlers)
[GIN-debug] GET    /api/v1/far_map/:id       --> github.com/edgecomllc/eupf/cmd/api/rest.(*ApiHandler).getFarValue-fm (4 handlers)
[GIN-debug] PUT    /api/v1/far_map/:id       --> github.com/edgecomllc/eupf/cmd/api/rest.(*ApiHandler).setFarValue-fm (4 handlers)
[GIN-debug] GET    /api/v1/pfcp_associations --> github.com/edgecomllc/eupf/cmd/api/rest.(*ApiHandler).listPfcpAssociations-fm (4 handlers)
[GIN-debug] GET    /api/v1/pfcp_associations/full --> github.com/edgecomllc/eupf/cmd/api/rest.(*ApiHandler).listPfcpAssociationsFull-fm (4 handlers)
[GIN-debug] GET    /api/v1/pfcp_sessions     --> github.com/edgecomllc/eupf/cmd/api/rest.(*ApiHandler).listPfcpSessionsFiltered-fm (4 handlers)
[GIN-debug] GET    /swagger/*any             --> github.com/swaggo/gin-swagger.CustomWrapHandler.func1 (4 handlers)
[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.

[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:   export GIN_MODE=release
 - using code:  gin.SetMode(gin.ReleaseMode)

[GIN-debug] GET    /metrics                  --> github.com/edgecomllc/eupf/cmd/api/rest.(*ApiHandler).InitMetricsRoute.(*ApiHandler).InitMetricsRoute.func1.func2 (4 handlers)
2024/10/16 01:02:45 INF running on :8080
2024/10/16 01:02:45 INF running on :9090

The link status of the network interfaces N3(ens20) and N6(ens22) is as follows.

# ip link show
...
4: ens20: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 xdp qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
    link/ether bc:24:11:74:fe:7b brd ff:ff:ff:ff:ff:ff
    prog/xdp id 39 
    altname enp0s20
...
6: ens22: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 xdp qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
    link/ether bc:24:11:bf:d4:23 brd ff:ff:ff:ff:ff:ff
    prog/xdp id 39 
    altname enp0s22
...

<a id="setup_dn"></a>

Setup Data Network Gateway on VM-DN

First, uncomment the next line in the /etc/sysctl.conf file and reflect it in the OS.

net.ipv4.ip_forward=1

# sysctl -p

Next, configure NAPT and routing to N6 IP address of eUPF.

# iptables -t nat -A POSTROUTING -s <DN> -j MASQUERADE
# ip route add <DN> via 192.168.16.151 dev ens20

Note. Set <DN> according to the core network.
ex) 10.45.0.0/16

With the above steps, eUPF(eBPF/XDP UPF) has been constructed. You will be able to work eUPF with Open5GS and free5GC. I would like to thank the excellent developers and all the contributors of eUPF.

<a id="sample_conf"></a>

Sample Configurations

<a id="5g_conf"></a>

For 5G

• Open5GS 5GC & UERANSIM UE / RAN Sample Configuration - eUPF(eBPF/XDP UPF)
• free5GC 5GC & UERANSIM UE / RAN Sample Configuration - eUPF(eBPF/XDP UPF)

<a id="4g_conf"></a>

For 4G

• Open5GS EPC & srsRAN 4G with ZeroMQ UE / RAN Sample Configuration - eUPF(eBPF/XDP UPF(PGW-U))

<a id="changelog"></a>

Changelog (summary)

• [2024.10.14] Changed the VM environment from Virtualbox to Proxmox VE.
• [2024.05.11] Changed the eUPF OS from Ubuntu 22.04 to 24.04.
• [2024.05.04] Updated to v0.6.4.
• [2024.02.11] Updated to v0.6.1.
• [2023.12.05] The version confirmed to work in the changelog on 2023.12.04 has been tagged as v0.6.0.
• [2023.12.04] Updated as FTUP feature has been merged into main branch.
• [2023.11.24] Updated to 120-upf-ftup-fteid branch that supports FTUP.
• [2023.10.29] Initial release.