feat: oai core support

README.md

@@ -12,7 +12,7 @@
 
 eUPF is the opensource User Plane Function (UPF) project for using inside or "outside" of any 3GPP 5G core. The goal of the project is to provide high-observability and easily-deployed software for a various cases like multi-access edge computing (MEC) and local traffic breakout. eUPF is built with eBPF to provide high observability and performance.
 
- eUPF is tested with the Free5GC and Open5GS 5G cores.
+ The eUPF has been tested with three different 5G cores: Free5GC, Open5GS and OpenAirInterface. The OpenAirInterface gNB was also used during testing.
 
 ## What is 5G core and CUPS
 

docs/deployments/README.md

@@ -9,9 +9,9 @@ eUPF outbound connections is pure routed at the node. There is no embedded NAT,
 | ------- | --- | ------- | ---------------------- |
 | Open5GS | UERANSIM | - | [Open5GS](./to-be-done/README.md) |
 | Open5GS | OpenAirInterface | - | Comming soon... |
-| Free5GC | UERANSIM | - | [Free5GC](./to-be-done/README.md) |
+| Free5GC | UERANSIM | - | [Free5GC](https://github.com/edgecomllc/free5gc-compose/blob/master/README.md) |
 | Free5GC | UERANSIM | ULCL | [Free5GC & ULCL](./to-be-done/README.md) |
-
+| OpenAirInterface 5G Core | OpenAirInterface 5G RAN	 | - | [OAI 5G SA mode with L2 nFAPI simulator](./oai-nfapi-sim-compose/README.md) |
 
 ## K8s deployments
 

docs/deployments/oai-nfapi-sim-compose/README.md

@@ -0,0 +1,85 @@
+# OpenAir Core + OpenAir RAN as a docker-compose
+We will use 5G L2 nFAPI simulator to test L2 and above Layers. Let's pull Eurecom's deployment for 5G SA mode with 1 User: [OAI Full Stack 5G-NR L2 simulation with containers and a proxy](https://gitlab.eurecom.fr/oai/openairinterface5g/-/tree/develop/ci-scripts/yaml_files/5g_l2sim_tdd) and replace the UPF with our eUPF.
+
+📝This deploy uses `network_mode: "host"` for communications over `lo` interface of the host between containers oai-gnb, proxy, oai-nr-ue0.
+
+We will add two services `edgecom-upf` and `edgecom-nat` with a dedicated network between. We have configured routing between the both, so that eUPF passes packets from the UE towards the NAT container, and the replies come through NAT to eUPF and down to the UE.
+
+## How to deploy:
+1. Deploy  the whole project "OAI Full Stack 5G-NR L2 simulation with containers and a proxy" 
+    following instructions https://gitlab.eurecom.fr/oai/openairinterface5g/-/blob/develop/ci-scripts/yaml_files/5g_l2sim_tdd/README.md
+
+    <details><summary>TLDR: Look at our example where host interface name is `ens3` with IP addr `188.120.232.247`</summary>
+    <p>
+
+    ```ruby
+    sergo@edgecom:~/gitlab$ git clone https://gitlab.eurecom.fr/oai/openairinterface5g.git
+    sergo@edgecom:~/gitlab$ cd openairinterface5g/ci-scripts/yaml_files/5g_l2sim_tdd/
+
+    nano docker-compose.yaml
+                - DEFAULT_DNS_IPV4_ADDRESS=169.254.25.10  #172.21.3.100
+
+    nano ../../conf_files/gnb.sa.band78.106prb.l2sim.conf
+        NETWORK_INTERFACES :
+        {
+            GNB_INTERFACE_NAME_FOR_NG_AMF            = "ens3";
+            GNB_IPV4_ADDRESS_FOR_NG_AMF              = "188.120.232.247";
+            GNB_INTERFACE_NAME_FOR_NGU               = "ens3";
+            GNB_IPV4_ADDRESS_FOR_NGU                 = "188.120.232.247";
+            GNB_PORT_FOR_NGU                         = 2152; # Spec 2152
+        };
+
+    nano ../../conf_files/nrue.band78.106prb.l2sim.conf
+    MACRLCs = (
+            {
+            num_cc = 1;
+            tr_n_preference = "nfapi";
+            local_n_if_name  = "ens3";
+            remote_n_address = "127.0.0.1"; //Proxy IP
+            local_n_address  = "127.0.0.1";
+
+
+    sudo docker pull oaisoftwarealliance/proxy:develop
+    sudo docker image tag oaisoftwarealliance/proxy:develop oai-lte-multi-ue-proxy:latest
+
+    sudo ifconfig lo: 127.0.0.2 netmask 255.0.0.0 up
+    ```
+
+    </p>
+    </details> 
+    </p>
+
+    **Don't start it.**
+
+2. Copy content of folder `edgecomllc/eupf/docs/deployments/oai-nfapi-sim-compose` from https://github.com/edgecomllc/eupf repository to your `openairinterface5g/ci-scripts/yaml_files/5g_l2sim_tdd`
+
+    📝 There is `docker-compose.override.yml` file which should happen to be in the same folder with `docker-compose.yml`. Check it before continue.
+3. Start containers in the following order:
+    ```ruby
+    sudo docker-compose up -d mysql oai-nrf oai-amf oai-smf edgecom-nat edgecom-upf
+    sudo docker-compose up -d  oai-gnb
+    sudo docker-compose up -d proxy oai-nr-ue0
+    ```
+4. Check the interface `oaitun_ue1` is created and ip address is set: `ip a |grep oaitun`
+    ```ruby
+    ip a |grep oaitun
+    8808: oaitun_ue1: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 500
+        inet 12.1.1.5/24 brd 12.1.1.255 scope global oaitun_ue1
+    ```
+5. Check the Internet is accessible over created GTP tunnel
+    ```ruby
+    ping -I oaitun_ue1 -c 3 1.1.1.1
+    PING 1.1.1.1 (1.1.1.1) from 12.1.1.5 oaitun_ue1: 56(84) bytes of data.
+    64 bytes from 1.1.1.1: icmp_seq=1 ttl=54 time=192 ms
+    64 bytes from 1.1.1.1: icmp_seq=2 ttl=54 time=68.0 ms
+    64 bytes from 1.1.1.1: icmp_seq=3 ttl=54 time=150 ms
+
+    --- 1.1.1.1 ping statistics ---
+    3 packets transmitted, 3 received, 0% packet loss, time 2000ms
+    rtt min/avg/max/mdev = 67.998/136.369/191.533/51.290 ms
+    ```
+6. `sudo docker-compose down oai-nr-ue0 proxy` to free up resources before the system sluggish.
+
+### Undeploy all
+`sudo docker-compose down`
+

docs/deployments/oai-nfapi-sim-compose/docker-compose.override.yml

@@ -0,0 +1,118 @@
+version: '3.8'
+
+services:
+  edgecom-nat:
+    container_name: nat
+    build:
+      context: ./nat
+    sysctls:
+      - net.ipv4.conf.all.forwarding=1
+    cap_add:
+      - NET_ADMIN
+    networks:
+      privnet_n6:
+        aliases:
+          - natn6.free5gc.org
+
+  oai-spgwu:
+    command: echo 'stop'
+    networks: !reset
+        traffic_net:
+            ipv4_address: 192.168.72.134
+
+
+  oai-smf:
+    image: oaisoftwarealliance/oai-smf:v1.5.1
+    environment:
+      - DISCOVER_UPF=no
+      - USE_FQDN_DNS=no
+
+  oai-gnb:
+    depends_on: !reset
+      - edgecom-upf
+      - oai-amf
+
+  edgecom-upf:
+    container_name: eupf
+    image: ghcr.io/edgecomllc/eupf:7086b71cb1dbc254db84c30e14fa4a68d3beb132
+    privileged: true
+    command: --iface eth0 --iface eth1 --n3addr 192.168.71.134 --nodeid 192.168.71.134 --paddr 192.168.71.134:8805 --attach generic --loglvl trace --feature_ftup true --teid_pool 65535
+    volumes:
+      - /sys/fs/bpf:/sys/fs/bpf
+      - ./eupf/entrypoint.sh:/app/bin/entrypoint.sh
+    environment:
+      GIN_MODE: release
+    ulimits:
+      memlock: -1
+    cap_add:
+# Granular caps support depends on kernel version
+#      - BPF
+##      - TRACING
+#      - PERFMON
+      - NET_ADMIN
+      - SYS_ADMIN
+      - SYS_RESOURCE # setrlimit
+    ports:
+      - "8880:8080"
+      - "9090:9090"
+    sysctls:
+      - net.ipv4.conf.all.forwarding=1
+    networks:
+      public_net:
+        ipv4_address: 192.168.71.134
+        aliases:
+          - upf.free5gc.org
+      privnet_n6:
+        aliases:
+          - upfn6.free5gc.org
+
+  prometheus:
+    image: prom/prometheus
+    container_name: prometheus
+    command:
+      - '--config.file=/etc/prometheus/prometheus.yml'
+    ports:
+      - 9091:9090
+    restart: unless-stopped
+    volumes:
+      - ./prometheus:/etc/prometheus
+      - prom_data:/prometheus
+    networks:
+      public_net:
+
+  grafana:
+    image: grafana/grafana
+    container_name: grafana
+    ports:
+      - 3000:3000
+    restart: unless-stopped
+    environment:
+      - GF_SECURITY_ADMIN_USER=admin
+      - GF_SECURITY_ADMIN_PASSWORD=grafana
+    volumes:
+      - ./grafana/provisioning:/etc/grafana/provisioning
+      - ./grafana/dashboards:/etc/grafana/dashboards
+    networks:
+      public_net:
+
+  edgecom-upf-ui:
+    container_name: ui
+    image: ghcr.io/edgecomllc/eupf-ui:74adead324e00eb83f97adfec40c7188c5f57a4a
+    command: >
+      sh -c "
+      sed -i 's#__API_PORT__#'\"$$API_PORT\"'#g' /usr/share/nginx/html/env.js
+      && nginx -g 'daemon off;'
+      "
+    ports:
+      - "8081:80"
+    environment:
+      - API_PORT=8080
+
+networks:
+  privnet_n6:
+    ipam:
+      driver: default
+      config:
+        - subnet: 10.100.250.0/24
+    driver_opts:
+      com.docker.network.bridge.name: br-oainfapi-n6

docs/deployments/oai-nfapi-sim-compose/eupf/entrypoint.sh

@@ -0,0 +1,23 @@
+#!/bin/env sh
+
+# Mount bpffs and debugfs if not present already
+#if [[ $(/bin/mount | /bin/grep /sys/fs/bpf -c) -eq 0 ]]; then
+#    /bin/mount bpffs /sys/fs/bpf -t bpf;
+#fi
+#if [[ $(/bin/mount | /bin/grep /sys/kernel/debug -c) -eq 0 ]]; then
+#    /bin/mount debugfs /sys/kernel/debug -t debugfs;
+#fi
+
+apk add iproute2
+
+iptables -A FORWARD -j ACCEPT
+#ip route add 10.1.0.0/16 via `nslookup upf.free5gc.org | awk '/^Address: / { print $2 }'` dev eth0
+
+echo "1200 n6if" >> /etc/iproute2/rt_tables
+ip rule add from 10.60.0.0/16 table n6if
+ip rule add from 12.1.1.0/24 table n6if
+#ip route add default via `nslookup nat.free5gc.org | awk '/^Address: / { print $2 }'` dev eth0 table n6if
+ip route add default via `nslookup natn6.free5gc.org | awk '/^Address: / { print $2 }'` table n6if
+
+# Run app replacing current shell (to preserve signal handling) and forward cmd arguments
+exec /app/bin/eupf "$@"

docs/deployments/oai-nfapi-sim-compose/nat/Dockerfile

@@ -0,0 +1,10 @@
+FROM alpine:3.17
+
+WORKDIR /app
+
+RUN apk add --no-cache iptables
+
+COPY ./busy-poll.sh /app/bin/busy-poll.sh
+
+# CMD is overridden if arguments are passed.
+ENTRYPOINT [ "sh", "/app/bin/busy-poll.sh" ]

docs/deployments/oai-nfapi-sim-compose/nat/busy-poll.sh

@@ -0,0 +1,30 @@
+#!/bin/bash 
+
+terminate=0
+#_term() { 
+#  echo "Caught SIGTERM signal!" 
+#  terminate=1
+#}
+#trap _term SIGTERM
+
+
+iptables -t nat -A POSTROUTING -s 10.60.0.0/16 -o eth0 -j MASQUERADE
+iptables -t nat -A POSTROUTING -s 12.1.1.0/24 -o eth0 -j MASQUERADE
+iptables -A FORWARD -j ACCEPT
+
+#echo "1200 n6if" >> /etc/iproute2/rt_tables
+#ip rule add to 10.1.0.0/16 table n6if
+#ip route add default via upf.free5gc.org dev eth0 table n6if
+
+#ip route add 10.60.0.0/16 via `nslookup upf.free5gc.org | awk '/^Address: / { print $2 }'` dev eth0
+ip route add 10.60.0.0/16 via `nslookup upfn6.free5gc.org | awk '/^Address: / { print $2 }'`
+ip route add 12.1.1.0/24 via `nslookup upfn6.free5gc.org | awk '/^Address: / { print $2 }'`
+
+
+while [ $terminate -ne 1 ]
+do
+    sleep 10;
+done
+
+
+echo "Exit busy-polling script" 

docs/install.md

@@ -1,6 +1,6 @@
 # How to install and run eUPF
 The easyest way to install eUPF is to use helm charts for one of the supported opensource 5G core projects in your own kubernetes cluster.
-Alternatively, eUPF could be deployed with docker-compose (only with free5gc config is ready at the moment).
+Alternatively, eUPF could be deployed with docker-compose (with free5gc or OpenAirInterface configs is ready at the moment).
 
 We have prepared templates to deploy with two opensource environments: **open5gs** and **free5gc**, for you to choose.
 
@@ -107,17 +107,17 @@ networks:
 
 ### Set eUPF configuration parameters
 
-[See](Configuration.md) configuration guide
+See [configuration guide](Configuration.md)
 
 ### Run eUPF
 
 ```
 docker-compose up -d
 ```
 
-## Examples
+## Examples in docker-compose
 
-See docker-compose deployment examples with **open5gs** and **free5gc** [here](./deployments/README.md).
+See docker-compose deployment examples with **open5gs**, **free5gc** and **OpenAirInterface** in [the Deployment examples table](./deployments/README.md#docker-compose-deployments).
 
 # Deploy with Kubernetes
 
@@ -207,6 +207,9 @@ UE can send packet to internet and get response
 
 # Information for troubleshooting
 
+<details><summary>For builds with trace enabled, not production one: See details under the spoiler.</summary>
+<p>
+
 To see debug log from eBPF programs, at the **node** console start command:
 `sudo cat /sys/kernel/debug/tracing/trace_pipe`
 
@@ -229,6 +232,9 @@ sergo@edgecom:~$ sudo cat /sys/kernel/debug/tracing/trace_pipe
           <idle>-0       [007] dNs.1 266111.396998: bpf_trace_printk: upf: bpf_redirect: if=4 18446669071771765924 -> 18446669071771765930
 ```
 
+</p>
+</details> 
+
 ## Components logs then successfully connected:
 <details><summary>eUPF successfull connections log output (stdout)</summary>
 <p>