You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

In the previous article #003 "Are you P4 compliant ?" we exposed a setup where RARE/freeRouter was controlling BMv2 P4 dataplane called simple_switch_grpc. In this article we replace the open source BMv2 target by a commercial virtual target provided by INTEL/BAREFOOT. As a side note, we will show that this setup can be integrated with real networks. (with inherent software limitations) 

Requirement

  • Basic Linux/Unix knowledge
  • Basic networking knowledge

Overview

I'm repeating the core message from #003: For those who are not familiar with data plane programming and especially with P4, "P4 is a domain-specific programming language for specifying the behaviour of the dataplanes of network-forwarding elements." (from p4.org) in short it helps you to write a "program specifying how a switch processes packets".

Article objective

In this article we'll using freeRouter setup deployed in #003 and replace bmv2/simple_switch_grpc providing freeRouter P4Lang's dataplane by INTEL BAREFOOT/bf_switchd. Actually the effective dataplane is ensured by INTEL/BAREFOOT virtual bf_switchd model running RARE P4 program called: bf_router.p4.

Diagram

[ #003 ] - Cookbook

In our example we will use the OpenNetworkLinux KVM image (ONL9) this is the recommended build from INTEL/BAREFOOT for SDE-9.2.0.

and we add a bridge network interface to or laptop RJ45 connection.

mkdir -p ~/freeRouter/bin ~/freeRouter/lib ~/freeRouter/etc ~/freeRouter/log
cd ~/freeRouter/lib
wget http://freerouter.nop.hu/rtr.jar
Update & Upgrade system
╭─[11:11:54]floui@debian ~ 
╰─➤ tree freeRouter
freeRouter
├── bin   # binary files      
├── etc   # configuration files      
├── lib   # library files      
└── log   # log files
get freeRouter net-tools tarball
wget freerouter.nop.hu/rtr.tar
Install build tools
tar xvf rtr.tar -C ~/freeRouter/bin/

For those you would like to rebuild these binaries you can find the compilation shell script in freeRouter cloned git repository in: ~/freeRouter/src/native/c.sh

In that section, you'll need to get access to INTEL/BAREFOOT Software Development Environment. For Research & Academia institution, you can apply here in order to become a FASTER member and access to INTEL/BAREFOOT resources. You can find here, a document installing INTEL/BAREFOOT SDE on ONL for a WEDGE100BF32X system. In our case, we are setting up the following environment:

  • ONL9 as VM guest with kernel 8192 Mb of RAM abd 2 vCPU
  • SDE 9.2.0
  • VirtualBox is running on MACOSX host

Just for the sake of example, SDE 9.2.0 is installed in root home directory:

SDE installation environment
export SDE=/root/bf-sde-9.2.0
export SDE_INSTALL=/root/bf-sde-9.2.0/install
export PATH=$PATH:$SDE_INSTALL/bin:$SDE/tools

bitbucket is a private repository. It is currently being reworked in order to make it public as per INTEL/BAREFOOT decision to grant P4 code related to TOFINO architecture. It is thus inaccessible for now but will be soon opened.

Clone RARE code from repository 
cd ~/
git clone https://bitbucket.software.geant.org/scm/rare/rare.git
compile RARE bf_router.p4
root@bionic:~/rare/p4src# p4_build.sh -I /root/rare/p4src/ -DHAVE_MPLS /root/rare/p4src/bf_router.p4 
Using SDE          /root/bf-sde-9.2.0
Using SDE_INSTALL /root/bf-sde-9.2.0/install
Using SDE version bf-sde-9.2.0

OS Name: Ubuntu 18.04.4 LTS
This system has 8GB of RAM and 1 CPU(s)
Parallelization:  Recommended: -j1   Actual: -j1

Compiling for p4_16/tna
P4 compiler path:    /root/bf-sde-9.2.0/install/bin/p4c
P4 compiler version: 9.2.0 (SHA: 639d9ec) (p4c-based)
Build Dir: /root/bf-sde-9.2.0/build/p4-build/bf_router
 Logs Dir: /root/bf-sde-9.2.0/logs/p4-build/bf_router

  Building bf_router        CLEAR CONFIGURE MAKE INSTALL ... DONE

FreeRouter uses 2 configuration files in order to run, let's write these configuration files for R1 in ~/freeRouter/etc

freeRouter hardware configuration file: p4-freerouter-hw.txt
int eth0 eth 0000.1111.00fb 127.0.0.1 22710 127.0.0.1 22709
tcp2vrf 2323 v1 23
tcp2vrf 9080 v1 9080
freeRouter software configuration file: p4-freerouter-sw.txt
hostname p4-freerouter
buggy
!
!
vrf definition v1
 exit
!
interface ethernet0
 description freerouter@P4_CPU_PORT[veth251]
 no shutdown
 no log-link-change
 exit
!
interface sdn1
 description freerouter@sdn1[enp0s3]
 mtu 9000
 macaddr 0072.3e18.1b6f
 vrf forwarding v1
 ipv4 address 192.168.0.131 255.255.255.0
 ipv6 address 2a01:e0a:159:2850::666 ffff:ffff:ffff:ffff::
 ipv6 enable
 no shutdown
 no log-link-change
 exit
!
!
!
!
!
!
!
!
!
!
!
!
!
!
server telnet tel
 security protocol telnet
 no exec authorization
 no login authentication
 vrf v1
 exit
!
server p4lang p4
 export-vrf v1 1
 export-port sdn1 0 10
 interconnect ethernet0
 vrf v1
 exit
!
client tcp-checksum transmit
!
end
Setup bf_switchd dataplane communication channel via veth pair and interface adjustment (disable IPv6 at VM guest level, MTU 10240, disable TCP offload etc.)
echo 1 > /proc/sys/net/ipv6/conf/all/disable_ipv6
echo 1 > /proc/sys/net/ipv6/conf/default/disable_ipv6

ip link add veth251 type veth peer name veth250
ip link set veth250  up 
ip link set veth251  up 

ifconfig enp0s3 promisc
ifconfig veth250 promisc
ifconfig veth251 promisc

ip link set dev veth250 up mtu 10240
ip link set dev veth251 up mtu 10240
ip link set dev enp0s3 up mtu 10240
export TOE_OPTIONS="rx tx sg tso ufo gso gro lro rxvlan txvlan rxhash"

for TOE_OPTION in $TOE_OPTIONS; do
    /sbin/ethtool --offload veth250 "$TOE_OPTION" off &> /dev/null
    /sbin/ethtool --offload veth251 "$TOE_OPTION" off &> /dev/null
    /sbin/ethtool --offload enp0s3 "$TOE_OPTION" off &> /dev/null

sudo ip link add veth251 type veth peer name veth250
sudo ip link set veth250 up  
sudo ip link set veth251 up
freeRouter launch with supplied p4-freerouter-hw.txt and p4-freerouter-sw.txt with a console prompt
╭─[6:06:13]floui@debian ~/freeRouter  
╰─➤  java -jar lib/rtr.jar routersc etc/p4-freerouter-hw.txt etc/p4-freerouter-sw.txt
info cfg.cfgInit.doInit:cfgInit.java:556 booting
info cfg.cfgInit.doInit:cfgInit.java:680 initializing hardware
info cfg.cfgInit.doInit:cfgInit.java:687 applying defaults
info cfg.cfgInit.doInit:cfgInit.java:695 applying configuration
info cfg.cfgInit.doInit:cfgInit.java:721 done
welcome
line ready
freerouter#
launch freeRouter pcapInt in order to stitch control plane and P4 BMv2 dataplane communication
╭─[1:00:53]floui@debian[1]  ~/freeRouter/bin  
╰─➤  sudo ./pcapInt.bin veth251 22709 127.0.0.1 22710 127.0.0.1
binded to local port 127.0.0.1 22709.
will send to 127.0.0.1 22710.
pcap version: libpcap version 1.8.1
opening interface veth251 with pcap1.x api
serving others
>
Run RARE P4 dataplane - simple_switch_grpc router.p4 
export P4_RARE_ROOT=/home/floui/RARE/02-PE-labs
sudo simple_switch_grpc --log-file $P4_RARE_ROOT/run/log/p4-freerouter.log \                                               
                        -i 1@enp0s9 \                        
                        -i 64@veth250 \                                                
                        --thrift-port 9090 --nanolog ipc://$P4_RARE_ROOT/run/bm-0-log.ipc --device-id 0 $P4_RARE_ROOT/build/simple_switch_grpc.json \
                        -- --grpc-server-addr 127.0.0.1:50051 > $P4_RARE_ROOT/run/log/p4-freerouter.out 2>&1 &
Launch forwarder.p4 (p4runtime GRPC based interface)
╭─[2:07:10]floui@debian[1]  ~/RARE/02-PE-labs/p4src  ‹master*› 
╰─➤  ./forwarder.py
rx:  ['myaddr4_add', '224.0.0.0/4', '0', '1', '\n']
rx:  ['myaddr4_add', '255.255.255.255/32', '0', '1', '\n']
rx:  ['myaddr6_add', 'ff00::/8', '0', '1', '\n']
rx:  ['myaddr4_add', '192.168.1.0/24', '-1', '1', '\n']
rx:  ['myaddr4_add', '192.168.1.131/32', '-1', '1', '\n']
rx:  ['myaddr6_add', 'fd7d:a59c:650b::/60', '-1', '1', '\n']
rx:  ['myaddr6_add', 'fd7d:a59c:650b::666/128', '-1', '1', '\n']
rx:  ['myaddr6_add', 'fe80::/64', '-1', '1', '\n']
rx:  ['mylabel6_add', '270549', '1', '\n']
rx:  ['mylabel4_add', '606864', '1', '\n']
rx:  ['state', '1', '1', '0', '\n']
rx:  ['mtu', '1', '9000', '\n']
rx:  ['portvrf_add', '1', '1', '\n']
rx:  ['neigh4_add', '14252', '192.168.1.1', '34:ce:00:67:18:c2', '1', '00:72:3e:18:1b:6f', '1', '\n']
rx:  ['neigh4_add', '52194', '192.168.1.143', '9c:eb:e8:d5:2c:51', '1', '00:72:3e:18:1b:6f', '1', '\n']
rx:  ['keepalive', '\n']
rx:  ['keepalive', '\n']
...

Verification

FreeRouter telnet access from Virtualbox VM guest via port 2323
╭─[7:07:41]floui@debian[1]  ~/freeRouter/etc  
╰─➤  telnet localhost 2323
Trying ::1...
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
welcome
line ready
freerouter#
freerouter running configuration
p4-freerouter#sh run                                                           
hostname p4-freerouter
buggy
!
vrf definition v1
 exit
!
interface ethernet0
 description freerouter@P4_CPU_PORT[veth251]
 no shutdown
 no log-link-change
 exit
!
interface sdn1
 description freerouter@sdn1[enp0s9]
 mtu 9000
 macaddr 0072.3e18.1b6f
 vrf forwarding v1
 ipv4 address 192.168.1.131 255.255.255.0
 ipv6 address fd7d:a59c:650b::666 ffff:ffff:ffff:fff0::
 ipv6 enable
 no shutdown
 no log-link-change
 exit
!
!
!
!
!
!
!
!
!
!
!
!
!
!
server telnet tel
 security protocol telnet
 no exec authorization
 no login authentication
 vrf v1
 exit
!
server p4lang p4
 export-vrf v1 1
 export-port sdn1 1 0
 interconnect ethernet0
 vrf v1
 exit
!
client tcp-checksum transmit
!
end
Check control plane is communicating with bmv2 p4 dataplane
p4-freerouter#show interfaces summary                                          
interface  state  tx     rx    drop
ethernet0  up     10616  9243  0
sdn1       up     10340  9069  0
Ping IPv4 from freerouter -> LAN router gateway
p4-freerouter#ping 192.168.1.131 /vrf v1                                       
pinging 192.168.1.131, src=null, cnt=5, len=64, tim=1000, ttl=255, tos=0, sweep=false
!!!!!
result=100%, recv/sent/lost=5/5/0, rtt min/avg/max/total=0/0/0/0
p4-freerouter#ping 192.168.1.1 /vrf v1                                         
pinging 192.168.1.1, src=null, cnt=5, len=64, tim=1000, ttl=255, tos=0, sweep=false
!!!!!
result=100%, recv/sent/lost=5/5/0, rtt min/avg/max/total=3/5/10/28
IPv4 arp check ( 192.168.1.1 is the gateway, 192.168.1.143 is the VM host)
p4-freerouter#sh ipv4 arp sdn1                                                 
mac             address        time      static
34ce.0067.18c2  192.168.1.1    00:00:35  false
9ceb.e8d5.2c51  192.168.1.143  00:00:35  false
6420.0c65.437b  192.168.1.173  00:01:35  false

Ping IPv6 from freerouter -> LAN router gateway
p4-freerouter#ping fd7d:a59c:650b::666 /vrf v1                                 
pinging fd7d:a59c:650b::666, src=null, cnt=5, len=64, tim=1000, ttl=255, tos=0, sweep=false
!!!!!
result=100%, recv/sent/lost=5/5/0, rtt min/avg/max/total=0/0/0/1
p4-freerouter#ping fd7d:a59c:650b::1 /vrf v1                                   
pinging fd7d:a59c:650b::1, src=null, cnt=5, len=64, tim=1000, ttl=255, tos=0, sweep=false
!!!!!
result=100%, recv/sent/lost=5/5/0, rtt min/avg/max/total=2/2/3/13
IPv6 neighbor discovery check ( 192.168.1.1 is the gateway, 192.168.1.143 is the VM host)
p4-freerouter#show ipv6 neighbors sdn1                                         
mac             address                    time      static  router
34ce.0067.18c2  fd7d:a59c:650b::1          00:01:22  false   false
9ceb.e8d5.2c51  fe80::10e6:87a7:6a9:f14a   00:01:22  false   false
34ce.0067.18c2  fe80::36ce:ff:fe67:18c2    00:01:22  false   false
b6be.fdcf.d0f9  fe80::b4be:fdff:fecf:d0f9  00:01:22  false   false
Initiate IPv4 ssh from freerouter -> LAN router gateway
p4-freerouter#ssh 192.168.1.1 /vrf v1 /user root                               
 - connecting to 192.168.1.1 22
password: ***************
                
 - securing connection



BusyBox v1.28.4 () built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 18.06.2, r7676-cddd7b4c77
 -----------------------------------------------------
root@OpenWrt:~#


Initiate IPv6 ssh from freerouter -> LAN router gateway
p4-freerouter#ssh fd7d:a59c:650b::1 /vrf v1 /user root                         
 - connecting to fd7d:a59c:650b::1 22
password: ***************
                
 - securing connection



BusyBox v1.28.4 () built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 18.06.2, r7676-cddd7b4c77
 -----------------------------------------------------

Conclusion

In this article you:

  • had a demonstration of how to integrate freeRouter into a local area network (Similar to article #002)
  • However instead of using pcapInt you are now using a software P4 dataplane from P4lang project: bmv2 
  • BMv2 simple_switch_grpc target is used an run RARE router.p4
  • communication between freeRouter control plane and bmv2 is ensured by pcapInt via veth pair [ veth250 - veth251 ]
  • This communication is possible via RARE forwarder.py based on GRPC P4Lang P4Runtime python binding
  • In this example the BMv2 P4 switch has only 1 dataplane interface that is bound to enp0s9 VM interface exposed to the local network as a bridged interface

[ #003 ] RARE/FreeRouter-101 - key take-away

  • FreeRouter is using UNIX socket in order to forward packet dedicated to control plane + dataplane communication.

This essential paradigm is used to ensure communication between freeRouter and BMv2 P4 dataplane. It is ensured by pcapInt binary from freeRouter net-tools that will bind freeRouter socket (veth251@locathost:22710) to a virtual network interface (veth250@localhost:22709)  connected to CPU_PORT 64.

  • freeRouter control plane and dataplane communication is enabled by RARE forwarder.py 

forwarder.py is a simple python script based on GRPC P4Runtime python library.

freeRouter is doing all the control plane route computation and write/modify/remove message entry via P4Runtime so that P4 entries are created/modified/removed accordingly from P4 tables

While BMv2 target is a very good choice for packet processing algorithm validation, it is not an ideal target for production use. We will see in next articles how we can reach a higher rate throughput related required by use cases defined by network operators.  


  • No labels