WiFiMon Hardware Probes (WHP) are used to gather performance measurements in a WiFi network from dedicated small form factor devices which are installed in fixed points. WiFiMon tested its operation and recommends the use of Raspberry Pi’s v3 Model B+ or v4. WiFiMon Hardware Probe will work in the following configuration:
There are two options for the WHP installation:
The following steps apply for both installation options. WiFiMon users who will use the prepared WHP image (installation option 1) should simply edit the crontab and wireless.py and twping_parser.py files as discussed in the following. WiFiMon users who will not use the prepared WIFiMon WHP image (installation option 2) should follow the steps 2 up to 5.
Follow the instructions at the official Raspberry Pi site. Skip the "Download the image" step and use the WiFiMon Raspberry Pi operating system image instead (download size is approx. 3 GB).
WiFiMon Raspberry Pi image given above is a custom version of Raspberry Pi OS (Buster) with desktop, with the default Raspberry Pi credentials (user: pi, password: raspberry).
We advise the user to always secure Raspberry Pi by changing the default password.
Follow the simple steps below:
You should see a red light on the Raspberry Pi and raspberries on the monitor. The WiFiMon Hardware Probe will boot up into a graphical desktop.
Secure the Raspberry Pi by changing the default password. Optionally, you may enable SSH to access the command line of a Raspberry Pi remotely or setup remote desktop. Next, you have to connect to the wireless network you want to measure.
First, the following programs should be downloaded:
sudo apt-get update sudo apt-get install -y xvfb firefox-esr |
The WiFiMon Hardware Probe (WHP) performs performance tests towards the WiFiMon Test Server (WTS) in an automated manner. It uses crontab to schedule the tests. To do that, open the terminal (as user "pi") and enter the command: crontab -e. You will have to pick the text editor. Then scroll to the bottom of the file and add the following code block (which you will modify as explained below):
00,10,20,30,40,50 * * * * Xvfb :100 & 02,12,22,32,42,52 * * * * export DISPLAY=:100 && firefox-esr --new-tab URL_TO_nettest.html >/dev/null 2>&1 04,14,24,34,44,54 * * * * export DISPLAY=:100 && firefox-esr --new-tab URL_TO_speedworker.html >/dev/null 2>&1 06,16,26,36,46,56 * * * * export DISPLAY=:100 && firefox-esr --new-tab URL_TO_boomerang.html >/dev/null 2>&1 59 * * * * sudo killall firefox-esr |
You have to modify the following parts of the crontab in lines 2-4:
You should put the URL or IP address of the WTS in which the NetTest, speedtest and boomerang JS scripts are injected. Details about the configuration of the WiFiMon testtools are included in the WiFiMon Test Server (WTS) installation documentation. Following the assumptions/notations of the WTS guide, examples of the URLs for NetTest, speedtest and boomerang respectively are (i) https://WTS_FQDN/wifimon/measurements/nettest.html, (ii) https://WTS_FQDN/wifimon/measurements/speedworker.html and (iii) https://WTS_FQDN/wifimon/measurements/boomerang.html.
Furthermore, open the terminal (as user "root") and enter the command: crontab -e. Add the following lines:
03,13,23,33,43,53 * * * * python3 wireless.py 07,17,27,37,47,57 * * * * python3 twping_parser.py |
Line 1 of the crontab is related to the streaming of wireless network interface metrics to the WiFiMon Analysis Server (WAS). Optionally, the intervals of the WHP measurements could be altered by appropriately configuring the crontab so that measurement are more or less frequent. The configuration of the crontab config given above sets up 10-minute intervals between the measurements of each test tool in a way in which there are no overlapping measurements.
Line 2 of the crontab is related to the streaming of TWAMP measurement results to the WiFiMon Analysis Server (WAS).
In /home/pi, you will find the Python script wireless.py. The contents of the script are the following:
#!/usr/bin/python3
import subprocess
import datetime
import requests
from requests.packages.urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)
import json
def return_command_output(command):
proc = subprocess.Popen(command, stdout = subprocess.PIPE, shell = True)
(out, err) = proc.communicate()
output = out.rstrip('\n'.encode('utf8'))
return output
def get_mac(iface):
command = "cat /sys/class/net/" + str(iface) + "/address"
mac = return_command_output(command).decode('utf8')
mac = mac.replace(":", "-")
return mac
def find_wlan_iface_name():
command = "printf '%s\n' /sys/class/net/*/wireless | awk -F'/' '{print $5 }'"
wlan_iface_name = return_command_output(command)
return wlan_iface_name.decode('utf8')
def parse_iwconfig(iface):
bit_rate = return_command_output("sudo iwconfig " + iface + " | grep Bit | awk '{print $2}' | sed 's/Rate=//'").decode('utf8')
tx_power = return_command_output("sudo iwconfig " + iface + " | grep Bit | awk '{print $4}' | sed 's/Tx-Power=//'").decode('utf8')
link_quality = return_command_output("sudo iwconfig " + iface + " | grep Link | awk '{print $2}' | sed 's/Quality=//'").decode('utf8')
link_quality = link_quality.split("/")[0]
signal_level = return_command_output("sudo iwconfig " + iface + " | grep Link | awk '{print $4}' | sed 's/level=//'").decode('utf8')
accesspoint = return_command_output("sudo iwconfig " + iface + " | grep Mode | awk '{print $6}' | sed 's/Point: //'").decode('utf8')
accesspoint = accesspoint.replace(":", "-")
essid = return_command_output("sudo iwconfig " + iface + " | grep ESSID | awk '{print $4}' | sed 's/ESSID://'").decode('utf8')
essid = essid.replace("\"", "")
return bit_rate, tx_power, link_quality, signal_level, accesspoint, essid
def parse_iwlist(iface, accesspoint):
information = {}
command = "sudo iwlist " + iface + " scan | grep -E \"Cell|Frequency|Quality|ESSID\""
aps = return_command_output(command).decode("utf8")
aps = aps.split("\n")
cell_indices = list()
for index in range(0, len(aps)):
line_no_whitespace = ' '.join(aps[index].split())
parts = line_no_whitespace.split()
if parts[0] == "Cell":
cell_indices.append(index)
for index in cell_indices:
line0 = ' '.join(aps[index].split())
ap_mac = line0.split()[-1]
ap_mac = ap_mac.replace(":", "-")
information[ap_mac] = {}
line1 = ' '.join(aps[index + 1].split())
frequency = line1.split()[0].split(":")[1]
information[ap_mac]["frequency"] = str(frequency)
line2 = ' '.join(aps[index + 2].split())
parts = line2.split()
information[ap_mac]["drillTest"] = float(parts[2].split("=")[1])
line3 = ' '.join(aps[index + 3].split())
parts = line3.split(":")
information[ap_mac][str(parts[1].replace('"', ''))] = information[ap_mac]["drillTest"]
return information
def convert_info_to_json(accesspoint, essid, mac, bit_rate, tx_power, link_quality, signal_level, probe_no, information, location_name, test_device_location_description, nat_network, system_dictionary):
overall_dictionary = {}
overall_dictionary["macAddress"] = "\"" + str(mac) + "\""
overall_dictionary["accesspoint"] = "\"" + str(accesspoint) + "\""
overall_dictionary["essid"] = "\"" + str(essid) + "\""
bit_rate = int(float(bit_rate))
overall_dictionary["bitRate"] = str(bit_rate)
tx_power = int(float(tx_power))
overall_dictionary["txPower"] = str(tx_power)
link_quality = int(float(link_quality))
overall_dictionary["linkQuality"] = str(link_quality)
signal_level = int(float(signal_level))
overall_dictionary["signalLevel"] = str(signal_level)
overall_dictionary["probeNo"] = str(probe_no)
information = json.dumps(information)
overall_dictionary["monitor"] = information
overall_dictionary["locationName"] = "\"" + str(location_name) + "\""
overall_dictionary["testDeviceLocationDescription"] = "\"" + str(test_device_location_description) + "\""
overall_dictionary["nat"] = "\"" + str(nat_network) + "\""
system_dictionary = json.dumps(system_dictionary)
overall_dictionary["system"] = system_dictionary
json_data = json.dumps(overall_dictionary)
return json_data
def processing_info():
command = '''echo "$(iostat | head -1 | awk '{print $1}')"'''
operating_system = return_command_output(command).decode('utf8')
command = '''echo "$(iostat | head -1 | awk '{print $2}')"'''
driver_version = return_command_output(command).decode('utf8')
command = '''echo "$(iostat | head -1 | awk '{print $6}' | cut -c 2-)"'''
total_cores = return_command_output(command).decode('utf8')
command = '''echo "$(vmstat 1 2|tail -1|awk '{print $15}')"'''
cpu_utilization = 100 - int(return_command_output(command).decode('utf8'))
command = '''echo "$(vmstat --stats | grep 'total memory' | tail -1 | awk '{print $1}')"'''
total_memory = return_command_output(command).decode('utf8')
command = '''echo "$(vmstat --stats | grep 'used memory' | tail -1 | awk '{print $1}')"'''
used_memory = return_command_output(command).decode('utf8')
command = '''echo "$(df -h / | tail -1 | awk '{print $2}')"'''
total_disk_size = return_command_output(command).decode('utf8')
command = '''echo "$(df -h / | tail -1 | awk '{print $3}')"'''
used_disk_size = return_command_output(command).decode('utf8')
system_dictionary = {}
system_dictionary["operatingSystem"] = str(operating_system)
system_dictionary["driverVersion"] = str(driver_version)
system_dictionary["totalCores"] = str(total_cores)
system_dictionary["cpuUtilization"] = str(cpu_utilization)
system_dictionary["totalMemory"] = str(total_memory)
system_dictionary["usedMemory"] = str(used_memory)
system_dictionary["totalDiskSize"] = str(total_disk_size)
system_dictionary["usedDiskSize"] = str(used_disk_size)
return system_dictionary
def stream_data(data):
headers = {'content-type':"application/json"}
try:
session = requests.Session()
session.verify = False
session.post(url='https://WAS_FQDN:443/wifimon/probes/', data=data, headers=headers, timeout=30)
except:
pass
def set_location_information():
location_name = ""
test_device_location_description = ""
nat_network = ""
return location_name, test_device_location_description, nat_network
def wireless_info():
system_dictionary = processing_info()
location_name, test_device_location_description, nat_network = set_location_information()
iface_name = find_wlan_iface_name()
mac = get_mac(iface_name)
bit_rate, tx_power, link_quality, signal_level, accesspoint, essid = parse_iwconfig(iface_name)
information = parse_iwlist(iface_name, accesspoint)
probe_no = ""
json_data = convert_info_to_json(accesspoint, essid, mac, bit_rate, tx_power, link_quality, signal_level, probe_no, information, location_name, test_device_location_description, nat_network, system_dictionary)
stream_data(json_data)
if __name__ == "__main__":
wireless_info() |
The following values should be set:
For the disk and memory statistics, you need to install iostat and vmstat packages with the following command:
sudo apt install -y systat |
In /home/pi, you will find the Python script twping_parser.py. The contents of the script are the following:
'''
Sample twping output (MIND THE NAMING OF THE LINES)
line 0: --- twping statistics from [192.168.1.1]:9706 to [192.168.1.2]:19642 ---
line 1: SID: c0a80102e5e36a42b8a73f74cec8780e
line 2: first: 2022-03-21T23:18:58.819
line 3: last: 2022-03-21T23:19:10.456
line 4: 100 sent, 0 lost (0.000%), 0 send duplicates, 0 reflect duplicates
line 5: round-trip time min/median/max = 0.109/0.3/1.07 ms, (err=3.8 ms)
line 6: send time min/median/max = 936/936/936 ms, (err=1.9 ms)
line 7: reflect time min/median/max = -936/-936/-935 ms, (err=1.9 ms)
line 8: reflector processing time min/max = 0.00191/0.021 ms
line 9: two-way jitter = 0.1 ms (P95-P50)
line 10: send jitter = 0.1 ms (P95-P50)
line 11: reflect jitter = 0 ms (P95-P50)
line 12: send hops = 0 (consistently)
line 13:reflect hops = 0 (consistently)
'''
import subprocess
import json
import requests
from requests.packages.urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)
def return_command_output(command):
'''
Execute a command and return its output
'''
proc = subprocess.Popen(command, stdout = subprocess.PIPE, shell = True)
(out, err) = proc.communicate()
output = out.rstrip('\n'.encode('utf8'))
return output
def perform_twping(twamp_server_ip):
'''
Perform the twping command and retrieve its output in milliseconds
'''
command = "twping " + str(twamp_server_ip) + " -n m -B wlan0"
twping_results = return_command_output(command).decode('utf8')
return twping_results
def locate_twping_data(twping_output):
'''
Find the line at which the important part of the twping output starts
'''
twping_output_parts = twping_output.split('\n')
line_to_start = 0
for line in twping_output_parts:
initial_three_chars = line[0:3]
if initial_three_chars == "---":
break
line_to_start += 1
return line_to_start
# Parse lines one by one. Look at the top for the numbering of the lines
def parse_line4(line4):
parts = line4.split(" ")
sent, lost, send_dups, reflect_dups = parts[0], parts[2], parts[5], parts[8]
return sent, lost, send_dups, reflect_dups
def parse_times(line):
parts = line.split(" ")
min_median_max = parts[4].split("/")
minimum, median, maximum = min_median_max[0], min_median_max[1], min_median_max[2]
err = parts[6].split("=")[1]
return minimum, median, maximum, err
def parse_line8(line):
parts = line.split(" ")
time_unit = parts[-1]
minimum = parts[-2].split("/")[0]
maximum = parts[-2].split("/")[1]
return minimum, maximum
def parse_jitter(line):
parts = line.split(" ")
value = parts[3]
characterization = parts[5][1:-1]
return value, characterization
def parse_hops(line):
parts = line.split(" ")
value = parts[3]
characterization = parts[4][1:-1]
return value, characterization
def form_json(probe_number, twamp_server, sent, lost, send_dups, reflect_dups,
min_rtt, median_rtt, max_rtt, err_rtt, min_send, median_send, max_send,
err_send, min_reflect, median_reflect, max_reflect, err_reflect,
min_reflector_processing_time, max_reflector_processing_time,
two_way_jitter_value, two_way_jitter_char, send_jitter_value, send_jitter_char,
reflect_jitter_value, reflect_jitter_char, send_hops_value, send_hops_char,
reflect_hops_value, reflect_hops_char):
'''
Create a json object with the parsed values. Values are first stored in a dictionary.
'''
overall_dictionary = {}
overall_dictionary["probeNumber"] = probe_number
overall_dictionary["twampServer"] = twamp_server
overall_dictionary["sent"] = sent
overall_dictionary["lost"] = lost
overall_dictionary["sendDups"] = send_dups
overall_dictionary["reflectDups"] = reflect_dups
overall_dictionary["minRtt"] = min_rtt
overall_dictionary["medianRtt"] = median_rtt
overall_dictionary["maxRtt"] = max_rtt
overall_dictionary["errRtt"] = err_rtt
overall_dictionary["minSend"] = min_send
overall_dictionary["medianSend"] = median_send
overall_dictionary["maxSend"] = max_send
overall_dictionary["errSend"] = err_send
overall_dictionary["minReflect"] = min_reflect
overall_dictionary["medianReflect"] = median_reflect
overall_dictionary["maxReflect"] = max_reflect
overall_dictionary["errReflect"] = err_reflect
overall_dictionary["minReflectorProcessingTime"] = min_reflector_processing_time
overall_dictionary["maxReflectorProcessingTime"] = max_reflector_processing_time
overall_dictionary["twoWayJitterValue"] = two_way_jitter_value
overall_dictionary["twoWayJitterChar"] = two_way_jitter_char
overall_dictionary["sendJitterValue"] = send_jitter_value
overall_dictionary["sendJitterChar"] = send_jitter_char
overall_dictionary["reflectJitterValue"] = reflect_jitter_value
overall_dictionary["reflectJitterChar"] = reflect_jitter_char
overall_dictionary["sendHopsValue"] = send_hops_value
overall_dictionary["sendHopsChar"] = send_hops_char
overall_dictionary["reflectHopsValue"] = reflect_hops_value
overall_dictionary["reflectHopsChar"] = reflect_hops_char
json_data = json.dumps(overall_dictionary)
return json_data
def parse_twping(twping_output, line_to_start, probe_number):
'''
Parse twping output line by line
'''
twping_output_parts = twping_output.split('\n')
sent, lost, send_dups, reflect_dups = parse_line4(twping_output_parts[line_to_start + 4])
min_rtt, median_rtt, max_rtt, err_rtt = parse_times(twping_output_parts[line_to_start + 5])
min_send, median_send, max_send, err_send = parse_times(twping_output_parts[line_to_start + 6])
min_reflect, median_reflect, max_reflect, err_reflect = parse_times(twping_output_parts[line_to_start + 7])
min_reflector_processing_time, max_reflector_processing_time = parse_line8(twping_output_parts[line_to_start +8])
two_way_jitter_value, two_way_jitter_char = parse_jitter(twping_output_parts[line_to_start + 9])
send_jitter_value, send_jitter_char = parse_jitter(twping_output_parts[line_to_start + 10])
reflect_jitter_value, reflect_jitter_char = parse_jitter(twping_output_parts[line_to_start + 11])
send_hops_value, send_hops_char = parse_hops(twping_output_parts[line_to_start + 12])
reflect_hops_value, reflect_hops_char = parse_hops(twping_output_parts[line_to_start + 13])
json_data = form_json(probe_number, twamp_server, sent, lost, send_dups, reflect_dups,
min_rtt, median_rtt, max_rtt, err_rtt, min_send, median_send, max_send, err_send,
min_reflect, median_reflect, max_reflect, err_reflect, min_reflector_processing_time,
max_reflector_processing_time, two_way_jitter_value, two_way_jitter_char,
send_jitter_value, send_jitter_char, reflect_jitter_value, reflect_jitter_char,
send_hops_value, send_hops_char, reflect_hops_value, reflect_hops_char)
return json_data
def stream_data(json_data):
'''
Stream JSON data to the WiFiMon Analysis Server
Set the FQDN of the WiFiMon Analysis Server
'''
headers = {'content-type' : "application/json"}
try:
session = requests.Session()
session.verify = False
session.post(url = 'https://WAS_FQDN:443/wifimon/twamp/', data = json_data, headers = headers, timeout = 30)
except:
pass
return None
if __name__ == "__main__":
# Define the number of the WiFiMon Hardware Probe
PROBE_NO = "PROBE_NUMBER"
# Define the FQDN of the TWAMP Server
twamp_server = "TWAMP_SERVER_FQDN"
# Perform twping against the TWAMP Server
twping_results = perform_twping(twamp_server)
# Parse twping results
line_to_start = locate_twping_data(twping_results)
json_data = parse_twping(twping_results, line_to_start, PROBE_NO)
# Stream data to the WiFiMon Analysis Server
stream_data(json_data) |
The following values should be set:
For the above script to work, you need to install perfsonar-tools from the perfSONAR repository. The installation process is detail in the following link. In the sequel we summarize the necessary installation steps:
cd /etc/apt/sources.list.d/ curl -o perfsonar-release.list http://downloads.perfsonar.net/debian/perfsonar-release.list curl http://downloads.perfsonar.net/debian/perfsonar-official.gpg.key | apt-key add - sudo apt update sudo apt install perfsonar-tools |
We suggest that you take additional efforts to safeguard the security of your probes:
set +o history wpa_passphrase YOUR_ESSID YOUR_PASSWORD set -o history |
A "psk=....." line will be generated. Add this line in /etc/wpa_supplicant/wpa_supplicant.conf under your ESSID and delete the plaintext password.
delgroup pi sudo rm /etc/sudoers.d/010_pi-nopasswd |