...
- A Raspberry Pi 3 Model B+ or a newer model
- A micro SD card with at least 16GB
- WiFiMon Raspberry Pi operating system image (Installation option 1) or Raspberry Pi with installed Raspberry Pi OS (Installation option 2)
Setting up the WHP
There are two options for the WHP installation:
- Installation and configuration from the prepared WiFiMon WHP image (Installation option 1)
- Installation and configuration on the Raspberry Pi with already installed Raspberry Pi OS (Stretch or later) (Installation option 2)
Installation and configuration
The following steps apply for both installation options. WiFiMon administrators who will use the prepared WHP image (installation option 1) should simply edit the crontab and wireless.py and twping_parser.py files as discussed in the following. WiFiMon administrators who will not use the prepared WIFiMon WHP image (installation option 2) should follow the steps 2 up to 5.
Step 1: Write the image to the micro SD card
Follow the instructions at the official Raspberry Pi site. Skip the "Download the image" step and use the WiFiMon Raspberry Pi operating system image instead (download size is approx. 3.5 GB).
...
We advise the WiFiMon administrator to always secure Raspberry Pi by changing the default password.
Step 2: Start the Raspberry Pi
Follow the simple steps below:
...
You should see a red light on the Raspberry Pi and raspberries on the monitor. The WiFiMon Hardware Probe will boot up into a graphical desktop.
Step 3: Configure the Raspberry Pi
Secure the Raspberry Pi by changing the default password. Optionally, you may enable SSH to access the command line of a Raspberry Pi remotely or setup remote desktop. Next, you have to connect to the wireless network you want to measure.
...
The WiFiMon Hardware Probe (WHP) performs performance tests towards the WiFiMon Test Server (WTS) in an automated manner. It uses crontab to schedule the tests. To do that, open the terminal (as user "pi") and enter the command: crontab -e. You will have to pick the text editor that you prefer. Then scroll to the bottom of the file and add the following code block (which you will modify as explained below):
...
:
...
|
...
You have to modify the following parts of the crontab in lines 2-4:
...
appropriate lines within nettest.sh, boomerang.sh and speedtest.sh scripts to point to the testpages URL.
You should put the URL or IP address of the WTS in which the NetTest, LibreSpeed Speedtest and Akamai Boomerang JS scripts are injected. Details about the configuration of the WiFiMon testtools are included in the WiFiMon Test Server (WTS) installation documentation. Following the assumptions/notations of the WTS guide, examples of the URLs for NetTest, speedtest and boomerang respectively are (i) https://WTS_FQDN/wifimon/measurements/nettest.html, (ii) https://WTS_FQDN/wifimon/measurements/speedworker.html and (iii) https://WTS_FQDN/wifimon/measurements/boomerang.html.
...
Line 2 of the crontab is related to the streaming of TWAMP measurement results to the WiFiMon Analysis Server (WAS).
Step 4: Streaming Wireless Network Interface Metrics to the WiFiMon Analysis Server (WAS)
In /home/pi, you will find the Python script wireless.py. The contents of the script are the following:
wireless.py
...
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
#!/usr/bin/python3 import sys import subprocess import datetime import requests from requests.packages.urllib3.exceptions import InsecureRequestWarning requests.packages.urllib3.disable_warnings(InsecureRequestWarning) import json import pingparsing def return_command_output(command): proc = subprocess.Popen(command, stdout = subprocess.PIPE, shell = True) (out, err) = proc.communicate() output = out.rstrip('\n'.encode('utf8')) return output def get_mac(iface): command = "cat /sys/class/net/" + str(iface) + "/address" mac = return_command_output(command).decode('utf8') mac = mac.replace(":", "-") return mac def find_wlan_iface_name(): command = "printf '%s\n' /sys/class/net/*/wireless | awk -F'/' '{print $5 }'" wlan_iface_name = return_command_output(command) return wlan_iface_name.decode('utf8') def parseget_iwconfigencryption(iface): bit_ratecommand = return_command_output("sudo iwconfigwpa_cli status" + iface + " | grep Bit | grep key_mgmt" + "|awk -F '\=' {'print $2'}" enc = return_command_output(command).decode('utf8') return enc def parse_iwconfig(iface): bit_rate = return_command_output("sudo iwconfig " + iface + " | grep Bit | awk '{print $2}' | sed 's/Rate=//'").decode('utf8') tx_power = return_command_output("sudo iwconfig " + iface + " | grep Bit | awk '{print $4}' | sed 's/Tx-Power=//'").decode('utf8') link_quality = return_command_output("sudo iwconfig " + iface + " | grep Link | awk '{print $2}' | sed 's/Quality=//'").decode('utf8') link_quality = link_quality.split("/")[0] signal_level = return_command_output("sudo iwconfig " + iface + " | grep Link | awk '{print $4}' | sed 's/level=//'").decode('utf8') accesspoint = return_command_output("sudo iwconfig " + iface + " | grep Mode | awk '{print $6}' | sed 's/Point: //'").decode('utf8') accesspoint = accesspoint.replace(":", "-") essid = return_command_output("sudo iwconfig " + iface + " | grep ESSID | awk '{print $4}' | sed 's/ESSID://'").decode('utf8') essid = essid.replace("\"", "") return bit_rate, tx_power, link_quality, signal_level, accesspoint, essid def parse_iwlist(iface, accesspoint): information = {} command = "sudo iwlist " + iface + " scan | grep -E \"Cell|Frequency|Quality|ESSID\"" aps = return_command_output(command).decode("utf8") aps = aps.split("\n") cell_indices = list() for index in range(0, len(aps)): line_no_whitespace = ' '.join(aps[index].split()) parts = line_no_whitespace.split() if parts[0] == "Cell": cell_indices.append(index) for index in cell_indices: line0 = ' '.join(aps[index].split()) ap_mac = line0.split()[-1] ap_mac = ap_mac.replace(":", "-") information[ap_mac] = {} line1 = ' '.join(aps[index + 1].split()) frequency = line1.split()[0].split(":")[1] information[ap_mac]["frequency"] = str(frequency) line2 = ' '.join(aps[index + 2].split()) parts = line2.split() information[ap_mac]["drillTest"] = float(parts[2].split("=")[1]) line3 = ' '.join(aps[index + 3].split()) parts = line3.split(":") information[ap_mac][str(parts[1].replace('"', ''))] = information[ap_mac]["drillTest"] return information def convert_info_to_json(accesspoint, essid, mac, enc, bit_rate, tx_power, link_quality, signal_level, probe_no, information, location_name, test_device_location_description, nat_network, system_dictionary, number_of_users, pingparser_result): overall_dictionary = {} # values from ping received through pingparser github tool overall_dictionary["wts"] = str(pingparser_result["destination"]) packet_transmit = int(float(pingparser_result["packet_transmit"])) overall_dictionary["pingPacketTransmit"] = str(packet_transmit) packet_receive = int(float(pingparser_result["packet_receive"])) overall_dictionary["pingPacketReceive"] = str(packet_receive) packet_loss_rate = int(float(pingparser_result["packet_loss_rate"])) overall_dictionary["pingPacketLossRate"] = str(packet_loss_rate) packet_loss_count = int(float(pingparser_result["packet_loss_count"])) overall_dictionary["pingPacketLossCount"] = str(packet_loss_count) try: rtt_min = int(float(pingparser_result["rtt_min"])) rtt_avg = int(float(pingparser_result["rtt_avg"])) rtt_max = int(float(pingparser_result["rtt_max"])) rtt_mdev = int(float(pingparser_result["rtt_mdev"])) packet_duplicate_rate = int(float(pingparser_result["packet_duplicate_rate"])) packet_duplicate_count = int(float(pingparser_result["packet_duplicate_count"])) except: # -1 indicates failure to reach the wts and calculate the above values rtt_min = -1 rtt_avg = -1 rtt_max = -1 rtt_mdev = -1 packet_duplicate_rate = -1 packet_duplicate_count = -1 overall_dictionary["pingRttMin"] = str(rtt_min) overall_dictionary["pingRttAvg"] = str(rtt_avg) overall_dictionary["pingRttMax"] = str(rtt_max) overall_dictionary["pingRttMdev"] = str(rtt_mdev) overall_dictionary["pingPacketDuplicateRate"] = str(packet_duplicate_rate) overall_dictionary["pingPacketDuplicateCount"] = str(packet_duplicate_count) # values from iw* commands overall_dictionary["macAddress"] = "\"" + str(mac) + "\"" overall_dictionary["encType"] = "\"" + str(enc) + "\"" overall_dictionary["accesspoint"] = "\"" + str(accesspoint) + "\"" overall_dictionary["essid"] = "\"" + str(essid) + "\"" bit_rate = int(float(bit_rate)) overall_dictionary["bitRate"] = str(bit_rate) tx_power = int(float(tx_power)) overall_dictionary["txPower"] = str(tx_power) link_quality = int(float(link_quality)) overall_dictionary["linkQuality"] = str(link_quality) signal_level = int(float(signal_level)) overall_dictionary["signalLevel"] = str(signal_level) overall_dictionary["probeNo"] = str(probe_no) information = json.dumps(information) overall_dictionary["monitor"] = information # values defined by administrator overall_dictionary["locationName"] = "\"" + str(location_name) + "\"" overall_dictionary["testDeviceLocationDescription"] = "\"" + str(test_device_location_description) + "\"" overall_dictionary["nat"] = "\"" + str(nat_network) + "\"" # values received through arp-scan command overall_dictionary["numberOfUsers"] = "\"" + str(number_of_users) + "\"" system_dictionary = json.dumps(system_dictionary) # values received from system commands (memory, cpu, disk) overall_dictionary["system"] = system_dictionary json_data = json.dumps(overall_dictionary) return json_data def processing_info(): command = '''echo "$(iostat | head -1 | awk '{print $1}')"''' operating_system = return_command_output(command).decode('utf8') command = '''echo "$(iostat | head -1 | awk '{print $2}')"''' driver_version = return_command_output(command).decode('utf8') command = '''echo "$(iostat | head -1 | awk '{print $6}' | cut -c 2-)"''' total_cores = return_command_output(command).decode('utf8') command = '''echo "$(vmstat 1 2|tail -1|awk '{print $15}')"''' cpu_utilization = 100 - int(return_command_output(command).decode('utf8')) command = '''echo "$(vmstat --stats | grep 'total memory' | tail -1 | awk '{print $1}')"''' total_memory = return_command_output(command).decode('utf8') command = '''echo "$(vmstat --stats | grep 'used memory' | tail -1 | awk '{print $1}')"''' used_memory = return_command_output(command).decode('utf8') command = '''echo "$(df -h / | tail -1 | awk '{print $2}')"''' total_disk_size = return_command_output(command).decode('utf8') command = '''echo "$(df -h / | tail -1 | awk '{print $3}')"''' used_disk_size = return_command_output(command).decode('utf8') system_dictionary = {} system_dictionary["operatingSystem"] = str(operating_system) system_dictionary["driverVersion"] = str(driver_version) system_dictionary["totalCores"] = str(total_cores) system_dictionary["cpuUtilization"] = str(cpu_utilization) system_dictionary["totalMemory"] = str(total_memory) system_dictionary["usedMemory"] = str(used_memory) system_dictionary["totalDiskSize"] = str(total_disk_size) system_dictionary["usedDiskSize"] = str(used_disk_size) return system_dictionary def stream_data(data): headers = {'content-type':"application/json"} try: session = requests.Session() session.verify = False session.post(url='https://INSERT_WAS_FQDN:443/wifimon/probes/', data=data, headers=headers, timeout=30) except: pass def parse_arpscan(result): lines = result.split("\n") lines.pop(0) lines.pop(0) space_line = lines.index('') return space_line def arpscanner(): command = "sudo arp-scan --localnet" arpscan_result = return_command_output(command).decode('utf8') number_of_users = parse_arpscan(arpscan_result) return number_of_users def pingparser(wts): #command See: https://github.com/thombashi/pingparsing ping_parser = pingparsing.PingParsing(= 'ping -c 10 ' + str(wts) transmitterping_results = pingparsing.PingTransmitter(return_command_output(command).decode('utf8') transmitter.destinationping_parts = str(wtsping_results.split("\n") transmitter.countfor = 3 item in ping_parts: resultif item[0:3] = transmitter.ping()= "---": result_json = json.dumps(ping_parser.parse(result).as_dict(), indent=4) resultstart_parsing_dictfrom = json.loads(result_json) 1 + ping_parts.index(item) return result_dict def set_location_information():packets_part = ping_parts[start_parsing_from].split(",") locationpacket_nametransmit = "INSERT_LOCATION_NAME" int(packets_part[0].split(" ")[0], 10) packet_receive = int(packets_part[1].split(" ")[1], 10) testpacket_deviceloss_location_descriptioncount = "INSERT_TEST_DEVICE_LOCATION_DESCRIPTION"packet_transmit - packet_receive natpacket_loss_networkrate = "INSERTpacket_True_OR_False"loss_count / packet_transmit return location_name, test_device_location_description, nat_network def general_info():timing_part = ping_parts[start_parsing_from + 1].split("=")[1] system_dictionarytiming_part_no_whitespace = processing_info()timing_part[1:] location_name, test_device_location_description, nat_networktiming_without_unit = settiming_part_locationno_information()whitespace.split(" ")[0] iface_nametimes = findtiming_wlan_iface_name(without_unit.split("/") macrtt_max = get_mac(iface_name)times[0] bit_rate, tx_power, link_quality, signal_level, accesspoint, essid = parse_iwconfig(iface_name)rtt_min = times[1] informationrtt_avg = parse_iwlist(iface_name, accesspoint)times[2] probertt_nomdev = "INSERT_PROBE_NUMBER"times[3] wts# = "INSERT_WTS_FQDN"unused values numberpacket_ofduplicate_userscount = arpscanner()-1 pingparserpacket_duplicate_resultrate = pingparser(wts) -1 # construct json_dataa =dict convert_info_to_json(accesspoint, essid, mac, bit_rate, tx_power, link_quality, signal_level, probe_no, information, location_name, test_device_location_description, nat_network, system_dictionary, number_of_users, pingparser_result)to hold the results result_dict = {} result_dict["destination"] = wts stream_data(json_data) if __name__ == "__main__":result_dict["packet_transmit"] = packet_transmit result_dict["packet_receive"] = packet_receive result_dict["packet_loss_count"] = packet_loss_count result_dict["packet_loss_rate"] = packet_loss_rate general_info() |
The following values should be set:
result_dict["rtt_max"] = rtt_max
result_dict["rtt_min"] = rtt_min
result_dict["rtt_avg"] = rtt_avg
result_dict["rtt_mdev"] = rtt_mdev
result_dict["packet_duplicate_count"] = packet_duplicate_count
result_dict["packet_duplicate_rate"] = packet_duplicate_rate
return result_dict
def set_location_information():
location_name = "INSERT_LOCATION_NAME"
test_device_location_description = "INSERT_TEST_DEVICE_LOCATION_DESCRIPTION"
nat_network = "INSERT_True_OR_False"
return location_name, test_device_location_description, nat_network
def general_info():
system_dictionary = processing_info()
location_name, test_device_location_description, nat_network = set_location_information()
iface_name = find_wlan_iface_name()
mac = get_mac(iface_name)
enc = get_encryption()
bit_rate, tx_power, link_quality, signal_level, accesspoint, essid = parse_iwconfig(iface_name)
information = parse_iwlist(iface_name, accesspoint)
probe_no = "INSERT_PROBE_NUMBER"
wts = "INSERT_WTS_FQDN"
number_of_users = arpscanner()
pingparser_result = pingparser(wts)
json_data = convert_info_to_json(accesspoint, essid, mac, enc, bit_rate, tx_power, link_quality, signal_level, probe_no, information, location_name, test_device_location_description, nat_network, system_dictionary, number_of_users, pingparser_result)
stream_data(json_data)
if __name__ == "__main__":
general_info() |
The following values should be set:
- "probe_no" (line 246) should match the description "probe_no" (line 207) should match the number assigned to the testtools of the particular WiFiMon Hardware Probe (WHP), e.g. for the WHP assigned the number 1, the value should be "1" and for WHP assigned the description 'wifimon-5' it should be 'wifimon-5 . Assigning numbers to WHPs is possible by appropriately setting the testtool attribute included in the websites monitored by them. More information related to assigning number to WHPs is available in the WiFiMon Test Server installation guide.
- "WAS_FQDN" (line 166171) should match the FQDN of the WiFiMon Analysis Server (WAS) responsible for processing the wireless performance metrics of the WHP. The above code block assumes that the WAS uses https and port 443.
- "WTS_FQDN" (line 208247) should match the FQDN of the WiFiMon Test Server (WTS) or the IP of the WTS.
- LInes 195 Lines 233 to 197 235 can be filled with more information regarding the location of the WHP.
For the disk and memory statistics, you need to install iostat and vmstat packages with the following command:
|
Step 5: Streaming TWAMP Measurement Results to the WiFiMon Analysis Server (WAS)
In /home/pi, you will find the Python script twping_parser.py. The contents of the script are the following:
twping_parser.py
...
For the above script to work, you need to install perfsonar-tools from the perfSONAR repository. The installation process is detail in the following link. In the sequel we summarize the necessary installation steps:
...
cd
/etc/apt/sources
.list.d/
curl -o perfsonar-release.list http:
//downloads
.perfsonar.net
/debian/perfsonar-release
.list
curl http:
//downloads
.perfsonar.net
/debian/perfsonar-official
.gpg.key | apt-key add -
sudo
apt update
sudo
apt
install
perfsonar-tools
we summarize the necessary installation steps:
|
Moreover, you also need to install "ntpstat" via the following commands:
sudo apt update |
Step 6: Support for distributed control
sudo apt install -y salt-minion
Edits: In file /etc/salt/minion specify the IP/FQDN of the Salt master at the line "master:". In file /etc/salt/minion_id specify the description of the WHP. This description is "x" as included in WiFiMon testtools (see WTS installation guide)
Note: This setup requires you to name WHP testpages with names matching the WHP description, e.g. for "wifimon-5", testpages should be named nettestwifimon-5.html, boomerangwifimon-5.html and speedworkerwifimon5.html
Security Issues
Moreover, you also need to install "ntpstat" via the following commands:
...
sudo apt update
sudo apt install -y ntpstat
Security Issues
We suggest that you take additional efforts to safeguard the security of your probes:
...