This section page describes the key architectural components and the measurement workflow of the WiFiMon system:.
WiFiMon Architectural Components
Mobile User
client:A mobile client user is the user's device connected to the WiFi network. WiFiMon can be used in any WiFi network, but is optimised optimized to be used in the eduroam Eduroam environment, as through the analysis of the eduroam Eduroam logs very detailed performance assessments can be obtained. If the eduroam Eduroam is usedu2sed, the mobile client user will be authenticated at the home organisation (IdP) and authorised authorized at the end-user's location (SP) on an IEEE 802.1X based WiFi network.
WiFiMon Admin
Data sources: The data sources (see Figure 1) include:
WiFiMon admin is the network administrator managing the WiFiMon system. Connected at WiFiMon UI, he/she can perform queries to the Elastic ELK Cluster relatable to the wireless performance of the network.
| Captioned Item | ||||||
|---|---|---|---|---|---|---|
| ||||||
 |
Data Source
- Collected Raw Data
Data Collected raw data generated by network performance tools (Boomerang, NetTest and HTML5 SpeedTest) with the use of JavaScript code embedded on chosen web pages; tests . Tests are run by hardware probes and performance data are retrieved when the hardware probes visit the web sources with the embedded code, and. - Exported raw data from data sources (collectors) like Syslog, Server Log, RADIUS Accounting, L2/L3 address binding. Wireless controllers and Access Point identifiers (AP-IDs) may provide the minimum amount of information in a smaller implementation, at the most basic level for performance verification and monitoring on a wireless network. Therefore, the data sources will vary, depending on the vendor specific tools integrated in the network
- Raw Data
Data generated from the interaction of mobile users with services like Eduroam and DHCP, whose logs are fetched from Filebeat agents and are then sent to Logstash pipelines for filtering and editing them according to the needs of WiFiMon, before storing them in the Elastic ELK Cluster.
Elastic ELK Cluster
WiFiMon makes use of Elastic cluster: The raw data is automatically collected and fed into the WiFiMon agent, which uses Elasticsearch, a full-text, distributed NoSQL search engine. Elasticsearch uses documents rather than schema or tables (that used in SQL databases use), thus allowing leveraging and accessing data at very high speeds and making it appropriate for WiFiMon. In addition, the Elastic cluster uses tools such as logstash and filebeat to pipeline the raw data from the data sources simultaneously, transform these data and send it to Elasticsearch.Data Analysis: Within the Elastic cluster, raw data is processed to data objects relevant to performance measurements. These raw data are then correlated with information from collected RADIUS logs Exported Raw Data and the results of correlating them with the Collected Raw Data processed at WiFiMon Agent, are stored in the cluster. Part of Elastic ELK is the Kibana platform (accessible from the browser) which provides cluster management and powers the WiFiMon UI. Another component of Elastic Stack is the Filebeat, which is stored in the severs holding the Exported Raw Data.
WiFiMon Agent / UI
- WiFiMon Agent processes Collected Raw Data to objects relevant to performance measurements, which are correlated with Exported Raw Data collected from RADIUS and DHCP logs, to provide insight on the wireless network performance per client or AP. The
...
- results of correlations are
...
- stored in
...
- the ELK Cluster.
...
- WiFiMon UI is powered by Kibana platform to perform queries, for example, to retrieve information about the measurements initiated to a specific AP and within a specific time period. Other queries could be made to retrieve information about specific IP ranges, test tools etc.
...
Data Security
WiFiMon respects the privacy of data. All the traffic in WiFiMon is encrypted with SSL/TLS certificates. Furthermore, sensitive data such as mobile users IP or MAC address is hash-ed before stored in the ELK Cluster. The correlation procedure mentioned at WiFiMon Agent above, is performed over the hash-ed data.
WiFiMon Measurement Workflow
WiFiMon
Visualisation by web UI Kibana, the Network Administrator Frontend: Raw and reference data from the Elastic cluster are accessible through a network admin web UI that allows data querying; it is the looking glass of wireless performance on the network, which allows customised status checks of the wireless network. This architectural block projects the collected data and allows real-time visualisation options, such as: collected data for a specific time period, collected data for a specific AP, min-max-mean values of download/upload/latency measurements, anomalies of measurements on time, location, AP, top/bottom five performing locations, and others. The Kibana web UI respects the privacy of end-users by not exposing their personal identifiable information such as their IP address and/or MAC address.
...
Test Server (WTS)
The WiFiMon test server includes configured templates from NetTest, Boomerang and SpeedTests (HTML5), which provide information about RTTs and ping (ICMP) network relevant data like bandwidth and latency (represented in Figure 2B as lines marked as 2.1, 2.2, and 2.5). A standard image size is downloaded for NetTest, and for Boomerang it can be customisable to a maximum of 5 MByte size. The location of the WTS and its topological distance from the monitored WiFi network has a major impact on the crowdsourced measurement accuracy and reliability. You can read more about this here.
| Captioned Item | ||||||
|---|---|---|---|---|---|---|
| ||||||
 |
WiFiMon Software Probe (WSP)
The WiFiMon Software Probe takes the form of JavaScript integrated in often visited web pages, Figure 3. It includes the domain of the WiFiMon agent (e.g., wifimon.switch.ch), the listening agent port (8443) and the image location/path (e.g. https://eipa19.eipa.ttu.ee/wifimon/images/) as well as the cookie expiration time. The default value of this cookie is 1.5 min (currently) if it is not explicitly set in the WiFiMon test server.
WiFiMon Hardware Probe (WHP)
The WiFiMon hardware probes are set up on small form factor devices - such as the Raspberry Pi. The HW probe may be viewed as an end user logged in to the eduroam wireless network, but monitoring continuously from a fixed point. It measures bandwidth, latency, the average values of bit rate, the signal level, the link quality and the transmission power. The WiFiMon team recommends to set up a WHP on Raspberry Pi’s v3 Model B+ or v4
WiFiMon Analysis Server (WAS)
The Analysis Server includes the WiFiMon agent, Elasticsearch, Logstash and the WiFiMon UI with Kibana for customising reports and their visualisation. The WiFiMon agent checks the subnet (2.3, 2.4) of a monitored device visiting the website that includes the JavaScript lines (see Fig. B) and downloads available images from the WiFiMon Test server (2.5). Network performance metrics are calculated and streamed from the end user to the WiFiMon Analysis Server (2.6). Furthermore, the WAS correlates performance data with client IPs and AP-IDs (2.7/2.8).
Filebeat and RADIUS Server
By monitoring the log files defined in advance, Filebeat collects the logs that satisfy specific requirements and sends them to Elasticsearch or Logstash for further indexing. In WiFiMon, it is installed as an agent on the RADIUS server and forwards RADIUS logs. For details about Filebeat see footnote 7.
Logstash and Elasticsearch
In WiFiMon, Logstash is used to take data from the RADIUS server, transform it and send it to the predefined database for further processing. For details about Logstash see footnote 6.