This section describes key architectural components of the WiFiMon system:
Mobile client: A mobile client is user's device connected to the WiFi network. WiFiMon can be used in any WiFi network, but is optimised to be used in the eduroam environment, as through the analysis of the eduroam logs very detailed performance assessments can be obtained. If the eduroam is used, the mobile client will be authenticated at the home organisation (IdP) and authorised at the end-user's location (SP) on an IEEE 802.1X based WiFi network.
Data sources: The data sources (see Figure 1) include:
- Collected raw data generated by network performance tools (Boomerang, NetTest and HTML5 SpeedTest) with the use of JavaScript code embedded on chosen web pages; tests are run by hardware probes and performance data are retrieved when the hardware probes visit the web sources with the embedded code, and
- Exported raw data from data sources (collectors) like Syslog, Server Log, RADIUS Accounting, L2/L3 address binding. Wireless controllers and Access Point identifiers (AP-IDs) may provide the minimum amount of information in a smaller implementation, at the most basic level for performance verification and monitoring on a wireless network. Therefore, the data sources will vary, depending on the vendor specific tools integrated in the network
Elastic cluster: The raw data is automatically collected and fed into the WiFiMon agent, which uses Elasticsearch, a full-text, distributed NoSQL search engine. Elasticsearch uses documents rather than schema or tables (that SQL databases use), thus allowing leveraging and accessing data at very high speeds and making it appropriate for WiFiMon. In addition, the Elastic cluster uses tools such as logstash and filebeat to pipeline the raw data from the data sources simultaneously, transform these data and send it to Elasticsearch.
Data Analysis: Within the Elastic cluster, raw data is processed to data objects relevant to performance measurements. These raw data are then correlated with information from collected RADIUS logs to provide insight on the wireless network performance per client or AP. The correlated measurements are also stored in Elasticsearch.
Data Search: The Query or Report Generator is used to get specific information from the WiFiMon agent (data aggregation/analysis). Its main purpose is to have the opportunity through the web UI to search the indices that match the current index pattern by entering the search criteria in the query bar. By default, Kibana’s standard query language, which features autocomplete and a simple, easy to use syntax, can be used. When the administrator submits a search request, the histogram, documents table, and fields list are updated to reflect the search results. In the WiFiMon case, search requests could be made, for example, to retrieve information about the measurements initiated to a specific AP and within a specific time period. Other queries could be made to retrieve information about specific IP ranges, test tools etc.
Visualisation by web UI Kibana, the Network Administrator Frontend: Raw and reference data from the Elastic cluster are accessible through a network admin web UI that allows data querying; it is the looking glass of wireless performance on the network, which allows customised status checks of the wireless network. This architectural block projects the collected data and allows real-time visualisation options, such as: collected data for a specific time period, collected data for a specific AP, min-max-mean values of download/upload/latency measurements, anomalies of measurements on time, location, AP, top/bottom five performing locations, and others. The Kibana web UI respects the privacy of end-users by not exposing their personal identifiable information such as their IP address and/or MAC address.
WiFiMon Test Server (WTS)
The WiFiMon test server includes configured templates from NetTest, Boomerang and SpeedTests (HTML5), which provide information about RTTs and ping (ICMP) network relevant data like bandwidth and latency (represented in Figure 2B as lines marked as 2.1, 2.2, and 2.5). A standard image size is downloaded for NetTest, and for Boomerang it can be customisable to a maximum of 5 MByte size. The location of the WTS and its topological distance from the monitored WiFi network has a major impact on the crowdsourced measurement accuracy and reliability. You can read more about this here.
WiFiMon Software Probe (WSP)
The WiFiMon Software Probe takes the form of JavaScript integrated in often visited web pages, Figure 3. It includes the domain of the WiFiMon agent (e.g., wifimon.switch.ch), the listening agent port (8443) and the image location/path (e.g. https://eipa19.eipa.ttu.ee/wifimon/images/) as well as the cookie expiration time. The default value of this cookie is 1.5 min (currently) if it is not explicitly set in the WiFiMon test server.
WiFiMon Hardware Probe (WHP)
The WiFiMon hardware probes are set up on small form factor devices - such as the Raspberry Pi. The HW probe may be viewed as an end user logged in to the eduroam wireless network, but monitoring continuously from a fixed point. It measures bandwidth, latency, the average values of bit rate, the signal level, the link quality and the transmission power. The WiFiMon team recommends to set up a WHP on Raspberry Pi’s v3 Model B+ or v4
WiFiMon Analysis Server (WAS)
The Analysis Server includes the WiFiMon agent, Elasticsearch, Logstash and the WiFiMon UI with Kibana for customising reports and their visualisation. The WiFiMon agent checks the subnet (2.3, 2.4) of a monitored device visiting the website that includes the JavaScript lines (see Fig. B) and downloads available images from the WiFiMon Test server (2.5). Network performance metrics are calculated and streamed from the end user to the WiFiMon Analysis Server (2.6). Furthermore, the WAS correlates performance data with client IPs and AP-IDs (2.7/2.8).
Filebeat and RADIUS Server
By monitoring the log files defined in advance, Filebeat collects the logs that satisfy specific requirements and sends them to Elasticsearch or Logstash for further indexing. In WiFiMon, it is installed as an agent on the RADIUS server and forwards RADIUS logs. For details about Filebeat see footnote 7.
Logstash and Elasticsearch
In WiFiMon, Logstash is used to take data from the RADIUS server, transform it and send it to the predefined database for further processing. For details about Logstash see footnote 6.