Introduction

To achieve its purpose, correlating user information with network performance data, WiFiMon needs RADIUS and/or DHCP logs to be streamed in an Elasticsearch structure.

The sources generating log files are a FreeRadius and a DHCP server where Filebeat was installed as an agent. Thus the data flow starts with Filebeat collecting log events and forwarding them to Logstash. At Logstash, logs are filtered/enriched according to the needs of WiFiMon, before sending them toward Elasticsearch nodes in the cluster.

Package Installation

The filebeat package was installed in the DHCP and the FreeRadius server which implements the eduroam Service Provider. For more information see Repositories for APT and YUM.

...

All of the following commands should be executed as "root".

Filebeat Configuration

Filebeat monitors log files for new content, collects log events, and forwards them to Elasticsearch, either directly or via Logstash. In Filebeat terms one speaks about a) the input which looks in the configured log data locations, b) the harvester which reads a single log for new content and sends new log data to libbeat, and c) the output which aggregates and sends data to the configured output. For more information see Filebeat overview.

The configuration of Filebeat is done by editing the /etc/filebeat/filebeat.yml file. Filebeat will be configured to forward the data toward Logstash.

RADIUS Server

The following is the Filebeat configuration on the RADIUS server that forwards data to Logstash:

...

The important settings here are the multiline.* ones which manage multiline formatted logs. The .pattern matches lines starting with white-space. The .negate and .match work together, and combined as false and after make consecutive lines that match the pattern to be appended to the previous line that doesn't match it. This makes all the lines starting with white-space to be appended to the line that hold the date, actually the first line in the radius_sample_logs. For more information see Manage multiline messages.

DHCP Server

The following is the Filebeat configuration on the DHCP server that forwards data to Logstash:

...

The above command loads the template from FQDN-elastic.example.org node where elasticsearch is installed. Detailed information is written in the Filebeat log file.

Log Format

Below are the sample log files used in tests. It's about a log event of a user interacting with the Eduroam Service Provider and another one interacting with the DHCP server. 

...

Jun 18 19:15:20 centos dhcpd[11223]: DHCPREQUEST for 192.168.1.200 from a4:c4:94:cd:35:70 (galliumos) via wlp6s0
Jun 18 19:15:20 centos dhcpd[11223]: DHCPACK on 192.168.1.200 to a4:c4:94:cd:35:70 (galliumos) via wlp6s0

References

The following links were very useful while writing this material and performing the tests mentioned in it.

...