Log Inspector

Linux has long mastered the art of log forwarding and remote logging, which are prerequisites for external log analysis. From the beginning, security was the focus: An attacker who compromises a system most likely would also try to manipulate or delete the syslog files to cover his tracks. However, if the administrator uses a loghost, the files are less likely to fall into the hands of hackers and, thus, can still be analyzed after an attack.

As the number of servers increases, so do the size of logfiles and the risk of overlooking security-relevant entries. Security information and event management (SIEM) products usually determine costs by the size of logs. The Graylog [1] open source alternative discussed in this article processes many log formats; however, if the volume exceeds 5GB per day, license fees kick in.

Why SIEM?

As soon as several servers need to be managed, generating overall statistics or detecting problems that affect multiple servers becomes more and more complex, even if all necessary information is available. Because of the sheer quantity of information from different sources, the admin has to rely on tools that allow all logs to be viewed in real time and help with the evaluation.

SIEM products and services help you detect correlations in a jumble of information by enabling:

• Access to logfiles, even without administrator rights on the production system.
• Accumulation of the logfiles of all computers in one place.
• Analysis of logs with support for correlation analysis.
• Automatic notification for rule violations.
• Reporting on networks, operating systems, databases, and applications.
• Monitoring of user behavior.

Installing and configuring Graylog is quite easy. The Java application uses resources sparingly and stores metadata in MongoDB and logs in an Elasticsearch cluster. Graylog consists of a server and a web interface that communicate via a REST interface (Figure 1).

Figure 1: Graylog comprises a web interface, a server, MongoDB, and Elasticsearch.

Installation

Prerequisites for the installation of Graylog 2.4 – in this example, under CentOS 7 – are Java version 1.8 or higher, Elasticsearch 5.x [2], and MongoDB 3.6 [3]. If not already present, installing Java (as root or using sudo) before Elasticsearch and MongoDB is recommended:

yum install java-1.8.0-openjdk-headless.x86_64

You should remain root or use sudo for the following commands, as well. To install Elasticsearch and MongoDB, create a file named elasticsearch.repo and mongodb.repo in /etc/yum.repos.d (Listings 1 and 2); then, install the RPM key and the packages for MongoDB, Elasticsearch, and Graylog (Listing 3) to set up the basic components.

[elasticsearch-5.x]
name=Elasticsearch repository for 5.x packages
baseurl=https://artifacts.elastic.co/packages/5.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

[mongodb-org-3.0]
name=MongoDB Repository
 **
baseurl=http://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/3.0/x86_64/
gpgcheck=0
enabled=1

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
 **
yum install elasticsearch
yum -y install mongodb-org
 **
rpm -Uvh https://packages.graylog2.org/repo/packages/graylog-2.4-repository_latest.rpm
yum install graylog-server

Configuration

The best place to start the configuration is with Elasticsearch. In its configuration file, you specifically need to assign the cluster.name parameter. The only configuration file for Graylog itself is server.conf, which is located in the /etc/graylog/ directory and uses ISO 8859-1/Latin-1 character encoding. This extensive file begins with the definition of the master instance and ends with the encrypted password of the Graylog root user. The most important parameters define email, TLS, and the root password.

To start Graylog at all, the parameters password_secret and root_password_sha2 must be set. The password_secret parameter should be set with a string of at least 64 characters. Graylog uses this string for salting and encoding the password. The pwgen command generates a password and encrypts with sh256sum:

pwgen -N 1 -s 100
echo -n <Password> | sha256sum

The encrypted password is then assigned to the parameter root_password_sha2. Table 1 gives a summary of the important parameters.