OSSEC on Various Platforms

OSSEC works across all platforms and can be installed on Linux, Windows, and macOS, which makes the environment a useful choice as an additional security tool on hybrid networks. Linux distributions such as Debian, Ubuntu, CentOS, Red Hat Enterprise Linux, and Fedora all support OSSEC. You can use the package manager for the install or compile from the source code.

On Windows, you must make sure that the software supports your version and that all the required updates are in place before downloading and installing the OSSEC agent for Windows from the official website. You cannot install the OSSEC server itself on Windows unless you use the Windows Subsystem for Linux (WSL); obviously, this is not ideal. For macOS, the OSSEC install works in a similar way to Linux. You can download the source code from the official website and compile the program or use a package manager such as Homebrew.

When installing OSSEC, you need to coordinate the components, which include a central server, the OSSEC manager, and the OSSEC agents on the systems to be monitored. The manager collects and analyzes data from the agents and runs appropriate actions when the tool detects threats or security breaches. Make sure you choose the right configuration for your requirements and set up the communication between manager and agent correctly. Adjustments are made in the configuration file.

The OSSEC server can also work without agents, but then it does not read all the information. Agents can also be deployed on Windows servers to monitor the registry for changes. The server can also read data from system logfiles and aggregates all this information to carry out actions or deliver notifications in the event of trouble.

OSSEC can be used in the cloud, as well. In cloud environments, you can install the application on both virtual machines and cloud-based servers. Parallel bookable services are available directly in the cloud for Microsoft Azure, Amazon Web Services (AWS), and Google Cloud Platform (GCP). You need to install and configure the OSSEC manager on a central instance or a dedicated server then set up the agents on the individual virtual machines or cloud servers that you want to monitor. Here, too, the agents communicate with the central manager and send information about the system activities, which the manager analyzes before triggering appropriate actions.

If you use OSSEC in the cloud, it is important to safeguard the communication path between the agents and the manager. You need to use protected connections such as encrypted virtual private network (VPN) tunnels or private network connections to keep the communication between the systems private.

In many cases, cloud providers also offer their own security products and monitoring tools. It is advisable to use these in combination with OSSEC to maximize the security of your cloud infrastructure.

Installation and Setup

You can either install OSSEC or download it as a virtual appliance; the former option is preferable. Servers and agents can be downloaded separately and connected to an OSSEC server. In contrast to many live DVDs or simpler open source tools, OSSEC is not easy to install, set up, and operate. You need to take time to familiarize yourself fully with the product. As a general rule, it makes sense to reference the documentation [3] when you are setting up and managing the system.

In the following discussion, I concentrate on the free OSSEC version 3.7.0 install on Ubuntu 22.04. After the mandatory Linux update, install the required dependencies:

sudo apt-get update
sudo apt-get -y upgrade
sudo apt-get install -y build-essential
sudo apt-get install -y zlib1g-dev libpcre2-dev libevent-dev libssl-dev libsystemd-dev jq

The certificates and key files are then downloaded and installed and the repository is integrated:

wget http://www.ossec.net/files/OSSEC-ARCHIVE-KEY.asc
wget https://github.com/ossec/ossec-hids/releases/download/ 3.7.0/ossec-hids-3.7.0.tar.gz.asc
gpg --import OSSEC-ARCHIVE-KEY.asc

Next, download the current OSSEC version, and unzip and execute the installation script:

wget https://github.com/ossec/ossec-hids/archive/3.7.0.tar.gz
gpg --verify ossec-hids-3.7.0.tar.gz.asc 3.7.0.tar.gz
tar -zxvf 3.7.0.tar.gz && cd ossec-hids-3.7.0/
wget https://github.com/PCRE2Project/pcre2/releases/download/pcre2-10.40/pcre2-10.40.tar.gz
tar -zxvf pcre2-10.40.tar.gz -C src/external/

No errors should appear in the build process at this point.

After starting the installation script, the next step is to select the language in which you want to install OSSEC. English is preselected and I do not recommend changing this setting, because some of the translations were inaccurate when I tried them out in the lab.

Adapting the Environment

A wizard helps you customize the installation (Figure 1). The server selection installs the first OSSEC server on the network; you can connect more agents running on Linux, Windows, or macOS to the server later on. Next, select the installation directory; the default is var/ossec.

Figure 1: The basic setup of OSSEC is handled by a text-based wizard.

You can also enable email notifications now by entering an email address. In many cases, the system finds the SMTP server, but you can change the configuration at any time. You will always want to install the integrity check daemon, which is also preselected, as is the rootkit detection engine.

I also recommend adding active response as part of the setup. This function lets the system run predefined commands on the basis of incoming events. For example, you can block an IP address or block access for a specific user. In the next step, it makes sense to activate the firewall drop response, which allows OSSEC to block the host in iptables (Linux) or ipfilter (Solaris, FreeBSD, or NetBSD) if it detects an attacker. This step helps fend off brute force scans, port scans, and some other forms of attack.

You then have the option of adding IP addresses to allowlists if you do not want OSSEC to check and block communication from some IP addresses. You will also want to enable remote syslog over UDP port 514 so that the system can send logfiles to syslog servers. Once you have made your choices, the wizard installs OSSEC accordingly. At the end of the installation you will see a Configuration finished properly message.