Collector

Many different tools on the market help enterprise security teams monitor security-related log and network data, with a view to detecting and analyzing acute threats and attacks on their infrastructures. Back in 2008, the open source Security Onion [1] project was launched with the aim of bundling open and free software to analyze threats, establish security monitoring in the sense of an intrusion detection system (IDS), and support central log management on the corporate network.

The idea behind Security Onion was to provide a Linux-based operating system that would include a full set of useful tools and give users a suitable environment for their daily work. Security Onion was initially based on Ubuntu. In version 2, though, the installation of the individual tools was shifted to containers so that Security Onion now runs on basically any distribution that supports Docker. That said, it officially only supports the Ubuntu and CentOS distributions. For this article, I use the downloadable ISO file, but you can always try out one of the other variants, such as one of the prebuilt images available for AWS or Azure.

Intrusion Detection

The motivation for using Security Onion is intrusion detection. You need to distinguish between host-based IDS (HIDS) and network-based IDS (NIDS). Both methods have their advantages and disadvantages in terms of possible monitoring points. On a host, you will mainly check the running processes, settings, registry entries, files, and users, whereas checks on the network let you monitor communications, communication partners, content, and metadata.

Although you can access the data on the network centrally (e.g., at the monitoring or mirror port of a switch) without having to configure the monitored computers yourself, you need to find a manageable way of transporting the data for analysis from the host systems to a central log server that aggregates the data from all of your computers for monitoring.

You have two main approaches to detecting signs of unfriendly activities in what can be very large volumes of data: searching for known patterns (signature-based detection) and searching for unknown activities that deviate from the typical use of your systems (anomaly detection). Both approaches have their strengths and weaknesses in terms of accurate detection and event coverage, which means your best option is to combine the two and make sure you don't miss out on any anomalies in your infrastructure. To further support you in your analysis, Security Onion comes with in-house functions on top of established tools.

Before you get started with Security Onion, I'll first look at the included toolkit and define the tasks these tools have in the Security Onion environment. Depending on the specific purpose and size of your organization, you can either run the instances on a single computer or divide them up across multiple systems on your network for better scaling. I look at the basic approaches to generating data and the analytics tools with which you interact.

Network Data

Security Onion comes with five tools for collecting network data and for initial analysis: Stenographer, Suricata, Zeek, Strelka, and OpenCanary.

Google developed Stenographer for bulk logging of network traffic, storing the results on disk, managing the space available for the data, and quickly and selectively accessing individual fragments of logged communication. Therefore, the developers had to shrink these fragments to fractions of a percent and exclude complex packet processing, such as the analysis of entire connections. As a query language, Stenographer uses some of the Berkley Packet Filter (BPF) syntax expressions that you might be familiar with from tcpdump or sniffing (not analyzing) traffic with Wireshark.

Suricata is a complete NIDS and intrusion prevention system (IPS), which means you can use it offline (i.e., as a monitor on a switch) as well as online (e.g., on a firewall system). Much like the slightly more widespread Snort, Suricata is rules based; that is, you use signatures of malicious behavior that you apply to the metadata or content of the monitored communications. Suricata allows the use of Snort rules in part, but only if they were created for the older 2.x version. You can also use the IPS function for rules in the context of Security Onion, but what you will mainly be interested in here is the log data generated for comprehensive monitoring.

Zeek was initially developed under the name Bro; it offers an approach to monitoring large volumes of data and performing complex analyses. To this end, Zeek comes with its own scripting language, along with policy scripts, which you can use to describe and delimit network traffic for analysis. Zeek writes the metadata for monitored communications to a log that can be processed downstream for exactly the kind of thing that Security Onion is looking to do. Moreover, Zeek supports separate storage of files crossing the network for further analysis.

Strelka adds a tool to Security Onion for analyzing files and creating extensive metadata for these files. Similar to Bro, the goal primarily is to provide further information and pass it on to analysis systems, which relies on the log data generated by Strelka. In the use case here, it works with the files that Zeek extracts from the network traffic. The two tools do not necessarily have to be on the same system – the Strelka developers even explicitly recommend separating the two systems for the best possible performance, all the more so because Zeek performs time-critical activities, whereas Strelka will certainly take a bit more time with its CPU-intensive analysis of the files. The tool offers north of 40 different scanners that can be applied to more than 60 different file types.

OpenCanary is the intrusion detection honeypot in Security Onion. This low-interaction honeypot does not offer much in the way of full-fledged services, but it is mainly concerned with an attacker's first attempts to contact a system. An attacker who lands in this honeypot will notice what has happened pretty quickly and refrain, one hopes, from further attempts. According to the authors of OpenCanary, the attacker has already connected to the honeypot by this time, which is precisely the intention. The list of supported protocols is quite extensive and includes SSH, FTP, HTTP (and HTTP proxy), MySQL, Telnet, and Redis. You can freely configure different details of the services, like the string with the SSH version or the used ports. Each connection attempt is documented in the log data. Because the honeypot's services have no legitimate use, every single request is suspicious.

Entering Host Data

Collecting network data is often very simple, but the requirements for installing a HIDS or similar sensor technology are sometimes more complex. Security Onion offers three tools: Wazuh, osquery, and Beats.

Wazuh is a fork of OSSEC – a well-known HIDS – that began life back in 2015, mainly with the aim of improving the development of the tool. What the developers above all criticized was the lack of ongoing development of OSSEC and the hesitance to adopt new features. In fact, the current OSSEC version from 2020 is already more than two years old. Wazuh, on the other hand, has evolved significantly and now boasts an active community and broader support for current developments. Wazuh offers a client-server architecture so that events can be evaluated centrally. You need to install Wazuh agents on each host. In addition to Windows, Linux, and macOS, Unix systems such as Solaris, AIX, and HP-UX are also supported.

On the server side, you can use a cluster setup and run the Wazuh master node separately from any number of worker nodes for optimal load balancing. Analysts also have access to a dashboard with an extensive search function in Wazuh. In addition to classic log data analysis, rootkit and vulnerability detection, and file integrity checking, you can also use Wazuh to inventory your clients. Log data is collected from the computers, and depending on the configuration, initial events or alarms are generated and logged in Security Onion for further evaluation.

The osquery tool comes from Meta (formerly Facebook). It allows continuous or case-based querying of system properties. The tool abstracts the underlying systems and makes them available in a form much like a relational database. You write your queries for osquery in SQL, where you can currently access 273 different tables. The advantage is that you can simultaneously search for properties and indicators of compromise (IoCs) on all connected systems. Osquery offers its own service for continuous monitoring, providing query packs with preconfigured queries created in JSON (e.g., to detect rootkits). With your own extensions, you can add more tables for your queries to the schema as needed.

Emerging from the familiar Elastic Stack, Beats is available in Security Onion as Winlogbeat for Windows-specific logs and as the cross-platform Filebeat for monitoring and transferring various log data to a Logstash server. Both tools offer different fields for selecting the relevant data. The Logstash server is also provided, of course, as a component of the Security Onion installation (in standalone or cluster variants). Filebeat offers different modules that either already fully support existing logfiles (e.g., from NGINX and Apache as well as AWS and Google Cloud) or that can be easily adapted to match logfiles. Filebeat is also responsible for transferring the data of the other tools, which deliver the results of their evaluation or alarms and events in logfiles.

Filebeat provides an easy way of transferring further log data from the monitored systems to the central Logstash instance; the data can include classic Linux syslog or evaluations from Microsoft tools belonging to the Sysinternals Suite, such as Autoruns or Sysmon.