Mix It Up

As an administrator, you will be familiar with the need for automation and have probably already automated updates and backups, creating new users, distributing software, and scaling your infrastructure. Shuffle [1] gives you an automation platform ideal for linking the REST APIs of popular security tools for automation with a view to security orchestration, automation, and response (SOAR).

Shuffle fetches the input from your monitoring tools (e.g., an intrusion detection system) and passes this input on to any number of other tools for further action, such as to your network management tool to isolate an affected host. Ultimately, the faster your response, the more difficult you make it for attackers to navigate your infrastructure successfully.

Installing Shuffle

Even during installation, you can benefit from the advantages of automation because the Shuffle developers give you a ready-made configuration for Docker Compose. To load the Git project and prepare to launch the tool, use the commands:

git clone https://github.com/Shuffle/Shuffle
cd Shuffle
sudo install -d -m 0755 -o 1000 -g 1000 shuffle-database

Before you can launch Shuffle, you need to configure the settings for your instance in the .env file. What you definitely have to edit is the specifications for SHUFFLE_DEFAULT_USERNAME and SHUFFLE_DEFAULT_PASSWORD, where you save the username and password for your initial admin user. You can also assign an API key directly in the next line to access Shuffle with the REST API. If you want to run the tool behind a proxy, do not forget to specify the proxy, too. Take a quick look at the other settings and adjust them to your environment, if needed. To call Shuffle, use the command:

docker compose up -d

Docker then loads the required images from the registries and creates the containers. After a short time, you can access the web interface on https://localhost:3443 – unfortunately, you have to accept the certificate warning first – and log in with the previously assigned login data. The next task is to update the available apps in the Apps menu. In the background, the software already starts providing the available apps, which noticeably increases the load on your computer and the required storage space.

Automating Processes

Shuffle lets you define processes in the Workflows menu to map out your automation preferences. You can launch various use cases, which Shuffle provides directly. The use cases are assigned to different categories: Collect, Enrich, Detect, Respond, and Verify. Many of these use cases require other services on the network.

You will find apps to match the supported services in the selection list. For example, you can analyze email in your inbox, download attachments and check them with Yara rules, and deposit information in The Hive for downstream analysis. Many good examples already exist to integrate Shuffle with your infrastructure.

To give you an initial insight into the basic functionality of Shuffle, I'll create a workflow. The idea is to analyze the IT-Administrator magazine RSS feed and save the information if the security keyword appears in the feed. To begin, you need to click on Workflows at top left, click the + symbol below the use cases, enter a name and description, and click Submit . You are now taken to the Edit view. First remove all the items by mousing over them and clicking on the trash bin.

Next, search for RSS in the apps overview on the left side and drag the icon into the editing area. Assign a name such as IT-Administrator_RSS and add the URL https://www.it-administrator.de/rss.xml under Parameters . You can then save and launch the workflow at the bottom of the window. The Results field appears on the right side, and you can see the information available in JSON.

Filtering Data

To filter this RSS data, look for the Shuffle Tools entry (a wrapper for various text editing functions) in the Apps and drag it into the editing area. This should automatically create a connection from the RSS app to this element. If this is not the case, drag the blue dot at the top of the RSS reader to Shuffle Tools and create the connection. Now change the Name of the Shuffle tool to Filter_Security .

Now you can change the functions the tools perform by selecting the Filter list Action, and in the Parameters input mask that now appears, use the output from the RSS app as the input list by clicking on the "+" symbol and mousing over the IT-Administrator_RSS entry; you can now see sample data from the last call in the area that opens (Figure 1). Because you need to select the list, click the list item below entries .

Figure 1: Selecting variables for filtering.

Populate the Field parameter with the value summary (i.e., the summary of an article in RSS). Select contains for Check and type Security in the Value field. After saving again and running the workflow, you will see two results areas. At the bottom you will now find a JSON section that divides the RSS reader entries into valid and invalid categories. If you don't see an entry in the valid list, no entry currently exists with the term Security in the summary. Feel free to try other terms here.

Of course, you now want to process the filtered entries. In principle, entries could now be created in a ticket system such as Zammad, email could be sent, messages could be sent in Slack, and so on. You would then specify the URL in the app parameters, define the matching function, and save the login credentials or an API key with the required permissions. To keep the example here as simple as possible, just write the matches to a file.

To do this, again select Shuffle Tools on the left and drag a connection from Filter_Security to this element. Change the name to something like Write_to_file and select Create file as the Find Action . In the parameters that now appear, enter a file name and search for the list in Data with the + symbol and the Filter_Security entry below the valid entry. After saving, you can run the workflow again and read out the file name of the file in the output. Because the file has now been created in the back-end container, you can find the file there. To access the container namespace in the folder, use the docker-compose.yml file and the command:

docker compose exec shuffle-backend /bin/sh

Now navigate through the folder structure under /shuffle-files/, and you will find the corresponding file on the lowest level. The cat command lets you output the contents of the file and makes sure that only the filtered entries are present.

Of course, Shuffle is not designed to trigger processes manually like this once only, but to start them regularly or respond to external triggers. To do this, you can define a scheduler in your workflow or create a REST API URL that you can use to trigger execution from the outside. The example gives you a simple configuration from which you can work out how to set up more complex configurations in Shuffle. Take a look at the apps on offer and think about where you could benefit from further automation in your daily work routine.