Aggregating information with Huginn

Smart Collection

Getting Huginn

Once the database is running, run the following commands as the huginn user to load the source code for the software into the matching home directory:

cd /home/huginn
sudo -u huginn -H git clone https://github.com/cantino/huginn.git -b master huginn

The Huginn developers do not release stable versions; they only provide the current source code on GitHub (in the master branch). The second command therefore always automatically downloads the current Huginn version.

No fewer than two sample configuration files included with the distribution simplify the setup, which you simply copy to the appropriate location. Listing 4 displays the required commands and creates some subdirectories needed by Huginn – including directories for the logs – and then modifies the access permissions. The command

sudo -u huginn -H bundle install --deployment --without development test

finally installs the additional packages required for Ruby (the Gems).

Listing 4

Creating Subdirectories for Huginn

cd huginn
sudo -u huginn -H cp .env.example .env
sudo -u huginn -H cp config/unicorn.rb.example config/unicorn.rb
sudo -u huginn mkdir -p log tmp/pids tmp/sockets
sudo chown -R huginn log/ tmp/
sudo chmod -R u+rwX,go-w log/ tmp/
sudo chmod -R u+rwX,go-w log/
sudo chmod -R u+rwX tmp/
sudo -u huginn -H chmod o-rwx .env

Well Met

After completing the installation marathon, it's time to configure Huginn. To begin, determine the MySQL version number by typing mysql --version. Note whether the number displayed here is greater than or equal to 5.5.3; then, call

sudo -u huginn -H rake <secret>

This command returns a long cryptic string that is copied to the clipboard for use later. Now, working as the huginn user, open the .env configuration file in a text editor:

sudo -u huginn -H nano.env

At the start of the file, replace REPLACE_ME_NOW! with the cryptic string mentioned above. In the Database Setup section, replace huginn_development with huginn_production:

DATABASE_NAME=huginn_production

Additionally, the Huginn operator needs to replace root with huginn:

DATABASE_USERNAME=huginn

To the right of DATABASE_PASSWORD=, type the password in quotes used by the Huginn account to log in to MySQL (i.e., <secret> in this example).

If the MySQL version you are using is version 5.5.3 or newer, then replace utf8 with utf8mb4:

DATABASE_ENCODING=utf8mb4

The results should look like Figure 1. At the bottom of the config file, remove the leading hash (#) in the line:

# RAILS_ENV=production

Figure 1: These settings give Huginn access to the huginn_production database.

For Huginn to send email, you need to store the access credentials for an SMTP server: Search for the Email Configuration section in the lower part of the configuration files. Enter the domain name of an SMTP server after SMTP_SERVER= and the port after SMTP_PORT=.

If Huginn needs a username and password to log in to the SMTP server, enter these after SMTP_USER_NAME= and SMTP_PASSWORD=. If necessary, adjust the authentication method after SMTP_AUTHENTICATION= and the encryption after SMTP_ENABLE_STARTTLS_AUTO=. Comments describing the settings can be helpful. Finally, add the sender address after EMAIL_FROM_ADRESS=.

In Nano, the keyboard shortcut Ctrl+O saves the changes, and Ctrl+X takes you back to the command line. The next step is to set up the database with the commands from Listing 5. The first line creates the database, and the second line updates it. So that you can immediately log into Huginn in the web interface, the third command sets up an admin account with the password supersecret. The final command in Listing 5 prepares all the files required by the web interface, including the JavaScript code.

Listing 5

Setting Up the Database

sudo -u huginn -H bundle exec rake db:create RAILS_ENV=production
sudo -u huginn -H bundle exec rake db:migrate RAILS_ENV=production
sudo -u huginn -H bundle exec rake db:seed RAILS_ENV=production SEED_USERNAME=admin SEED_PASSWORD=supersecret
sudo -u huginn -H bundle exec rake assets:precompile RAILS_ENV=production

If the first command prompts an error message or requires the password for the root MySQL user, the access credentials in the .env configuration file are incorrect. In this case, check whether the correct password typed after DATABASE_PASSWORD= is in quotes.

Init Scripts

The next step is to create a number of init scripts. To begin, type

sudo -u huginn -H nano Procfile

to create the configuration file, or Procfile. Comment out the following two lines in the file, by adding hash signs at the start:

# web: bundle exec rails server -p ${PORT-3000} -b ${IP-0.0.0.0}
# jobs: bundle exec rails runner bin/threaded.rb

Conversely, remove the hash signs from the following two lines in the PRODUCTION section:

web: bundle exec unicorn -c config/unicorn.rb
jobs: bundle exec rails runner bin/threaded.rb

After saving the files, generate the scripts, set up log rotation, and check whether Huginn is running:

sudo bundle exec rake production:export
sudo cp deployment/logrotate/huginn /etc/logrotate.d/huginn
sudo bundle exec rake production:status

If you want to change one of the Huginn configuration files – .env, unicorn.rb, or Procfile – you need to regenerate the init scripts with:

sudo bundle exec rake production:export

Nginx delivers the web interface, and Huginn at least comes with a few prebuilt configuration files, which you only need to sign for the web server. If Huginn is the only website served up by Nginx, the fourth command disables the default Nginx page (Listing 6).

Listing 6

Installing and Configuring Nginx

sudo apt install nginx
sudo cp deployment/nginx/huginn /etc/nginx/sites-available/huginn
sudo ln -s /etc/nginx/sites-available/huginn /etc/nginx/sites-enabled/huginn
sudo rm /etc/nginx/sites-enabled/default

Just one small setting is missing in the Nginx configuration file. In Nano, open the file

sudo nano /etc/nginx/sites-available/huginn

and replace the YOUR_SERVER_FQDN placeholder with the domain name of the host running Huginn. When in doubt, or on a test system, this is usually localhost. In any case, make sure you close the line with a semicolon. After saving, entering