Safely Behind Bars

Locking a Process

In FreeBSD, each process is represented by a C struct (struct proc), which is described in /usr/include/sys/proc.h. It contains a pointer that points to the prison structure. The p_flag field with the value P_JAILED shows the process manager that a process belonging to the structure must be run in a jail.

The high degree of protection has its roots in the way the FreeBSD kernel process manager handles the C struct proc. Once a process has been assigned compute time by the time-slice procedure, proc->prison->p_flag is checked to see whether the process belongs to a jail. For processes that were started within a jail, this flag is set unconditionally.

Many other kernel services decide on access on the basis of these flags and on whether and how resources may be accessed. An excerpt from the source code illustrates the sequence (Listing 2). Figure 1 schematically represents the communication pathways.

01 int jail(struct thread *td,
02   struct jail_args *uap)
03 {
04   struct prison *pr, *tpr;
05   struct jail j;
06   struct jail_attach_args jaa;
07   [...]
08   error = copyin(uap->jail, &j,
09     sizeof(j));
10   [...]
11   MALLOC(pr, struct prison *,
12     sizeof(*pr),
13     M_PRISON, M_WAITOK | M_ZERO);
14   [...]
15   error = copyinstr(j.path,
16     &pr->pr_path, sizeof(pr->pr_path), 0);
17   [...]
18   error = copyinstr(j.hostname,
19     &pr->pr_host, sizeof(pr->pr_host), 0);
20   [...]
21   pr->pr_ip = j.ip_number;
22   pr->pr_linux = NULL;
23   pr->pr_securelevel = securelevel;
24   [...]
25   error = jail_attach(td, &jaa);
26   [...]
27   FREE(pr, M_PRISON);
28   [...]
29   return (error);
30 }
31
32 int jail_attach(struct thread *td,
33   struct jail_attach_args *uap)
34 {
35   struct proc *p;
36   struct prison *pr;
37
38   [...]
39   p = td->td_proc;
40   [...]
41   error = change_dir(pr->pr_root, td));
42   [...]
43   change_root(pr->pr_root, td);
44   [...]
45   return (error);
46 }

Figure 1: Communication between userland and the kernel on creating a jail.

Installation, Configuration, and Updates

Installing jails is not difficult, but the configuration requires care. After all, it's all about security. Commands can only be executed by the root user.

To set up a jail, you need at least 150MB of free disk space – as you add utilities, you will obviously need more. As an example, I'll look at a jail that houses web services. You need approximately 4GB for this setup, which makes it possible to run a FAMP server (FreeBSD-Apache-MySQL-PHP) or a content management system (CMS). Furthermore, the operating system and kernel source code must be installed. The approach is described in detail in the manual [1]. To create a jail, complete the steps shown in Listing 3.

01 # mkdir -p /jail/www
02 # cd /usr/src
03 # make world DESTDIR=/jail/www
04 # cd etc
05 # make distribution DESTDIR=/jail/www
06 # mount_devfs devfs /jail/www/dev
07 # cd /jail/www
08 # ln -sf dev/null kernel

After everything is compiled and installed, the jail can access devices via the device filesystem (devfs). This mount must complete before the jail starts. Because some programs expect a /kernel file, you can simulate this with a symbolic link. In fact, the kernel on the host system is responsible for all jails.

Before the jail is launched for the first time, you need to check some of the utilities on the host to see whether they are bound to the host IP address. This is done with the simple command:

# sockstat | grep "\*:[0-9]"

The description for rc.conf(5) lists the services that need to be bound to a fixed IP address via parameters. One example is the inetd service:

inetd_flags="-wW -a <IP_address> <host_system>"

The Sendmail service on the host system should be started so that it only listens on the localhost address (sendmail_enable= "NO"). To complete the installation of the jail, you can now configure the network interface on the host system.

For a routing-enabled jail, run the following command to assign an IP address,

# ifconfig netif0 inet alias  <IP_address jail>/32

where netif0 is the network interface. You should add this to the /etc/rc.conf file on the host system:

ifconfig_netif0_alias0="inet <IP_address jail> netmask255.255.255.255"

In contrast, an internal jail is always assigned the IP address of the loopback interface,

# ifconfig lo0 inet alias 127.0.0.1/32

where lo0 is the loopback interface. Again, you should add this to the /etc/rc.conf file on the host system:

ifconfig_lo0_alias0=  "inet  127.0.0.1 netmask 255.255.255.255"

For the following steps, it doesn't matter whether you are setting up an internal or a routing-enabled jail.

After completing the installation to this point, manually launch the jail with the hostname www.homenet.net , an IP address of 192.168.1.200, and the shell /bin/sh for the configuration:

# jail /jail/www www.homenet.net 192.168.1.200 /bin/sh

From now on, the administrator is inside the jail. First, set the password for the root user. You will also want to create a user who belongs to the wheel group, like root, for logging in later via SSH.

To avoid warnings about a non-existing filesystem table, create an empty /etc/fstab. The /etc/rc.conf file should now look like Listing 4.

01 rpcbind_enable="NO"
02 network_interfaces=""
03 hostname="www.homenet.net"
04 sshd_enable="YES"
05 sshd_flags="-p <Port Jail>"
06 sendmail_enable="NO"
07 syslogd_enable="YES"
08 syslogd_program="/usr/sbin/syslogd"
09 syslogd_flags="-ss"
10 defaultrouter="<IP Address Router>"

Additionally, you need to change the configuration for the SSH daemon in the /etc/ssh/sshd_config file

ListenAddress <IP_address jail>

and add the domain name server(s) to the /etc/resolv.conf file:

nameserver <IP_address DNS>

The syslogd daemon creates logfiles in a jail just as on the host system. You would not want to log in to every single jail to check what is happening inside. It's easier to redirect the syslogd output to the host. In the system configuration for the jail (/etc/rc.conf), you enable the daemon with syslogd_enable="YES". In /etc/syslog.conf, you can enter:

*.* @<address of Syslog host>

and then restart the daemon by typing /etc/rc.d/syslogd restart. This step completes the configuration within the jail, and you can get out of jail free by typing exit. For a final test, start the jail in the same way it will start later when you boot the system

# jail /jail/www www.homenet.net 192.168.1.200 /bin/sh /etc/rc

To log in to the jail, type:

ssh root@www.homenet.net -p<Port Jail>

If everything was configured correctly, the jail prompt should appear.

On the host system, you still need to complete some final configuration steps. Because the jail will be started the next time the system boots, you need to add the lines shown in Listing 5 to /etc/rc.conf.

01 jail_enable="YES" jail_list="www" jail_www_rootdir="/jails/www"
02 jail_www_hostname="<hostname jail>" jail_www_ip="<IP address jail>"
03 jail_www_exec="/bin/sh /etc/rc" jail_www_devfs_enable="YES"
04 jail_www_devfs_ruleset="devfsrules_jail"

The jail_enable="YES" entry must be set to start a jail. If you want to launch multiple jails on the host system, add them to a space-separated list under the keyword jail_list. They are then launched in the specified order. The last line is interesting; it defines the ruleset for the device filesystem service.

For everything to work correctly, you need to add the contents of /etc/defaults/devfs.rules to the existing configuration in /etc/devfs.rules. The other details are specific to each individual jail. However, you should note that the name specified in jail_list must match the name used elsewhere, as you can see in the example.

You also need to replace or add the details for the syslog daemon in the /etc/rc.conf configuration:

syslogd_flags=  "-a <network IP address>/24 -4-b <host_address>"

Then, restart the syslog daemon.

After entering all the parameters in /etc/rc.conf, you can start the jail:

# /etc/rc.d/jail start <name of jail>

To stop a jail, type stop<name of jail>. Omit the name to stop all jails.

Software can be installed in the same way as on the host system – either in the form of binaries using pkg_add -r <package_name> or via the ports. However, this approach requires the ports tree from the host system to be mounted in the jail, and the /usr/ports directory must exist in the jail. The nullfs filesystem exists for this purpose. On the host system, run

# mount_nullfs -o ro /usr/ports /jail/www/usr/ports

and add the following entry to /etc/fstab on the host:

/jails/www/usr/ports /usr/ports  nullfs rw 0 0

You also need to make two changes to /etc/make.conf in the jail:

WRKDIRPREFIX=/tmp
DISTDIR=/tmp/distfiles

If you update the host system, you also need to update the jail. To do so, change to the /usr/src directory as the root user. Because make buildworld has typically already been run on the host, it is no longer necessary for the jail. You just need:

# cd /usr/src
# mergemaster -p -D <path to jail>
# make installworld DESTDIR= <path to jail>
# mergemaster -D <path to jail>

This completes the jail update.

Sometimes, it is important to see whether processes are actually running in the jail and – if so – which ones. The ps -ax command can help you here. The following excerpt from the process list shows which processes are running in a jail. You can see this by the additional J in the status (STAT) column.

# ps -ax
708 ?? SsJ 0:00,42 /usr/sbin/syslogd ...
722 ?? SsJ 0:17,19 /usr/sbin/named ...
774 ?? SsJ 0:00,12 /usr/sbin/sshd ...
781 ?? IsJ 0:00,87 /usr/sbin/cron -s
...

The jls command lets you view all the currently active jails, as shown in Listing 6.

# jls
JID IP Address    Hostname            Path
1   192.168.1.201 dns.domain.tld      /home/jail/dns
2   192.168.1.202 www.domain.tld      /home/jail/www
3   192.168.1.203 xwindow.domain.tld  /home/jail/xwindow

Jails, Simplified

An alternative and much easier way of creating jails on a host relies on the ezjail script collection. It is located in the sysutils section and can be quickly installed using portinstall or pkg_add -r. The following steps are performed as root. In the first step, you create a basic jail, which acts as a template for all further jails. To do this, run:

# ezjail-admin update -pP

This process takes about two to five hours, depending on the performance of the system, because running make world recompiles everything. Using the -i parameter instead of -pP works around this procedure and only runs a make installworld. For the next steps, again note that the difference between internal and routing-enabled jails lies only in the IP address assignments. The next step creates a jail:

# ezjail-admin create <jail_name> <IP_address>

As described previously, you need to enter the appropriate parameters for the network interface and the jails in /etc/rc.conf. Incidentally, there is no difference. Next, enter:

# Start jails with ezjail
ezjail_enable="YES"

Now you can start the jails using

/usr/local/etc/rc.d/ezjail.sh start

The alternative is ezjail-admin start.

The update process is a breeze with ezjail. A simple call to ezjail-admin update -i is all it takes to update the jail. However, the applications in the jail must be updated in a further step.

As experienced administrators will quickly see, the ezjail script collection takes much of the work off your hands. You no longer have to enter the individual make commands and run them.