Encrypting Files
Encrypting your data is becoming increasingly important, but you don’t always have to use an encrypted filesystem. Sometimes just encrypting files is enough.
The revelations of Edward Snowden caused a big upsurge in the use of encryption for protecting data from inappropriate access. People are now using encrypted filesystems as well as self-encrypted hard drives (SEDs). However, not everyone is using encryption.
Recent revelations about accessing the data of individuals includes the story about how the NSA and Britain’s Government Communications Headquarters (GHCQ) supposedly gained access to SIM cards from Gemalto, allowing them to access any cell phone communications that use these cards. There is also the story about how Lenovo installed malware on its laptops that allows the software to steal web traffic using man-in-the-middle attacks.
When you use an encrypted filesystem or SEDs, all of the data is encrypted. However, if you forget the password, you lose all of the data on the filesystem or drive. It may be easier to encrypt files individually so that if you forget the password, you only lose a single file and not the entire filesystem or drive. Moreover, you might be casually copying your files to the cloud or other backup systems from your desktop, laptop, or cellphone. If you do not encrypt these files yourself, more likely than not, these files are not encrypted.
Using simple tools to encrypt files individually and then copy them to your backup is an easy process. As previously mentioned, by encrypting the files individually, if you forget the password, then theoretically you will lose only a single file (unless you use the same passphrase for all files, in which case you might lose access to all data).
Before you proceed with the rest of this article, realize that I’m not a security or cryptography expert, nor do I play one on TV. Please do your own research. Given that, in the sections below, I review a few file encryption/decryption tools and finish with some personal recommendations on using them.
GPG
To start, I’ll look at the probably the most popular encryption tool, GNU Privacy Guard (GPG). The tool has become popular because it’s fast, the encryption is very good if used correctly, the code is open-source, and it follows the OpenPGP specification that is an IETF standard. GPG was really designed as a command-line encryption tool for files but has been incorporated into email tools for encrypting email.
GPG uses a hybrid encryption approach with a combination of two approaches: symmetric-key encryption and public-key cryptography. Symmetric key encryption/decryption means that both the sender and the receiver share the same key. Typically symmetric-key encryption is used for speed and public-key cryptology is used because of easy secure key exchange.
As mentioned, GPG can be used for encrypting messages such as email. To do this, GPG uses asymmetric key-pairs that are individually generated for each user. From this key pair, you can exchange the public keys with other users using Internet key servers or something similar, allowing them to decrypt the email you have sent.
A variety of encryption options are available with GPG. By default it uses the symmetric encryption algorithm, called CAST5 that is a 128-bit symmetric-key block cipher. Other encryption algorithms available are listed below along with the public key techniques and compression algorithms.
- Public key:
- Cipher:
- Hash
- MD5
- SHA-1
- RIPEMD-160
- SHA-256
- SHA-384
- SHA-512
- SHA-224
- Compression
- ZIP
- ZLIB
- BZIP2
Note: For AES, GPG always use block sizes of 128 bits and a varying key length of 128, 192, and 256 bits.
For some cipher algorithms such as AES-256, the number indicates the length of the hash key used in the algorithm. A general rule of thumb is that the larger the hash key, the more “protected” your data will be (if your passphrase is sufficiently long). However, it also means that it takes more resources, such as CPU, memory, and time, to encrypt the file. If you want to encrypt the file and very rarely decrypt it, you might want to use an algorithm with a very long hash key. If you’re going to be decrypting the file fairly often, then you might want to try a shorter key to improve encryption/decryption time at the expense of somewhat “weaker” encryption. Ultimately the choice is yours but personally I like to encrypt my data with a very long cipher key (almost as large as I can get).
According to evil32.com, using modern GPUs, 32-bit key IDs can be decoded. They say that it only takes four seconds to generate a colliding 32-bit key ID on a GPU. In fact, they claim that they found collisions for every 32-bit key ID in the Web of Trust (WOT) strong set. Breaking the 32-bit key ID doesn’t compromise GPG’s encryption according to the site, but “… it further erodes the usability of GPG and increases the chance of human error.”
Key IDs are not typically used in encrypting data, but you should definitely be aware of them, particularly if you use GPG in every day use. Therefore the researchers highly recommend using 64-bit key IDs.
Using GPG is very easy. You begin with a file and use gpg to encrypt it using the -c option which uses a symmetric key as well as the default CAST5 cipher. The example below encrypts the text file hpc_001.html:
[laytonjb@home4 TEMP]$ ls -s total 11228 11032 Flying_Beyond_the_Stall.pdf 196 hpc_001.html [laytonjb@home4 TEMP]$ gpg -c hpc_001.html [laytonjb@home4 TEMP]$ ls -s total 11256 11032 Flying_Beyond_the_Stall.pdf 196 hpc_001.html 28 hpc_001.html.gpg
Notice that the gpg command leaves the original file in place and creates a new file with a .gpg extension. Also notice that encrypting a simple text file produced a much smaller encrypted file than the plain text original.
During the encryption I had to enter my passphrase twice. You have to remember this passphrase because without it, you cannot decrypt the file. Please remember this. The data cannot be recovered without a massive amount of CPU time to crack the encryption. This is no joke – cracking the file could potentially take years (many years). Therefore do not forget the passphrase, but also don’t write it down and leave it somewhere.
You can also compress the text file before you encrypt it as shown here:
[laytonjb@home4 TEMP]$ gzip -9 hpc_001.html [laytonjb@home4 TEMP]$ ls -s total 11084 11032 Flying_Beyond_the_Stall.pdf 28 hpc_001.html.gpg 24 hpc_001.html.gz [laytonjb@home4 TEMP]$ gpg -c hpc_001.html.gz [laytonjb@home4 TEMP]$ ls -s total 11108 11032 Flying_Beyond_the_Stall.pdf 28 hpc_001.html.gpg 24 hpc_001.html.gz 24 hpc_001.html.gz.gpg
Notice that the compressed file hpc_001.html.gz is encrypted this time. [Note: GPG typically has the option of compressing the file as well as encrypting it, but I like to keep things separate.]
To decrypt the encrypted file to another file, you just use the -d -o (decrypted file) options. The -o option directs the output to a file, and the -d option tells GPG to decrypt the file. In the example below, I decrypt the compressed file hpc_001.html.gz:
[laytonjb@home4 TEMP]$ gpg -o hpc_001.html.gz -d hpc_001.html.gz.gpg gpg: 3DES encrypted data gpg: encrypted with 1 passphrase gpg: WARNING: message was not integrity protected [laytonjb@home4 TEMP]$ ls -s total 11108 11032 Flying_Beyond_the_Stall.pdf 28 hpc_001.html.gpg 24 hpc_001.html.gz 24 hpc_001.html.gz.gpg
During the decryption I had to give the passphrase that I used to encrypt the file. Notice that the decrypted file is called hpc_001.hml.gz. (I erased the original hpc_001.html.gz before I decrypted the file.) You can check that the file is correct by uncompressing it and then looking at the first few lines, which should be text:
[laytonjb@home4 TEMP]$ gunzip hpc_001.html.gz [laytonjb@home4 TEMP]$ ls -s total 11280 11032 Flying_Beyond_the_Stall.pdf 28 hpc_001.html.gpg 196 hpc_001.html 24 hpc_001.html.gz.gpg [laytonjb@home4 TEMP]$ head -n 5 hpc_001.html HPC Storage - Getting Started with IO profiling applications
Looks like plain text to me and it matches the original file.
You can also choose a cipher other than CAST5. In the example below, the AES-256 cipher is used to encrypt the PDF file in the directory.
[laytonjb@home4 TEMP]$ ls -s total 11228 11032 Flying_Beyond_the_Stall.pdf 196 hpc_001.html [laytonjb@home4 TEMP]$ gpg -c -crypto-algo=AES256 Flying_Beyond_the_Stall.pdf gpg: WARNING: recipients (-r) given without using public key encryption [laytonjb@home4 TEMP]$ ls -s total 20940 11032 Flying_Beyond_the_Stall.pdf 196 hpc_001.html 9712 Flying_Beyond_the_Stall.pdf.gpg
The option -crypto-algo=AES256 tells GPG to use the AES-256 cipher to encrypt the file. Again, I had to enter my passphrase twice to encrypt the file.
GPG is very flexible and powerful. For example, you have options for handling keys so that you don’t have to enter a passphrase (unattended key generation), but keep in mind that these should be 64-bit and not the typical 32-bit keys. Articles on the Internet can walk you through these options and how you can use them.
ZIP
ZIP is an archive file format, something along the lines of TAR. In addition to collecting files in a single archive file as tar does, zip can also compress the resulting archive or components of the archive. It supports several compression methods including the following:
- Shrink
- Reduce (levels 1-4)
- Implode
- Deflate
- Deflate64
- bzip2
- LZMA (EFS)
- WavPack
- PPMd
According to the Wikipedia link, the most popular compression method is Deflate.
In addition to creating an archive and compression, zip is also capable of encrypting the archive. It can use AES methods, which are documented in the .zip file format specification. Also, starting in version 6.2 of the ZIP format, file name encryption was introduced so that metadata was encrypted in what is called the “Central Directory” portion of the ZIP. However, there are portions of the archive where the file names are not encrypted.
Using zip to encrypt files is very similar to using gpg, as shown in the example below:
[laytonjb@home4 TEMP]$ ls -s total 11228 11032 Flying_Beyond_the_Stall.pdf 196 hpc_001.html [laytonjb@home4 TEMP]$ zip --password MY_SECRET file.zip hpc_001.html adding: hpc_001.html (deflated 88%) [laytonjb@home4 TEMP]$ ls -s total 11252 24 file.zip 11032 Flying_Beyond_the_Stall.pdf 196 hpc_001.html
In the command line, the option --password MY_SECRET specifies the passphrase as MY_SECRET. (You can use the -P option instead of --password.) If you want to use a longer passphrase with blanks, enclose it in quotes:
[laytonjb@home4 TEMP]$ zip --password 'Help me Watson' file.zip hpc_001.html adding: hpc_001.html (deflated 88%) [laytonjb@home4 TEMP]$ ls -s total 11252 24 file.zip 11032 Flying_Beyond_the_Stall.pdf 196 hpc_001.html
However, specifying the passphrase on the command line means that it will be in the “history” of the shell. This is probably not the most secure way to encrypt files with zip. Perhaps a better way is just to use the encrypt option (-e); then, it will prompt you for the passphrase, which you have to enter twice.
[laytonjb@home4 TEMP]$ zip -r -0 -e files.zip ./ Enter password: Verify password: adding: Flying_Beyond_the_Stall.pdf (stored 0%) adding: hpc_001.html (stored 0%) [laytonjb@home4 TEMP]$ ls -s total 22456 11228 files.zip 11032 Flying_Beyond_the_Stall.pdf 196 hpc_001.html
The options used are,
- -r: recursively zip
- -0: no compression (for faster execution)
- -e: encrypt (prompts the user for a passphrase)
The command takes all of the files in the current directory and sub-directories and creates a single archive without compression. However, if you compressed the archive, during the compression, zip will post the list of files in the archive. Depending on your level of paranoia, you might not want this to happen. In that case, it might be better to use tar to create the archive and then compress and encrypt it with zip (i.e., zip -e).
7-Zip
7-Zip is an open source tool for creating archives, compressing them, and encrypting them (much like zip). It has several algorithms for data compression:
- LZMA – Default; an improved and optimized version of the LZ77 algorithm.
- LZMA2 – An improved version of LZMA.
- PPMD – Dmitry Shkarin’s PPMdH with small changes.
- PCJ – A converter for 32-bit x86 executables.
- PCJ2 – A converter for 32-bit x86 executables.
- Bzip2 – The standard BWT algorithm.
- Deflate – The standard LZ77-based algorithm.
7-Zip also supports AES-256 for encryption and can encrypt file names and directory names.
Using 7-Zip is pretty easy and is very similar to using zip. Here, I encrypt the simple text file hpc_001.html:
[laytonjb@home4 TEMP]$ ls -s total 7288 196 hpc_001.html 7092 MFS2007.pdf [laytonjb@home4 TEMP]$ 7z a -p hpc_001.html.7z hpc_001.html 7-Zip [64] 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18 p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,1 CPU) Scanning Creating archive hpc_001.html.7z Enter password (will not be echoed) : Verify password (will not be echoed) : Compressing hpc_001.html Everything is Ok [laytonjb@home4 TEMP]$ ls -s total 7308 196 hpc_001.html 20 hpc_001.html.7z 7092 MFS2007.pdf
The options I used are: a, create archive, and -p, set password. By just specifying -p, 7-Zip (p7zip, the command-line version of 7-Zip) will prompt for the passphrase so that it won’t be copied into the shell history. However, you can input the passphrase on the command line.
A key point to note is that p7zip leaves the original file in place and creates a copy with a .7z extension. This might seem subtle, but it can be important. I like leaving the original file alone because if the encryption process goes sideways, I still have it available. I also like to decrypt the file and do a diff between the original file and the decrypted file. It might seem pointless to do this, but I like to make sure that the encryption and decryption processes worked correctly, AND I remember my passphrase.
To decrypt the file, you just use the -e (extract) option:
[laytonjb@home4 TEMP]$ 7z e hpc_001.html.7z 7-Zip [64] 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18 p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,1 CPU) Processing archive: hpc_001.html.7z Enter password (will not be echoed) : Extracting hpc_001.html Everything is Ok Size: 198510 Compressed: 18945
As you can tell, p7zip gives you some detail about the decryption of the file. Also don’t forget that as part of the extraction, p7zip also uncompresses the file.
OpenSSL
SSL and its successor TLS were protocols developed to provide communication security over a network using cryptography. You are probably most familiar with the protocol in web browsers for websites beginning with https. You can take advantage of the encryption in SSL or TLS to encrypt your data.
The most common implementation of SSL is OpenSSL, an open-source community project for a full-featured toolkit implement of SSL and TLS, as well as general-purpose cryptography. It was subject of the infamous Heartbleed vulnerability that primarily affected the communication encryption aspect of OpenSSL. The cryptography library aspect of OpenSSL is still extremely useful.
OpenSSL has a number of ciphers, cryptographic hash functions, and public key encryption algorithms.
- Ciphers
- AES
- Blowfish
- Camellia
- SEED
- CAST-128
- DES
- IDEA
- RC2
- RC4
- RC5
- Triple DES
- GOST 28147-89
- Cryptographic hash functions
- MD5
- MD4
- MD2
- SHA-1
- SHA-2
- RIPEMD-160
- MDC-2
- GOST R 34.11-94
- Public-key cryptography
- RSA
- DSA
- Diffie–Hellman key exchange
- Elliptic curve
- GOST R 34.10-2001
OpenSSL really focuses on encryption and decryption and not compression. Consequently, you shouldn’t expect the encrypted file to be smaller than the original file.
Using OpenSSL requires a few more arguments than the typical encryption tool:
[laytonjb@home4 TEMP]$ ls -s total 7288 196 hpc_001.html 7092 MFS2007.pdf [laytonjb@home4 TEMP]$ openssl aes-256-cbc -salt -in hpc_001.html -out hpc_001.html.enc enter aes-256-cbc encryption password: Verifying - enter aes-256-cbc encryption password: [laytonjb@home4 TEMP]$ ls -s total 7484 196 hpc_001.html 196 hpc_001.html.enc 7092 MFS2007.pdf
The first option I use is aes-256-cbc, which tells OpenSSL to use the 256-bit key with OpenSSL, along with the AES cipher. The -in option specifies the input file, and -out specifies the output (encrypted) file.
The option -salt is added to the command line because it can be very important for improving security. Classically, a salt is a random bit of data that is used as an additional input to a one-way function that hashes the passphrase. It protects against dictionary attacks and against precomputed rainbow table attacks. The reason is that without the salt, the same password always generates the same encryption key. When the salt is used with OpenSSL, the first 8 bytes of the encrypted data are reserved for the salt (i.e., the random bit of data). When the file is decrypted, the salt is read from the encrypted file and used for decryption.
Notice that OpenSSL does not echo the passphrase, so it can’t be captured in the shell history. Also, notice that OpenSSL doesn’t have a standard file extension. I chose .enc to show that the file is encrypted.
As I mentioned earlier, OpenSSL is just an encryption tool. It doesn’t do file compression. Consequently, the file size of the encrypted text file in the previous example is roughly the same as the original text file. OpenSSL can operate on a compressed file as well, but in a step that is done separately:
[laytonjb@home4 TEMP]$ openssl aes-256-cbc -salt -in hpc_001.html.gz \ -out hpc_001.html.gz.enc enter aes-256-cbc encryption password: Verifying - enter aes-256-cbc encryption password: [laytonjb@home4 TEMP]$ ls -s total 7336 196 hpc_001.html.enc 24 hpc_001.html.gz 24 hpc_001.html.gz.enc 7092 MFS2007.pdf
In this case, I used gzip to compress the file before using OpenSSL. Otherwise the process is the same. Notice the size difference between the encrypted compressed file, and the encrypted but uncompressed file.
Decrypting a file is also fairly easy using the -d option on the command line:
[laytonjb@home4 TEMP]$ openssl aes-256-cbc -d -in hpc_001.html.enc -out hpc_001.html.2 enter aes-256-cbc decryption password: [laytonjb@home4 TEMP]$> ls -s total 7680 196 hpc_001.html 196 hpc_001.html.2 196 hpc_001.html.enc 7092 MFS2007.pdf
Crypt Replacements
The *nix of yore came with a command named crypt that could be used for encrypting data. However, its level of security wasn’t very good, so it disappeared from the scene. Even if you can find the source for it, several tools break the encryption, so it should be avoided at all costs. However, the popularity of Crypt was already in place, and some older scripts used it. Today you have the choice of several replacements for Crypt.
Ccrypt
Ccrypt is based on the Rijndael block cipher. The same cipher is the basis of the AES specification. Internally ccrypt takes the specified password, which can be of any length, and hashes the key to 256 bits. As with almost all ciphers, the longer the password, the better the security.
Ccrypt is not symmetric, which means you have to specify whether you are encrypting a file or decrypting it. If you want to encrypt a file, you typically use the command ccrypt:
[laytonjb@home4 TEMP]$ ls -s total 7288 196 hpc_001.html 7092 MFS2007.pdf [laytonjb@home4 TEMP]$ ccrypt hpc_001.html Enter encryption key: Enter encryption key: (repeat) [laytonjb@home4 TEMP]$ ls -s total 7288 196 hpc_001.html.cpt 7092 MFS2007.pdf
Notice that the encrypted file size is about the same size as the unencrypted file for this text example.
One thing you should pay particular attention to is that ccrypt encrypts the file but does not leave the original file in place. I worry about this because if a problem crops up during the encryption process, the file may be corrupted.
The final thing to notice is that ccrypt does not echo the passphrase to stdout, so the shell history cannot capture it.
To decrypt the file, you can run the following command:
[laytonjb@home4 TEMP]$ ccrypt -d hpc_001.html.cpt Enter decryption key: [laytonjb@home4 TEMP]$ ls -s total 7288 196 hpc_001.html 7092 MFS2007.pdf
The -d option just means “decrypt.”
Bcrypt
Another encryption option is bcrypt. Bcrypt uses the blowfish encryption algorithm with passphrases of between 8 and 56 characters. It also uses an internal 448-bit hashed key. The blowfish algorithm itself seems to provide a good level of encryption if you don’t use weak keys (use longer passwords), but the bcrypt code itself hasn’t been updated in a while. However, various versions of Bcrypt exist for many operating systems, including Linux, *nix, Windows, OS X, and others.
Encrypting a file using bcrypt is very simple:
[laytonjb@home4 TEMP]$ bcrypt hpc_001.html Encryption key: Again: [laytonjb@home4 TEMP]$ ls -s total 7116 24 hpc_001.html.bfe 7092 MFS2007.pdf
Notice that the encryption process did not leave the original file in place, but it reduced the size of the encrypted file relative to the original file size.
You should also note that Bcrypt does not echo the passphrase to stdout. This means the shell history will not capture it (always a good thing).
Decrypting is also very similar:
[laytonjb@home4 TEMP]$ bcrypt hpc_001.html.bfe Encryption key: [laytonjb@home4 TEMP]$ ls -s total 7288 196 hpc_001.html 7092 MFS2007.pdf
Bcrypt is a symmetric cipher because it can detect whether the file is encrypted and then decrypts; hence, there is no need for a decrypt option.
MCrypt
Another replacement option for crypt is mcrypt. It has a very large number of cryptography algorithms. A few of which include:
- Blowfish
- DES
- Loki
- Mars
- Rijndael (up to 256 block size)
- Twofish
- Triple DES
It has a number of modes of encryption that provide additional capability beyond just a straight block cipher. You can read about that at an on-line MCrypt man page.
Using mcrypt is very similar to the other Crypt replacement tools:
[laytonjb@home4 TEMP]$ ls -s total 7288 196 hpc_001.html 7092 MFS2007.pdf [laytonjb@home4 TEMP]$ mcrypt hpc_001.html Enter the passphrase (maximum of 512 characters) Please use a combination of upper and lower case letters and numbers. Enter passphrase: Enter passphrase: File hpc_001.html was encrypted. [laytonjb@home4 TEMP]$ ls -s total 7484 196 hpc_001.html 196 hpc_001.html.nc 7092 MFS2007.pdf
In contrast to ccrypt and bcrypt, mcrypt creates an encrypted file that is different from the original file; however, the encrypted text file is basically the same size as the unencrypted file.
Also note that like the other Crypt tools, MCrypt does not echo the passphrase to stdout.
Summary
I didn’t want this article to be just a survey of command-line tools for encrypting files, but I think it’s important to compare the tools with one another so you can see the various features (or quirks) and how they might affect your workflow and your security. It’s not a complete list of all tools available. For example, I didn’t cover the ability of Vim to encrypt files while editing them. Nor did I cover commercial tools – only open-source. However, I hope I have covered the tools that illustrate the various capabilities.
As I mentioned earlier, I’m not a security expert, but I do take security seriously. In examining command-line tools that encrypt and decrypt tools I’ve developed a few general principles I follow.
- Passphrase length: The length of the passphrase is very important. Try to make it as long as you can because a long length makes it more difficult to crack. One suggestion I have is to take a long-ish sentence or quote or movie line that you can remember and use that as your passphrase. You can also take two shorter phrases and combine them into a single phrase.
- Passphrase variety: If your data is important to you, I would recommend using several different passphrases and rotating them. You don’t have to have a large number of passphrases, but you shouldn’t use the same passphrase all of the time. By having different passphrases, if one of them happens to be cracked, then you won’t lose all of your data.
- Character combinations: As with passwords, I would also suggest you use a combination of uppercase, lowercase, numbers, and special characters in your passphrase. Some rules of thumb are floating around the web that you should not use groups of words followed by a short group of numbers such as “111” because some cracking tools look for these patterns. However, don’t complicate your passphrases to the point you will easily forget them.
- Practice: Before you start encrypting files on a regular basis as part of a process, practice with your passphrases. Make sure you can remember what the passphrase is so that you won’t lose any data.
- Keys: Consider using keys rather than passphrases to make life a bit easier. You can also put the keys on a simple USB stick, encrypt them, and put them in a safe location (or lock them up). In the case of GPG, you should also use 64-bit key IDs.
- Echo passphrase: If possible, do not put the passphrase on the command line because the shell history will pick it up. If someone can gain access to the history file, they will have your passphrase. Also make sure that the encryption tool does not echo the passphrase to stdout.
- Large hash key: Use a cipher with the longest possible hash key for the best encryption. In general, the longer the hash key, the more difficult it will be to decrypt a file. I tend to like 256 bits, but I would like to go larger. However, just be aware that the larger the key, the more CPU, and possibly memory, it will take to encrypt the file.
- What to encrypt: What files you encrypt or not is completely up to you. Personally, I like to use command-line encryption rather than encrypted filesystems because I don’t want nor need to encrypt everything. For example I’m not going to encrypt my desktop background pictures since I don’t view them as sensitive in any way. However, I will encrypt special tax information or email I consider sensitive and personal. Just please don’t encrypt your cat pictures.
Notice in this list that I didn’t mention anything about data compression. It’s really your choice if you want to encrypt a file as well as compress it using tools such as zip or p7zip, or to use compression tools before and separate from encrypting the file. I like to compress my files before encrypting them so I can save as much space as possible. I will also use tar as often as possible to collect the files into a single archive.
In my opinion, encryption can be a very important tool to protect your privacy. Think about making encryption a part of your everyday processes.
Tags:
encryption , Privacy , security , security tools