Tools for incident response and memory analysis

Protecting your network environment with the latest virus protection, controlling what software is installed and allowed to run, restricting network access, protecting web browsing, limiting user account access, updating security patches … . All these efforts are critical, but all will fall short if you don't have the proper monitoring to detect villainy on your network and respond quickly. When your network has the proper monitoring in place, and knowledgeable engineers are monitoring for outbreaks, you will have a better view of how traffic flows in your environment. When you understand how traffic flows, you can respond better when something bad happens.

Incident response involves addressing and managing the security events on a network and providing a proper response to those events. The end game is to limit the damage and reduce recovery time and costs. The best way to achieve these goals is with an incident response plan that includes a step-by-step process that investigators should follow when an attack has taken place.

One important step for in the investigation process is to analyze the memory of the infected computer. A well-designed computer intrusion investigation is not limited to data on disk: Evidence is collected from additional data sources, including network traffic and volatile memory. This paper describes procedures for making an accurate and reliable copy of volatile memory content so that the investigator can examine the data for further evidence. Looking directly at memory can turn up clues that might be hidden in a view of the disk. For instance, a trojaned version of the ps command might not report the presence of a malware process running on the system, but a close look at memory reveals the presence of that hidden process.

The only way to get a perfect and undisturbed view of memory is to capture a snapshot of memory to an image file and then extract that file to another system for viewing. This process requires several steps, and it is a good idea to have all the necessary tools ready in advance, so if an incident occurs, you'll be in position to start your investigation before the attack spreads to other systems.

In this article, I focus on the following tools:

• F-Response TACTICAL [1] – a tool used to create a secure connection between the infected computer and the computer you will use as the examining system. TACTICAL, which comes in the form of two USB sticks, is designed to streamline live analysis, collection, and authentication. When used correctly, this tool provides direct, live, read-only access to the remote target computer's disks, volumes, and physical memory. An executable runs directly from the provided USB storage. No software is installed on the target computer. F-Response TACTICAL works with all RAID disks, physical drives, logical volumes, and physical memory (32- and 64-bit windows).
• dc3dd [2] – a tool that captures the content of memory on the infected system; dc3dd is basically the standard *nix dd utility upgraded for forensic use, which allows you to take hashes and split an image, all from one command.
• Volatility [3] and Mandiant Redline  [4]  – free tools used to analyze the memory image file.

I'll describe how these tools work together to capture, transfer, and analyze a memory image file. In this case, the analysis turns up some hidden processes running on the infected system. Volatility and Mandiant Redline are very versatile tools capable of discovering many other kinds of clues. In a later article, I will take a deeper look at memory analysis with Volatility and Mandiant Redline.

TACTICAL: Connecting to the Infected System

In this section, I show you how to investigate a malware-infected Windows system using a Linux-based examiner platform to acquire the image.

F-Response TACTICAL is a simple and inexpensive tool that helps you gather an image file from an infected computer. The TACTICAL solution requires only two USB sticks. One USB stick, labeled TACTICAL Subject , goes on the infected computer. The other stick, labeled TACTICAL Examiner , goes on another computer on the network that you will use as the platform for launching and monitoring the investigation.

Once the USB stick is loaded on the Windows system, start the program so it can listen on its external interface (see Figure 1). Because the tools run from the USB stick, you don't have to install any software, which means the intruder won't see any evidence of its presence on the system.

Figure 1: Running the TACTICAL Subject program on the infected Windows box; remember to enable physical memory.

On the examiner platform, insert the Examiner USB stick and make sure it shows up as loaded (Figure  2). Next, start the executable on the F-RESPONSE Examiner USB stick (Figure 3),

$ ./f-response-tacex-lin.exe -s 192.168.1.129 -p 3260

specifying that the target Windows computer with the Subject USB stick is at 192.168.1.129 and listening on port 3260. The system connects to targets

• iqn.2008-02.com.f-response.cr0wn-d00e37654:disk-0
• iqn.2008-02.com.f-response.cr0wn-d00e37654:disk-1
• iqn.2008-02.com.f-response.cr0wn-d00e37654:vol-c
• iqn.2008-02.com.f-response.cr0wn-d00e37654:vol-e
• iqn.2008-02.com.f-response.cr0wn-d00e37654:pmem

Figure 2: Make sure the Examiner USB stick is loaded on the examiner workstation.

Figure 3: Perform the connection between the Examiner workstation and the infected windows box.

In this case, I am looking for the running memory (pmem) on the target computer, so I can ignore disk and vol targets.

Next, log in to iqn.2008-02.com.f-response.cr0wn-d00e37654:disk-0 with the command (Figure 4):

# iscsiadm -m node -targetname=iqn.2008-02.com.f-response.cr0wn-d00e37654:disk-0 --login

Figure 4: Successfully connected to the Windows box at 192.168.1.129.

The iscsiadm command is an open-iscsi administration utility that allows discovery and login to iSCSI targets, as well as access and management of the open-iscsi database. The -m option specifies the mode, which is node in this case.

The mode could also be defined as: discoverydb, fw, host iface, or session. With the mode selected as node, use the -targetname= parameter to specify the location of the target drive. After successfully connecting to the remote machine, run fdisk -l and see the new device located at /dev/sdd1 (see Figure 5).

Figure 5: Results after running fdisk -l.

When the login command completes, you can mount the device nodes to your machine locally for analysis using the mount command. Next, mount the partition /dev/sdd1 with the command:

# mount -o ro,show_sys_files,streams_interface=windows /dev/sdd1 /mnt/windows_mount

Using mount with the option -o ro will mount the filesystem read-only, show_sys_files shows all system files as normal files, and streams_interface=windows controls how named data streams in WIM files are made available with windows as the named data stream. This command will mount the memory from the Windows system to /mnt/windows_mount. After changing into that directory and listing the files, you will see the list of files followed by a login to the pmem location (Figure 6).

Figure 6: List of files after mounting the memory from the target Windows box, followed by a login to the pmem location.

The pmem location is where the memory (RAM) resides on the target. I have made a forensic image of that memory and placed it on the examining computer for analysis.

Now I need to log in to the process memory of the target system, which is the pmem location. I will use the iscsiadm open-iscsi administration utility to perform this task with the following command:

# iscsiadm -m node -targetname=iqn.2008-02.com.f-response.cr0wn-d00e37654:pmem -login

Again, I will use the isciadm utility and the node option with -targetname to specify the location of pmem. Now I run fdisk -l and see the partition tables (Figure 7).

Figure 7: Results after running fdisk -l notice the HPFS/NTFS system at /dev/sdd1. This result is after login to the pmem location.

Imaging Memory with dc3dd

I can image the remote system's memory with the use of dc3dd, which was developed by Jesse Komblum at the DoD Cyber Crime Center. Dc3dd is similar to dd but is designed for forensic work, allowing you to take hashes and split an image all from one command. Taking hashes in a forensic examination ensures authenticity of the file – in this case, the memory image. The ability to split an image into workable, fixed-size chunks is also an advantage.

Open up a terminal and type:

# dc3dd if=/dev/sde of=/cases/remote-system-memory8.img progress=on hash=md5 hashlog= /cases/remote-system-memory8.md5

The command breaks down as:

• if=DEVICE or FILE – Read input from a device or a file, in this case /dev/sde
• of=FILE or DEVICE – Write output to a file or device, in this case /cases/remote-system-memory8.img
• progress=on – Show progress on-screen
• hash=ALGORITHM – Compute a hash of the input and also of any outputs (specified using hof=, hofs=, phod=, or fhod=. ALGORITHM options include md5, sha1, sha256, or sha512.
• hashlog=FILE – Log total hashes and piecewise hashes to FILE.

This command does a forensic copy of the Windows memory file to your computer; you can see a screenshot of the progress in Figure 8. (A forensic copy is a copy that shows the hash, therefore proving authenticity.)

Figure 8: Performing a forensic copy of the Windows memory file using dc3dd.

Analyzing with Volatility

Now that I have an image file of the Windows memory, I can analyze the file for the existence of malware. Several tools are available for this task, including Redline by Mandiant. An important open source analysis tool is Volatility, implemented in Python for the extraction of digital artifacts from volatile memory (RAM) samples.

If the memory image was acquired from an unknown system, you can use Volatility to identify the operation system (Figure 9).

Figure 9: Using Volatility to identify the operation system associated with the memory image file.

Volatility's imageinfo plug-in finds the operating system to which the memory dump belongs. In Figure 9, the suggested profile portion of the output shows a WinXP SP2x86 system; you will need this information to perform more work on this memory image file.

To look at the running processes, use:

$ vol.py -profile=WinXPSP2x86 pslist -f remote-system-memory8.img

You can use the psscan plugin to scan the memory image for EPROCESS blocks with the command:

$ vol.py -profile=WinXPSP2x86 psscan -f remote-system-memory8.img

Use psscan to enumerate processes using pool tag scanning, which finds processes that have terminated and that have been hidden or unlinked by a rootkit (Figure 10).

Figure 10: Volatility with the psscan invoked.