Data Panda

On and On

Pandas offers a plethora of auxiliary functions for data manipulation. The DataFrame methods stack() and unstack(), for example, rotate a DataFrame so that the columns become rows and vice versa.

To clean up existing data, Pandas provides drop_duplicates(), which dedupes Series and DataFrame objects. In contrast, replace() searches for all entries with a certain value and replaces the matches with another value:

series.replace('a', 'b')

The map() method is more generic: It expects a function or a dictionary and automatically converts the entries of a data object. For example, the following example uses the str.lower() function to convert all entries in a column to lowercase:

dataframe['a'].map(str.lower)

In true Python style, Pandas again allows an anonymous lambda function to be passed in. At this point, you can also appreciate the power of vectorization that NumPy supports. The Series class provides, among other things, a separate str property that does not require line-by-line iteration to process strings. For example:

series.str.contains("ADMIN")

finds all entries that contain the ADMIN string.

Future

Pandas offers many data manipulation methods that I was unable to cover in this article and many equally unmentioned arguments that change functions into useful helpers in more or less everyday application cases. Moreover, Pandas uses the plot() method (Figure 3) from the Matplotlib [5] library to visualize DataFrames and Series. The Pandas documentation contains a complete reference (Figure 4).

Figure 3: Using Matplotlib to visualize Pandas records.

Figure 4: The Pandas documentation lists all of the library's resources.

The Pandas data library shows that Python is mature enough – mainly thanks to the NumPy foundation – to take on compiled languages in terms of speed, while offering the benefits of intuitive syntax and a variety of interactive shells.