pandas 之 交叉表-透视表

数据分析最重要的技巧, 没有之一. 我的分析工作都是用Excel和pandas 来整透视表, 如论深入了解, 还得Excel + 业务分析

import numpy as np 
import pandas as pd

认识

A pivot table is a data summarization tool(数据汇总工具) frequently found in spreadsheet programs and other data analysis software(广泛应用于数据分析中). It aggregates a table of data by one or more keys, arranging the data in a rectangle(矩形) with some of the group keys along the rows and some along the columns.
Pivot tables in Python with pandas are made possible through the groupby facility(促进) described in this chapter combined with reshape operations utilizing hierarchical indexing.
DataFrame has a pivot_table method, and there is also a top-level pandas.pivot_table function. In addition to providing a convenience interface to groupby, pivot_table can add partial totals , also known as margins.

Returning to the tipping dataset, suppose you wanted to compute a table of group means(the default pivot_table aggregation type) arranged by day and smoker on the rows: (对分组计算组内平均)

tips = pd.read_csv('../examples/tips.csv')

"新增一列 tip_pct"

tips['tip_pct'] = tips['tip'] / tips['total_bill']

tips[:6]

'新增一列 tip_pct'

"默认的aggregation 是 mean"
tips.pivot_table(index=['day', 'smoker'])

'默认的aggregation 是 mean'

This could have been produced with groupby directly. Now, suppose we want to aggregate only tip_pct and size, and additionally group by time. I'll put smoker in the table columns and day in the rows:

tips.pivot_table(['tip_pct', 'size'], index=['time', 'day'],
                 columns='smoker')

We could augment this table to include partial totals by passing margins=True. This has the effect of adding all row and column labels, with corresponding values being the group statistics for all the data within a single tier:

tips.pivot_table(['tip_pct', 'size'], index=['time', 'day'],
                columns='smoker', margins=True)

Here, the All values are means without taking into account smoker versus non-smoker or any of the two levels of grouping on the rows.

To use a different aggregation function, pass it to aggfunc. For example, count or len will give you a cross-tabulation of group sizes:

tips.pivot_table('tip_pct', index=['time', 'smoker'],
                columns='day', aggfunc=len, margins=True)

If some combinations are empty, you may wish to pass a fill_value

tips.pivot_table('tip_pct', index=['time', 'size', 'smoker'],
                columns='day', aggfunc='mean', fill_value=0)

See Table 10-2 for a summary of pivot_table methods.

交叉表: Crosstab

• 是透视表的一部分, aggfunc=count而已
  A cross-tabulation (or crosstab for short) is a special case of a pivot table that computes group frequencies.Here is an example:

As part of some survey analysis, we might want to summarize this data nationality and handedness. You could use pivot_table to do this, but the pandas.crosstab function can be more convenient:

pd.crosstab(data.Nationality, data.Handedness, margins=True)

The first two arguments to crosstab can each either be an array or Series or a list of arrays. As in the tips data:

"根据 day, time 对 smoker 进行统计"
pd.crosstab([tips.time, tips.day], tips.smoker, margins=True)

'根据 day, time 对 smoker 进行统计'

小结

Mastering pandas's data grouping tools can help both with data cleaning as well as modeling or statistical analysis work.
(熟练掌握 groupby 对 数据清洗, 建模统计等都是有认识和实操方面的帮助的.)

耐心和恒心, 总会获得回报的.