python DataFrame

1. 说明

 DataFrame是Pandas库中处理表的数据结构，可看作是python中的类似数据库的操作，是Python数据挖掘中最常用的工具。下面介绍DataFrame的一些常用方法。

2. 遍历

1) 代码

1. import pandas as pd

2. import math

3.  

4. df=pd.DataFrame({'key':['a','b','c'],'data1':[1,2,3],'data2':[4,5,6]})

5. print(df)

6. for idx,item in df.iterrows():

7. print(idx)

8. print(item)

2) 结果

1. data1 data2 key

2. 0 1 4 a

3. 1 2 5 b

4. 2 3 6 c

5. 0

6. data1 1

7. data2 4

8. key a

9. Name: 0, dtype: object

10. … 略

3. 同时遍历两个数据表

1) 代码

1. import pandas as pd

2. import math

3.  

4. df1=pd.DataFrame({'key':['a','b'],'data1':[1,2]})

5. df2=pd.DataFrame({'key':['c','d'],'data2':[4,5]})

6. for (idx1,item1),(idx2,item2) in zip(df1.iterrows(),df2.iterrows()):

7. print("idx1",idx1)

8. print(item1)

9. print("idx2",idx2)

10. print(item2)

2) 结果

1. ('idx1', 0)

2. data1 1

3. key a

4. Name: 0, dtype: object

5. ('idx2', 0)

6. data2 4

7. key c

8. Name: 0, dtype: object

9. ('idx1', 1)

10. data1 2

11. key b

12. Name: 1, dtype: object

13. ('idx2', 1)

14. data2 5

15. key d

16. Name: 1, dtype: object

4. 取一行或多行

1) 代码

1. import pandas as pd

2. import math

3.  

4. df1=pd.DataFrame({'key':['a','b','c'],'data1':[1,2,3]})

5. df2=df1[:1]

6. print(df2)

2) 结果

1. data1 key

2. 0 1 a

5. 取一列或多列

1) 代码

1. import pandas as pd

2. import math

3.  

4. df1=pd.DataFrame({'key':['a','b','c'],'data1':[1,2,3]})

5. df2=pd.DataFrame()

6. df2['key2']=df1['key']

7. print(df2)

2) 结果

1. key2

2. 0 a

3. 1 b

4. 2 c

6. 列连接（横向：变宽）：merge

1) 代码

 

1. import pandas as pd

2.  

3. df1=pd.DataFrame({'key':['a','b','c'],'data1':[1,2,3]})

4. df2=pd.DataFrame({'key':['a','b','c'],'data2':[4,5,6]})

5. df3=pd.merge(df1,df2)

 

2) 结果

1. data1 key

2. 0 1 a

3. 1 2 b

4. 2 3 c

5. data2 key

6. 0 4 a

7. 1 5 b

8. 2 6 c

9. data1 key data2

10. 0 1 a 4

11. 1 2 b 5

12. 2 3 c 6

7. 行连接（纵向：变长）：concat

1) 代码

1. import pandas as pd

2.  

3. df1=pd.DataFrame({'key':['a','b','c'],'data':[1,2,3]})

4. df2=pd.DataFrame({'key':['d','e','f'],'data':[4,5,6]})

5. df3=pd.concat([df1,df2])

2) 结果

1. data key

2. 0 1 a

3. 1 2 b

4. 2 3 c

5. data key

6. 0 4 d

7. 1 5 e

8. 2 6 f

9. data key

10. 0 1 a

11. 1 2 b

12. 2 3 c

13. 0 4 d

14. 1 5 e

15. 2 6 f

8. 对某列做简单变换

1) 代码

1. import pandas as pd

2.  

3. df=pd.DataFrame({'key':['a','b','c'],'data1':[1,2,3]})

4. print(df)

5. df['data1']=df['data1']+1

6. print(df)

2) 结果

1. data1 key

2. 0 1 a

3. 1 2 b

4. 2 3 c

5. data1 key

6. 0 2 a

7. 1 3 b

8. 2 4 c

9. 对某列做复杂变换

1) 代码

1. import pandas as pd

2. import math

3.  

4. df=pd.DataFrame({'key':['a','b','c'],'data1':[1,2,3]})

5. print(df)

6. df['data1']=df['data1'].apply(lambda x: math.sin(x))

7. print(df)

2) 结果

1. data1 key

2. 0 1 a

3. 1 2 b

4. 2 3 c

5. data1 key

6. 0 0.841471 a

7. 1 0.909297 b

8. 2 0.141120 c

10. 对某列做函数处理

1) 代码

1. import pandas as pd

2.  

3. def testme(x):

4. print("???",x)

5. y = x + 3000

6. return y

7.  

8. df=pd.DataFrame({'key':['a','b','c'],'data1':[1,2,3]})

9. print(df)

10. df['data1']=df['data1'].apply(testme)

11. print(df)

2) 结果

1. data1 key

2. 0 1 a

3. 1 2 b

4. 2 3 c

5. ('???', 1)

6. ('???', 2)

7. ('???', 3)

8. data1 key

9. 0 3001 a

10. 1 3002 b

11. 2 3003 c

11. 用某几列计算生成新列

1) 代码

1. import pandas as pd

2.  

3. df=pd.DataFrame({'key':['a','b','c'],'data1':[1,2,3],'data2':[4,5,6]})

4. print(df)

5. df['data3']=df['data1']+df['data2']

6. print(df)

2) 结果

1. data1 data2 key

2. 0 1 4 a

3. 1 2 5 b

4. 2 3 6 c

5. data1 data2 key data3

6. 0 1 4 a 5

7. 1 2 5 b 7

8. 2 3 6 c 9

12. 用某几列用函数生成新列

1) 代码

1. import pandas as pd

2. import math

3.  

4. def testme(x):

5. print(x['data1'],x['data2'])

6. return x['data1'] + x['data2']

7.  

8. df=pd.DataFrame({'key':['a','b','c'],'data1':[1,2,3],'data2':[4,5,6]})

9. print(df)

10. df['data3']=df.apply(testme, axis=1)

11. print(df)

2) 结果

1. data1 data2 key

2. 0 1 4 a

3. 1 2 5 b

4. 2 3 6 c

5. (1, 4)

6. (2, 5)

7. (3, 6)

8. data1 data2 key data3

9. 0 1 4 a 5

10. 1 2 5 b 7

11. 2 3 6 c 9

13. 删除列

1) 代码

1. import pandas as pd

2. import math

3.  

4. df=pd.DataFrame({'key':['a','b','c'],'data1':[1,2,3],'data2':[4,5,6]})

5. print(df)

6. df=df.drop(['data2'],axis=1)

7. print(df)

2) 结果

1. data1 data2 key

2. 0 1 4 a

3. 1 2 5 b

4. 2 3 6 c

5. data1 key

6. 0 1 a

7. 1 2 b

8. 2 3 c

14. One-Hot变换

（把一列枚举型变为多列数值型）

1) 代码

1. import pandas as pd

2. import math

3.  

4. df1=pd.DataFrame({'key':['a','b','c'],'data1':[1,2,3]})

5. print(df1)

6. df2=pd.get_dummies(df1['key'])

7. print(df2)

8. df3=pd.get_dummies(df1)

9. print(df3)

2) 结果

1. data1 key

2. 0 1 a

3. 1 2 b

4. 2 3 c

5. a b c

6. 0 1 0 0

7. 1 0 1 0

8. 2 0 0 1

9. data1 key_a key_b key_c

10. 0 1 1 0 0

11. 1 2 0 1 0

12. 2 3 0 0 1

15. 其它常用方法

1) 求均值方差，中位数等

df[f].describe()

2) 求均值

df[f].mean()

3) 求方差

df[f].std()

4) 清除空值

df.dropna()

5) 填充空值

df.fillna()

16、获取行数、列数、索引及第几行第几列的值

df=DataFrame([{‘A’:’11’,’B’:’12’},{‘A’:’111’,’B’:’121’},{‘A’:’1111’,’B’:’1211’}])

print df.columns.size#列数 2
print df.iloc[:,0].size#行数 3
print df.ix[[0]].index.values[0]#索引值 0
print df.ix[[0]].values[0][0]#第一行第一列的值 11
print df.ix[[1]].values[0][1]#第二行第二列的值 121