文本分类

本文详细介绍如何在Python中使用Pandas处理分类数据,包括创建、排序、比较分类变量,以及使用cut函数进行区间划分。通过实例演示了不同创建方法、属性获取与修改、排序操作,并提供了地震数据深度分类的实践案例。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

第8章 分类数据

import pandas as pd
import numpy as np
df = pd.read_csv('data/table.csv')
df.head()
SchoolClassIDGenderAddressHeightWeightMathPhysics
0S_1C_11101Mstreet_11736334.0A+
1S_1C_11102Fstreet_21927332.5B+
2S_1C_11103Mstreet_21868287.2B+
3S_1C_11104Fstreet_21678180.4B-
4S_1C_11105Fstreet_41596484.8B+

一、category的创建及其性质

1. 分类变量的创建

(a)用Series创建
pd.Series(["a", "b", "c", "a"], dtype="category")
0    a
1    b
2    c
3    a
dtype: category
Categories (3, object): [a, b, c]
(b)对DataFrame指定类型创建
temp_df = pd.DataFrame({'A':pd.Series(["a", "b", "c", "a"], dtype="category"),'B':list('abcd')})
temp_df.dtypes
A    category
B      object
dtype: object
(c)利用内置Categorical类型创建
cat = pd.Categorical(["a", "b", "c", "a"], categories=['a','b','c'])
pd.Series(cat)
0    a
1    b
2    c
3    a
dtype: category
Categories (3, object): [a, b, c]
(d)利用cut函数创建
默认使用区间类型为标签
pd.cut(np.random.randint(0,60,5), [0,10,30,60])
[(10, 30], (0, 10], (10, 30], (30, 60], (30, 60]]
Categories (3, interval[int64]): [(0, 10] < (10, 30] < (30, 60]]
可指定字符为标签
pd.cut(np.random.randint(0,60,5), [0,10,30,60], right=False, labels=['0-10','10-30','30-60'])
[10-30, 30-60, 30-60, 10-30, 30-60]
Categories (3, object): [0-10 < 10-30 < 30-60]

2. 分类变量的结构

一个分类变量包括三个部分,元素值(values)、分类类别(categories)、是否有序(order)
从上面可以看出,使用cut函数创建的分类变量默认为有序分类变量
下面介绍如何获取或修改这些属性
(a)describe方法
该方法描述了一个分类序列的情况,包括非缺失值个数、元素值类别数(不是分类类别数)、最多次出现的元素及其频数
s = pd.Series(pd.Categorical(["a", "b", "c", "a",np.nan], categories=['a','b','c','d']))
s.describe()
count     4
unique    3
top       a
freq      2
dtype: object
(b)categories和ordered属性
查看分类类别和是否排序
s.cat.categories
Index(['a', 'b', 'c', 'd'], dtype='object')
s.cat.ordered
False

3. 类别的修改

(a)利用set_categories修改
修改分类,但本身值不会变化
s = pd.Series(pd.Categorical(["a", "b", "c", "a",np.nan], categories=['a','b','c','d']))
s.cat.set_categories(['new_a','c'])
0    NaN
1    NaN
2      c
3    NaN
4    NaN
dtype: category
Categories (2, object): [new_a, c]
(b)利用rename_categories修改
需要注意的是该方法会把值和分类同时修改
s = pd.Series(pd.Categorical(["a", "b", "c", "a",np.nan], categories=['a','b','c','d']))
s.cat.rename_categories(['new_%s'%i for i in s.cat.categories])
0    new_a
1    new_b
2    new_c
3    new_a
4      NaN
dtype: category
Categories (4, object): [new_a, new_b, new_c, new_d]
利用字典修改值
s.cat.rename_categories({'a':'new_a','b':'new_b'})
0    new_a
1    new_b
2        c
3    new_a
4      NaN
dtype: category
Categories (4, object): [new_a, new_b, c, d]
(c)利用add_categories添加
s = pd.Series(pd.Categorical(["a", "b", "c", "a",np.nan], categories=['a','b','c','d']))
s.cat.add_categories(['e'])
0      a
1      b
2      c
3      a
4    NaN
dtype: category
Categories (5, object): [a, b, c, d, e]
(d)利用remove_categories移除
s = pd.Series(pd.Categorical(["a", "b", "c", "a",np.nan], categories=['a','b','c','d']))
s.cat.remove_categories(['d'])
0      a
1      b
2      c
3      a
4    NaN
dtype: category
Categories (3, object): [a, b, c]
(e)删除元素值未出现的分类类型
s = pd.Series(pd.Categorical(["a", "b", "c", "a",np.nan], categories=['a','b','c','d']))
s.cat.remove_unused_categories()
0      a
1      b
2      c
3      a
4    NaN
dtype: category
Categories (3, object): [a, b, c]

二、分类变量的排序

前面提到,分类数据类型被分为有序和无序,这非常好理解,例如分数区间的高低是有序变量,考试科目的类别一般看做无序变量

1. 序的建立

(a)一般来说会将一个序列转为有序变量,可以利用as_ordered方法
s = pd.Series(["a", "d", "c", "a"]).astype('category').cat.as_ordered()
s
0    a
1    d
2    c
3    a
dtype: category
Categories (3, object): [a < c < d]
退化为无序变量,只需要使用as_unordered
s.cat.as_unordered()
0    a
1    d
2    c
3    a
dtype: category
Categories (3, object): [a, c, d]
(b)利用set_categories方法中的order参数
pd.Series(["a", "d", "c", "a"]).astype('category').cat.set_categories(['a','c','d'],ordered=True)
0    a
1    d
2    c
3    a
dtype: category
Categories (3, object): [a < c < d]
(c)利用reorder_categories方法
这个方法的特点在于,新设置的分类必须与原分类为同一集合
s = pd.Series(["a", "d", "c", "a"]).astype('category')
s.cat.reorder_categories(['a','c','d'],ordered=True)
0    a
1    d
2    c
3    a
dtype: category
Categories (3, object): [a < c < d]
#s.cat.reorder_categories(['a','c'],ordered=True) #报错
#s.cat.reorder_categories(['a','c','d','e'],ordered=True) #报错

2. 排序

先前在第1章介绍的值排序和索引排序都是适用的
s = pd.Series(np.random.choice(['perfect','good','fair','bad','awful'],50)).astype('category')
s.cat.set_categories(['perfect','good','fair','bad','awful'][::-1],ordered=True).head()
0       good
1       fair
2        bad
3    perfect
4    perfect
dtype: category
Categories (5, object): [awful < bad < fair < good < perfect]
s.sort_values(ascending=False).head()
29    perfect
17    perfect
31    perfect
3     perfect
4     perfect
dtype: category
Categories (5, object): [awful, bad, fair, good, perfect]
df_sort = pd.DataFrame({'cat':s.values,'value':np.random.randn(50)}).set_index('cat')
df_sort.head()
value
cat
good-1.746975
fair0.836732
bad0.094912
perfect-0.724338
perfect-1.456362
df_sort.sort_index().head()
value
cat
awful0.245782
awful0.063991
awful1.541862
awful-0.062976
awful0.472542

三、分类变量的比较操作

1. 与标量或等长序列的比较

(a)标量比较
s = pd.Series(["a", "d", "c", "a"]).astype('category')
s == 'a'
0     True
1    False
2    False
3     True
dtype: bool
(b)等长序列比较
s == list('abcd')
0     True
1    False
2     True
3    False
dtype: bool

2. 与另一分类变量的比较

(a)等式判别(包含等号和不等号)
两个分类变量的等式判别需要满足分类完全相同
s = pd.Series(["a", "d", "c", "a"]).astype('category')
s == s
0    True
1    True
2    True
3    True
dtype: bool
s != s
0    False
1    False
2    False
3    False
dtype: bool
s_new = s.cat.set_categories(['a','d','e'])
#s == s_new #报错
(b)不等式判别(包含>=,<=,<,>)
两个分类变量的不等式判别需要满足两个条件:① 分类完全相同 ② 排序完全相同
s = pd.Series(["a", "d", "c", "a"]).astype('category')
#s >= s #报错
s = pd.Series(["a", "d", "c", "a"]).astype('category').cat.reorder_categories(['a','c','d'],ordered=True)
s >= s
0    True
1    True
2    True
3    True
dtype: bool

四、问题与练习

【问题一】 如何使用union_categoricals方法?它的作用是什么?
#分类数据经过 pd.concat 合并等操作后类型转为了 object 类型,如果想要保持分类类型的话,可以借助 union_categoricals 来完成
from pandas.api.types import union_categoricals 
union_categoricals([type1, type2])
【练习一】 现继续使用第四章中的地震数据集,请解决以下问题:
现在将深度分为七个等级:[0,5,10,15,20,30,50,np.inf],请以深度等级Ⅰ,Ⅱ,Ⅲ,Ⅳ,Ⅴ,Ⅵ,Ⅶ为索引并按照由浅到深的顺序进行排序。
data = pd.read_csv('data/Earthquake.csv')
data.head()
日期时间维度经度方向距离深度烈度
02003.05.2012:17:44 AM39.0440.38west0.110.00.0
12007.08.0112:03:08 AM40.7930.09west0.15.24.0
21978.05.0712:41:37 AM38.5827.61south_west0.10.00.0
31997.03.2212:31:45 AM39.4736.44south_west0.110.00.0
42000.04.0212:57:38 AM40.8030.24south_west0.17.00.0
data.describe()
维度经度距离深度烈度
count10062.00000010062.00000010062.00000010062.00000010062.000000
mean38.80942133.6234193.17501512.5020571.678513
std1.2372425.7966794.71546115.3166342.068193
min35.77000025.7000000.1000000.0000000.000000
25%37.86000028.9600001.4000005.0000000.000000
50%38.85000030.9800002.3000008.1000000.000000
75%39.60000039.2800003.60000012.0000003.800000
max42.77000045.00000095.400000172.0000007.200000
data.set_index(pd.cut(data['深度'], [-np.inf,5,10,15,20,30,50,np.inf], right=True, labels=['Ⅰ','Ⅱ','Ⅲ','Ⅳ','Ⅴ','Ⅵ','Ⅶ'])).sort_values(by='深度')
日期时间维度经度方向距离深度烈度
深度
2000.03.3112:24:51 AM39.4943.00north_east2.70.00.0
2000.01.1712:19:56 AM38.0731.94north_west3.20.00.0
2000.06.0612:47:36 AM40.4433.09west5.70.00.0
1997.04.0412:03:38 AM40.1729.27north_west2.10.00.0
1996.08.1412:17:56 AM40.6935.41north_west2.10.00.0
1995.07.2312:05:04 AM37.6129.29north_east3.20.00.0
1997.06.1612:18:04 AM37.9229.17north_east3.20.00.0
1982.10.3112:48:24 AM39.2827.73east0.90.00.0
1998.11.1812:32:24 AM39.1740.07east0.90.00.0
1976.03.0612:20:32 AM36.9430.38north_east5.60.00.0
1996.08.1412:54:32 AM40.6735.37west1.00.00.0
2000.03.0412:50:33 AM37.2427.59west1.00.00.0
1982.01.1612:06:47 AM39.2729.00south2.10.00.0
1997.07.1712:29:07 AM36.8930.40north_west5.60.00.0
1976.08.2412:53:30 AM39.2029.50south_west1.00.00.0
1977.02.2112:36:15 AM37.6429.10south_west1.00.00.0
1996.04.2612:12:36 AM39.7139.20north_west5.60.00.0
1976.09.3012:21:14 AM41.8026.30north_west5.60.00.0
1981.06.0412:40:37 AM38.6127.20south_east1.00.03.9
1979.10.1612:27:05 AM37.2027.80south_east1.00.00.0
1981.03.2812:59:18 AM37.2029.90south_east5.60.00.0
1976.05.0512:12:21 AM39.4029.01south_east1.00.03.5
1995.05.1512:42:49 AM39.4526.23south5.60.00.0
1978.07.0512:18:24 AM39.4933.20south_east2.10.00.0
1995.01.0612:52:26 AM36.9429.02north_west3.20.00.0
1980.04.2112:58:00 AM39.2630.12south_east2.10.00.0
1977.03.1612:17:43 AM37.2028.10west3.30.00.0
1975.03.2812:11:07 AM40.3926.39north_west5.60.00.0
1996.08.1412:24:56 AM40.7135.28south_west1.00.00.0
1996.04.1012:29:40 AM40.4132.25south_west1.00.00.0
...........................
2011.06.2312:29:07 AM37.0530.98south_east2.1110.40.0
1965.03.2612:29:23 AM36.8230.94south_east6.6111.05.1
1990.02.2612:00:45 AM37.1730.21north_east2.8111.00.0
2011.08.1712:27:48 AM36.8930.99west1.8111.40.0
1991.03.1112:33:43 AM37.0130.96south_west0.8113.00.0
2005.07.2012:42:08 AM36.9830.91north_west2.1116.00.0
2010.02.2312:26:37 AM36.8731.30south_west1.5117.60.0
1991.01.2412:32:33 AM37.1131.00south4.8118.00.0
1951.02.0212:59:23 AM36.8330.54south_east2.7120.00.0
2013.12.2512:11:35 AM36.9731.10north_west1.1123.24.4
1989.07.1212:42:54 AM37.1631.09north_east3.5125.00.0
1981.07.1012:39:32 AM37.1031.16north_east4.1126.00.0
1964.12.3012:14:03 AM36.4034.20south_east8.7128.04.9
1936.08.1212:24:28 AM37.4429.44north_east2.9130.04.9
1925.09.0112:16:30 AM37.5629.17north5.4130.05.3
1966.08.3112:54:16 AM38.9041.50north_west1.6131.04.1
1966.05.0912:51:10 AM37.0530.98south_east2.2132.04.9
1989.01.1612:33:12 AM37.1730.95north_east1.4132.00.0
1964.08.2212:14:05 AM36.9330.60north_east3.2133.04.1
2004.04.2112:41:47 AM36.7331.49west4.3133.00.0
1992.10.2012:26:13 AM37.2031.21north_east5.6136.00.0
1985.06.0412:15:17 AM37.1531.08north_east2.2139.00.0
1978.01.1712:09:03 AM39.4041.40south_west7.3139.00.0
1962.08.1812:29:07 AM36.9732.52south_east1.4140.04.7
2007.12.0212:40:22 AM37.3231.09north_west3.6141.10.0
1982.01.2412:37:03 AM36.6127.52south_east10.1146.03.7
1936.02.0212:08:26 AM37.6938.82north_west2.7160.04.9
1975.09.0612:11:02 AM38.6040.80south_west1.8166.00.0
1969.07.2312:54:11 AM38.9041.00north_east2.5169.04.2
1966.08.1912:38:56 AM38.4041.20south_east1.6172.04.6

10062 rows × 8 columns

内容概要:本文档详细介绍了基于MATLAB实现的无人机三维路径规划项目,核心算法采用蒙特卡罗树搜索(MCTS)。项目旨在解决无人机在复杂三维环境中自主路径规划的问题,通过MCTS的随机模拟与渐进式搜索机制,实现高效、智能化的路径规划。项目不仅考虑静态环境建模,还集成了障碍物检测与避障机制,确保无人机飞行的安全性和效率。文档涵盖了从环境准备、数据处理、算法设计与实现、模型训练与预测、性能评估到GUI界面设计的完整流程,并提供了详细的代码示例。此外,项目采用模块化设计,支持多无人机协同路径规划、动态环境实时路径重规划等未来改进方向。 适合人群:具备一定编程基础,特别是熟悉MATLAB和无人机技术的研发人员;从事无人机路径规划、智能导航系统开发的工程师;对MCTS算法感兴趣的算法研究人员。 使用场景及目标:①理解MCTS算法在三维路径规划中的应用;②掌握基于MATLAB的无人机路径规划项目开发全流程;③学习如何通过MCTS算法优化无人机在复杂环境中的飞行路径,提高飞行安全性和效率;④为后续多无人机协同规划、动态环境实时调整等高级应用打下基础。 其他说明:项目不仅提供了详细的理论解释和技术实现,还特别关注了实际应用中的挑战和解决方案。例如,通过多阶段优化与迭代增强机制提升路径质量,结合环境建模与障碍物感知保障路径安全,利用GPU加速推理提升计算效率等。此外,项目还强调了代码模块化与调试便利性,便于后续功能扩展和性能优化。项目未来改进方向包括引入深度强化学习辅助路径规划、扩展至多无人机协同路径规划、增强动态环境实时路径重规划能力等,展示了广阔的应用前景和发展潜力。
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值