python collections模块使用详解(1)

• collections 共涉及到以下几个模块:
  [‘deque’, ‘defaultdict’, ‘namedtuple’, ‘UserDict’, ‘UserList’,
  ‘UserString’, ‘Counter’, ‘OrderedDict’, ‘ChainMap’]

namedtuple详解

tuple

tuple 拆包特性

$ ipython
Python 3.7.3 (default, Mar 28 2019, 10:38:38) [MSC v.1915 32 bit (Intel)]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.4.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: user_tuple = ('laoliu', 30, 175, 'beijing')

# python的拆包特性
In [2]: name, *other = user_tuple     

In [3]: name
Out[3]: 'laoliu'

In [4]: other
Out[4]: [30, 175, 'beijing']
# 拆包
In [5]: name, age, hight, address = user_tuple

In [6]: print(name, age, hight, address)
laoliu 30 175 beijing

In [7]: d = {}
# 元组可以作为dict的键, 但list不可以,这是因为tuple是不可变对象
In [8]: d[user_tuple] = 'shanghai'

In [9]: d
Out[9]: {('laoliu', 30, 175, 'beijing'): 'shanghai'}

namedtuple的使用方法示例

Python 3.7.3 (default, Mar 28 2019, 10:38:38) [MSC v.1915 32 bit (Intel)]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.4.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from collections import namedtuple

In [2]: User = namedtuple("User",['name', 'age'])

In [3]: user = User(name='liu',age=18)

In [4]: print(user.name, user.age)
liu 18

In [5]: user_tuple = ('zhang', 19)
#  以 * 传参, 也是位置参数, 详解可参照函数的传参方式 
In [6]: user2 = User(*user_tuple)    

In [7]: print(user.name, user.age)
liu 18

In [8]: print(user2.name, user2.age)
zhang 19

In [9]: user_dict = {'name': 'wang', 'age':20}
#  以 ** 传参, 也就是关键字参数, 详解可参照函数的传参方式 
In [10]: user3 = User(**user_dict)

In [11]: print(user3.name, user3.age)
wang 20
#  新增属性之后的解决办法
In [12]: User = namedtuple('User',['name', 'age', 'height'])

In [13]: user = User(name='liu',age=18)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-13-7d01a0548062> in <module>
----> 1 user = User(name='liu',age=18)

TypeError: __new__() missing 1 required positional argument: 'height'
# 第一种解决办法就是添加关键字
In [14]: user = User(name='liu',age=18, height=170)

In [15]: print(user.name, user.age, user.height)
liu 18 170

In [16]: print(user2.name, user2.age, user2.height)
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-16-d9f0ea18d8bf> in <module>
----> 1 print(user2.name, user2.age, user2.height)

AttributeError: 'User' object has no attribute 'height'

In [17]: user2
Out[17]: User(name='zhang', age=19)
# 这个就是函数传参中的位置参数与关键字参数混用的例子
In [18]: user2 = User(*user_tuple, height=171)

In [19]: user2
Out[19]: User(name='zhang', age=19, height=171)

In [20]: user3
Out[20]: User(name='wang', age=20)
# 错误传参示例
In [21]: use3 = User(**user_dict, 'height':172)
  File "<ipython-input-22-9441aec8cf18>", line 1
    use3 = User(**user_dict, 'height':172)
                                     ^
SyntaxError: invalid syntax

# 错误传参示例
In [22]: use3 = User(**user_dict, {'height': 172})
  File "<ipython-input-23-55ac47e668b6>", line 1
    use3 = User(**user_dict, {'height': 172})
                            ^
SyntaxError: positional argument follows keyword argument unpacking

In [23]: user_dict
Out[23]: {'name': 'wang', 'age': 20}

In [24]: user_dict.update({'height':172})

In [25]: user_dict
Out[25]: {'name': 'wang', 'age': 20, 'height': 172}

In [26]: user3 = User(**user_dict)

In [27]: user3
Out[27]: User(name='wang', age=20, height=172)

defaultdict 使用详解

Python 3.7.3 (default, Mar 28 2019, 10:38:38) [MSC v.1915 32 bit (Intel)]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.4.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from collections import defaultdict

In [2]: users = ['liu1', 'liu2', 'liu3', 'liu1', 'liu1', 'liu2']

In [3]: # 统计users中每个名字出现的次数

In [4]: user_dict = {}
# 这是我们常用的方法来解决 
In [5]: for user in users:
   ...:     if user in user_dict:
   ...:         user_dict[user] += 1
   ...:     else:
   ...:         user_dict[user] = 1
   ...:

In [6]: user_dict   # 结果
Out[6]: {'liu1': 3, 'liu2': 2, 'liu3': 1}
# 第二种解决办法, 使用setdefault()
In [7]: user_dict2 = {}

In [8]: for user in users:
    ...:     user_dict2.setdefault(user, 0)   
    ...:     user_dict2[user] += 1
    ...:

In [9]: user_dict2
Out[9]: {'liu1': 3, 'liu2': 2, 'liu3': 1}
# 第三种解决办法 使用defaultdict()  (推荐使用)
In [10]: user_dict3 = defaultdict(int)

In [11]: for user in users:
    ...:     user_dict3[user] += 1
    ...:

In [12]: user_dict3
Out[12]: defaultdict(int, {'liu1': 3, 'liu2': 2, 'liu3': 1})
# defaultdict() 扩展使用, 创建一些复杂的数据结构
# 求如下数据结构:
{
    'group1':{
        'name': '',
        'nums': 0
    }
}

In [13]: def gen_default():
    ...:     return {'name': '', 'nums': 0}
    ...:

In [14]: default_dict = defaultdict(gen_default)

In [15]: default_dict['group1']
Out[15]: {'name': '', 'nums': 0}

deque 使用详解

• deque 是线程安全的, list是非线程安全的,在多线程编程的情况下要多注意
• queue (队列)其实是deque实现在的
• deque是使用C语言编写的, 速度很快, 可以经常使用

Python 3.7.3 (default, Mar 28 2019, 10:38:38) [MSC v.1915 32 bit (Intel)]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.4.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from collections import deque

In [2]: a = ['b', 'c', 'd']
# 将list转为deque
In [3]: a_duque = deque(a)

In [4]: a_duque
Out[4]: deque(['b', 'c', 'd'])
# 将tuple转为deque
In [5]: b = (1,2, 3)

In [6]: b_duque = deque(b)

In [7]: b_duque
Out[7]: deque([1, 2, 3])

In [8]: c = {"a": 1, "b": 2, "c": 4}
# 将dice转为deque
In [9]: c_deque = deque(c)

In [10]: c_deque
Out[10]: deque(['a', 'b', 'c'])
# deque的append操作,同list
In [11]: c_deque.append('d')

In [12]: c_deque
Out[12]: deque(['a', 'b', 'c', 'd'])
# deque.appendleft() 将元素添加至左侧第0个位置
In [13]: c_deque.appendleft('e')

In [14]: c_deque
Out[14]: deque(['e', 'a', 'b', 'c', 'd'])
# 浅拷贝
In [15]: c_deque_copy = c_deque.copy()

In [16]: c_deque_copy.count()
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-16-6cfcef3ac009> in <module>
----> 1 c_deque_copy.count()

TypeError: count() takes exactly one argument (0 given)
# 查找某个元素的出现的次数
In [17]: c_deque_copy.count('a')
Out[17]: 1

In [18]: c_deque_copy
Out[18]: deque(['e', 'a', 'b', 'c', 'd'])

In [19]: c_deque_copy[1] = 'a1'

In [20]: c_deque_copy
Out[20]: deque(['e', 'a1', 'b', 'c', 'd'])

In [21]: c_deque
Out[21]: deque(['e', 'a', 'b', 'c', 'd'])
# 合并两个deque, 将后者deque合并至前者的右侧,原址修改,返回None 
In [22]: c_deque.extend(a_duque)

In [23]: c_deque
Out[23]: deque(['e', 'a', 'b', 'c', 'd', 'b', 'c', 'd'])
# 合并两个deque, 将全者的deque合并至前者的左侧, 原地修改, 返回None
In [24]: c_deque.extendleft(b_duque)

In [25]: c_deque
Out[25]: deque([3, 2, 1, 'e', 'a', 'b', 'c', 'd', 'b', 'c', 'd'])
# 返回元素的索引位置
In [26]: c_deque.index('b')
Out[26]: 5
# 在给出的位置插入元素
In [27]: c_deque.insert(2,'f')

In [28]: c_deque
Out[28]: deque([3, 2, 'f', 1, 'e', 'a', 'b', 'c', 'd', 'b', 'c', 'd'])

In [29]: c_deque.rotate('a')
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-29-fc907ea6932a> in <module>
----> 1 c_deque.rotate('a')

TypeError: 'str' object cannot be interpreted as an integer
# 将队尾的指定数量的元素放到队前, 默认为1
In [30]: c_deque.rotate()

In [31]: c_deque
Out[31]: deque(['d', 3, 2, 'f', 1, 'e', 'a', 'b', 'c', 'd', 'b', 'c'])
# 反转整个deque
In [32]: c_deque.reverse()

In [33]: c_deque
Out[33]: deque(['c', 'b', 'd', 'c', 'b', 'a', 'e', 1, 'f', 2, 3, 'd'])

In [34]:

Counter 使用详解

Python 3.7.3 (default, Mar 28 2019, 10:38:38) [MSC v.1915 32 bit (Intel)]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.4.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from collections import Counter

In [2]: users = ['liu1', 'liu2', 'liu3', 'liu1', 'liu1', 'liu2']

# 对列表中的数据进行统计
In [3]: users_counter = Counter(users)

In [4]: users_counter
Out[4]: Counter({'liu1': 3, 'liu2': 2, 'liu3': 1})

# 统计字符中,每个字符出现的次数
In [5]: test = Counter('abcddfdefadsfasdjfoaisdfjasdjfasdfasdfasdfgfhdf')

In [6]: test
Out[6]:
Counter({'a': 8,
         'b': 1,
         'c': 1,
         'd': 11,
         'f': 11,
         'e': 1,
         's': 7,
         'j': 3,
         'o': 1,
         'i': 1,
         'g': 1,
         'h': 1})
# 统计两个字符串中的字符出现次数, 方法1
In [7]: test.update('aadfd')

In [8]: test
Out[8]:
Counter({'a': 10,
         'b': 1,
         'c': 1,
         'd': 13,
         'f': 12,
         'e': 1,
         's': 7,
         'j': 3,
         'o': 1,
         'i': 1,
         'g': 1,
         'h': 1})
# 统计两个字符串中的字符出现次数, 方法2
In [9]: test2 = Counter('abcde')

In [10]: test.update(test2)

In [11]: test
Out[11]:
Counter({'a': 11,
         'b': 2,
         'c': 2,
         'd': 14,
         'f': 12,
         'e': 2,
         's': 7,
         'j': 3,
         'o': 1,
         'i': 1,
         'g': 1,
         'h': 1})
# TOP n 的统计方法
In [12]: test.most_common(3)
Out[12]: [('d', 14), ('f', 12), ('a', 11)]

OrderedDict 使用详解

Python 3.7.3 (default, Mar 28 2019, 10:38:38) [MSC v.1915 32 bit (Intel)]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.4.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from collections import OrderedDict

In [2]: user_dict = OrderedDict()

In [3]: user_dict['b'] = 'liu'

In [4]: user_dict['a'] = 'liu1'

In [5]: user_dict['c'] = 'liu2'

In [6]: user_dict
Out[6]: OrderedDict([('b', 'liu'), ('a', 'liu1'), ('c', 'liu2')])
# 弹出最后一个item.
In [7]: user_dict.popitem()
Out[7]: ('c', 'liu2')

In [8]: user_dict
Out[8]: OrderedDict([('b', 'liu'), ('a', 'liu1')])
# 移动元素至最后 使用场景优先级
In [9]: user_dict.move_to_end('b')

In [10]: user_dict
Out[10]: OrderedDict([('a', 'liu1'), ('b', 'liu')])
# 弹出最后一个key,返回key对应的值
In [11]: user_dict.pop('b')
Out[11]: 'liu'

In [12]: user_dict
Out[12]: OrderedDict([('a', 'liu1')])

ChainMap 使用详解

Python 3.7.3 (default, Mar 28 2019, 10:38:38) [MSC v.1915 32 bit (Intel)]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.4.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from collections import ChainMap

In [2]: dict1 = {'a': 'liu', 'b': "liu1"}

In [3]: dict2 = {"c": "liu3", 'd': 'liu4'}
# 最常使用的方法就是连接多个dict, 若键重复, 则只取第一个dict中的键值
In [4]: new_dict = ChainMap(dict1, dict2)

In [5]: for key, value in new_dict.items():
   ...:     print(key, value)
   ...:
c liu3
d liu4
a liu
b liu1
# 修改dict2, 使其中一个键与dict1的键重复
In [6]: dict2 = {"b": "liu3", 'd': 'liu4'}

In [7]: new_dict = ChainMap(dict1, dict2)

In [8]: for key, value in new_dict.items():
   ...:     print(key, value)
   ...:
b liu1
d liu4
a liu

In [9]: new_dict
Out[9]: ChainMap({'a': 'liu', 'b': 'liu1'}, {'b': 'liu3', 'd': 'liu4'})
# 返回dict的list, 但只是指向原来dict, 不是原dict的copy
In [10]: new_dict.maps 
Out[10]: [{'a': 'liu', 'b': 'liu1'}, {'b': 'liu3', 'd': 'liu4'}]

In [11]: new_dict.maps[0]['a'] = 'liu333'

In [12]: new_dict
Out[12]: ChainMap({'a': 'liu333', 'b': 'liu1'}, {'b': 'liu3', 'd': 'liu4'})

In [13]: dict1
Out[13]: {'a': 'liu333', 'b': 'liu1'}