python之多线程 threading.Lock() 和 threading.RLock()

最新推荐文章于 2025-01-21 11:46:08 发布
最新推荐文章于 2025-01-21 11:46:08 发布
阅读量413 收藏
点赞数
CC 4.0 BY-SA版权
文章标签： python
原文链接：http://www.cnblogs.com/my8100/p/7399600.html
本文详细探讨了Python中线程同步的重要性和实现方法，包括使用threading.Lock()和threading.RLock()进行资源保护，避免多线程环境下的竞态条件，并通过实例展示了不同情况下锁的使用效果。

0.目录

2. threading.Lock() 的必要性
3.观察block
4.threading.RLock() 的应用场景

1.参考

Thread Synchronization Mechanisms in Python

count += 1 不是原子操作，三步操作可能被中断，通过lock将三步操作“封装”为一步操作，要么执行，要么不执行。

counter = 0

def process_item(item):
    global counter
    ... do something with item ...
    counter += 1

# The reason for this is that the increment operation is actually executed in three steps; 
#first, the interpreter fetches the current value of the counter,
# then it calculates the new value,
# and finally, it writes the new value back to the variable.
View Code

Atomic Operations #

The simplest way to synchronize access to shared variables or other resources is to rely on atomic operations in the interpreter.

An atomic operation is an operation that is carried out in a single execution step, without any chance that another thread gets control.

 

 What kinds of global value mutation are thread-safe?

python的原子操作

A global interpreter lock (GIL) is used internally to ensure that only one thread runs in the Python VM at a time. In general, Python offers to switch among threads only between bytecode instructions; how frequently it switches can be set via sys.setcheckinterval(). Each bytecode instruction and therefore all the C implementation code reached from each instruction is therefore atomic from the point of view of a Python program.

In theory, this means an exact accounting requires an exact understanding of the PVM bytecode implementation. In practice, it means that operations on shared variables of built-in data types (ints, lists, dicts, etc) that “look atomic” really are.

For example, the following operations are all atomic (L, L1, L2 are lists, D, D1, D2 are dicts, x, y are objects, i, j are ints):

L.append(x)
L1.extend(L2)
x = L[i]
x = L.pop()
L1[i:j] = L2
L.sort()
x = y
x.field = y
D[x] = y
D1.update(D2)
D.keys()

These aren’t:

i = i+1
L.append(L[-1])
L[i] = L[j]
D[x] = D[x] + 1

Operations that replace other objects may invoke those other objects’ __del__() method when their reference count reaches zero, and that can affect things. This is especially true for the mass updates to dictionaries and lists. When in doubt, use a mutex!
View Code

 

2. threading.Lock()  的必要性

#!usr/bin/env python
#coding:utf-8
import sys
import time
import random
import logging

import threading
import Queue


lock = threading.Lock()     #'function-call ownership'
rlock = threading.RLock()   #thread ownership

logging.basicConfig(level=logging.DEBUG,
                    format = '%(asctime)s - %(threadName)-10s - %(levelname)s - %(message)s')
logger = logging.getLogger()


count = 0

class MyThread(threading.Thread):
    def __init__(self):
        threading.Thread.__init__(self)
        
    def run(self):
        global count
        
        for i in range(100):
            count += 1
        logger.debug('{} finished, count is {}'.format(self.name, count))     

def main():
    logger.debug('initial count: {}'.format(count))
    
    thread_list = [MyThread() for i in range(2)]
    for t in thread_list:
        t.start()
    for t in thread_list:
        t.join()   

    logger.debug('final count: {}'.format(count))


if __name__ == '__main__':
    main()
View Code

修改run函数代码的不同输出：

    def run(self):
        global count

        for i in range(100):
            count += 1
        logger.debug('{} finished, count is {}'.format(self.name, count))     
# 在切换线程之前，某一线程已经完成，两个线程顺序完成，结果几乎不会有误
# 2017-08-20 12:19:30,857 - MainThread - DEBUG - initial count: 0
# 2017-08-20 12:19:30,858 - Thread-1   - DEBUG - Thread-1 finished, count is 100
# 2017-08-20 12:19:30,858 - Thread-2   - DEBUG - Thread-2 finished, count is 200
# 2017-08-20 12:19:30,858 - MainThread - DEBUG - final count: 200        
        
        time.sleep(0.001)
        for i in range(100):
            count += 1
        logger.debug('{} finished, count is {}'.format(self.name, count)) 
# 开头sleep导致两个线程几乎同时启动，结果可能有误
# 2017-08-20 12:24:59,046 - MainThread - DEBUG - initial count: 0
# 2017-08-20 12:24:59,048 - Thread-2   - DEBUG - Thread-2 finished, count is 124
# 2017-08-20 12:24:59,048 - Thread-1   - DEBUG - Thread-1 finished, count is 153
# 2017-08-20 12:24:59,048 - MainThread - DEBUG - final count: 153
        
        for i in range(10000):
            count += 1
        logger.debug('{} finished, count is {}'.format(self.name, count))  
# bytecodes足够导致两个线程交替运行，结果大概率有误      
# 2017-08-20 12:20:17,719 - MainThread - DEBUG - initial count: 0
# 2017-08-20 12:20:17,723 - Thread-1   - DEBUG - Thread-1 finished, count is 12438
# 2017-08-20 12:20:17,723 - Thread-2   - DEBUG - Thread-2 finished, count is 12616
# 2017-08-20 12:20:17,723 - MainThread - DEBUG - final count: 12616        
        
            
        with lock:
            for i in range(10000):
                count += 1
        logger.debug('{} finished, count is {}'.format(self.name, count))    
# lock直到某一线程完成，结果正确
# 2017-08-20 12:20:37,630 - MainThread - DEBUG - initial count: 0
# 2017-08-20 12:20:37,631 - Thread-1   - DEBUG - Thread-1 finished, count is 10000
# 2017-08-20 12:20:37,632 - Thread-2   - DEBUG - Thread-2 finished, count is 20000
# 2017-08-20 12:20:37,634 - MainThread - DEBUG - final count: 20000
        

        for i in range(10000):
            with lock:
                count += 1
        logger.debug('{} finished, count is {}'.format(self.name, count)) 
# 两个线程交替lock，结果正确        
# 2017-08-20 12:21:03,921 - MainThread - DEBUG - initial count: 0
# 2017-08-20 12:21:03,973 - Thread-1   - DEBUG - Thread-1 finished, count is 19979
# 2017-08-20 12:21:03,973 - Thread-2   - DEBUG - Thread-2 finished, count is 20000
# 2017-08-20 12:21:03,973 - MainThread - DEBUG - final count: 20000

 

3.观察block

    def run(self):
        global count

        all = range(10000)  #确保每个线程 +1 的次数
        while all != []: 
            if not lock.acquire(False):  #假设没有参数会导致block，则马上返回false当不block；否则返回true且acquire
                logger.debug('{} wait...{}'.format(self.name, len(all)))   
            else:
                try:
                    count += 1
                    all.pop()
                except Exception as err:
                    logger.debug('{} err, count is {}'.format(self.name, count))   
                finally:
                    # logger.debug('{} release {} {}'.format(self.name, count, len(all)))  #导致两个线程顺序执行？？？
                    lock.release()
        logger.debug('{} finished, count is {}'.format(self.name, count))

输出：

2017-08-20 12:32:55,204 - MainThread - DEBUG - initial count: 0
2017-08-20 12:32:55,210 - Thread-1   - DEBUG - Thread-1 wait...9925
2017-08-20 12:32:55,210 - Thread-1   - DEBUG - Thread-1 wait...9925
2017-08-20 12:32:55,210 - Thread-1   - DEBUG - Thread-1 wait...9925
2017-08-20 12:32:55,210 - Thread-1   - DEBUG - Thread-1 wait...9925
2017-08-20 12:32:55,210 - Thread-1   - DEBUG - Thread-1 wait...9925
2017-08-20 12:32:55,210 - Thread-1   - DEBUG - Thread-1 wait...9925
2017-08-20 12:32:55,210 - Thread-1   - DEBUG - Thread-1 wait...9925
2017-08-20 12:32:55,211 - Thread-1   - DEBUG - Thread-1 wait...9925
2017-08-20 12:32:55,213 - Thread-1   - DEBUG - Thread-1 wait...9925
2017-08-20 12:32:55,213 - Thread-1   - DEBUG - Thread-1 wait...9925
2017-08-20 12:32:55,213 - Thread-1   - DEBUG - Thread-1 wait...9925
2017-08-20 12:32:55,213 - Thread-1   - DEBUG - Thread-1 wait...9925
2017-08-20 12:32:55,214 - Thread-1   - DEBUG - Thread-1 wait...9925
2017-08-20 12:32:55,214 - Thread-1   - DEBUG - Thread-1 wait...9925
2017-08-20 12:32:55,214 - Thread-1   - DEBUG - Thread-1 wait...9925
2017-08-20 12:32:55,214 - Thread-1   - DEBUG - Thread-1 wait...9925
2017-08-20 12:32:55,216 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,216 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,216 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,216 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,216 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,217 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,217 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,217 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,217 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,217 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,217 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,217 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,217 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,217 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,219 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,219 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,219 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,219 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,219 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,220 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,220 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,220 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,220 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,220 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,220 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,220 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,220 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,220 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,220 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,221 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,221 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,221 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,221 - Thread-2   - DEBUG - Thread-2 wait...6036
2017-08-20 12:32:55,226 - Thread-1   - DEBUG - Thread-1 finished, count is 13964
2017-08-20 12:32:55,236 - Thread-2   - DEBUG - Thread-2 finished, count is 20000
2017-08-20 12:32:55,236 - MainThread - DEBUG - final count: 20000

 

4.threading.RLock() 的应用场景

Problems with Simple Locking

lock = threading.Lock()

def get_first_part():
    with lock: # any thread that attempts to acquire the lock will block, even if the same thread is already holding the lock. 
        ... fetch data for first part from shared object
    return data    
 
def get_second_part():
    with lock:
        ... fetch data for second part from shared object
    return data  
    
def get_both_parts():
    with lock:  # other thread may modify the resource between the two calls
        first = get_first_part()
        # between the two calls
        second = get_second_part()
    return first, second    
    
# While simple locks will block if the same thread attempts to acquire the same lock twice, 
# a re-entrant lock only blocks if another thread currently holds the lock. 
rlock = threading.RLock()

 

转载于:https://www.cnblogs.com/my8100/p/7399600.html

深入理解 Python 中的 threading.Lock：并发编程的基石
这家伙很懒，什么都没有留下
10-25 1598
threading.Lock 是 Python 中用于实现线程间互斥锁的基本工具。通过正确使用锁，可以保护共享资源免受数据竞争和不一致问题的困扰。本文介绍了 threading.Lock 的基本概念、使用方法、注意事项以及两个使用案例，希望能帮助读者深入理解并掌握这一工具。在实际编程中，应该根据具体需求选择合适的同步原语和策略来实现并发控制。
python多线程之线程锁（Lock）和递归锁（RLock）实例
专业资深测试者，多年质量管理经验，浪迹网游、金融、房产、招聘及海外行业多年，CDSN百万原创阅读，公众号500+原创文章，提倡测试左移、风险、越权、假设、探索测试，喜欢用抽象思维模式思考和技术探讨，自动化、性能、安全测试实施者，感谢关注！
03-15 1735
一、线程锁 Threading模块为我们提供了一个类，Threading.Lock锁。我们创建一个该类对象，在线程函数执行前，“抢占”该锁，执行完成后，“释放”该锁，则我们确保了每次只有一个线程占有该锁。这时候对一个公共的对象进行操作，则不会发生线程不安全的现象了。 1、我们先建立了一个threading.Lock类对象lock,在run方法里，我们使用lock.acquire()...
Python线程锁 threading.Lock()介绍
qq_42310545的博客
01-21 2276
在 Python 中，线程锁（Thread Lock）是一种同步原语，用于在多线程环境中协调线程之间的访问，避免多个线程同时访问共享资源时出现竞态条件（Race Condition）。线程锁确保在任何时刻只有一个线程可以访问特定的资源。为了实现线程锁，Python的threading模块提供了Lock类，下面将先简单介绍下Lock类的一些基础概念和用法。
python threading lock_python之多线程 threading.Lock() 和 threading.RLock()
weixin_39862716的博客
11-30 517
python之多线程 threading.Lock() 和 threading.RLock()
python threading lock和rlock_正确使用threading.RLock
weixin_39999730的博客
12-22 363
I'm creating a utility in Python that reads data in from a file on startup on a separate thread so that the rest of the GUI components can load. The data gets stored into a list and then appended to a...
python threading lock_python的threading模块中threading.RLock和threading.Lock | 学步园
weixin_39972777的博客
11-30 310
python的threading模块中threading.RLock和threading.Lock | 学步园
Python 线程,with的作用(自动获取和释放锁Lock)
houyanhua1的专栏
10-14 1万+
import threading import time num=0 #全局变量多个线程可以读写，传递数据 mutex=threading.Lock() #创建一个锁 class Mythread(threading.Thread): def run(self): global num with mutex: #with Lock的作用相当于自动获
Python应用实战：python多线程-多线程安全问题&lock与rlock.zip
03-21
总的来说，理解并正确使用锁和可重入锁是解决Python多线程安全问题的关键。在编写多线程程序时，合理地引入锁机制可以防止数据竞争，确保程序的正确性。但同时，过度使用锁可能导致程序的复杂性和性能下降，因此在...
Python线程协作threading.Condition实现过程解析
09-17
总结来说，`threading.Condition`在Python多线程环境中提供了一种灵活的同步机制，通过控制线程的执行顺序和等待状态，确保线程安全地共享资源。它特别适用于那些需要等待特定条件满足才能继续执行的场景。在实际...
python使用threading获取线程函数返回值的实现方法
09-21
`threading`模块是Python的标准库之一，它提供了线程管理的功能，包括创建、同步和控制线程。然而，Python的`threading`模块本身并不直接支持获取线程函数的返回值，这需要我们通过一些额外的方式来实现。以下将详细...
【python】详解threading模块：lock、Rlock的使用（二）
brucewong0516的博客
08-08 2万+
在之前的【python】多线程threading详解一文中，是有对多线程进行一个详细的梳理的。其中就提到了线程锁这一功能。主要基于Rlock实现。本文将进一步总结，丰富线程锁内容。 在使用多线程的应用下，如何保证线程安全，以及线程之间的同步，或者访问共享变量等问题是十分棘手的问题，也是使用多线程下面临的问题，如果处理不好，会带来较严重的后果，使用python多线程中提供Lock 、Rlock 、...
python threading.Lock()
robert198837的专栏
12-08 2561
''' Created on 2013-12-7 @author: Administrator ''' import threading,time b=50 l=threading.Lock() def threadcode():     global b     print "thread %s invoked " % threading.current
python模块学习（threading模块中的Lock类和RLock类）
去海里喂鱼的博客
11-09 618
学习版本3.5.2 Lock类：一个线程对资源进行加锁操作之后其它线程就必须等它释放锁之后才能对该资源进行操作。acquire()与acquire_lock()都是加锁操作，release()与release_lock()都是释放锁操作，locked()与locked_lock()都是查询当前锁的状态。 RLock类：线程对资源进行加锁操作后，该线程还能继续对该资源加锁，加锁和释放锁的数量需要
Python threading中lock的使用
热门推荐
锅炉房刘大爷的博客
03-28 3万+
在多线程中使用lock可以让多个线程在共享资源的时候不会“乱”，例如，创建多个线程，每个线程都往空列表l中添加一个数字并打印当前的列表l，如果不加锁，就可能会这样：# encoding=utf8 import threading import time lock = threading.Lock() l = [] def test1(n): lock.acquire() l.append...
python中thread.Lock()和thread.RLock()的区别
IT之一小佬的博客
09-03 1097
python中thread.Lock()和thread.RLock()的区别
python threading Lock
yuanzhoulvpi的博客
10-21 357
python threading Lock
python threading.lock
Claroja
08-07 478
threading.Lock() threading.RLock() 递归所，同一线程可以被多次acquire，如果使用RLock，那么acquire和release必须成对出现，调用了n次acquire锁请求，则必须调用n次的release才能在线程中释放锁对象 Lock.acquire([blocking]) Lock.release() thread.Lock thread.RLock 参考： https://www.jb51.net/article/154462.htm ...
线程池的使用步骤
weixin_49578980的博客
09-17 2843
线程池的使用步骤 使用线程池的工厂类Executors里面提供的静态方法newFixedThreadPool生产一个指定线程数量的线程池 创建一个类,实现Runnable接口,重写run方法,设置线程任务 调用ExecutorService中的方法submit,传递线程任务(实现类),开启线程,执行run方法 调用ExecutorService中的方法shutdown销毁线程池(不建议执行) public class ThreadPool{ public static void main(Str
10、第七 - 网络编程基础 - 递归锁--threading.RLock()
weixin_30369087的博客
04-08 225
　　Python 中的递归锁，其实就是大锁里面包含了小的锁。Python中为了支持在同一线程中多次请求同一资源，python提供了可重入锁RLock 举例说明 --（可以尝试把threading.RLock() 改为threading.Lock() 进行验证）如下： import threading,time def run1(): print ("grab t...
self.lock = threading.Lock() with self.lock
最新发布
04-30
比如Lock与RLock的区别，处理死锁的方法，以及多线程数据共享的技术。这些扩展问题可以帮助用户深入理解。 检查引用内容是否适用。引用[1]和[3]提到了RLock和Lock的区别，但用户的问题是关于Lock的，所以主要参考...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值