结合线程与异步实时获取爬虫爬取状态status

最新推荐文章于 2024-08-24 10:07:52 发布

原创最新推荐文章于 2024-08-24 10:07:52 发布 · 297 阅读

1 ·

CC 4.0 BY-SA版权

文章标签：

#python #异步 #线程

python 同时被 3 个专栏收录

16 篇文章

订阅专栏

多线程

3 篇文章

订阅专栏

异步

1 篇文章

订阅专栏

本文介绍了一个银行爬虫模型的实现，模型包括登录、获取信息和爬取数据等状态。通过多线程和异步任务，跟踪模型状态变化，确保爬虫运行效率和稳定性。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

import time
from threading import Thread
import asyncio

class bankcrawl_model():
    def __init__(self):
        self.status="sleeping"

    def login(self):
        self.status="logging"
        print("this is status:","logging")
        time.sleep(3)
    def getinfo(self):
        self.status="getinfo"
        print("this is status:","getinfo")
        time.sleep(3)
    def crawl(self):
        print("this is status:","crawl")
        self.status="crawl"
        time.sleep(3)
        self.status="sleeping"
        print("this is status:","None")

class ModelThread(Thread):
    def __init__(self,model):
        Thread.__init__(self)
        self.model=model

    def run(self):
        self.model.login()
        self.model.getinfo()
        self.model.crawl()
        
async def get_status(model):
    old = model.status
    print(old)
    while True:
        new=model.status
        if new!=old:
            old=new
            print(new)
        if new=="sleeping":
            break

if __name__=='__main__':
    print("main start!")
    model=bankcrawl_model()
    modellist=[model]
    start_thread(modellist)
    loop=asyncio.get_event_loop()
    loop.run_until_complete(get_status(model))
    print("main end")